AntFleet

Disagreement · be00b9c6-openai-0

Inconsistent JSON snapshot schema between Step 9 (read) and Step 13 (write) breaks week-over-week delta logic

mismatch
repo 6f7fc663·PR #5·reviewed 1 week ago

Primary finding

Inconsistent JSON snapshot schema between Step 9 (read) and Step 13 (write) breaks week-over-week delta logic

highapi-contracthigh
  • skills/fork-skill-digest/SKILL.md:132-147
  • skills/fork-skill-digest/SKILL.md:298-304
  • skills/fork-skill-digest/SKILL.md:310
Step 9 defines the on-disk state schema that the skill will read on next run (e.g., DEFAULT_FLIP_ENABLE is an array of strings). Step 13 specifies a different schema to persist (objects with forks/pct). Persisting schema B and reading schema A on the next run will cause delta computation to misinterpret or fail, violating the stated contract that the JSON is the source of truth.

Recommendation

Pick one schema and use it consistently for both reading and writing. Recommend adopting the richer Step 13 schema and updating Step 9 to read that shape, including explicit keys (skill, forks, pct, model, var). Add a version field (e.g., schema_version) to allow future evolution.

Counterpart finding

VAR_HOTSPOT and bucket priority can shadow MODEL_CONSENSUS / EMERGING for the same skill, but spec says 'first match wins' inconsistently

lowapi-contractmedium
  • skills/fork-skill-digest/SKILL.md:122-137
  • skills/fork-skill-digest/SKILL.md:174-186
Step 7 mandates that a skill appears in only one bucket (first match wins, with MODEL_CONSENSUS before VAR_HOTSPOT). But the article template in step 11 lists separate Model overrides and Var hotspots sections as if a skill could appear in both. A skill that hits both MODEL_CONSENSUS and VAR_HOTSPOT will only appear under Model overrides, silently dropping the var-hotspot signal from the digest. This is a real loss of analytic signal that the doc never acknowledges.

Recommendation

Either allow MODEL_CONSENSUS and VAR_HOTSPOT (and EMERGING) to coexist for the same skill (since they describe orthogonal dimensions), or explicitly state in the article template that each skill appears in at most one section so the omission is intentional.

Why this didn't post

This finding didn't meet AntFleet's unanimous agreement threshold. Both frontier models review every PR independently; only findings they both flag with the same severity and category are posted to the PR. This one fell through.

read the methodology →
← back to all disagreementsview public receiptssee unanimous findings + anatomies →
Tweet ↗