AntFleet

Disagreement · 32b75da9-anthropic-3

First-ever-run backfill writes announced_at=today, so a fork that activates on day-2 will never alert

mismatch
repo 6f7fc663·PR #27·reviewed 1 week ago

Primary finding

First-ever-run backfill writes announced_at=today, so a fork that activates on day-2 will never alert

mediumbughigh
  • skills/fork-first-run-alert/SKILL.md:264-268
  • skills/fork-first-run-alert/SKILL.md:138-142
The backfill rule says 'populate seen for every current ACTIVE/POWER fork'. That's correct for suppression on day one. However, the spec elsewhere says the seen-list is keyed by `full_name` and the only way to re-alert is manual reset. Combined with the constraint 'A fork that goes ACTIVE → STALE → ACTIVE again does NOT re-fire', this means: every fork that was already ACTIVE on the first day this skill ran is *permanently* suppressed, even if its very first workflow run actually happened weeks later. That's intentional for already-active forks, but the doc frames this only as 'no per-fork alerts emitted' on day one — it doesn't acknowledge that the entire pre-existing fleet is silently lost forever, including the very signal this skill exists to catch when fork-cohort hasn't run yet. Operators reading the doc may be surprised.

Recommendation

Either emit a one-line summary notification on first-ever-run listing the backfilled fleet (named alert by batch), or document the trade-off explicitly in the 'Why this exists' section so operators understand day-one silence is intentional.

Counterpart finding

Documented 80-fork cap and pushed_at sorting not reflected in fallback snippet (risk of budget overrun or inconsistent order)

lowdocs-gaphigh
  • skills/fork-first-run-alert/SKILL.md:124-131
  • skills/fork-first-run-alert/SKILL.md:267
  • skills/fork-first-run-alert/SKILL.md:118-122
The text promises a hard 80-fork cap and sorting by pushed_at desc, but the provided command neither sorts nor trims to 80. This inconsistency can lead to scanning more forks than intended and unpredictable ordering for trimming decisions.

Recommendation

Amend the snippet to explicitly sort and cap: after collecting forks, apply jq 'sort_by(.pushed_at) | reverse | .[:80]' before iterating. Also note logging 'truncated_at=80' when applied.

Why this didn't post

This finding didn't meet AntFleet's unanimous agreement threshold. Both frontier models review every PR independently; only findings they both flag with the same severity and category are posted to the PR. This one fell through.

read the methodology →