AntFleet

Disagreement · 11ac0e29-openai-6

overallSuccessRate includes in-progress runs; per-skill metric excludes them (inconsistent definition)

solo GPT-5
repo 6f7fc663·PR #26·reviewed 1 week ago

GPT-5 finding

overallSuccessRate includes in-progress runs; per-skill metric excludes them (inconsistent definition)

lowbughigh
  • dashboard/app/api/analytics/route.ts:111-113
  • dashboard/app/api/analytics/route.ts:189-194
Per-skill successRate divides by total - inProgress, but overallSuccessRate divides by totalRuns including in-progress, leading to inconsistent rates and misleading summaries.

Recommendation

Align the definitions: either exclude in-progress from overallSuccessRate or document the intentional difference. Prefer success/(success+failure).

Other reviewer

The other reviewer flagged nothing in this file/line range.

Why this didn't post

This finding didn't meet AntFleet's unanimous agreement threshold. Both frontier models review every PR independently; only findings they both flag with the same severity and category are posted to the PR. This one fell through.

read the methodology →
← back to all disagreementsview public receiptssee unanimous findings + anatomies →
Tweet ↗