AntFleet

Disagreement · 847fbd10-openai-3

Large-request detection stringifies structured content, misclassifying request size

solo GPT-5
repo 7e34f3ef·PR #3·reviewed 1 week ago

GPT-5 finding

Large-request detection stringifies structured content, misclassifying request size

mediumbugmedium
  • python/smart_router.py
In Anthropic-style messages, content is often a list of blocks. Stringifying arrays/objects inflates character counts (including JSON syntax), or may undercount when images/tool calls exist. This can route to a 'large' model unnecessarily, affecting cost/latency.

Recommendation

Compute length using message semantics: if content is a list, sum text lengths of text blocks; ignore non-text blocks or approximate with better heuristics. Consider token-based estimation if available.

Other reviewer

The other reviewer flagged nothing in this file/line range.

Why this didn't post

This finding didn't meet AntFleet's unanimous agreement threshold. Both frontier models review every PR independently; only findings they both flag with the same severity and category are posted to the PR. This one fell through.

read the methodology →