GPT-5 finding
Large-request detection stringifies structured content, misclassifying request size
mediumbugmedium
- python/smart_router.py
In Anthropic-style messages, content is often a list of blocks. Stringifying arrays/objects inflates character counts (including JSON syntax), or may undercount when images/tool calls exist. This can route to a 'large' model unnecessarily, affecting cost/latency.
Recommendation
Compute length using message semantics: if content is a list, sum text lengths of text blocks; ignore non-text blocks or approximate with better heuristics. Consider token-based estimation if available.