AntFleet

Disagreement · 8ff8c1af-anthropic-3

OpenAIProvider does not check options.signal.aborted before estimating output tokens on aborted streams

mismatch
repo 56f59a0d·PR #2·reviewed 4 days ago

Primary finding

OpenAIProvider does not check options.signal.aborted before estimating output tokens on aborted streams

lowbughigh
  • src/providers/openai.ts:148-161
If a request is aborted before the server returned a usage block, both inputTokens and outputTokens default to 0 and are then estimated from messages.length / responseText.length. For aborted requests, this leads to charging the user for ~all of their input even though the request may not have been billed by the upstream provider. Conversely, an unestimated aborted output looks like 'zero output' which pricing.calculateCost will then treat as truly zero.

Recommendation

Skip estimation (or mark usage as undefined/unknown) when options.signal.aborted is true; only estimate on successful completions lacking usage info.

Counterpart finding

Token usage estimation in OpenAI provider ignores system prompt, undercounting input tokens

lowmaintainabilityhigh
  • src/providers/openai.ts:154-157
  • src/providers/openai.ts:231-233
Both streaming and non-streaming estimations exclude the system prompt text, which can be large and materially affect cost tracking. This under-reports input tokens when the API does not return exact usage.

Recommendation

Include options.systemPrompt.length in the estimation. For example: baseLength = (options.systemPrompt?.length ?? 0) + messages.reduce(...); Then divide by an appropriate chars-per-token heuristic per model.

Why this didn't post

This finding didn't meet AntFleet's unanimous agreement threshold. Both frontier models review every PR independently; only findings they both flag with the same severity and category are posted to the PR. This one fell through.

read the methodology →

From the same review

These findings passed the unanimous gate on the same PR review. The disagreement above was filtered out; the findings below were posted.

← back to all disagreementsview public receiptssee unanimous findings + anatomies →
Tweet ↗