The $400 Infinite Loop That Taught Me to Watch Token Budgets
An autonomous retry loop + a silent schema mismatch burned through a week's API budget in twenty minutes. Here is how I set guardrails I could actually see.
Autonomous agents are wonderful right up until the moment they become very determined to fail.
I shipped a background workflow designed to enrich customer records by calling an external search tool, parsing the result, and retrying if a required field was missing. In local testing with three mocked records, it felt magical.
In staging, one real record came back from the third-party API with null in a field my validator expected as a string.
What happened in the twenty minutes I wasn’t looking
The agent didn’t crash. Instead, it politely reasoned:
- “The field is missing, let me call the search tool again with a slightly different query.”
- The tool returned the exact same
null. - The agent packed the growing conversation history—plus the full tool schema—into the next prompt and tried again.
Every iteration added thousands of input tokens. Because each HTTP call succeeded, no alert fired until my billing notification hit my inbox.
Guardrails need visibility, not just maxRetries
Setting a hard cap on iterations (maxSteps) was the obvious fix, but the deeper lesson was observing token velocity and loop patterns in real time.
Now, every agent workflow I ship tracks:
- Tokens per trace (alerting when a single run crosses 3× the p95 baseline)
- Repeated tool signatures (detecting when the same tool is called twice with identical arguments)
- Time-to-First-Token vs. total duration so users get honest UI feedback while the system stays within budget
That mistake paid for itself many times over: it turned cost and latency from an afterthought on an invoice into first-class signals in my traces.