Pre-Execution Budget Enforcement: A Circuit-Breaker Architecture for Autonomous AI Agents

Most teams running LangChain, CrewAI, or AutoGen pipelines in production have some form of cost observability: token counters logged per call, a dashboard aggregating spend by pipeline or by day, maybe a Slack alert when a threshold is crossed. All of that is downstream of the spend. By the time a billing alert fires, the calls have already been made and the money is already gone. In the LangChain A2A case, the Verifier agent never approved the Analyzer's output and never issued a clearly bounded request. It kept asking for "further analysis" in open-ended terms, the Analyzer kept complying, and the exchange repeated for eleven days. The post-mortem's own framing is worth stating plainly: the team had observability, not enforcement. Observability let them see the problem after enough cost had accrued to trip a billing threshold. It did nothing to prevent the problem, and nothing to halt the agents before the next call completed.

This matters because agent-to-agent loops do not look like traditional runaway processes. A crashed server pegs a CPU graph you can see instantly. A looping agent pair looks like normal, expensive, ongoing work: valid API calls, valid tokens, valid responses, just going nowhere. Nothing about the traffic pattern looks anomalous to a system built to detect anomalies; it only looks anomalous once you ask whether the work is converging, and almost nothing in a standard orchestration stack asks that question by default. If you are also thinking through the broader risk surface an autonomous agent introduces once it can call external tools or the network, the companion piece on the AI agent enterprise security threat model covers that ground; this guide is narrowly about stopping the spend before it happens, not diagnosing why a specific incident occurred.

A circuit breaker cannot be bolted onto a pipeline that has no visibility into its own cost or iteration count. Before you write any trip logic, four things need to already exist.

Start by naming the ceilings explicitly, in writing, before you touch code. A single "budget" number is not enough; you need at least four dimensions, because a runaway loop can blow through any one of them while looking fine on the others.

1. Token ceiling per agent invocation. How many input plus output tokens can one agent call consume before it is refused.
2. Dollar ceiling per pipeline run. The aggregate cost across every agent in a single logical task, not just one agent's slice of it.
3. Iteration ceiling per agent pair or loop. The maximum number of times two agents (an Analyzer and a Verifier, for example) can exchange messages before the loop is forcibly ended regardless of whether either side thinks it is done.
4. Wall-clock ceiling per run. A hard time limit independent of cost, because a slow, cheap loop can still tie up compute and downstream resources indefinitely.

Some orchestration frameworks already expose native primitives for a subset of these. CrewAI's Agent class accepts max_iter (default 20 iterations per agent task), max_rpm (a requests-per-minute rate limit), and max_execution_time (a wall-clock cap in seconds) directly as constructor arguments, no custom middleware required. AutoGen's ConversableAgent exposes max_consecutive_auto_reply, which caps how many auto-replies an agent will send in a row before requiring human input; setting it to 0 is a hard stop on autonomous replies entirely. Treat these as your floor, not your ceiling: a default max_iter=20 is not a considered decision, it is whatever number the framework maintainers picked as a reasonable-sounding default, and it may be far too loose for a Verifier that has no fixed rubric for "good enough." Set your own numbers based on what a legitimate task actually costs, not on what the framework ships with.

The critical design decision is where the check runs. It has to run before the call is dispatched, not after the response comes back and not inside error handling that a retry loop can route around. In LangChain, the natural attachment point is a custom subclass of BaseCallbackHandler, registered against the lifecycle methods that fire around each call: on_llm_start, on_chain_start, and on_agent_action. Setting raise_error=True on the handler means an exception raised inside one of these methods propagates out of the run rather than being silently swallowed, which is what turns a callback from a passive logger into an active gate.

class BudgetCircuitBreaker(BaseCallbackHandler):
    raise_error = True

    def __init__(self, ledger, ceiling_usd, ceiling_calls):
        self.ledger = ledger          # external, tamper-resistant counter store
        self.ceiling_usd = ceiling_usd
        self.ceiling_calls = ceiling_calls

    def on_llm_start(self, serialized, prompts, **kwargs):
        spent = self.ledger.get_spend(kwargs["run_id"])
        calls = self.ledger.get_call_count(kwargs["run_id"])
        if spent >= self.ceiling_usd or calls >= self.ceiling_calls:
            self.ledger.record_trip(kwargs["run_id"], spent, calls)
            raise BudgetExceededError(
                f"Ceiling reached before call: ${spent} spent, {calls} calls"
            )

For CrewAI and AutoGen pipelines, or for custom orchestrators that do not expose a LangChain-style callback surface, the equivalent pattern is a middleware wrapper function that every agent-to-model or agent-to-agent call is routed through, checking the same external ledger before forwarding the call and raising before any network request is issued. The specific hook name changes by framework; the requirement does not: the check has to execute synchronously, before the outbound call, and its failure mode has to be a hard exception, not a log line.

Borrow the standard circuit-breaker state machine from distributed systems: closed (calls proceed normally), open (calls are refused outright), and optionally half-open (a single test call is allowed through to check whether conditions have changed). For agent budget enforcement, keep it simple: closed and open are usually sufficient, because unlike a flaky downstream service, a budget ceiling does not heal itself, a human has to raise it deliberately.

Two details separate a real trip from a cosmetic one. First, the trip has to raise a distinct, typed exception (BudgetExceededError, not a generic RuntimeError) so that generic retry-on-error logic elsewhere in the pipeline cannot catch it and try again, which would silently defeat the entire mechanism. Second, the trip event has to be logged and alerted through a channel separate from ordinary errors, because an operator scanning logs for "anything red" needs to be able to tell instantly that a budget ceiling fired, as opposed to a transient API timeout that resolved on its own.

Once a breaker trips, resist the urge to auto-reset it on a timer. A ceiling that quietly re-opens after five minutes reintroduces exactly the failure mode this architecture exists to close. Require an explicit, logged, human-initiated reset before the pipeline can run again.

Cost ceilings catch loops that spend money on every iteration. They do not catch a hook that recursively invokes other hooks, or a sub-agent that spawns further sub-agents, each of which may be individually cheap but collectively unbounded. Operators working through this exact problem in multi-agent coordination failures have converged on two complementary fixes.

The first is a depth counter that travels with the call itself rather than living only in process memory: an incrementing value (something like _hop_count or a depth field passed via an environment variable) that each hook or sub-agent checks and rejects once it exceeds a fixed maximum, before doing any real work. This survives a process restart in a way that an in-memory counter does not, because the counter is attached to the message, not to the process that happens to be running it.

The second is a wall-clock watchdog owned by a process one layer above the agent, not by the agent or its own hooks. A hook has no authority to bound its own runtime; if it hangs, it hangs. The pattern that has proven reliable is a parent supervisor process that enforces a hard timeout and, on expiry, kills the entire process group, not just the direct child. Killing only the direct child leaves any MCP or tool-call descendants free to keep mutating state and issuing calls after the parent believes the run has stopped, which is precisely how a "stopped" pipeline keeps spending. This is the same default-deny posture that governs network egress for autonomous agents, covered in more depth in the guide to default-deny egress proxies for AI agent SSRF prevention: nothing proceeds unless an explicit check upstream has already allowed it, and the absence of an allow is treated as a deny, not as a pass-through.

A breaker you have not deliberately tripped is a breaker you do not actually have. Build a sandboxed test harness with a deliberately misconfigured Verifier-style agent, one with an open-ended, no-fixed-rubric approval criterion that will never say yes, and run it against the real orchestration path with intentionally tiny ceilings: a handful of iterations, a single dollar of budget, a short wall-clock window.

The validation bar is simple to state and worth holding to precisely: reproduce a two-agent, no-termination-predicate loop in a controlled environment, run it against your instrumented pipeline with production-representative ceilings, and confirm the run halts within one call of the configured limit, with a logged trip event, a terminated process group, and zero further ledger writes afterward. Compare the wall-clock time from loop start to trip against the 264 hours it took a billing dashboard to surface the LangChain A2A incident. If your breaker takes anywhere close to that long to fire, the check is not actually pre-execution, it is observability wearing a disguise.

Run this test on a schedule, not just once at build time. Framework upgrades, refactors to the callback wiring, or a well-meaning change that adds a retry wrapper around the model call can all quietly route traffic around the check without anyone noticing until the next incident.

Hard ceilings are not a complete answer, and treating them as one creates its own blind spot.

Every ceiling tight enough to stop a fast runaway loop will, at some point, also stop a legitimate long-running job: a large document synthesis task, a big batch analysis, a genuinely complex multi-step research request that happens to need more iterations than the ceiling allows. That is not a bug in the architecture, it is the actual tradeoff the architecture makes, and it needs to be made explicitly rather than tuned away by accident.

The asymmetry favors erring tight. A false positive costs a retried task and an operator's few minutes reviewing why it tripped. A false negative, a ceiling set loose enough that the real runaway case slips under it, costs exactly what the LangChain A2A incident cost: eleven days and tens of thousands of dollars before anyone noticed. Set ceilings closer to what legitimate work actually requires, and build an explicit, logged, time-boxed override path for the cases that genuinely need more room, rather than raising the default ceiling for everyone to avoid the friction of the rare edge case. An override that requires a human to type a reason and a duration is friction by design; a ceiling nobody ever hits is not doing its job.