Count the Uncertain Decisions, Not the Agents
A field guide to placing a model inside deterministic workflows—and testing whether that boundary actually helps.
When a workflow becomes unreliable, “Should we add another agent?” is usually the wrong first question. Start with a more diagnostic one: which decision cannot be expressed safely as ordinary, testable software? If the answer is “none,” adding an agent adds another moving part without solving the bottleneck. If one step requires interpreting messy language or adapting to human feedback, that is a candidate for a bounded model call—not a reason to hand the whole process to an autonomous loop.
That is the useful idea behind HumanLayer’s 12 Factor Agents: treat an agent as software with an LLM at selected decision points, not as a prompt surrounded by an unlimited bag of tools. This is the author’s engineering framework and reported experience, not a controlled study showing that this architecture is always safer or better.
A deployment example makes the boundary visible
HumanLayer describes a deployment assistant built around deterministic code. After a person merges a change, software deploys it to staging and runs end-to-end tests. A focused agent then receives a specific commit and can propose a production action. The application—not the model—checks the proposed tool call and asks a human to approve it. In the reported example, the human redirects the order (“deploy the backend first”); the agent proposes the backend action, approval is requested, and deterministic code executes only after approval. The frontend follows through the same gate. Production tests then determine whether the workflow is complete or needs a rollback review.
The important design choice is not “one agent.” It is where interpretation happens and who retains authority . The model translates a human correction into a structured next action. Code owns the deployment sequence, validates the action, pauses for approval, records state, and runs tests. The source presents this as a real project example; it does not publish comparative failure rates or prove that every deployment should use an agent.
Find the smallest useful decision boundary
For your own workflow, draw the steps before choosing an agent framework. Mark each step as one of three kinds:
- Rule-governed: the input and correct action can be stated explicitly. Keep this in ordinary code where practical: permission checks, required test runs, fixed ordering, and retries.
- Language-dependent: the system must interpret a request, explanation, or correction whose wording varies. Consider a model here, but make its output a small, typed choice such as
deploy_backend,ask_for_clarification, orstop. - Authority-bearing: the step changes production, money, access, or another consequential state. The model may propose; deterministic policy and an authorized person decide whether it executes.
Then connect the model step to a narrow action schema. Reject unknown actions and malformed arguments in code. Keep checks such as “is this commit approved?” outside the model’s discretion. If the proposal is incomplete or conflicts with policy, return a concrete error or ask for clarification; do not let an open-ended retry loop silently invent a workaround.
This boundary also tells you when not to use an agent. If requests come from a stable form, rules cover the cases, and exceptions can be routed to a person, a deterministic workflow may be simpler. If the model cannot be given a bounded action vocabulary, or if its decision cannot be reviewed before an irreversible step, the design is not ready for more autonomy.
A small test you can actually run
Hypothesis—not a result: a bounded language step can handle varied human corrections without making the surrounding workflow harder to inspect. To test it, prepare a fixed set of representative requests and corrections, including clear instructions, ambiguous wording, unauthorized actions, and requests that contradict the required order. Compare two implementations on the same cases: an end-to-end agent and a deterministic workflow with one constrained decision step.
Save the cases, model inputs and outputs, proposed actions, validation errors, approval decisions, final state, and human review time. Before running anything, define success: no unauthorized action executes; ambiguous or invalid requests are stopped or clarified; each outcome can be reconstructed from the trace; and the constrained version does not lose required task correctness. A single safety violation is a stop condition. If the bounded design is no easier to audit or operate, the experiment has not justified the added model step.
The lesson is conditional, not “fewer agents are always better”: keep predictable transitions in software; spend model autonomy only on a language decision you can name, constrain, and inspect. Count those decisions before you count agents.
Practice card
- Inputs: one real workflow, five to ten representative cases, and its current rules.
- Steps: map fixed transitions; mark uncertain language decisions; define typed actions; place policy and approval checks outside the model; log every transition.
- Artifact: a workflow diagram, action schema, case set, and replayable trace.
- Quality check: unauthorized actions never execute, ambiguity is surfaced, and every result has an inspectable reason.
- Do not generalize from: a small local exercise to production reliability or universal agent-count advice.
Sources
- First-party source: HumanLayer, “12 Factor Agents”
- Chinese companion: 别数智能体,先找出不确定的决策
- Related field note: Find the First Wrong Step