AI Investment Agents Need Near-Miss Logs Before They Earn More Autonomy

As agentic finance moves from pilots to linked tasks, investment teams should measure near misses and handoffs before granting systems more authority.

Minimal editorial illustration of an AI investment workflow passing through evidence and human-control checkpoints.

The next control for an AI investment agent should not be a larger model or a longer task list. It should be a near-miss log: a record of where the system almost used the wrong source, crossed an authority boundary, missed a handoff, or produced an answer that a reviewer could not reproduce.

That is the practical implication of two October signals. BCG’s Global Asset Management Report 2026 says agentic systems are pushing firms toward redesigned workflows, not just isolated pilots. The UK Financial Conduct Authority, meanwhile, has said that agentic finance needs clear accountability, operational resilience, failure and near-miss reporting, and continuous conditional assurance. Neither source demonstrates better investment returns. Together they point to a more useful test: can an investment workflow show where the agent was uncertain and how a human contained the uncertainty?

The workflow changed from answer review to control review

Traditional research automation usually ends with an analyst checking a summary or a spreadsheet. An agentic workflow links retrieval, extraction, comparison, calculation, and routing. The agent may decide which document to open next, ask another tool for data, or send an exception to a reviewer. That makes the handoffs—not just the final prose—the object of review.

For an investment-data or research team, the new unit of work is a supervised case. Each case should have a frozen question, permitted sources, data timestamp, model and prompt version, tools called, intermediate artifacts, confidence or uncertainty flags, human decisions, and final disposition. The agent can propose a comparison of filings or flag a change in a risk disclosure. It should not silently expand the source set, redefine the sample, or turn an unverified signal into an allocation instruction.

The near-miss log makes those boundaries measurable. A near miss is not only a harmful event. It includes a wrong-document retrieval caught before use, a stale data input identified by a reviewer, a calculation that failed a reconciliation, an unsupported citation, an escalation that arrived too late, or a human override that was necessary but not recorded. Counting only successful cases creates reporting and survivorship bias; counting only severe failures hides the small frictions that predict unsafe scaling.

A reproducible control loop

Start with a narrow research queue, such as comparing the latest two public filings for changes in stated risk factors. Define the source allowlist and the fields the agent may extract. Require the system to return source passages, document dates, a structured change record, and an explicit “unable to verify” state.

Then place controls at four points. First, retrieval control checks that the document is on the allowlist and that its date is inside the intended window. Second, extraction control checks the quoted passage against the structured field. Third, calculation control reconciles counts, units, and changes against a deterministic script or a second human-readable calculation. Fourth, disposition control requires a human to accept, reject, or escalate the result before it enters a research note.

Every exception gets a short code: source, freshness, extraction, calculation, authority, or handoff. Record expected behavior, observed behavior, detection point, reviewer, correction, and whether the same failure appeared again. This is not a claim that an exception rate predicts investment performance. It is an operational measure of whether the workflow is becoming more reliable, more reproducible, or merely more fluent.

The FCA’s October account is relevant because it treats accountability and resilience as design requirements, and says significant failures and near misses should be reported. Its examples concern financial services more broadly, not an equity-research benchmark. BCG’s report is an industry perspective, not causal evidence. The investment-team inference is therefore limited: if the workflow is gaining autonomy, its evidence and authority transitions need telemetry before its task scope expands.

Practice lab

Exercise: For 60 minutes, build a paper-only near-miss log for 10 public filing comparisons. Use the same frozen prompt, source allowlist, and extraction schema for all cases. Do not connect a brokerage account or create an order.

Baseline: Have a human complete two comparisons manually before running the agent on the remaining eight. Metrics are: source-date accuracy, quoted-passage support, reconciliation errors, escalation rate, near misses per case, and median minutes to a reviewable record. The observation window is the session plus a seven-day recheck of three cases for stale links or changed source availability.

Stop condition: Stop the exercise if the agent invents a source, cannot expose the passage behind a field, repeatedly crosses the allowlist, or produces an actionable recommendation instead of a research record. Treat any unlogged human correction as a process failure, not a clean pass.

What the log cannot prove

A lower near-miss rate may mean the task became easier, the reviewer became less attentive, or the system learned to suppress visible exceptions. A higher rate may reflect better detection. Selection bias appears if teams log only high-profile cases; measurement bias appears if reviewers disagree about what counts; non-stationarity appears when filings, model behavior, tools, or prompts change. Vendors have incentives to emphasize throughput, while internal teams have incentives to show successful deployment.

The judgment is consequently narrow. Near-miss telemetry is a gate for workflow authority, not a score for stock selection and not evidence of alpha. Discretionary investors should inspect the evidence and handoff trail. Systematic teams should version the agent, data, prompts, and control rules together. Builders should make uncertainty, refusal, and escalation first-class outputs. More autonomy is justified only when the record shows what changed, what was checked, and where a human still had to decide.


阅读中文版本 →