AI Investment Workflows Need an Exception-Rate Test
AI adoption in asset management is changing research operations, but the useful unit of progress is fewer unresolved exceptions—not faster summaries.
AI is entering investment work through the back office of judgment: sorting disclosures, comparing evidence, routing exceptions, and preparing research queues. The important question is not whether a model can produce a fluent summary. It is whether the new workflow leaves fewer unresolved exceptions without hiding more of them.
That is the practical implication of recent evidence. Chen, Sialm, and Xu’s NBER Working Paper 35273 documents the growth and performance of AI-driven investing using investment-adviser disclosures, while a recent industry operating-model analysis argues that AI-embedded workflows should be measured with cycle-time and exception-rate reductions. The first is a research study; the second is an industry claim. Neither proves that a faster or more automated desk produces better investment decisions.
The workflow change is therefore a measurement change. An analyst can use a model to extract a company’s guidance revisions, classify the evidence behind each change, and place uncertain items in a review queue. The human still decides what counts as a comparable period, whether a document is authoritative, and whether an apparent change is economically meaningful. The model’s output is a set of dated claims with source passages and confidence labels—not a recommendation.
This is useful because investment research contains many low-value repetitions but also a small number of high-cost mistakes. A system that reduces reading time while increasing unsupported claims has not improved the workflow. A system that surfaces more exceptions may look slower at first while making review more auditable. The distinction matters for discretionary analysts, systematic researchers, and data teams building internal tools.
An implementable version has four steps. First, freeze the input set and timestamp every document. Second, ask the model to extract claims into a fixed schema: metric, period, source passage, transformation, and unresolved question. Third, run deterministic checks for missing periods, unit changes, duplicate sources, and contradictions. Fourth, send only exceptions to a human reviewer, who records the disposition and reason. The audit trail should preserve the original passage and the model version; otherwise later reviewers cannot tell whether an apparent improvement came from better data, a prompt change, or selective reporting.
Try this safely in 60 minutes. Choose five public filings or earnings releases from one sector and create a manual baseline: how many claims can you extract, and how many require correction? Run the same packet through an AI assistant with the fixed schema. Measure extraction completeness, unsupported-claim rate, contradiction detection, reviewer minutes per document, and the share of items that remain unresolved. Observe only the current packet; do not use the output to trade or to make an allocation decision. Stop if the model cannot quote the source passage for a material claim, if inputs become unavailable, or if the reviewer cannot reproduce the disposition from the log.
The test has serious causal limits. Five documents are not a representative sample. A carefully chosen sector can make the model look better or worse, and human reviewers may unconsciously change the baseline after seeing the AI output. Reported exception-rate gains can also reflect a vendor’s definition of an exception. More fundamentally, cleaner evidence handling does not establish predictive power, and predictive power in a historical sample does not survive costs, latency, regime change, or crowding automatically.
The judgment is simple: investment AI should earn its place by reducing the number of unresolved, unauditable decisions—not by maximizing generated text or minimizing analyst minutes. Treat speed as a hypothesis. Treat exception quality, provenance, calibration, and reproducibility as the evidence.
For context, compare this workflow with AI Investment Research Needs Expiry Dates for Its Evidence and AI Monitoring Is Useful Only When It Shows What Changed.
Sources: NBER Working Paper 35273; F2 Strategy, “The Next Generation Operating Model”.