AI Can Classify Investment Disclosures—If Humans Audit the Measurement
A new study shows how AI can turn adviser disclosures into a measurable research workflow, while also showing why the measurement design matters as much as the model.
Investment teams have a new way to use AI that is more defensible than asking a model to “find alpha”: classify a large, messy set of disclosures, then measure where the classification breaks. A study published in Frontiers in Artificial Intelligence applies that workflow to Form ADV Part 2A brochures from U.S. investment advisers. Its important result is not that an LLM can read filings. It is that the research conclusion changes materially when the team changes the definition, sample, weighting, or validation design.
That is the practical frontier. AI can make document-based investment research more systematic, but only if the output is treated as a measurement instrument rather than ground truth. For a research desk, the deliverable is not a percentage in a slide. It is a dated, reproducible dataset with a construct, an error audit, and a record of which humans checked the edge cases.
The workflow changed from reading to measuring
The study drew a stratified random sample of 400 SEC-registered advisers from a population of 16,223 firms and successfully classified 388 available brochures. The researchers prespecified five labels: investment-process use, operational or client-service use, AI-related risk, explicit non-use, and named AI vendors. An LLM performed the first pass. The team then tested same-family reproducibility, commissioned a blinded cross-family recoding of 180 brochures, and performed human verification on 158 cases. Read the study.
This replaces a familiar analyst workflow—search, read, copy a quote, count mentions—with a pipeline: define the unit of observation, classify, sample for review, estimate error, and publish the measurement rules alongside the result. The same architecture applies to investment research on earnings-call language, risk-factor changes, manager letters, fund mandates, or alternative-data vendors. The model handles repetitive coding; the analyst owns the ontology and the adjudication.
The reported headline is useful but easy to misuse. AI use appeared in 23.7% of observed brochures, or 22.7% after survey weighting, while 27.6% contained AI-related risk language. Disclosure rose from 10.4% in the smallest AUM quartile to 43.8% in the largest. Against human coding, the composite any-use measure had 0.76 precision and 0.81 sensitivity. A literal keyword rule missed or mislabeled roughly a quarter of cases.
Those numbers describe this measurement design, not the true adoption rate of investment AI. The study reports that the same corpus can produce estimates of 9.9%, 13.1%, or 23.7% under different constructs, adviser universes, and weighting schemes. A separate industry survey may report a much higher adoption figure because it includes pilots or internal uses that do not appear in a fiduciary brochure. That gap is a comparison of objects, not proof of undisclosed adoption.
What an investment desk can reproduce
Start with a document set whose availability date is recorded. Write a short label rubric before looking at outcomes. Decide whether the question concerns any mention of AI, material investment-process use, operational use, risk language, or a named vendor. These are different research questions and should not be collapsed into one score.
Use a model for a bounded first pass. Require a label, quoted passage, document location, confidence, and “not enough evidence” option. Keep the raw text and the model version. Stratify the human review rather than reviewing only the model’s uncertain cases: sample positive, negative, and borderline classifications across firm size and adviser type. Record precision, sensitivity, disagreement, and the error pattern. If the classification changes after a rubric revision, retain both versions and explain the change.
This is close to the evidence-ledger discipline described in Agentic Trading: Why LLM Trading Agents Need an Evidence Ledger and the point-in-time concerns in Time-Series Foundation Models and Financial Return Priors. In each case, the investment value comes from preserving provenance and timing, not from making the output sound more certain.
A 60-minute research-only test
Use 30–50 publicly available adviser brochures, or another fixed document set, and do not use the result to select securities, clients, or trades. Baseline: a keyword count for “artificial intelligence,” “machine learning,” and named vendors. AI workflow: apply a five-label rubric, require a quote and page reference, and randomly send 20% of each label to human review. Compare keyword-versus-rubric agreement, human precision and sensitivity on the reviewed sample, the share of “uncertain” cases, and the number of material classifications that lack a directly supporting passage. Observation window: one session for the pilot, followed by a seven-day recheck of the same documents for reproducibility. Stop if the model cannot cite the passage, if reviewers cannot agree on the label definition, or if changing the rubric moves the headline result by more than a predeclared tolerance. The output is a measurement memo, not an investment recommendation.
The judgment
The durable opportunity is not autonomous investment interpretation. It is auditable compression of a document universe. The model may reduce search and coding labor, but it does not remove selection bias, survivorship, stale disclosures, document-length effects, or the incentives firms have to describe AI as either a capability or a risk. Even the study’s website comparison is sensitive to document length, and its sample is anchored to a 2024 adviser population while observing later disclosures.
For discretionary investors, the practice is to ask for quoted evidence and an uncertainty bucket. For systematic teams, it is to version the construct and test stability across strata and time. For investment-data builders, it is to sell the review trail—inputs, labels, adjudication, and expiry—not an apparently precise adoption number. AI has changed the workflow from reading everything to measuring a sample. The human responsibility is to make sure the measurement remains about the question the portfolio team actually asked.