Ordering Legacy Code for AI Agents: Lessons and a Practical Playbook
A practical sequence for bringing AI agents into legacy codebases: map reality, protect behavior, encode architectural boundaries, and automate evidence-based cleanup.
Ryan Lopopolo —lead author of OpenAI's "harness engineering" blueprint and, as of September 2026, Principal Engineer for Agentic Google Cloud Platform—is the closest technical and philosophical peer to SpaceXSI Principal Engineer Lauren Tan ("Potato")1. Following Tan's November 11, 2026 talk *"All Code Is Slop (Except Mine)"*1, studying Lopopolo offers the most direct operational counterpart to her core thesis: that an engineer's highest leverage comes not from writing longer AGENTS.md prompt manuals, but from encoding architectural constraints into codebase boundaries, custom linters, and runtime verification1. While other prominent practitioners at AI Engineer 2026 share parts of this mindset, Lopopolo uniquely mirrors Tan's exact combination of Electron-client runtime inspection, mechanical boundary lints, and repository-as-input discipline at high pull-request volume1.
Sorting what you can learn from Lopopolo yields four progressive layers of practice: replacing monolithic instruction files with a ~100-line AGENTS.md routing map over versioned documentation, enforcing strict dependency layering through custom linters whose error messages guide agent repairs, running automated background "garbage collection" pull requests to arrest codebase entropy, and giving agents isolated runtime observability to prove their own changes2.
Applying these lessons to bring order to an old (brownfield) codebase , however, requires bridging an important gap. Lopopolo's flagship OpenAI harness experiment started from an empty greenfield repository with a ban on human-written code6, and he explicitly describes maintaining long-term architectural coherence across years as an unsolved problem3. Furthermore, the productivity and quality metrics reported by both Tan and Lopopolo come from first-party conference talks, engineering blog posts, podcasts, and secondary vendor case studies rather than independent controlled benchmarks1. To translate their target architecture onto aging, messy code without triggering context collapse, the playbook at the end of this report pairs Lopopolo and Tan's guardrails with brownfield workflows from Dexter Horthy , Mitchell Hashimoto , and Boris Cherny , alongside deterministic tools including ast-grep, baseline, and knip9.
Why Ryan Lopopolo Is Lauren Tan's Closest Peer
Tan and Lopopolo independently converged on the same structural diagnosis of agent-generated "slop." Both observed that coding agents treat the existing repository as their largest behavioral input, meaning any messy workaround or deprecated pattern left in the code gets copied and multiplied1. Tan frames this through a garden analogy—noting that her daily review load shifted from 3 colleague pull requests in 2024 to 300+ agent-generated pull requests per engineer, making structural guardrails essential so weeds do not overrun the codebase1. Lopopolo frames the same dynamic as repository "entropy" that requires continuous automated cleanup2. Both engineers also built around Electron desktop applications and wired the Chrome DevTools Protocol (CDP) directly into agent loops so models can verify live UI behavior instead of relying on human diff inspection alone1.
Comparing their published accounts reveals strong alignment in mechanism alongside useful differences in emphasis, particularly around formal verification versus automated background refactoring.
| Dimension | Lauren Tan (SpaceXSI / Grabbot) | Ryan Lopopolo (OpenAI Frontier & Symphony / Google Cloud) |
|---|---|---|
| Core Thesis | Trust Pyramid: place strongest constraints in codebase design, static analysis, and verification before prose1. | Harness Engineering: humans steer via context and tools while agents write 100% of production code2. |
| Self-Reported Scale (2026) | 3,000 individual PRs/month (~11,000 team PRs/month) and ~0.5% relative revert rate after 6 months of setup; notes most tokens go to verification, matching a peer's 70% figure1. | ~3.5 to 70 PRs/engineer/week (per a ZenML case study); ~1M LOC, 0% human code/review, and ~1B tokens/day or ~$2,000–$3,000/engineer/month (per OpenAI and podcast summaries)3. |
| Starting Context | Evolving Grabbot product codebase structured with the internal Dune framework1. | Empty greenfield repository starting mid-2025 for an Electron-based data analyst agent5. |
| Role of AGENTS.md | Narrow top layer reserved strictly for judgment calls that cannot be encoded mechanically1. | Concise ~100-line table of contents routing agents to a structured docs/ tree; monolithic manuals banned2. |
| Mechanical Guardrails | Dune framework constraints: useEffect banned in product code, electron-main/renderer import lints, 99% of product comments banned1. | Forward-only layers (Types to UI) plus single Providers boundary; custom lints with self-remediating error prose2. |
| Codebase Decay Defense | Keeping the codebase "garden" clean so agents imitate sound patterns rather than justified hacks1. | Scheduled background Codex scans updating domain quality grades and opening refactoring PRs taking under 1 minute to review2. |
| Verification & Review | CDP runtime checks, user-facing feature maps, custom CLIs, plus TLA+, Lean, and Bombadil UI property testing1. | Per-worktree app booting, CDP snapshots, ephemeral local telemetry (LogQL/PromQL/TraceQL), and persona-based CI review agents2. |
Because Lopopolo's OpenAI team started from an empty repository6, studying him alone leaves a practical gap when your immediate goal is ordering an existing codebase. Three complementary practitioners fill in the brownfield mechanics. Dexter Horthy (CEO of HumanLayer) focuses explicitly on complex and brownfield repositories, introducing the Research-Plan-Implement (R.P.I.) workflow and Frequent Intentional Compaction to keep agents out of the "Dumb Zone" of context pollution, while advising teams to scale planning and verification depth to the cost of rework10. Mitchell Hashimoto , who named "Engineer the Harness" in his February 2026 adoption framework, explains why chat interfaces break down on brownfield projects and advocates turning every observed mistake into a verification script or targeted AGENTS.md line while supervising one high-craft background agent at a time14. Finally, Boris Cherny —creator of Claude Code, author of Programming TypeScript, and former Meta engineering leader who ran large Python-to-monolith and REST-to-GraphQL migrations at Instagram—contributes practical cleanup tactics including Plan Mode, /techdebt duplication sweeps, code-simplifier subagents, and tagged-union types that make invalid states unrepresentable9.
Sorted Curriculum: What to Learn from Ryan Lopopolo
Lopopolo's writings, OpenAI's February 2026 engineering post, and his harness-engineering repository organize agent-first engineering into four distinct practices2. Studying them in order helps you prioritize durable structural leverage over fragile prompt tinkering.
1. Context Routing: Replace the Monolithic Manual with a Map
OpenAI's team initially tried maintaining a single comprehensive AGENTS.md instruction manual, and Lopopolo documents four failure modes that killed it2. First, a massive instruction file consumes scarce context capacity, crowding out the actual task and source files2. Second, marking everything as critical destroys prioritization, causing agents to fall back on local pattern-matching rather than intentional navigation2. Third, monolithic files rot immediately as code evolves2. Fourth, a giant text blob resists mechanical checks for freshness, ownership, coverage, and cross-linking2.
Lopopolo's fix is progressive disclosure 2. His team shrank AGENTS.md to roughly 100 lines serving strictly as a stable table of contents that teaches the agent's operating loop and points into a versioned docs/ system of record2. Architecture maps, per-domain quality grades, active execution plans, and completed decision logs live in the repository alongside code2. Automated tests fail if documentation drifts out of sync with code edits, and background "doc-gardening" tasks repair stale files2. This echoes Tan's placement of prose at the smallest tier of her trust pyramid1, while adding concrete CI mechanics to prevent documentation decay.
2. Mechanical Architecture and Self-Remediating Lints
Mirroring Tan's Dune framework—which bans useEffect in product code and forbids cross-boundary imports between electron-main and electron-renderer directories1—Lopopolo enforces rigid architectural boundaries centrally while giving agents broad local autonomy inside those boundaries2. On his OpenAI team, each business domain followed a strict forward dependency chain (Types -> Config -> Repo -> Service -> Runtime -> UI), and cross-cutting concerns such as authentication, telemetry, and feature flags could enter only through a single explicit Providers interface2. Structural tests and custom linters rejected any illegal dependency edge mechanically2.
For taming legacy code, Lopopolo's highest-leverage tactic is treating lint and test failures as context injection 2. Rather than emitting terse error codes, his custom linters print clear prose instructions explaining the invariant, identifying the mistake, and directing the agent to the exact runbook or module required to fix the root cause2. Tan applies the identical pattern in Grabbot, pairing a 99% ban on product code comments—which agents otherwise use to rationalize fragile hacks—with lint errors that instruct the agent to eliminate the underlying architectural flaw1.
3. Entropy Management and Automated Garbage Collection
Autonomous agents replicate whatever idioms already inhabit a repository, including suboptimal ones2. Early in OpenAI's experiment, engineers spent 20% of their week (every Friday) manually cleaning up agent drift, an approach that failed to scale2. Lopopolo's team replaced manual Friday cleanups with an automated garbage-collection loop anchored by explicit "golden principles" stored in the repository, such as requiring shared utility packages over hand-rolled helpers and enforcing typed boundary validation instead of guessing data shapes2.
Scheduled background Codex tasks continuously scanned the codebase against these principles, updated domain quality grades, and opened small, targeted refactoring pull requests that humans could review and automerge in under a minute2. Paying down technical debt daily in tiny increments kept anti-patterns from compounding over weeks2. For an engineer cleaning up old code, this reframes cleanup from a dreaded multi-month rewrite into continuous automated hygiene.
4. Runtime Legibility and Artifact-Driven Verification
As code generation becomes cheap, human QA and diff review become the primary bottlenecks1. Within his OpenAI frontier team, Lopopolo removed mandatory human code review by making application behavior directly observable to agents (while OpenAI internally estimated building the product in roughly 1/10th the time of manual coding)2. Each git worktree could boot an isolated application instance connected to CDP for DOM snapshots, screenshots, and UI clicks, alongside an ephemeral local observability stack (Vector fanning logs, metrics, and traces into Victoria Logs, Metrics, and Traces queryable via LogQL, PromQL, and TraceQL)2. Agents could reproduce bugs, verify latency or startup budgets (such as keeping startup under 800 ms), and pass matrixed CI review personas (frontendarchitect.md, reliabilityengineer.md, appsecengineer.md) before completion2.
Here, reading Lopopolo alongside Tan exposes a productive tension on testing depth. Tan pushes verification into formal and property-based methods—using TLA+ for cross-device flows, Lean to prove UI virtualization invariants, and Bombadil for UI property testing1—whereas Lopopolo warns against brittle, implementation-coupled tests that duplicate coverage and bloat maintenance cost22. Both agree that human attention should shift from reading raw code diffs to inspecting execution artifacts such as screenshots, recordings, and traces1.
Playbook: Bringing Order to Your Old Codebase
Translating Tan and Lopopolo's architecture onto an aging codebase requires strict sequencing. Tan cautions that automating an unreliable inner loop merely produces a high-volume slop factory1, and Horthy warns that dumping an unmapped brownfield repository into an unconstrained agent pushes the model straight into the "Dumb Zone" of context pollution10. The four-phase sequence below synthesizes practitioner workflows and vendor-documented patterns into a safe order of operations for existing code.
Phase 1: Map Ground Truth and Build a Safety Net
Audit actual dependencies before editing. Follow Horthy's rule that "docs lie, code doesn't"10: direct an agent to read the existing repository, inventory modules, and map real call paths rather than trusting outdated wikis10.Create a ~100-line AGENTS.md router. Place your exact build, lint, and fastest verification commands at the top7, then link out to a docs/ tree for architecture notes and execution plans instead of writing a monolithic rulebook2.Practice Frequent Intentional Compaction. When investigating tangled legacy modules fills a chat session with stale file reads and failed commands, have the agent write verified findings into an active plan file (docs/exec-plans/active/) and restart in a fresh context window before editing code10.Generate characterization tests before refactoring. As highlighted in legacy modernization workflows, untested code cannot be refactored safely23. Have agents generate characterization tests locking down current observable behavior on modules you plan to modify23, while keeping humans in charge of undocumented business rules (such as financial rounding or regulatory logic) where LLMs can alter semantics silently11.
Phase 2: Strip Dead Weight and Separate Mechanical from Semantic Refactors
Automate continuous dead-code removal. Rather than opening a giant cleanup PR that nobody can review, schedule a tool such as knip to open small, daily PRs deleting unused exports, files, and dependencies13.Use deterministic AST codemods for bulk syntax changes. Do not ask an LLM to rename symbols or swap deprecated APIs across thousands of files, as models are nondeterministic and hallucinate imports at scale11. Pair deterministic AST tools (ast-grep, jscodeshift, or OpenRewrite) for structural rewrites with semantic graph indexing (such as CodeGraph or LSP) for impact analysis11.Restrict LLM refactoring to bounded modules. Engineering analyses of migrations at scale indicate that LLMs struggle sharply with cross-class architectural reorgs, God-class splits, and database schema migrations11. Keep humans responsible for service boundaries and data models, and assign agents isolated, well-tested modules in separate git worktrees to refactor bottom-up7.
Phase 3: Install Ratchets and Self-Remediating Guardrails
Cap legacy patterns with a ratchet. When you cannot rewrite hundreds of legacy call sites immediately, prevent agents from copying them. Use a ratcheting CI tool such as baseline (a Rust CLI that counts banned pattern or dependency occurrences and enforces a ceiling that only moves downward) or grandfathering ignore pragmas so existing debt is isolated while any new violation fails CI12.Turn every second mistake into a deterministic check. Apply the shared rule of Tan, Lopopolo, and Hashimoto: correct a novel agent error once, and on the second occurrence encode the constraint into types (such as Cherny's tagged unions preventing invalid states), explicit directory boundaries, or a custom lint rule1.Write lint errors for the agent's context window. Customize lint messages so failures explain the architectural rule and point directly to the approved replacement API or runbook, allowing the agent to self-repair on its next loop1.Eliminate explanatory workaround comments. Follow Tan's insight that comments justifying quick fixes preserve technical debt and teach future agents bad habits; reject workaround rationalizations in favor of root-cause fixes or tracked baseline ceilings1.
Phase 4: Automate Verification and Continuous Garbage Collection
Keep local verification fast and isolated. If tests take 20 minutes to run, agents will stall or skip checks7. Build filtered test commands or lightweight CLI drivers (like Tan's custom CLIs and feature maps, or Hashimoto's targeted test scripts) inside per-task git worktrees1.Give agents runtime visibility. For applications with user interfaces or service logs, wire up CDP browser checks or local log/metric queries so agents must produce before-and-after behavioral evidence rather than asserting completion from code compilation alone1.Scale background cleanup to your review capacity. Start at Hashimoto's pace—supervising one high-certainty background cleanup agent while you focus on deep architectural work14—or run Cherny's /techdebt and code-simplifier steps at the end of each feature branch9. Once tests and ratchets reliably catch regressions, graduate to Lopopolo's scheduled background GC tasks that open tiny, sub-one-minute refactoring PRs continuously2.
Quick-Reference Mapping for Ordering Old Code
| Legacy Codebase Problem | Actionable Harness Mechanism | Primary Source to Study |
|---|---|---|
| Agents ignore bloated AGENTS.md or trust stale wikis | Trim AGENTS.md to ~100-line map; enforce doc-sync checks; treat code as ground truth | Ryan Lopopolo2, Dexter Horthy10 |
| Agents hallucinate across large, tangled brownfield modules | Research-Plan-Implement (R.P.I.) with Frequent Intentional Compaction across fresh sessions | Dexter Horthy10 |
| Agents copy legacy workarounds and spread "weeds" | Ban workaround-justifying comments; enforce baseline ratchets to cap and burn down debt | Lauren Tan1, baseline12 |
| Bulk API or framework renames fail under LLM nondeterminism | Run deterministic AST codemods (ast-grep, OpenRewrite) for syntax; scope LLMs to tested modules | Tian Pan11, CodeGraph/ast-grep24 |
| Accumulated dead code and duplicated helper functions | Daily automated knip PRs; /techdebt sweeps; scheduled background GC PRs (<1 min review) | knip13, Boris Cherny9, Ryan Lopopolo2 |
| Agents repeat identical architectural or API mistakes | Encode taste in structural layer tests and custom linters whose error prose guides the fix | Lauren Tan1, Ryan Lopopolo2, Mitchell Hashimoto14 |
| Reviewers drown in large diffs on fragile legacy modules | Shift to characterization tests, CDP/browser screenshots, and filtered local verification scripts | Lauren Tan1, Ryan Lopopolo2, Boris Cherny9 |
References
All Code Is Slop (Except Mine) | AI Engineer CODE 2026, https://ai.engineer/code/2026/talks/lauren-tanHarness engineering: leveraging Codex in an agent-first world, https://openai.com/index/harness-engineering/Ryan Lopopolo - Latent Space, https://latent.space/wiki/ryan-lopopolo/AI Engineer Code Summit 2026 Recordings, https://ai.engineer/code/2026Zero Human-Written Code: Harness Engineering for Autonomous AI, https://www.zenml.io/llmops-database/zero-human-written-code-harness-engineering-for-autonomous-ai-agents-at-scaleRyan Lopopolo (OpenAI): How PMs Ship 100K Lines of Code, https://www.news.aakashg.com/p/ryan-lapopolo-podcastHarness Engineering for Coding Agents: What It Is (2026) - Agentastic, https://www.agentastic.dev/harness-engineeringHow PMs Ship 100K Lines of Cod… - The Growth Podcast, https://podcasts.apple.com/us/podcast/how-pms-ship-100k-lines-of-code-at-openai-with/id1763555775?i=1000769555393Boris Cherny's Claude Code Tips — How He Actually Uses It (142+, https://howborisusesclaudecode.com/No Vibes Allowed: Solving Hard Problems in Complex Codebases, https://www.youtube.com/watch?v=rmvDxxNubIgAI-Assisted Codebase Migration at Scale: Automating the Upgrades, https://tianpan.co/blog/2026/04/17/ai-codebase-migration-agents-scalestewartjarod/baseline: Enforce architectural decisions AI coding, https://github.com/stewartjarod/baselineAutomate dead code removal using Ona and Knip - YouTube, https://www.youtube.com/watch?v=OuE0xZYnaCUMy AI Adoption Journey - Mitchell Hashimoto, https://mitchellh.com/writing/my-ai-adoption-journeyA software factory needs pressure in both directions - AI Engineer, https://ai.engineer/code/2026/talks/dexter-horthyThe Definitive Guide to Agent Harness Engineering - Medium, https://engineeratheart.medium.com/the-definitive-guide-to-agent-harness-engineering-5f5edf25fd73How the Creator of Claude Code Actually Uses It - DEV Community, https://dev.to/a_shokn/how-the-creator-of-claude-code-actually-uses-it-32dfBoris Cherny: Bio, Work & Ideas - AI Engineer, https://ai.engineer/speakers/boris-chernyBoris Cherny — Innovator Profile - Engatica, https://engatica.com/innovators/boris-cherny/lopopolo/harness-engineering: Ryan Lopopolo's anthology, field, https://github.com/lopopolo/harness-engineeringHarness Engineering — What I tried out? | by Onaopemipo Oluborode, https://medium.com/towardsdev/harness-engineering-what-i-tried-out-06cc4dfbfc93Ryan Lopopolo: Bio, Work & Ideas - AI Engineer, https://ai.engineer/speakers/ryan-lopopoloAI Agents for Legacy Code Modernization — The Developer's Week, https://amux.io/guides/legacy-code-modernization-ai-agents/codegraph-ast-grep - BrowserAct SkillHub - 100000+ AI Agent Skills, https://skills.browseract.com/skills/stark-ai-de-agent-skills-codegraph-ast-grepRatchets in software development (2021) - Hacker News, https://news.ycombinator.com/item?id=46814340
Read the Chinese companion edition.