Software Development as a Learning System: How AI Agents Could Plan, Build, Review, and Converge

A framework for treating AI-assisted software development as a governed feedback loop: specialized agents plan and build, independent reviews provide evidence, and humans set the goals and release boundaries.

A developer reviews code and test signals from an AI coding agent.

Introduction

The first generation of AI coding tools focused primarily on code generation. A developer described a function, asked a question, or supplied an error message, and the AI proposed an implementation.

The next generation may be fundamentally different.

Instead of asking an AI system to generate code once, we can create a governed development process in which multiple AI agents repeatedly:

  1. Interpret a human goal.
  2. Convert it into requirements and measurable outcomes.
  3. Create an implementation plan.
  4. Divide the plan into phases.
  5. Design the architecture.
  6. Build each phase.
  7. Test and review the result.
  8. Diagnose defects and weaknesses.
  9. Repair the implementation.
  10. Reconsider the plan or architecture when problems repeat.
  11. Continue until the product reaches acceptable technical and human-defined outcomes.

This would transform AI from a coding assistant into part of a feedback-driven digital production system .

The process has an important conceptual similarity to machine learning. Machine learning advances through repeated cycles of generation, evaluation, error analysis, and optimization. Agentic software development can use the same operating discipline even when the underlying AI models are not being retrained.

The central idea is therefore not that software development is identical to machine learning. It is that software development can be managed as a measurable learning and convergence process .


1. From AI Code Generation to Agentic Engineering

A traditional AI coding interaction often looks like this:

Human request
    ↓
AI-generated code
    ↓
Human review
    ↓
Manual correction

This can make an individual developer more productive, but the human still coordinates most of the engineering lifecycle.

A more advanced agentic process looks like this:

Human goal and constraints
    ↓
Requirements and acceptance criteria
    ↓
Planning and architecture
    ↓
Implementation
    ↓
Testing and independent review
    ↓
Failure classification
    ↓
Repair, replanning, or redesign
    ↓
Validation
    ↓
Controlled release
    ↓
Production feedback
    ↓
Next improvement cycle

The distinction is significant:

A code generator produces an output. An agentic engineering system manages a goal-directed process.

This direction is consistent with the emerging field of AI agentic programming. A recent survey describes software agents that plan, execute multi-step tasks, use compilers and debuggers, interact with version-control systems, and adapt their actions based on intermediate feedback. It also identifies planning, memory, context management, execution monitoring, evaluation, safety, and human collaboration as central components of trustworthy software-engineering agents. AI Agentic Programming survey

The change is not simply that AI can write more code. The nature of engineering work also changes.

Developers increasingly become responsible for:

  • Defining goals
  • Creating specifications
  • Designing constraints
  • Building evaluation systems
  • Reviewing agent decisions
  • Diagnosing repeated failures
  • Managing architectural tradeoffs
  • Deciding when a result is ready

A longitudinal study of professional software engineers characterizes this emerging category as supervisory engineering work , including the direction, evaluation, and correction of AI-generated output. The study found that developers reported spending less time on many implementation tasks, but it also identified risks involving cognitive load, flow, and the effort required to verify AI output. longitudinal study of AI coding assistants

Automation does not eliminate engineering. It moves engineering to a different level.


2. The Machine-Learning Analogy

The strongest analogy is not between software tests and machine-learning datasets. That comparison can be useful in limited situations, but it becomes difficult if pushed too far.

The more valuable analogy is between the overall improvement processes .

Simplified machine-learning process

Define an objective
    ↓
Produce or train a model
    ↓
Evaluate performance
    ↓
Analyze errors
    ↓
Adjust the model, data, or training strategy
    ↓
Repeat until performance stabilizes

Agentic software-development process

Define a product goal
    ↓
Produce a plan and implementation
    ↓
Evaluate behavior and quality
    ↓
Analyze defects and weaknesses
    ↓
Repair the implementation or revise the design
    ↓
Repeat until quality stabilizes

The common principles are:

  • Goal definition
  • Iterative production
  • Measurable evaluation
  • Error analysis
  • Feedback-driven improvement
  • Experiment tracking
  • Convergence monitoring
  • Strategy revision when progress stops

The analogy should be presented carefully:

Agentic software development does not necessarily train software as if it were a statistical model. It borrows the operating discipline of machine learning to improve the software artifact and the production process.

In many projects, the underlying language model does not change at all. What changes is:

  • The implementation
  • The project plan
  • The architecture
  • The test suite
  • The agent instructions
  • The tool configuration
  • The orchestration workflow
  • The project memory
  • The evaluation policy

This is system-level learning , even when no model weights are updated.


Andrej Karpathy is probably the best-known public figure whose ideas are closely related to this direction.

His earlier “Software 2.0” framing described systems in which developers define goals and use optimization to discover behavior rather than explicitly programming every rule. His more recent “Software 3.0” presentation describes large language models as a new computing layer that can be programmed using natural language. He also emphasizes partial autonomy, human-AI collaboration loops, and the importance of designing infrastructure for agents as consumers and producers of digital information. report on Karpathy’s software thesis, Karpathy’s “Software Is Changing Again” talk

Discussions of his agentic-engineering approach distinguish disciplined engineering from unconstrained “vibe coding.” The disciplined approach emphasizes:

  • Clear specifications
  • Controlled agent execution
  • Evaluation loops
  • Review of proposed changes
  • Human oversight of architecture
  • Multiple agents managed more like an engineering team

These themes closely support the idea of feedback-driven development. agentic-engineering playbook, agentic-engineering overview, agentic-engineering explainer

However, it would be too strong to say that Andrej Karpathy has endorsed this complete convergence framework.

A more accurate positioning is:

Andrej Karpathy has advanced closely related ideas involving Software 2.0, Software 3.0, partial autonomy, natural-language programming, human-AI collaboration, and agentic engineering. The convergence framework extends these ideas by formalizing review, repeated-defect detection, stopping rules, and escalation from code repair to plan or architecture revision.

The combination of these elements is where the framework becomes especially useful.


4. What “Learning” Means in Software Development

Learning can occur at several levels.

Level 1: Learning within an iteration

The system produces a candidate, evaluates it, and corrects it.

For example:

  1. A builder agent implements a feature.
  2. A test agent identifies incorrect behavior.
  3. A review agent finds inadequate error handling.
  4. A security agent identifies an unsafe input path.
  5. The builder modifies the implementation.
  6. The evaluation is repeated.

The software improves because evidence from one iteration influences the next.

This is the most direct form of learning.

Level 2: Learning across a project

As the project continues, the system can begin to identify patterns.

For example:

  • The same module repeatedly fails.
  • Repairs in one component frequently damage another.
  • Different agents interpret the same requirement differently.
  • A dependency creates repeated reliability problems.
  • The architecture makes testing unnecessarily difficult.
  • Local patches increase complexity without improving stability.
  • The original project plan omitted a critical capability.

At this level, the system should not simply apply another patch. It should reconsider the plan, component boundaries, interfaces, assumptions, or architecture.

Level 3: Learning across projects

An organization can preserve experience across development efforts.

Its engineering memory may contain:

  • Successful implementation plans
  • Failed approaches
  • Common defect patterns
  • Architecture decision records
  • Effective test strategies
  • Security findings
  • Human override decisions
  • Production incidents
  • User feedback
  • Cost and schedule results
  • Reusable components

Future agents can use this information to improve estimation, planning, test generation, architecture selection, and risk identification.

Level 4: Improvement of the agent system

The most advanced form of learning occurs when the system improves not only the product but also its own development workflow.

It might change:

  • Which agents are selected
  • How tasks are decomposed
  • What tools are available
  • How project memory is organized
  • When agents work in parallel
  • Which reviewer evaluates a candidate
  • How retry and escalation rules operate

Research is already exploring this direction. The paper “Self-Evolving Software Engineering Agents” describes a software-engineering agent that can modify its own scaffold while working on real software problems. The work treats the agent itself as software that can be refined in response to its experience.

Salesforce AI Research has presented SAGE, an approach in which an agent uses its own execution trajectory and environmental feedback to induce a better plan. The researchers describe this as a form of adaptation through grounded experience rather than conventional model retraining. Salesforce AI Research’s SAGE overview

These examples support an important part of the framework:

Feedback can improve the artifact, the plan, and eventually the agentic production process itself.

5. The Multi-Agent Production Model

A reliable production system should not depend on one general-purpose agent playing every role.

A builder that implements a solution, evaluates its own work, approves the architecture, and authorizes release has several conflicts of interest. It may also reproduce the same assumptions at every stage.

A stronger design uses specialized agents coordinated by a workflow orchestrator.

Orchestrator

The orchestrator manages the process.

Its responsibilities include:

  • Tracking project state
  • Assigning work
  • Enforcing phase boundaries
  • Monitoring time and cost
  • Recording evidence
  • Applying retry limits
  • Requesting approvals
  • Triggering escalation
  • Controlling the release path

The orchestrator should behave more like a stateful workflow engine than an unrestricted conversational agent.

Requirements agent

The requirements agent converts the human goal into:

  • User needs
  • Functional requirements
  • Nonfunctional requirements
  • Constraints
  • Assumptions
  • Acceptance criteria
  • Prohibited behavior
  • Open questions
  • Product-success measures

Its most important task is to translate an informal goal into a contract that can guide both production and evaluation.

Planning agent

The planning agent:

  • Divides the work into phases
  • Identifies dependencies
  • Creates milestones
  • Maps requirements to tasks
  • Identifies risks
  • Proposes validation activities
  • Estimates resources
  • Defines checkpoints

Architecture agent

The architecture agent defines:

  • Components
  • Interfaces
  • Data flows
  • Trust boundaries
  • Storage strategies
  • Failure-handling mechanisms
  • Performance assumptions
  • Integration approaches
  • Security controls

It should also record why major decisions were made.

Builder agent

The builder creates or modifies:

  • Source code
  • Configuration
  • Infrastructure definitions
  • Database definitions
  • Automated tests
  • Documentation
  • Packaging instructions

The builder can propose that work is complete, but it should not be authorized to approve its own release.

Test agent

The test agent evaluates:

  • Functional behavior
  • Integration behavior
  • Boundary conditions
  • Error handling
  • Performance
  • Compatibility
  • Regression risk
  • Reliability under abnormal conditions

It should look for missing test categories rather than merely executing an existing suite.

Review agent

The review agent asks broader engineering questions:

  • Is the implementation correct?
  • Does it satisfy the intended requirement?
  • Is it unnecessarily complicated?
  • Is it maintainable?
  • Are responsibilities clearly separated?
  • Are failure modes handled consistently?
  • Are important assumptions documented?
  • Is the implementation creating avoidable technical debt?

Security and compliance agent

This agent examines:

  • Authentication
  • Authorization
  • Secret handling
  • Data access
  • Input validation
  • Dependency risk
  • Privacy
  • Logging
  • Auditability
  • Applicable policy requirements

User-simulation agent

The user-simulation agent attempts realistic tasks from different perspectives:

  • New user
  • Experienced user
  • System administrator
  • User with an accessibility need
  • User who misunderstands the instructions
  • User under time pressure
  • Careless user
  • Potential attacker

Release agent

The release agent prepares:

  • Versioned artifacts
  • Release notes
  • Deployment instructions
  • Migration instructions
  • Rollback procedures
  • Traceability evidence
  • Known limitations

Learning agent

The learning agent studies the project history to identify:

  • Repeated defect categories
  • Repair strategies that did not work
  • Weak architectural choices
  • Missing requirements
  • Ineffective evaluation methods
  • Excessive disagreement
  • Repeated human intervention
  • Opportunities to improve the workflow

This is not simply a group conversation among AI models. It is a controlled production system with roles, states, evidence, permissions, and stopping rules.


6. Review Is the Learning Signal

Review is often treated as a final checkpoint after implementation.

In an agentic development system, review becomes the central learning mechanism.

It has at least four functions.

Function 1: Detect problems

Review agents may identify:

  • Incorrect behavior
  • Missing requirements
  • Security weaknesses
  • Poor usability
  • Performance problems
  • Fragile interfaces
  • Inconsistent behavior
  • Unnecessary complexity
  • Incomplete documentation
  • Unsupported claims

Function 2: Classify problems

Each issue should be recorded using structured attributes:

  • Severity
  • Category
  • Affected component
  • Relevant requirement
  • Probable root cause
  • New or recurring
  • Regression or original defect
  • Local or architectural
  • Confidence
  • Recommended next action

Without a failure taxonomy, agents may repeatedly repair symptoms without recognizing that several failures share the same cause.

Function 3: Guide the response

The review should recommend the appropriate correction level:

Implementation defect
    ↓ Local code repair
Repeated component defect
    ↓ Component or interface review
Repeated cross-component defect
    ↓ Architecture review
Missing or conflicting capability
    ↓ Plan and requirement review
Wrong product objective
    ↓ Human reconsideration of the goal

Function 4: Measure progress

Review should also determine whether the process is improving.

Useful questions include:

  • Is the number of defects declining?
  • Is severity declining?
  • Are resolved problems staying resolved?
  • Are changes introducing fewer regressions?
  • Are different reviewers reaching similar conclusions?
  • Are implementation changes becoming smaller?
  • Is the architecture stabilizing?
  • Is each additional iteration producing meaningful value?

This makes review more than a quality gate. It becomes the feedback signal driving the development process.

Google Developers has applied a related principle to coding-agent harnesses, distinguishing broad end-to-end scores from behavioral evaluations. Behavioral evaluations check whether an agent performs expected actions, such as clarifying an ambiguous request or running a validator before claiming completion. This type of evaluation helps explain not only whether an agent succeeded, but also how its behavior should improve. Google Developers’ coding-agent harness article


7. Development as an Optimization Process

Once development is organized as a feedback loop, every implementation can be treated as a candidate state.

  • Candidate 1: Initial implementation
  • Candidate 2: Correction after functional testing
  • Candidate 3: Correction after security and code review
  • Candidate 4: Revised implementation after architecture review
  • Candidate 5: Release candidate

For every candidate, the system can record:

  • Code and artifact version
  • Test results
  • Review findings
  • Root-cause classifications
  • Architecture decisions
  • Regressions
  • Performance
  • Cost
  • Time
  • Human interventions
  • User-oriented outcomes

This creates a traceable development history similar to an experiment history.

A simplified quality function might be written as:

Q = w_f F + w_r R + w_s S + w_u U − w_c C

Where:

  • F represents functional correctness
  • R represents reliability
  • S represents security
  • U represents user value
  • C represents cost or complexity
  • The w values represent project-specific priorities.

This does not mean every release decision should be reduced to one number. Some requirements, particularly security and compliance requirements, may be hard gates rather than weighted variables.

The model is useful because it makes tradeoffs explicit.

A system should not improve functional coverage by creating unacceptable security risk. It should not reduce cost by sacrificing a mandatory requirement. It should not maximize test scores while producing an unusable product.


8. Convergence, Plateau, Oscillation, and Divergence

A major contribution of the machine-learning analogy is the language of convergence.

Many software projects stop because the deadline arrives or visible tests pass. A learning-oriented approach asks whether the development process has reached a stable and acceptable state.

The process may be in one of four conditions.

Converging

  • Defect count is declining.
  • Defect severity is declining.
  • Regressions are becoming less frequent.
  • Architecture is stabilizing.
  • User-oriented evaluation is improving.
  • Each cycle moves the product closer to its goals.

Plateauing

  • Additional cycles produce little improvement.
  • The same quality scores remain unchanged.
  • Reviewers repeat similar comments.
  • Cost continues to increase without corresponding value.

A plateau may indicate that the current repair strategy has reached its limit.

Oscillating

  • Fixing one component repeatedly breaks another.
  • The system alternates between two incomplete solutions.
  • Different agents undo one another’s changes.
  • Previously resolved problems continuously return.

Oscillation often indicates coupled components, conflicting requirements, or unstable architecture.

Diverging

  • Defects are increasing.
  • The implementation is becoming more complex.
  • Reviewer disagreement is expanding.
  • Repairs introduce more problems than they resolve.
  • The architecture requires increasingly invasive changes.

Divergence should trigger a stop rather than additional uncontrolled retries.

Example convergence history

Iteration Critical defects Regressions User task success
1 5 0 45%
2 3 2 62%
3 1 1 76%
4 0 1 84%
5 0 0 86%

The fifth cycle produces only a small improvement in user success. Depending on the project’s threshold and cost, this may indicate convergence. It might also indicate a product-design limit that requires a different strategy rather than more local repair.


9. Repeated Failures Should Trigger Architectural Learning

One of the most important parts of the framework is recognizing that repeated defects may indicate a problem above the code level.

A useful escalation policy might be:

  1. First occurrence: Attempt a focused local correction.
  2. Second occurrence: Perform deeper root-cause analysis.
  3. Third occurrence: Stop ordinary repair and review the plan or architecture.
  4. Continued recurrence: Escalate to a human decision-maker.

For example:

If the same root-cause category appears three times:

  1. Freeze the current candidate.
  2. Preserve logs, tests, and decisions.
  3. Run an independent root-cause review.
  4. Compare local repair with partial redesign.
  5. Update the architecture decision record.
  6. Regenerate the affected project phase.
  7. Require human approval if scope, cost, or risk changes.

This creates a hierarchy of correction:

Code
  ↓
Component
  ↓
Interface
  ↓
Plan
  ↓
Architecture
  ↓
Product goal

The key principle is:

Repeated local failure is often evidence of a higher-level design problem.

An autonomous system that only edits code cannot fully learn. A more capable system must be able to reconsider how the problem was decomposed and whether the current architecture remains appropriate.


10. Defining Different Types of Convergence

“No issues found” is not a sufficient definition of success.

There are several distinct forms of convergence.

Technical convergence

Technical convergence occurs when:

  • Mandatory tests pass.
  • Builds are reproducible.
  • Critical security findings are resolved.
  • Performance meets defined thresholds.
  • Reliability targets are satisfied.
  • No unacceptable regression remains.

Review convergence

Review convergence occurs when:

  • Independent reviewers find no significant new failure category.
  • Previously identified defects remain resolved.
  • Reviewer disagreement has declined.
  • Additional review cycles produce little new information.

Architectural convergence

Architectural convergence occurs when:

  • Major components are stable.
  • Interfaces are no longer changing frequently.
  • Local corrections do not require repeated restructuring.
  • The design supports required quality attributes.
  • Remaining work can proceed without destabilizing the system.

Product convergence

Product convergence occurs when:

  • The software solves the intended problem.
  • Users can complete the relevant tasks.
  • The interface and output are understandable.
  • The feature improves an operational or business outcome.
  • Users find the product valuable enough to adopt.

AI agents can automate much of technical convergence. Product convergence is more difficult because usefulness depends on human context.

A technically correct implementation of the wrong requirement is still the wrong product.


11. Independent Validation Without Overemphasizing OOS

The distinction between in-sample and out-of-sample evaluation can offer one useful lesson, but it should not dominate the framework.

In machine learning, evaluating a model only on examples that shaped its training can create a misleading picture of performance.

A related risk exists in agentic development. If a builder repeatedly sees the same tests, it may create an implementation that satisfies those tests without fully satisfying the broader requirement.

Software can therefore benefit from:

  • Independently designed tests
  • Hidden qualification tests
  • Randomized input generation
  • Property-based testing
  • Fuzz testing
  • Adversarial security testing
  • Cross-platform testing
  • User-simulation tests
  • Controlled production trials

The point is not to create a perfect software equivalent of OOS evaluation. The point is simpler:

Do not allow a system to assess its success exclusively through the same examples, assumptions, and agents that shaped its implementation.

Once an independent test is disclosed and used for repair, it becomes part of the project’s development history. The organization should retain it as a regression test while continuing to create new and unfamiliar validation cases.

SWE-Gym illustrates how real repository tasks, executable environments, tests, and verifiers can support the training and evaluation of software-engineering agents. Its work also shows why verification environments are important for improving agents through experience.

SWE-agent similarly demonstrates a practical agent that can receive a software issue, interact with repository tools, modify code, and attempt to produce a verified solution.

Independent validation is therefore one component of the larger convergence framework, not its central organizing principle.


12. The Value for Software Developers

The framework helps developers by providing a more disciplined way to work with increasingly autonomous agents.

Better specifications

Developers must state:

  • What the product must do
  • What it must not do
  • Which properties must always hold
  • What evidence demonstrates success
  • Which uncertainties require clarification
  • Which decisions require human approval

A vague prompt may be acceptable for experimentation. It is not sufficient for governed autonomous development.

Better evaluations

Developers may spend more time creating:

  • Executable acceptance criteria
  • Test harnesses
  • Behavioral evaluations
  • Observability
  • Incident detection
  • Rollback mechanisms
  • Convergence thresholds
  • Escalation rules

As code generation becomes less expensive, reliable verification becomes more valuable.

Better failure diagnosis

The framework encourages developers to classify an agent failure as one of several types:

  • Requirement misunderstanding
  • Missing context
  • Planning failure
  • Architecture weakness
  • Incorrect implementation
  • Tool misuse
  • Verification failure
  • Environment failure
  • Permission failure
  • Orchestration problem

This is more effective than treating every failure as a prompting problem.

Better architectural discipline

Developers remain responsible for decisions that require broad context:

  • Component boundaries
  • Data ownership
  • Trust boundaries
  • Long-term maintainability
  • Technology selection
  • Operational risk
  • Technical debt
  • Product tradeoffs

The human role shifts from writing every line to designing the conditions under which reliable software can emerge.


13. The Value for Data Scientists

The framework also provides data scientists with a path into agentic software engineering.

Data scientists already understand:

  • Objective functions
  • Optimization
  • Error analysis
  • Experiment tracking
  • Convergence
  • Overfitting
  • Distribution changes
  • Evaluation leakage
  • Model monitoring

These concepts can be applied to the software-production process.

Development learning curves

Instead of tracking only model loss or accuracy, the system can track:

  • Defects per iteration
  • Severity per iteration
  • Regression rate
  • User-task success
  • Architecture changes
  • Reviewer disagreement
  • Agent cost
  • Human intervention
  • Improvement per cycle

Failure taxonomies

Data scientists can group failures into meaningful categories and study their relationships.

For example:

  • Requirement failures
  • Planning failures
  • Context failures
  • Architecture failures
  • Implementation failures
  • Tool failures
  • Verification failures
  • Environment failures
  • Human-agent coordination failures

This transforms anecdotal failures into analyzable evidence.

Agent-policy experiments

Data scientists can compare:

  • One builder versus several independent builders
  • One reviewer versus multiple reviewers
  • Immediate feedback versus delayed feedback
  • Short-term context versus persistent project memory
  • Fixed orchestration versus adaptive orchestration
  • Full evaluation disclosure versus protected evaluation
  • Different retry limits
  • Different escalation policies

Process-level optimization

Data scientists can help answer questions such as:

  • Which agent configuration produces the most reliable outcomes?
  • When does another repair cycle stop being cost-effective?
  • Which failure categories predict architectural problems?
  • How often should independent reviews run?
  • Which tasks are safe for greater autonomy?
  • What type of human intervention produces the greatest improvement?

The object of optimization becomes not only the model but also the complete human-agent production system.


14. A Shared Language for Developers and Data Scientists

The analogy creates a useful bridge between two disciplines.

Software engineering Machine-learning perspective
Product requirement Objective
Implementation candidate Candidate model or checkpoint
Build and test cycle Optimization iteration
Test and review results Evaluation signals
Defect classification Error analysis
Local code repair Local optimization
Plan revision Strategy revision
Architecture redesign Model-class or architecture change
Release threshold Convergence criterion
Production monitoring Post-deployment monitoring
Incident history New learning evidence

This mapping is not mathematically exact. Its value is organizational and operational.

It helps software engineers think more systematically about evaluation and convergence. It helps data scientists see software production as a measurable environment in which agents can act, receive feedback, and improve.


15. Extending the Framework Beyond Software

The same learning loop can apply to almost any digital artifact that can be generated, evaluated, and revised.

Human goal
  ↓
Specification
  ↓
Planning
  ↓
Generation
  ↓
Technical validation
  ↓
Independent review
  ↓
Correction or redesign
  ↓
Human or audience acceptance
  ↓
Release and monitoring

Reports

Agents can:

  • Gather approved evidence
  • Create an outline
  • Draft sections
  • Validate calculations
  • Check citations
  • Detect unsupported claims
  • Improve readability
  • Produce an executive summary

Automated review can detect technical defects. Humans remain important for interpretation, emphasis, sensitivity, and recommendations.

PDFs

Agents can evaluate:

  • Pagination
  • Fonts
  • Links
  • Rendering
  • Clipping
  • Tables
  • Image quality
  • Accessibility structure
  • Print behavior

Human judgment remains valuable for visual polish and reading experience.

Presentations

Agents can:

  • Build the narrative
  • Generate slide content
  • Create visuals
  • Check text density
  • Verify claims
  • Validate timing
  • Create speaker notes
  • Adapt the material for different audiences

Human reviewers remain important for persuasion, executive communication, and audience context.

Audio

Agents can review:

  • Script completeness
  • Pronunciation
  • Loudness
  • Clipping
  • Silence
  • Pacing
  • File format
  • Segment order

Humans may still need to judge emotional tone, trust, naturalness, and cultural appropriateness.

Video

Agents can coordinate:

  • Script development
  • Storyboarding
  • Scene production
  • Narration
  • Editing
  • Captions
  • Synchronization
  • Brand review
  • Rendering

Automated systems can identify technical failures, but human or audience feedback remains essential for storytelling and emotional impact.

The broader principle is:

Any digital product with measurable intermediate states can participate in a feedback-driven convergence process.

The degree of automation depends on how clearly success can be defined.


16. Human Involvement

The objective is not to remove humans entirely.

It is to move human involvement toward decisions involving purpose, context, priorities, risk, and accountability.

Before execution

Humans define:

  • The problem
  • Intended users
  • Desired outcomes
  • Constraints
  • Prohibited actions
  • Risk tolerance
  • Privacy boundaries
  • Cost and time budgets
  • Approval requirements

During execution

Humans intervene when:

  • Requirements are ambiguous.
  • Product priorities conflict.
  • Agents propose a major architecture change.
  • Reviewers disagree on a consequential issue.
  • Security or policy exceptions are requested.
  • The project exceeds its budget.
  • The system repeatedly fails to converge.
  • Scope must change materially.

Before release

Humans confirm:

  • Fitness for purpose
  • User value
  • Residual risk
  • Compliance
  • Release scope
  • Communication
  • Rollback readiness
  • Accountability

After release

Humans evaluate:

  • Real user outcomes
  • Unexpected consequences
  • Support requests
  • Complaints
  • Whether the selected metrics represent actual success
  • Whether greater autonomy is justified
  • Whether the product should be expanded, revised, or retired

The system automates high-volume production and verification. Humans retain authority over meaning and consequential decisions.


17. Progressive Autonomy

Organizations should increase autonomy gradually.

Level Agent authority Human responsibility
0. Recommend Suggests actions Human performs all work
1. Draft Creates code or content Human reviews every artifact
2. Sandbox Builds and tests in isolation Human approves movement beyond the sandbox
3. Limited execution Performs predefined reversible actions Human manages exceptions
4. Supervised autonomy Runs the workflow end to end Human handles escalation and release boundaries
5. Full autonomy Releases without individual approval Human defines policy and audits outcomes

Most organizations should begin at Level 2.

Greater autonomy should require evidence of:

  • Stable quality
  • Reliable evaluation
  • Effective escalation
  • Complete logging
  • Bounded permissions
  • Reversible actions
  • Safe rollback
  • Consistent product outcomes

Full autonomy should be limited to low-risk, measurable, reversible tasks.


18. A Practical Pilot

A useful first project might be an internal CSV validation tool.

Project contract

goal: Create an internal tool that validates uploaded CSV files.
users:
  - operations analysts
required_capabilities:
  - upload a CSV file
  - verify required columns
  - validate data types
  - identify invalid rows
  - explain validation errors
  - export an error report
quality_requirements:
  - all mandatory functional tests pass
  - no critical security findings remain
  - no unresolved high-severity defect remains
  - required accessibility checks pass
  - typical files process within three seconds
iteration_policy:
  maximum_local_repairs_per_issue: 3
  maximum_phase_iterations: 5
  maximum_architecture_revisions: 2
escalation_triggers:
  - repeated root-cause category
  - conflicting requirements
  - material scope increase
  - requested security exception
  - budget exceeded
human_approval:
  - requirements baseline
  - major architecture change
  - production deployment

Workflow

Requirements agent
  ↓
Planning and architecture agents
  ↓
Builder agent
  ↓
Functional, security, and usability evaluation
  ↓
Failure classification
  ↓
Local repair or architecture escalation
  ↓
Independent qualification
  ↓
Human acceptance
  ↓
Controlled release
  ↓
Production monitoring

This pilot is bounded, testable, useful, and reversible. It allows the organization to learn how to operate the agentic process before applying it to higher-risk systems.


19. Metrics for the Learning System

A useful dashboard should distinguish output volume from development quality.

Production metrics

  • Time per phase
  • Total iterations
  • Agent cost
  • Human review time
  • Percentage of tasks completed automatically

Quality metrics

  • Test pass rate
  • Defect severity
  • Regression rate
  • Reopened-issue rate
  • Security findings
  • Reliability
  • Performance
  • Accessibility results

Convergence metrics

  • Defects discovered per cycle
  • New failure categories per cycle
  • Percentage of recurring issues
  • Change size per iteration
  • Reviewer disagreement
  • Architecture-change frequency
  • Improvement per unit of cost
  • Cycles since the last material improvement

Product metrics

  • User task-completion rate
  • Time saved
  • Adoption
  • User error reduction
  • Satisfaction
  • Support requests
  • Operational or business impact

Human-oversight metrics

  • Escalation frequency
  • Human override rate
  • Approval rejection rate
  • Defects found only by humans
  • Reasons for intervention
  • Incidents associated with excessive autonomy

The goal is not to minimize human involvement at any cost.

The goal is to minimize unnecessary human effort while preserving or improving quality, safety, and usefulness.


20. What Is Distinctive About This Framework?

Many individual elements already exist:

  • Agile development
  • Continuous integration
  • DevOps
  • Test-driven development
  • Code review
  • AI coding agents
  • Agent orchestration
  • Automated evaluation
  • Root-cause analysis

The contribution is the way these elements are combined.

The framework proposes:

1. Agentic development as iterative optimization

Software emerges from repeated production, evaluation, and correction rather than one-pass generation.

2. Review as the learning signal

Review provides structured feedback that determines the next action.

3. Explicit convergence detection

The system measures whether defects, regressions, disagreement, and architectural instability are decreasing.

4. Hierarchical correction

A failure can trigger change at progressively higher levels:

Code → component → interface → plan → architecture → goal

5. Learning from repeated defects

Recurring failures are not treated as unrelated tickets. They become evidence that the current strategy may be wrong.

6. Improvement of both product and process

The system can improve the implementation, test strategy, plan, architecture, orchestration, and eventually its own agent scaffold.

7. Bounded human supervision

Humans define purpose, risk, policy, and consequential decisions without reviewing every minor agent action.

That combination moves the discussion beyond “AI can write code” toward a more important question:

Can we design a production system that recognizes failure, learns at the correct level, and knows when it has enough evidence to stop?

Conclusion

The future of AI-enabled development is unlikely to be a single agent producing a perfect application from one prompt.

A more realistic and powerful future is a governed system in which specialized agents continuously:

  • Plan
  • Build
  • Test
  • Review
  • Diagnose
  • Repair
  • Reconsider
  • Validate
  • Learn

The machine-learning analogy helps because it encourages software teams to think in terms of objectives, feedback signals, error analysis, experiment histories, convergence, plateaus, and strategy changes.

The analogy should not be taken literally. Software is not simply a statistical model, and software tests are not a perfect equivalent of training and validation datasets.

Its contribution is methodological:

Agentic software development can borrow the operating discipline of machine learning without pretending that the two fields are identical.

This way of thinking can help software developers build stronger specifications, evaluations, and architectural controls. It can help data scientists apply optimization, experimentation, and error-analysis methods to software production. It also gives both groups a shared language for governing increasingly autonomous development systems.

The final goal is not full automation for its own sake.

It is a more effective division of responsibility:

Humans define purpose, priorities, constraints, and acceptable risk. AI agents perform much of the planning, construction, evaluation, and repair required to move toward those goals.

If designed carefully, software development may evolve from a sequence of manually coordinated tasks into a continuous, evidence-driven learning system.

References