Stanford CS329A Part 1 — From Scaling to Self-Improving Agents

Stanford CS329A Part 1 provides the map for the entire course. It connects four layers that are often discussed separately: pretraining capability, post-training behavior, test-time search, and agent orchestration. The central lesson is that a strong generator is not yet a reliable agent. Reliability comes from feedback, verification, action, correction, and stopping.

This page is the detailed teaching companion to the public course-overview lecture. It is separate from CS329A Agent-System Literacy, which interprets the course for AI education. Here, the goal is to learn the lecture itself.

Download the Slides

The Four Frontiers

One useful teaching synthesis is to organize the lecture by where compute and engineering effort are spent.

pretraining
    -> post-training
        -> test-time search
            -> agent orchestration

These layers complement rather than replace one another.

1. Pretraining Expanded Capability

The lecture begins with the historical scaling axes:

• training compute;
• data volume;
• parameter count.

Scaling was associated with lower test loss and stronger few-shot and chain-of-thought benchmark behavior in selected model families.

The evidence needs a boundary. Observed benchmark emergence is not proof of a universal law, and it does not prove that the behavior was absent from training data. The lecture’s later discussion also acknowledges that modern reasoning behavior is deliberately shaped by curated data, instruction tuning, and reinforcement learning.

2. Post-Training Shapes Assistant Behavior

A pretrained model and an assistant-like model are not the same object.

pretraining
   -> fine-tuning on selected data
      -> instruction tuning
         -> preference optimization

A preference model approximates evaluator preferences. It is not automatically a truth detector.

3. Test-Time Scaling Searches a Fixed Model

At test time, model weights remain fixed while the system spends additional compute on a particular request.

Possible strategies include repeated sampling, longer reasoning, search over branches, and candidate selection. Tool use appears later as part of agent orchestration, though it can also consume inference-time compute in broader systems.

The simplest picture is:

fixed model
   -> candidate A
   -> candidate B
   -> candidate C
          |
          v
      verifier
          |
          v
     returned answer

Repeated sampling is one test-time method, not a synonym for every kind of test-time compute.

4. Coverage Is Not Reliability

For independent attempts with constant single-attempt success probability p, the intuitive coverage formula is:

coverage = 1 - (1 - p)^k

But coverage asks whether any candidate is correct. Deployed reliability asks whether the system returns a correct candidate.

Real model samples are correlated, and real selectors make errors. The formula is useful intuition, not a deployment guarantee.

5. Verification Is Domain-Dependent

The cost of checking an answer changes by domain.

A weak verifier can select a polished mistake. The source note identifies robust verification and the generator-verifier gap as active research problems.

6. Reasoning as a Search Process

The lecture presents reasoning operationally as an iterative process:

analyze
   -> decompose
      -> try a path
         -> inspect feedback
            -> correct or backtrack

This system view is useful without claiming that visible reasoning text is a faithful account of hidden model cognition. A reasoning trace is not proof that the reasoning is correct.

7. Two Meanings of Self-Improvement

The phrase “self-improving” can hide a persistence distinction.

If an agent fixes a mistake and forgets it next session, it improved the current trajectory but did not acquire durable learning.

8. Chatbot Versus Agent

A chatbot mainly returns information. An agent owns a bounded goal-directed loop.

chatbot
request -> response

agent
goal -> plan -> act -> observe -> correct -> stop

An agent typically needs:

• task state;
• tools or environment actions;
• feedback;
• memory or artifacts;
• a stopping or escalation rule.

Autonomy can remain bounded by human approval gates.

9. Open Loops Versus Workflow Graphs

The lecture distinguishes open-ended loops from practical workflow graphs.

Neither is universally better. Choose by task risk, uncertainty, and audit requirements.

This connects directly to harness, loop, and graph engineering:

harness = tools, memory, permissions, environment
loop    = feedback, retry, correction, stop
graph   = explicit routing through generation and checks

Three Questions for Any Self-Improving Agent

Before accepting the label, ask:

1. What generates alternatives?
2. What verifies success?
3. What persists after the task?

These questions separate capability claims from system behavior.

The Lecture in 12 Teaching Moves

Teaching Questions

1. Which layer changes model weights: pretraining, post-training, or test-time search?
2. Why is preference optimization not equivalent to truth optimization?
3. What is the difference between coverage and returned reliability?
4. Which tasks in your work have cheap verifiers?
5. If an agent fixes an error and forgets it next session, what improved?
6. Which system properties turn a chatbot into an agent?
7. When should a workflow graph replace an open-ended loop?
8. What would count as durable self-improvement in a student study agent?

Homework: Audit an Agent Claim

Choose one coding, research, study, or productivity agent.

Document:

Conclude whether “self-improving agent” is an accurate description or an overstatement.

Source Versus Teaching Synthesis

Directly sourced:

• scaling, post-training, and test-time-compute progression;
• repeated sampling and verifier selection;
• reasoning and agent workflow examples;
• verification bottlenecks and applications such as coding agents, customer-support assistance, research synthesis, and AI-scientist-style workflows.

Teaching synthesis:

• the four-frontier structure;
• the persistence boundary;
• the three-question diagnostic;
• the homework and education examples.

References

• Official course site
• Part 1 video
• Official CS329A playlist

Related Reading

• Stanford CS329A — Self-Improving AI Agents: Public Lecture Learning Path
• Stanford CS329A Part 2 — Test-Time Compute Scaling
• Stanford CS329A Part 3 — Robust Verification
• CS329A Agent-System Literacy — The Next Layer of AI Education
• Harness / Loop / Graph Engineering — Environment, Feedback, Flow