Stanford CS329A Part 7 — Self-Improvement and Deep Research Agents

Stanford CS329A Part 7 connects two apparently different systems: competitive-programming agents that search over code and deep-research agents that search over evidence. The durable lesson is not simply to generate more. A useful search loop must create non-redundant options, obtain feedback, select under a budget, and know when to stop.

This teaching companion is based on the complete official-video transcript, 14 visually inspected cues, and the three primary course readings. Cross-paper connections are explicitly labeled as teaching synthesis rather than paper findings.

Download the Slides

The Core Model

generator
   |
   v
diverse candidates or queries
   |
   v
external feedback --------+
   |                       |
   +--> rank / select <----+
             |
       return or refine
             |
        stopping rule

This is a teaching abstraction. Running code against tests and retrieving documents provide different evidence with different failure modes.

1. AlphaCode Turns Code Generation Into Search

Competitive programming requires more than translating an instruction into syntax. A solver must:

1. interpret a long problem statement;
2. select an algorithm;
3. implement it correctly;
4. satisfy public examples and unseen tests.

AlphaCode uses a pipeline rather than one answer:

training + problem
       |
large-scale code sampling
       |
execute public examples
       |
behavioral clustering
       |
small submission set

The 2022 paper reports an average top-54.3% rank in simulated Codeforces competitions with more than 5,000 participants. That statement belongs to those selected contests and that evaluation. It is not a claim that AlphaCode performs all professional software-engineering work.

2. pass@k and 10@k Ask Different Questions

A correct program can exist in a million-sample pool and still never be submitted. Coverage belongs to the set; practical reliability belongs to the generator, filters, clusters, scorer, and submission policy together.

3. AlphaCode 2 Improves Generation and Ranking

The AlphaCode 2 technical report describes five main components:

The report states:

• 43% of 77 selected problems solved, versus 25% for AlphaCode;
• estimated average performance at the 85th percentile;
• up to one million candidates sampled per problem;
• at most ten submissions per problem.

Those denominators matter. The report demonstrates a strong competitive-programming system, not a cheap or general coding assistant.

4. Wider Search Is Not Always Better Search

As sample count rises, three bottlenecks can dominate:

1. Redundancy: many programs repeat the same strategy or mistake.
2. Selection: a scorer must identify correctness from a huge pool.
3. Cost: generation, execution, clustering, and reranking all consume resources.

The lecture discussion suggests difficulty-aware budgets, decomposition, tree search, backtracking, and human help for unfamiliar patterns. These are research directions discussed in class, not AlphaCode 2 results presented in the technical report.

5. Search-o1 Searches During Reasoning

A long reasoning chain may encounter a fact the model does not reliably know. If it guesses, later steps can remain fluent while inheriting the error.

Static retrieval and agentic retrieval differ in control:

Search-o1 inserts search actions into the reasoning process. The paper’s Reason-in-Documents stage reads retrieved documents, extracts focused evidence, and returns that evidence to the main chain.

reasoning state
     |
knowledge gap detected
     |
search -> documents -> focused evidence
     |                     |
     +------ resume <------+

This can reduce irrelevant context, but it does not guarantee that retrieval found the right sources or that the extraction preserved every important fact.

6. One Diagnostic Across Two Domains

Use five questions to audit either a coding-search or deep-research system:

The shared questions are a teaching synthesis. Code tests are executable evidence; web documents remain claims that need provenance and interpretation.

The Lecture in 12 Teaching Moves

Homework: Design a Bounded Search Agent

Choose one university task: a coding assignment, literature review, data-analysis report, or teaching-plan design.

Specify:

1. the candidate or query generator;
2. the mechanism that creates useful diversity;
3. the feedback source and its blind spots;
4. the selector or reranker;
5. one false-positive feedback event;
6. a stopping rule with cost and latency limits;
7. what uncertainty must be disclosed to the user.

Then identify which parts are mechanically verifiable and which still require teacher or expert judgment.

Source Versus Teaching Synthesis

Directly sourced from the lecture and primary works:

• AlphaCode’s training, sampling, filtering, clustering, and evaluation setup;
• the distinction between candidate coverage and small-set selection;
• AlphaCode 2’s policy family, execution filtering, clustering, scoring model, and reported results;
• Search-o1’s agentic retrieval and Reason-in-Documents mechanism;
• the lecture’s discussion of adaptive search and calibration.

Teaching synthesis in this page:

• the five-question search diagnostic;
• the unified code-and-research loop;
• the university homework;
• the claim that feedback-channel design is the bridge between these examples.

References

1. Stanford CS329A course site.
2. Official Part 7 video.
3. Yujia Li et al., Competition-Level Code Generation with AlphaCode, 2022.
4. AlphaCode Team, Google DeepMind, AlphaCode 2 Technical Report, 2023.
5. Xiaoxi Li et al., Search-o1: Agentic Search-Enhanced Large Reasoning Models, 2025.

Related Reading

• Stanford CS329A — Self-Improving AI Agents: Public Lecture Learning Path
• Stanford CS329A Part 6 — Train-Time Scaling and Scaling RL
• Stanford CS329A Part 8 — Agentic Evaluations and Long-Horizon Tasks