The New Code

Posted on:

Jul 11, 2025, Video: The New Code — Sean Grove, OpenAI (YouTube)

  • Opening & Theme Setting

    • Thanks & Atmosphere: The past few days have been intense but inspiring.
    • Introducing the theme: “The New Code” — with a core focus on “specifications.”
    • The promise of specifications: write the intent once, run and reuse it everywhere.

  • Speaker & Agenda

    • Self-introduction: Sean, from OpenAI, working on alignment research.
    • Agenda preview:
      • Code vs. Communication: A value comparison, and why “specifications” might be better.
      • An “anatomy” of specifications using the “OpenAI Model Spec.”
      • How to convey intent to humans (human-to-human communication), using the “sycophancy problem” as a case study.
      • How to make specifications “executable”; how to convey intent to models (human-to-machine communication).
      • Thinking of specifications as “code.”
      • Several open questions.

  • Code vs. Communication: A Re-evaluation of Value

    • Interactive question: Who writes code, and who considers “code” the most valuable artifact they produce?
    • The argument: Code is only 10–20% of your value; the other 80–90% comes from “structured communication.”
    • The real process of engineering practice:
      • Talk to users, understand their challenges → Distill stories and requirements → Conceive goals and solutions.
      • Plan the implementation path → Share with colleagues → Translate the plan into code.
      • Test and validate: Focus on “whether the goal was achieved and the user’s challenge was mitigated,” not the code itself.
    • Conclusion: All these steps are essentially structured communication, and this is precisely the bottleneck.

  • The Coming Bottleneck & Role Shift

    • As AI models improve, “communication ability” will become an even more prominent bottleneck.
    • In the near future: The best communicators will be the most valuable “programmers.” Those who can communicate effectively can “program.”

  • Lessons from “Vibe Coding”

    • Why it feels good: You first communicate the intent and desired outcome, and the code is just a downstream product of that communication. The tedious implementation is left to the model.
    • The current weirdness: We write our intent into a prompt, get a code artifact from the model, and then throw the prompt away (it’s ephemeral).
    • Analogy: In traditional compilation, the binary is never the source artifact; we always regenerate the binary from the “source file.”
      • But in prompt engineering, we “keep the product (code) but discard the source (prompt),” which is like “shredding the source file while meticulously version-controlling the binary.”

  • Why We Need a “Written Specification”

    • A specification solidifies intent and value, becoming a common baseline for aligning human collaboration.
    • It is an artifact that is discussed, debated, referenced, and synchronized. Without a spec, you just have vague ideas.

  • Why Specifications Are Generally Superior to Code

    • Code is a “lossy projection” of the specification: Reading code often requires reverse-engineering why the author implemented it that way, and the goals and values behind it.
    • A complete and robust specification contains all the requirements needed to generate the implementation.
    • Compilation analogy: A source can target multiple platforms (ARM64/x86/WASM). Similarly, a good enough specification can generate:
      • TypeScript, Rust
      • Server and client
      • Documentation, tutorials, blog posts, and even podcasts.
    • Thought experiment: If you fed your entire codebase and documentation to a “podcast generator,” could it produce content that truly helps users succeed? If not, it means the key information isn’t in the code.
    • The new scarce skill: The person who can fully capture “intent and value” in a specification will become the most valuable “programmer.”
      • This doesn’t just apply to engineers; PMs, legislators, etc., are all writing “specifications.” This is a universal principle.

  • What a Specification Looks Like: The OpenAI Model Spec as an Example

    • The Model Spec: A “living document” that clearly expresses the intent and values OpenAI wants its models to internalize.
    • Iteration and Open Sourcing: It has since been updated and open-sourced.
    • Format: A set of Markdown files—
      • Human-readable, version-controllable, with a change history.
      • Written in natural language, allowing cross-functional teams (product, legal, safety, research, policy) to read, write, and discuss, sharing the same “source.”
    • Design detail: Each clause has a unique ID (e.g., sy73).
      • This ID can be used to find the corresponding file, which contains one or more “challenging prompts.”
      • The document has built-in “success criteria”: the model being tested must still comply with the clause under these challenges.

  • Case Study: The Sycophancy Problem and the Role of the Model Spec

    • A “4o” update once exhibited clear sycophantic behavior: the model pandered to the user, sacrificing objectivity and neutrality.
    • Impact: It damaged trust and raised questions about whether it was “intentional or accidental, and why it wasn’t caught.”
    • The Model Spec already had a clause against “sycophancy,” explaining that while it “feels good” in the short term, it is harmful in the long run.
    • From a specification-based perspective: If the behavior doesn’t match the spec, it’s a “bug.”
      • The actual actions: Rollback, publish research and a blog post, and fix the problem.
      • The value of the spec in the process: It serves as a “trust anchor,” clearly communicating expected and disallowed behaviors.

  • Making Specifications “Effective for Models”: Deliberative Alignment

    • Technical approach:
      • Take the specification and a set of difficult input prompts.
      • Sample responses from the “model to be tested/trained.”
      • Give the “original prompt + response + specification policy” to a “grader model” to score the response according to the spec.
      • The document is both training material and evaluation material. Reinforcement (updating weights) is done based on the scores.
    • Comparison with “injecting the spec at inference time”:
      • Putting the spec in the context for every inference can work, but it eats into the computational budget for solving the task.
      • A better way is to “bake it into the weights”: let the model internalize policies and styles (like safety, code style, testing requirements) through “muscle memory.”

  • Treating Specifications as “Code”: Toolchains and Engineering

    • Specifications are composable, executable, and testable; they have an “interface” with the real world; they can be released as “modules.”
    • “Type checking” analogy:
      • If Department A’s spec conflicts with Department B’s spec, it should be possible to expose this early and block the release.
    • “LLinter” concept:
      • If the language is too ambiguous, it will confuse both humans and models, leading to poor-quality output. There should be tools to flag this and encourage convergence.
    • Summary: Specifications bring a new engineering toolchain that is “intent-oriented rather than syntax-oriented.”

  • Legislators as Programmers: The Analogy of a Constitution

    • A constitution = a national-level “model spec”: the text strives to be clear and dependable.
    • It has:
      • A versioned update path (amendments).
      • “Judicial review” = the evaluator: determines the compliance of real-world situations with the policy.
      • “Out-of-distribution” cases due to world complexity: require a lot of “computation” (deliberation) to clarify applicability.
      • “Case law” = I/O pairs: form unit tests that disambiguate and reinforce the original spec.
      • Chain and execution: a long-term “training loop” that aligns the population to a common intent and value.
    • Inference: Legislators will become more like programmers, and vice versa.

  • Universal Principles & Role Redefinition

    • Programmers use “code specs” to align silicon; PMs use “product specs” to align teams; legislators use “legal specs” to align society.
    • Every prompt you write is a “proto-spec”: you are aligning an AI to a shared intent and value.
    • Therefore, everyone is a spec author; specs make delivery faster and safer; whoever writes the spec is the “programmer.”

  • Returning to the “Essence of Engineering”

    • Engineering has never been about “the code itself.” Coding is an important skill, but not the end goal.
    • Engineering = “the precise exploration of software solutions to human problems by humans.”
    • The shift: From diverse forms of “machine coding” to a unified form of human coding (expressing problem-solution pairs in a human-readable way).

  • Practical Advice & Future Vision

    • Four practical steps:
      • The first step for your next AI feature: write a spec.
      • Clarify expectations and success criteria, and debate and polish it for readability/clarity.
      • Make the spec executable: feed it to the model, and test and evaluate based on it.
      • Close the loop: use evaluation and feedback to improve the model and the spec itself.
    • Future “IDE” vision:
      • A shift from a “Development Environment” to an “Integrated Thought Clarifier”:
        • As you write a spec, it automatically extracts ambiguities and pushes for clarification, thereby improving both human-human and human-machine communication quality.
    • Recruitment & Call to Action:
      • A domain that urgently needs to be specified and is well-suited for it: large-scale alignment of “agents.”
      • A common symptom: “You only realize later that you never clearly told it what you wanted” — this calls for a spec.
      • The newly formed “Agent Robustness” team: an invitation to join and help advance safe AGI.
    • Closing thanks and willingness to chat.

  • [00:00–00:48] Opening & Theme

    • Thanks & atmosphere: The last few days have been intense but very inspiring.
    • Theme: “The New Code” — specifically, Specifications: write the intent once, run it everywhere.

  • [00:48–01:16] Self-Introduction & Agenda

    • Speaker: Sean Grove (OpenAI Alignment Research).
    • Agenda: Code vs. communication, the anatomy of a spec (using the Model Spec as an example), how to convey intent to humans/models, treating specs as “code,” and finally, some open questions.

  • [01:16–02:13] Value Output: Code or Communication?

    • Audience hand-raising: Many consider “code” their most valuable output.
    • The argument: Code is only 10–20% of the value; 80–90% comes from “structured communication.”

  • [02:13–03:02] The Real Workflow of Engineering

    • Talk to users → distill requirements into stories → devise goals → plan implementation → share with colleagues → translate to code → validate “the effect the code has on the world” (not the code itself).

  • [03:02–03:53] The Bottleneck is “Structured Communication”

    • Knowing what to build, why to build it, how to build it, and whether the goal was met are all communication problems.

  • [03:53–04:16] The Stronger the AI, the More Crucial the Communication

    • The most valuable programmer of the future = the person who can best communicate intent. If you can communicate, you can program.

  • [04:16–05:05] Lessons from “Vibe Coding”

    • Why it feels good: You communicate the intent first, and the code is a downstream product; the model does the “manual labor” for you.

  • [05:05–05:35] Our “Counter-intuitive” Handling of Prompts

    • Current practice: Write intent into a prompt → get code → throw the prompt away. This is like shredding the source code while meticulously version-controlling the binary.

  • [05:35–05:56] The Source Document is the “Regenerable” Asset

    • Just as you always recompile the binary from the source code, the source = the “specification” is the most valuable artifact.

  • [05:56–06:45] The Role of a Specification

    • A written specification aligns humans on intent/value and serves as an anchor for discussion, debate, and synchronization. No spec = just vague ideas.

  • [06:45–07:31] Why Specs Are Generally Better Than Code

    • Code is a “lossy projection” from the spec to the implementation (like how decompiling doesn’t recover comments/names/motivation); much of the intent and value is lost in the code.

  • [07:31–07:52] Specs Can “Generate” Code

    • The specification encodes all the requirements needed to generate the code.

  • [07:52–08:28] Write Once, “Compile” Everywhere

    • Just as source code can be compiled to ARM/x86/WASM, a sufficiently robust specification can have a model produce TS, Rust, server/client, docs, tutorials, blog posts, and even podcasts.

  • [08:28–09:01] Thought Experiment: Can You Generate a “Useful Podcast” from Just a Codebase?

    • If not, it means a lot of key information is not in the code.

  • [09:01–09:24] The New Scarce Skill: Writing Specs

    • The person who can write a spec that fully captures intent and value = the most valuable programmer. This includes not just engineers, but also PMs, legislators, etc.

  • [09:24–10:35] Case Study: The OpenAI Model Spec

    • The Model Spec is a living document that clearly expresses the intent and values OpenAI wants its models to embody. It was significantly updated and open-sourced on GitHub in February 2025, using Markdown to facilitate collaboration and review across cross-functional teams like legal, safety, policy, research, and product. (OpenAI, GitHub)

  • [10:35–11:02] Traceable Clauses & “Hard Cases”

    • Each clause has an ID (e.g., sy73). The repository contains corresponding challenging evaluation prompts as part of the success criteria/acceptance cases. (GitHub)

  • [11:02–12:09] Case Study: Sycophancy

    • A recent GPT-4o update led to “excessive sycophancy.” This behavior erodes trust. The Model Spec has explicitly stated “Don’t be sycophantic” since its initial version. (OpenAI, Model Spec)

  • [12:09–12:41] Questions and Response

    • Outsiders and researchers provided more worrying examples. The question was whether it was intentional or unintentional and why it wasn’t caught (highlighting the importance of pre-deployment evaluation/alignment processes). As an alignment anchor, if the behavior is inconsistent with the spec = it’s a bug. OpenAI rolled back the update and published an explanation/research. (OpenAI)

  • [12:41–13:18] The “Trust Anchor” Role of a Spec

    • It clearly communicates which behaviors are expected/not expected. Even if only for aligning “people,” a spec is already very useful.

  • [13:18–14:23] Aligning Models to the Spec Too: Deliberative Alignment

    • Method: Feed the spec + difficult inputs to the model being tested/trained → get a response → give it, along with the original prompt and policy, to a “grader model” to score it against the spec → use the score for reinforcement learning/updates. The document is both training and evaluation material. (OpenAI)

  • [14:23–14:54] Training vs. Loading Policy at Inference

    • Putting the spec in the system/developer message only at inference time also works, but it takes up compute budget. A better way is to “bake the spec into the weights,” giving the model “muscle memory.” (The spec can include requirements from code style, testing, to safety.) (OpenAI)

  • [14:54–15:36] Treating Specs Like “Code”

    • Specs are composable, executable, testable, have external interfaces, and can be distributed as modules. They can check for cross-departmental spec conflicts like a type checker and can have an “LL-lint” to flag ambiguous/misleading language.

  • [15:36–16:35] Legal Analogy: Legislators as “Programmers”

    • The U.S. Constitution is like a “national Model Spec”: it has clear text (striving for no ambiguity), versioned amendments, judicial review acts as the “grader model,” case law is like “unit tests,” and long-term enforcement is like a “training loop.” (National Archives)

  • [16:35–17:56] Universal Principle

    • Programmers use “code specs” to align silicon; PMs use “product specs” to align teams; legislators use “legal specs” to align people. Every time you write a prompt, you’re writing a “proto-spec.”

  • [17:56–19:23] Engineering Was Never “About Writing Code”

    • Engineering = the precise process of humans exploring software solutions for human problems. We are moving from many types of “machine coding” to a unified “human coding” (the expression of specs/intent).

  • [19:23–20:27] Actionable Advice

    • Before building your next AI feature: write a spec first. Define success criteria. Make the spec executable (feed it to the model and accept/reject based on the spec). A proposed idea: a future IDE might be an “Integrated Thought Clarifier,” helping you find and resolve ambiguity.

  • [20:27–21:15] Open Questions & Recruitment

    • What areas are both suitable for and in urgent need of specification? One answer: large-scale agent alignment (often you don’t clearly tell it what you want). The newly formed Agent Robustness team is hiring, with the goal of delivering safe AGI for the benefit of all humanity.

Supplementary & Corroborating External Links (Related to the talk’s key points)

  • OpenAI Model Spec (latest version and open-source repository): The “company-level specification” for defining model behavior, significantly updated and open-sourced on 2025-02-12. You can see clauses like “Don’t be sycophantic” and example evaluations. (OpenAI, GitHub, Model Spec)
  • Deliberative Alignment: OpenAI’s training paradigm that teaches an LLM human-written, interpretable safety rules and has the model explicitly reason about the rules and self-critique before answering. (OpenAI)
  • GPT-4o “Sycophancy” Incident & Rollback: OpenAI’s explanation of what happened, why it’s important, and how it was fixed (including the rollback and improvements to evaluation/feedback collection). (OpenAI)
  • Sycophancy Research Background: Anthropic’s research on the general tendency of RLHF models toward “sycophancy.” There is also recent research and media coverage on the controllability of “personality/trait vectors” for further reading. (Anthropic, The Verge)
  • Legal Analogy Materials: The U.S. National Archives’ official introduction and original text of the Constitution. (National Archives)

Note: The timeline above is strictly organized based on the per-paragraph transcript you provided. External links are for supplementary and verification purposes only (they do not alter the original meaning of the video).

Tags: AI-Agent