diff --git a/s21_workflow_runtime/README.ja.md b/s21_workflow_runtime/README.ja.md index 83412b38..24aa53b9 100644 --- a/s21_workflow_runtime/README.ja.md +++ b/s21_workflow_runtime/README.ja.md @@ -8,9 +8,7 @@ s01 → ... → s19 → s20 → `s21` → [s22](../s22_goal_loop/) > > **Harness 層**: Orchestration — single-agent loop の上に、決定的な multi-agent script runtime を追加します。 -> **情報源の境界：** この章の製品詳細は Claude Code 2.1.177 の clean-room 行動再構成に基づく。後続リリースで名称や制限は変わり得る。`code.py` はオフライン教材モデルであり、製品ソースの複製ではない。 -> -> 教材 CLI は `async_launched` を出した後、再現可能な出力のため同じプロセスで完了を待つ。示すのは lifecycle と journal であり、main loop の並行実行そのものではない。 +`code.py` は demo を決定的に保つため、`async_launched` を出した後、同じ process で完了を待ちます。常駐 background service を用意しなくても、lifecycle と journal を確認できます。 --- @@ -26,9 +24,9 @@ s01 から s20 まで、loop は常にモデル駆動で 1 step ずつ進みま ## 計画は chat のラウンドを重ねず、コードに書く -Claude Code の tool pool には `Workflow` ツールがあります。あなたが渡すか、モデルが high-intensity mode で起動した script は、`agent() / parallel() / pipeline() / phase()` という少数の primitive を使い、orchestration を決定的なコードとして表します。 +harness の tool pool に `Workflow` ツールを追加します。ユーザーまたはモデルが渡す script は、`agent() / parallel() / pipeline() / phase()` という少数の primitive を使い、orchestration を決定的なコードとして表します。 -main loop から見えるのは 1 回の `tool_use` だけで、すぐ「バックグラウンドで起動済み」という結果を受け取ります。本当の実行は background runtime で進み、進捗をリアルタイムに報告し、全過程をディスク上の journal へ記録します。script の中間結果は変数に保存され、会話履歴の場所を取りません。`resumeFromRunId` で再開すると、変更されていない `agent()` は journal cache に当たり、以前の結果を直接使って checkpoint から続行します。 +main loop から見えるのは 1 回の `tool_use` だけで、すぐ「バックグラウンドで起動済み」という結果を受け取ります。本当の実行は background runtime で進み、進捗をリアルタイムに報告し、全過程をディスク上の journal へ記録します。script の中間結果は変数に保存され、会話履歴の場所を取りません。`resume_from_run_id` で再開すると、変更されていない `agent()` は journal cache に当たり、以前の結果を直接使って checkpoint から続行します。 ![Workflow Runtime Overview](images/workflow-runtime-overview.svg) @@ -45,7 +43,7 @@ async def sample_workflow(ctx, args): ## Workflow ツール: バックグラウンド起動、main loop には 1 回の call だけ -`Workflow`（別名 `RunWorkflow`）は main Agent の tool pool にあります。明示的に「この workflow を実行」と頼む、保存済みの `/command` を使う、またはモデルが自動で high-intensity path へ入ると、モデルが `Workflow(...)` の tool call を出します。 +`Workflow` は main Agent の tool pool にあります。ユーザーが保存済み workflow の実行を求めるか、タスクが既知の orchestration に一致したときにモデルがこのツールを選びます。どちらも 1 回の `Workflow(...)` tool call になります。 ツールは argument を parse し、meta 情報を検証し、permission check を通し、local workflow task を登録すると、すぐ「非同期で起動済み」と返します。main loop は block せず別の仕事を続け、workflow は background で実行されます。これは s13 の引換券 pattern を拡大したものです。先に引換券を渡し、結果ができたら通知します。 @@ -60,11 +58,9 @@ class WorkflowTool: ... # 残りはバックグラウンドで進む ``` -> 実際の Claude Code は `{status:'async_launched', taskId, taskType:'local_workflow', runId, summary, transcriptDir, scriptPath}` をすぐ返し、background task の完了後に通知します。 +## Workflow metadata: 起動前に検証する -## Script と meta: 1 行目を正しく書く - -script の 1 行目は必ず `export const meta = { name, description, phases }` とし、変数、関数呼び出し、文字列連結を含まない純粋な literal でなければなりません。runtime はコードを一切実行する前に parse します。`name` と `description` は task と UI の表示に使い、`phases` は progress bar の group 名を定義します。 +各 workflow は `name`、`description`、任意の `phases` を持つ metadata object を登録します。runtime は workflow code を実行する前に検証します。`name` と `description` は task と UI の表示に使い、`phases` は progress bar の group 名を定義します。 不正な入力はすぐ `WorkflowInputError` になり、登録時に止まります。s14 の cron 式検証と同じ考えです。不正な script が実行時まで進んでから壊れないようにします。 @@ -86,8 +82,6 @@ def validate_meta(meta): return meta ``` -> 実際の Claude Code の `parseWorkflowScript` は、meta を 1 行目の純粋な literal に限定します。教材版は dict を直接受け取り、この部分を簡略化しています。 - ## Orchestration primitive: この少数だけで、すべての flow を書ける script は独立した context で動き、global variable として使えるのは少数の orchestration primitive だけです。script 自身はファイルを直接読み書きせず、shell も実行しません。実際のコード操作は、派遣された subagent が自分の tool permission で行います。primitive はすべて `ExecutionState` の method です。 @@ -113,8 +107,6 @@ async def pipeline(self, items, *stages): return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)]) ``` -> 実際の Claude Code は同名 primitive を script VM の context へ注入します。さらに `args`、total/spent/remaining を持つ `budget`、最大 1000 Agent の上限、concurrency semaphore も提供します。 - ## 構造化出力: Subagent に散文を返させない `agent({schema})` は、schema に一致する JSON object を subagent に要求します。内部では structured output call を 1 回使い、runtime が結果を schema で検証し、不一致なら 1 回 retry します。下流コードが受け取るのは規則的な object であり、再 parse が必要な長文ではありません。 @@ -132,8 +124,6 @@ if schema is not None: raise WorkflowInputError(f"agent({{schema}}) の出力が不正です: {err}") ``` -> 実際の Claude Code は `SimpleJsonSchema`、`StructuredOutput` ツール、schema-aware retry を組み合わせ、出力形式を保証します。 - ## Background task と progress event `LocalWorkflowTask` は status と token usage を管理し、SDK style の event stream を外へ出します。`task_started` → phase change、subagent start、log batch を含む一連の `task_progress` → 完了、失敗、停止に加え、output file、token 数、tool call 数、所要時間を含む最後の `task_notification` です。 @@ -147,11 +137,9 @@ class LocalWorkflowTask: print(f" progress {ptype} ...") ``` -> 実際の Claude Code は進捗を task state へまとめ、`task_progress.workflow_progress` として UI と SDK へ送ります。 - ## 保存: Snapshot + journal で中断から再開する -各 run は `~/.claude/projects///` に 5 種類を書きます。`.json` snapshot、`.output.json` output、`.journal.jsonl` journal、`scripts/.js` の script copy、`subagents/workflows//` の subagent transcript です。保存した再利用可能な workflow は project scope の `.claude/workflows/` または user scope の `~/.claude/workflows/` に置きます。 +この最小 runtime は各 run を `s21_workflow_runtime/.runtime/` に保存します。`.json` snapshot、`.output.json` output、`.journal.jsonl` journal です。production harness では workflow script や subagent transcript も保存できますが、snapshot と journal が安定した `runId` を共有することが重要です。 journal は checkpoint resume の中心で、各 `agent()` の結果を 1 行ずつ記録します。 @@ -165,7 +153,7 @@ class WorkflowJournal: ## Resume: runId から続行し、変更のないものを再利用する -`Workflow({scriptPath, resumeFromRunId, args})` を呼ぶと script を再実行しますが、各 `agent()` は決定的な semantic key を計算します。journal に key があれば、再実行せず cached result を返します。変更のない call はすべて cache hit し、変更された call とそれに依存する後続 step だけが本当に動きます。 +`resume_from_run_id` を渡して workflow を再度呼ぶと script を再実行しますが、各 `agent()` は決定的な semantic key を計算します。journal に key があれば、再実行せず cached result を返します。変更された call と、それに依存する後続 step だけが本当に動きます。 key は concurrency の完了順に依存してはいけません。`parallel` と `pipeline` の Agent は不定の順番で完了します。「何番目に完了したか」を key にすると、次回の cache が別の call へ対応してしまいます。そのため key は競合する counter ではなく、call の内容、つまり type、label、prompt、schema の stable hash です。 @@ -181,11 +169,9 @@ if cached is not MISS: return cached ``` -> 実際の Claude Code も「決定的 semantic key + journal cache」という考えです。同じ session で resume すると、完了済み `agent()` は cached result を直接返し、その後だけを実行します。 - ## 決定性: Resume に意味を持たせる再現性 -resume が動くには、まず script が再現可能でなければなりません。runtime は `Date.now()`、引数なしの `new Date()`、`Math.random()` などの非決定的なものを script context から取り除き、Node native API も渡しません。同じ script + 同じ argument → 同じ key → 100% cache hit になります。教材版は stable hash で同じ性質を得ます。実際の版は、非決定的な source を除いた sandbox VM で JavaScript 全体を実行します。 +resume が動くには、workflow が再現可能でなければなりません。この最小 Python runtime は stable hash と決定的な mock runner を使い、同じ workflow + 同じ argument から同じ key を作ります。production harness では workflow code も隔離し、制御されていない clock、randomness、filesystem access などの非決定的な source を除くべきです。 ## 実際に動かす diff --git a/s21_workflow_runtime/README.md b/s21_workflow_runtime/README.md index c1bc887a..20b9541c 100644 --- a/s21_workflow_runtime/README.md +++ b/s21_workflow_runtime/README.md @@ -8,9 +8,7 @@ s01 → ... → s19 → s20 → `s21` → [s22](../s22_goal_loop/) > > **Harness layer**: Orchestration — a deterministic multi-agent script runtime above the single-agent loop. -> **Source boundary:** Product details in this chapter are a clean-room behavioral reconstruction of Claude Code 2.1.177. Names and limits may change in later releases; `code.py` is an offline teaching model, not copied product source. -> -> The teaching CLI emits `async_launched` and then awaits completion in one process for deterministic output. It demonstrates the lifecycle and journal, not a concurrently running main loop. +`code.py` keeps the demo deterministic: it emits `async_launched` and then awaits completion in one process. This demonstrates the lifecycle and journal without requiring a long-running background service. --- @@ -26,9 +24,9 @@ Making the model drive this process one round at a time in the main loop is slow ## Put the Plan in Code, Not in a Sequence of Chat Turns -Claude Code includes a `Workflow` tool in its tool pool. You, or the model when it enters a high-intensity mode, provide a script that expresses deterministic orchestration through a few simple primitives: `agent()`, `parallel()`, `pipeline()`, and `phase()`. +Add a `Workflow` tool to the harness tool pool. The user or model provides a script that expresses deterministic orchestration through a few simple primitives: `agent()`, `parallel()`, `pipeline()`, and `phase()`. -The main loop sees only one `tool_use` and immediately receives a "started in the background" result. Real execution continues inside the background runtime, which reports progress in real time and records every step in a journal on disk. Intermediate script results live in variables instead of taking space in conversation history. When restarted with `resumeFromRunId`, unchanged `agent()` calls hit the journal cache and reuse previous results, resuming from the checkpoint. +The main loop sees only one `tool_use` and immediately receives a "started in the background" result. Real execution continues inside the background runtime, which reports progress in real time and records every step in a journal on disk. Intermediate script results live in variables instead of taking space in conversation history. When restarted with `resume_from_run_id`, unchanged `agent()` calls hit the journal cache and reuse previous results, resuming from the checkpoint. ![Workflow Runtime Overview](images/workflow-runtime-overview.svg) @@ -45,7 +43,7 @@ async def sample_workflow(ctx, args): ## The Workflow Tool: Start in the Background; the Main Loop Sees One Call -`Workflow`, also known as `RunWorkflow`, lives in the main agent's tool pool. You may explicitly ask to "run this workflow," invoke a saved `/command`, or let the model enter a high-intensity path automatically. In each case, the model emits a `Workflow(...)` tool call. +`Workflow` lives in the main agent's tool pool. The user can request a saved workflow, or the model can select the tool when a task matches a known orchestration. In either case, the model emits one `Workflow(...)` tool call. The tool parses the arguments, validates metadata, checks permissions, registers a local workflow task, and immediately returns "started asynchronously." The main loop does not block and can continue with other work while the workflow runs in the background. This is the claim-ticket pattern from s13 at a larger scale: hand over the ticket now, notify the user when the result is ready. @@ -60,11 +58,9 @@ class WorkflowTool: ... # The rest proceeds in the background ``` -> The real Claude Code immediately returns `{status:'async_launched', taskId, taskType:'local_workflow', runId, summary, transcriptDir, scriptPath}`, then sends a notification when the background task finishes. +## Workflow Metadata: Validate Before Launch -## Script and Meta: The First Line Must Be Correct - -The script's first line must be `export const meta = { name, description, phases }`, and it must contain only literals: no variables, function calls, or string concatenation. The runtime parses it before executing any code. `name` and `description` identify the task in the UI, while `phases` names groups in the progress display. +Each workflow registers a metadata object with `name`, `description`, and optional `phases`. The runtime validates it before executing any workflow code. `name` and `description` identify the task in the UI, while `phases` names groups in the progress display. Invalid input raises `WorkflowInputError` immediately and is rejected during registration. This is the same idea as validating cron expressions in s14: do not wait until execution to discover a bad script. @@ -86,8 +82,6 @@ def validate_meta(meta): return meta ``` -> The real Claude Code's `parseWorkflowScript` requires meta to be the first line and a pure literal. The teaching version accepts a dict directly to simplify this part. - ## Orchestration Primitives: A Small Set Is Enough for Every Flow A script runs in an isolated context with only a small set of orchestration primitives as globals. The script does not read files or run shell commands directly. All real code operations are performed by dispatched subagents under their own tool permissions. These primitives are methods on `ExecutionState`: @@ -113,8 +107,6 @@ async def pipeline(self, items, *stages): return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)]) ``` -> The real Claude Code injects same-named primitives into the script VM. It also exposes `args`, `budget` with total/spent/remaining values, an agent limit of up to 1000, and a concurrency semaphore. - ## Structured Output: Do Not Let Subagents Return Essays `agent({schema})` requires a subagent to return a JSON object matching the schema, internally through one structured-output call. The runtime validates the result and retries once if it does not match. Downstream code receives a regular object instead of a long essay that must be parsed again. @@ -132,8 +124,6 @@ if schema is not None: raise WorkflowInputError(f"agent({{schema}}) returned invalid output: {err}") ``` -> The real Claude Code combines `SimpleJsonSchema`, a `StructuredOutput` tool, and schema-aware retries to enforce the output format. - ## Background Tasks and Progress Events `LocalWorkflowTask` maintains status and token usage and emits an SDK-style event stream: `task_started` → a sequence of `task_progress` events containing phase changes, subagent starts, and log batches → one final `task_notification` reporting completion, failure, or stop, plus output files, token count, tool calls, and elapsed time. @@ -147,11 +137,9 @@ class LocalWorkflowTask: print(f" progress {ptype} ...") ``` -> The real Claude Code folds progress into task state and sends it to the UI and SDK as `task_progress.workflow_progress`. - ## Storage: Snapshot + Journal for Resuming after Interruptions -Each run writes five artifacts under `~/.claude/projects///`: a `.json` snapshot, `.output.json` output, `.journal.jsonl` journal, a `scripts/.js` script copy, and subagent transcripts under `subagents/workflows//`. Reusable workflows that you save live in `.claude/workflows/` at project scope or `~/.claude/workflows/` at user scope. +The minimal runtime stores each run under `s21_workflow_runtime/.runtime/`: a `.json` snapshot, `.output.json` output, and `.journal.jsonl` journal. A production harness may also persist the workflow script and subagent transcripts, but the key requirement is that the snapshot and journal share a stable `runId`. The journal is the core of checkpointed resume. It records every `agent()` result one line at a time: @@ -165,7 +153,7 @@ class WorkflowJournal: ## Resume: Continue by runId and Reuse Everything Unchanged -Calling `Workflow({scriptPath, resumeFromRunId, args})` reruns the script, but every `agent()` computes a deterministic semantic key. If that key is present in the journal, it returns the cached result without executing again. Every unchanged call hits the cache; only a changed call and the downstream steps that depend on it actually rerun. +Calling the workflow again with `resume_from_run_id` reruns the script, but every `agent()` computes a deterministic semantic key. If that key is present in the journal, it returns the cached result without executing again. Every unchanged call hits the cache; only a changed call and the downstream steps that depend on it actually rerun. The key detail is that keys cannot depend on concurrency order. Agents in `parallel` and `pipeline` finish in nondeterministic order. If "the nth completion" became the key, cache entries would map to the wrong calls on the next run. A key therefore uses a stable hash of call content, including type, label, prompt, and schema, rather than a shared counter: @@ -181,11 +169,9 @@ if cached is not MISS: return cached ``` -> The real Claude Code uses the same idea: deterministic semantic keys plus a journal cache. Resuming within the same session returns cached results for completed `agent()` calls and runs only the remaining ones. - ## Determinism: Reproducibility Makes Resume Meaningful -Resume works only if the script is reproducible. The runtime therefore removes nondeterministic sources such as `Date.now()`, no-argument `new Date()`, and `Math.random()` from the script context, and does not expose native Node APIs. The same script plus the same arguments produces the same keys and a 100% cache hit. The teaching version obtains the same property through stable key hashing; the real version runs the entire JavaScript inside a sandboxed VM with those sources removed. +Resume works only if the workflow is reproducible. The minimal Python runtime uses stable hashes and a deterministic mock runner, so the same workflow plus the same arguments produces the same keys. A production harness should also isolate workflow code and remove uncontrolled clocks, randomness, filesystem access, and other sources of nondeterminism. ## See It Run diff --git a/s21_workflow_runtime/README.zh.md b/s21_workflow_runtime/README.zh.md index 9a58ab6e..3983a45c 100644 --- a/s21_workflow_runtime/README.zh.md +++ b/s21_workflow_runtime/README.zh.md @@ -8,9 +8,7 @@ s01 → ... → s19 → s20 → `s21` → [s22](../s22_goal_loop/) > > **Harness 层**: 编排 — 在单 agent 循环之上，加一层确定的多 agent 脚本运行时。 -> **来源边界：** 本章产品细节来自对 Claude Code 2.1.177 的 clean-room 行为重建。后续版本可能更改名称与限制；`code.py` 是离线教学模型，不是产品源码复制。 -> -> 教学 CLI 会先发出 `async_launched`，随后在同一进程等待完成，以保证输出可复现。它演示的是生命周期与 journal，不是并发运行的主循环。 +`code.py` 为了让演示保持确定，会先发出 `async_launched`，随后在同一进程里等待执行完成。这样不用启动常驻后台服务，也能看清生命周期和 journal。 --- @@ -26,9 +24,9 @@ s01 → ... → s19 → s20 → `s21` → [s22](../s22_goal_loop/) ## 计划写在代码里，不是靠聊天一轮轮凑 -Claude Code 在工具池里放了一个 `Workflow` 工具。你（或者模型在高强度模式下触发）给它一段脚本，脚本用 `agent() / parallel() / pipeline() / phase()` 这几个简单的原语，把编排写成确定的代码。 +在 harness 的工具池里加入一个 `Workflow` 工具。用户或模型给它一段脚本，脚本用 `agent() / parallel() / pipeline() / phase()` 这几个简单的原语，把编排写成确定的代码。 -主循环这边只看到一次 `tool_use`，立刻拿到"已在后台启动"的返回：真正的执行在后台运行时里推进，实时上报进度，所有过程都写到磁盘的 journal 文件里。脚本里的中间结果存在变量里，不会塞进对话历史占地方。下次用 `resumeFromRunId` 重启时，没改过的 `agent()` 直接命中 journal 缓存，直接用之前的结果，断点续跑。 +主循环这边只看到一次 `tool_use`，立刻拿到"已在后台启动"的返回：真正的执行在后台运行时里推进，实时上报进度，所有过程都写到磁盘的 journal 文件里。脚本里的中间结果存在变量里，不会塞进对话历史占地方。下次用 `resume_from_run_id` 重启时，没改过的 `agent()` 直接命中 journal 缓存，直接用之前的结果，断点续跑。 ![Workflow Runtime 总览](images/workflow-runtime-overview.svg) @@ -45,7 +43,7 @@ async def sample_workflow(ctx, args): ## Workflow 工具：后台启动，主循环只看到一次调用 -`Workflow`（别名 `RunWorkflow`）就在主 agent 的工具池里。触发可能来自你显式说"跑一下这个 workflow"、一个保存好的 `/命令`，或者模型自动进入高强度路径，这时候模型会发一个 `Workflow(...)` 的工具调用。 +`Workflow` 就在主 agent 的工具池里。用户可以要求运行一个保存好的 workflow，模型也可以在任务匹配已知编排时选择这个工具；两种情况最终都只发出一次 `Workflow(...)` 工具调用。 工具收到后会解析参数、校验 meta 信息、过权限检查、注册一个本地 workflow 任务，然后立刻返回"已异步启动"。主循环不阻塞，该干嘛干嘛；workflow 自己在后台跑。这其实就是 s13 后台任务那套"凭条模式"的放大版：先给你个取件条，结果好了再通知你。 @@ -60,11 +58,9 @@ class WorkflowTool: ... # 剩下的后台慢慢跑 ``` -> 真实 Claude Code：工具会立刻返回 `{status:'async_launched', taskId, taskType:'local_workflow', runId, summary, transcriptDir, scriptPath}`，后台任务跑完了再通知。 +## Workflow 元数据：启动前先校验 -## 脚本和 meta：第一行必须写对 - -脚本的第一行必须是 `export const meta = { name, description, phases }`，而且必须是纯字面量，不能有变量、函数调用、字符串拼接。运行时在执行任何代码之前先解析它：`name` 和 `description` 用来显示任务和 UI，`phases` 给进度条分组命名。 +每个 workflow 都要注册一个元数据对象，包含 `name`、`description` 和可选的 `phases`。运行时会在执行任何 workflow 代码之前校验它：`name` 和 `description` 用来标识任务，`phases` 给进度条分组命名。 不对的输入直接抛 `WorkflowInputError`，注册的时候就拦住——这和 s14 校验 cron 表达式是一个思路：坏脚本别让它跑到执行的时候才炸。 @@ -86,8 +82,6 @@ def validate_meta(meta): return meta ``` -> 真实 Claude Code：`parseWorkflowScript` 强制 meta 必须是第一行且是纯字面量；教学版直接收一个 dict，简化了这部分。 - ## 编排原语：就这几个，够写所有流程 脚本跑在一个独立的上下文里，能用的全局变量就这几个编排原语。脚本本身不直接读写文件、不跑 shell，真正的代码操作都由派出去的子 agent 用它们自己的工具权限完成。这些原语都是 `ExecutionState` 上的方法： @@ -113,8 +107,6 @@ async def pipeline(self, items, *stages): return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)]) ``` -> 真实 Claude Code：同名原语由 VM 注入脚本上下文；还提供 `args`、`budget`（总预算/已花/剩余）、agent 数量上限（最多 1000 个）、并发信号量这些控制。 - ## 结构化输出：别让子 agent 回来写散文 `agent({schema})` 会强制子 agent 返回一个匹配 schema 的 JSON 对象（内部通过一次结构化输出调用实现），运行时会按 schema 校验结果，不对就重试一次。这样下游代码拿到的是规整的对象，不是需要再解析的一大段散文。 @@ -132,8 +124,6 @@ if schema is not None: raise WorkflowInputError(f"agent({{schema}}) 输出不合法: {err}") ``` -> 真实 Claude Code：用 `SimpleJsonSchema` + `StructuredOutput` 工具 + schema 重试机制保证输出格式。 - ## 后台任务和进度事件 `LocalWorkflowTask` 维护状态和 token 用量，向外发一条 SDK 风格的事件流：`task_started` → 一串 `task_progress`（包含阶段切换、子 agent 启动、日志输出这些批次）→ 最后一个 `task_notification`（完成/失败/停止，带输出文件、token 数、工具调用数、耗时）。 @@ -147,11 +137,9 @@ class LocalWorkflowTask: print(f" 进度 {ptype} ...") ``` -> 真实 Claude Code：进度会折叠进任务状态，作为 `task_progress.workflow_progress` 发给 UI 和 SDK。 - ## 存储：快照 + journal，断了能续 -跑完会写五样东西，都存在 `~/.claude/projects/<项目>/<会话>/` 目录下：快照 `.json`、输出 `.output.json`、journal `.journal.jsonl`、脚本副本 `scripts/.js`、子 agent 的对话记录 `subagents/workflows//`。你自己保存的常用 workflow 放在 `.claude/workflows/`（项目级）或 `~/.claude/workflows/`（用户级）。 +这个最小运行时把每次运行的数据存在 `s21_workflow_runtime/.runtime/`：快照 `.json`、输出 `.output.json` 和 journal `.journal.jsonl`。生产级 harness 还可以保存 workflow 脚本与子 agent 对话记录，但关键约束是快照和 journal 必须共享稳定的 `runId`。 journal 是断点续跑的核心，它一条一条记下来每个 `agent()` 的结果： @@ -165,7 +153,7 @@ class WorkflowJournal: ## resume：用 runId 续跑，没改的直接用缓存 -调用 `Workflow({scriptPath, resumeFromRunId, args})` 会重新跑脚本，但每个 `agent()` 会算一个确定的语义 key：key 在 journal 里有记录，就直接返回缓存的结果（不重跑），没改过的全部命中缓存；只有改过的那个以及它后面的步骤才会真的跑。 +带着 `resume_from_run_id` 再次调用 workflow 时，脚本会重新执行，但每个 `agent()` 都会计算一个确定的语义 key：key 在 journal 里有记录，就直接返回缓存结果；只有改过的调用以及依赖它的后续步骤才会真的运行。 这里有个关键点：key 不能依赖并发顺序。`parallel` 和 `pipeline` 里 agent 完成的顺序是不确定的，用"第几个完成"当 key，两次跑缓存就对错位了。所以 key 是根据调用内容（类型、标签、prompt、schema）算的稳定哈希，不是一个会竞争的计数器： @@ -181,11 +169,9 @@ if cached is not MISS: return cached ``` -> 真实 Claude Code：同样是"确定语义 key + journal 缓存"的思路；同会话内续跑时，已经完成的 `agent()` 直接返回缓存，后面的才实跑。 - ## 确定性：能复现，续跑才有意义 -续跑要能工作，脚本首先得可复现。所以运行时会把 `Date.now()`、无参 `new Date()`、`Math.random()` 这些不确定的东西从脚本上下文里去掉，也不给 Node 原生 API。同一份脚本 + 同样的参数 → 同样的 key → 100% 缓存命中。教学版用稳定哈希算 key 达到同样的效果（真实版是把整段 JS 脚本跑在去掉了这些不确定源的沙箱 VM 里）。 +续跑要能工作，workflow 首先得可复现。这个最小 Python 运行时使用稳定哈希和确定性的 mock runner，让同一份 workflow + 同样的参数产生同样的 key。生产级 harness 还应该隔离 workflow 代码，并移除不受控的时钟、随机数、文件系统访问等不确定来源。 ## 跑起来看看 diff --git a/s21_workflow_runtime/code.py b/s21_workflow_runtime/code.py index 108b54b2..e1e8a838 100644 --- a/s21_workflow_runtime/code.py +++ b/s21_workflow_runtime/code.py @@ -1,9 +1,5 @@ """ -s21_workflow_runtime — Dynamic Workflow runtime (teaching version) - -Clean-room behavioral reconstruction of Claude Code's `Workflow` tool / dynamic -workflow runtime. Grounded in @anthropic-ai/claude-code@2.1.177 observed -behavior (reverse-research/cc_workflow), NOT leaked source. +s21_workflow_runtime — minimal dynamic Workflow runtime for a teaching harness Idea: s01-s20 build a single, model-driven agent loop. s21 adds a deterministic @@ -16,14 +12,13 @@ Run: python code.py # run the sample workflow, print the event stream python code.py resume # resume the last run; unchanged agent() calls hit cache -Teaching simplifications (vs real runtime.mjs): +Implementation choices: - The "subagent" is a deterministic MockAgentRunner, not a real LLM. - - A workflow is a plain async Python function, not a sandboxed JS script - string. The real runtime runs the script in an isolated JS VM with - Date.now()/Math.random() removed so resume is reproducible. + - A workflow is a plain async Python function. A production harness may use a + declarative format or run user-authored scripts in an isolated VM. - The CLI emits `async_launched` and then awaits completion so the demo stays - deterministic. The real tool returns while execution continues in background. - - Storage is a local .runtime/ dir instead of ~/.claude/projects/.../workflows/. + deterministic. A long-running host can return while execution continues. + - Storage is a local .runtime/ directory beside this file. """ import asyncio @@ -33,7 +28,7 @@ import re import sys from pathlib import Path -# ---- knobs that mirror the real runtime's guards ---- +# ---- runtime guards ---- AGENT_CAP = 1000 # hard cap on agent() calls per run CONCURRENCY = 8 # parallelism cap (semaphore) STORE = Path(__file__).parent / ".runtime" # snapshots + journals live here @@ -50,7 +45,7 @@ def _stable_hash(s: str) -> int: def create_run_id(meta) -> str: # Deterministic in the teaching version so the journal path is predictable - # and `resume` lands on the same file. The real runtime mints a random id. + # and `resume` lands on the same file. return f"wf_{meta['name']}_{_stable_hash(meta['name']) % 10000:04d}" @@ -68,15 +63,14 @@ def validate_run_id(run_id): # Errors # ============================================================ class WorkflowInputError(Exception): - """Bad script / meta / schema input (mirrors WorkflowInputError).""" + """Bad workflow, metadata, or schema input.""" # ============================================================ # meta validation # ============================================================ def validate_meta(meta): - """Real runtime requires `export const meta = {...}` as the FIRST statement, - a pure literal, with name + description (+ optional phases). We take a dict.""" + """Validate name, description, and optional phases before launch.""" if not isinstance(meta, dict): raise WorkflowInputError("meta must be an object literal") if not meta.get("name") or not meta.get("description"): @@ -251,7 +245,7 @@ class WorkflowJournal: # ============================================================ class Budget: """budget.total / spent() / remaining(). Once spent reaches total, agent() - calls raise (the real runtime enforces the same ceiling).""" + calls raise instead of silently overspending.""" def __init__(self, total=None): self.total = total @@ -313,8 +307,7 @@ class ExecutionLimits: class ExecutionState: - """Injected into the workflow script. Provides the orchestration primitives. - Mirrors ExecutionState in runtime.mjs.""" + """Injected into the workflow script with the orchestration primitives.""" def __init__(self, task, journal, runner, budget, args, depth=0, limits=None): self.task = task @@ -415,9 +408,8 @@ class ExecutionState: # ============================================================ class WorkflowTool: """The Workflow tool. .call() validates meta, runs the permission check, - creates runId/taskId, registers a LocalWorkflowTask, and emits the same - lifecycle while this teaching CLI awaits the final result. Supports - resumeFromRunId. Mirrors WorkflowTool.call in runtime.mjs.""" + creates runId/taskId, registers a LocalWorkflowTask, and emits lifecycle + events while this teaching CLI awaits the final result. Supports resume.""" async def call(self, meta, script_fn, args=None, resume_from_run_id=None): validate_meta(meta) @@ -479,7 +471,6 @@ def _read_last_run(): # ============================================================ # Sample workflow: review changed code across dimensions, verify each finding. -# Mirrors cc_workflow/runtime/workflows/review_workflow.js (pipeline + parallel). # ============================================================ FINDINGS_SCHEMA = { "type": "object", "required": ["findings"], @@ -532,7 +523,7 @@ async def sample_workflow(ctx, args): return {"confirmed": confirmed} -# saved workflow registry (.claude/workflows/ analogue) +# Saved workflow registry WORKFLOWS = {SAMPLE_META["name"]: (SAMPLE_META, sample_workflow)} diff --git a/s21_workflow_runtime/images/workflow-runtime-overview.svg b/s21_workflow_runtime/images/workflow-runtime-overview.svg index 281cd2e3..edda0d37 100644 --- a/s21_workflow_runtime/images/workflow-runtime-overview.svg +++ b/s21_workflow_runtime/images/workflow-runtime-overview.svg @@ -40,7 +40,7 @@ Workflow({script, args}) - (or name | scriptPath) · resumeFromRunId + (or name | script) · resume_from_run_id @@ -104,7 +104,7 @@ - resumeFromRunId -> cached agent() + resume_from_run_id -> cached agent() diff --git a/s22_goal_loop/README.ja.md b/s22_goal_loop/README.ja.md index 3d5bbef4..1134ce9f 100644 --- a/s22_goal_loop/README.ja.md +++ b/s22_goal_loop/README.ja.md @@ -8,8 +8,6 @@ s01 → ... → s20 → s21 → `s22` > > **Harness 層**: Goal closure — turn 終端に program-controlled completion gate を追加します。 -> **情報源の境界：** この章の製品詳細は Claude Code 2.1.177 の clean-room 行動再構成に基づく。後続リリースで名称や制限は変わり得る。`code.py` はオフライン教材モデルであり、製品ソースの複製ではない。 - --- s01 から s21 まで、会話の 1 turn はどう終わったでしょうか。モデルが `tool_use` を出さなくなると、loop はそのまま `return` しました。one-shot task なら問題ありません。終わったら止まります。 @@ -20,7 +18,7 @@ s01 から s21 まで、会話の 1 turn はどう終わったでしょうか。 ## /goal: 各 turn の終端に gate を追加する -`/goal ` を入力すると session-scoped stopping condition を設定します。program は active goal として保存し、各 turn の後に独立した lightweight model を evaluator として使い、transcript 内の trusted evidence が condition を満たすか確認します。不足なら gate が停止を拒み、次ラウンドへ「作業を続ける」prompt を queue します。十分なら goal を消して complete とします。 +`/goal ` を入力すると session-scoped stopping condition を設定します。program は active goal として保存し、各 turn の後に evaluator が transcript 内の trusted evidence を condition と照合します。不足なら gate が停止を拒み、次ラウンドへ「作業を続ける」prompt を queue します。十分なら goal を消して complete とします。 ![Goal Loop Overview](images/goal-loop-overview.svg) @@ -40,8 +38,6 @@ if not has_tool_use(response): この gate を制御するのは program です。モデルが自分を律しているのではありません。モデルは gate の存在すら知らず、次のラウンドの入力を受け取って作業を続けるだけです。 -> 実際の Claude Code では `/goal` は session-scoped Stop hook で、workspace trust と hook restriction の管理下にあります。コードには `active_goal`、`goal_status`、`goal_met`、`tengu_goal_achieved` などの marker があります。 - ## Goal の設定: Evidence は command の後から数える `set_goal` は active goal として、goal text、最大 turn budget、counter、そして evidence window の開始点 `start_index` を保存します。現在の transcript length を使うため、`/goal` command 自身は window の外です。これが最初の防御です。command が自分自身の完了を証明することはできません。 @@ -55,8 +51,6 @@ def set_goal(self, objective, max_turns=20): } ``` -> 実際の Claude Code では `GoalRuntime.setGoal()` が active goal、開始位置、counter、budget を保存し、submit 後に `resetEvidenceStart()` で window を command 後へそろえます。 - ## Evaluator: 実在する evidence だけを信頼する ここが仕組み全体の core です。evaluator は会話全体を見ず、evidence window 内で trusted source から来た message だけを見ます。3 層の filter が、「完了したと言ったから完了」という内容をすべて外へ止めます。 @@ -79,9 +73,7 @@ def evidence_text(self): 効果は明確です。同じ `tests passed` でも、あなたが入力したものは数えず、background task notification が持ち帰ったものだけを数えます。モデルは「完了した」と自分で言うだけでは goal を complete にできません。これはコース全体に繰り返し現れた trust boundary の最後の登場です。s16 は protocol が理解ではなく field に依存すると言い、s19 は annotation が申告であり、申告は嘘をつけると言い、s22 は completion evidence を content ではなく origin で信頼します。 -教材版の `goal_satisfied()` は決定的な keyword matching です。実際の版は evidence window を別の lightweight model へ渡して判定します。 - -> 実際の Claude Code の evaluator は作業モデルとは別の lightweight model で、`evaluatorModel`、`default small fast model` と記されています。任意の text を信じず、会話内の evidence を判断します。 +最小版の `goal_satisfied()` は決定的な keyword matching を使い、demo を offline かつ再現可能に保ちます。production harness では、この policy を独立した lightweight evaluator model に置き換えられますが、trusted evidence boundary はそのまま維持します。 ## Gate の 3 状態: Completed / continuing / budget 超過 @@ -106,8 +98,6 @@ def evaluate_after_turn(self): continuation prompt には、わざわざ自身を evidence にしないよう書き、filter でも除外します。これで false positive を防ぐ 3 層がそろいます。command text、reminder text、ordinary conversation のいずれも数えません。budget は s11 の古い規則に従います。automatic retry mechanism には必ず上限が必要です。そうでなければ、永遠に satisfied にならない goal が費用を燃やし続けます。 -> 実際の Claude Code の `evaluateAfterTurn` は `goal_evaluated` event を出し、結果に応じて complete、continuation queue、gate の解除を行います。default budget は 20 turn です。 - ## Continuation prompt と外部 asynchronous message を分ける continuation prompt は同じ `CommandQueue` に入りますが、task completion notification や monitor line といった外部 asynchronous event とは別の方法で消費します。`dequeue` には switch があり、外部 inbox を消費するときは goal continuation を既定で skip します。 @@ -121,9 +111,7 @@ def dequeue(self, include_goal_continuations=True): return None ``` -なぜ分けるのでしょう。実際の model test では、モデルが continuation prompt を外部 notification と一緒に消費し、background evidence が到着する前に goal を complete と判定する bug が起きました。分離後は goal の進行が明示的な 1 step になり、asynchronous event に偶然運ばれません。 - -> 実際の Claude Code の `drainCommandQueue` は既定で `includeGoalContinuations=false` とし、goal continuation の消費を外部 asynchronous inbox から分けます。 +なぜ分けるのでしょう。同じ consumer が continuation prompt と外部 notification を一緒に取り出すと、background result が届く前に reminder text を新しい evidence と誤認する可能性があります。分離後は goal の進行が明示的な 1 step になり、asynchronous event に偶然運ばれません。 ## 実際に動かす diff --git a/s22_goal_loop/README.md b/s22_goal_loop/README.md index abd55e05..85cbd5f0 100644 --- a/s22_goal_loop/README.md +++ b/s22_goal_loop/README.md @@ -8,8 +8,6 @@ s01 → ... → s20 → s21 → `s22` > > **Harness layer**: Goal closure — a program-controlled completion gate at the end of each turn. -> **Source boundary:** Product details in this chapter are a clean-room behavioral reconstruction of Claude Code 2.1.177. Names and limits may change in later releases; `code.py` is an offline teaching model, not copied product source. - --- From s01 through s21, how does a conversation turn end? When the model stops emitting `tool_use`, the loop simply executes `return`. That is fine for one-shot work: finish and stop. @@ -20,7 +18,7 @@ This thread was present from the first chapter. s01 explained that exiting the l ## /goal: Add a Gate at the End of Every Turn -Entering `/goal ` sets a session-scoped stopping condition. The program stores it as the active goal. After each turn, an independent lightweight model acts as evaluator and checks whether trusted evidence in the transcript satisfies the condition. If evidence is insufficient, the gate blocks the attempted stop and queues a "keep working" prompt for the next round. If it is sufficient, the goal is cleared and marked complete. +Entering `/goal ` sets a session-scoped stopping condition. The program stores it as the active goal. After each turn, an evaluator checks whether trusted evidence in the transcript satisfies the condition. If evidence is insufficient, the gate blocks the attempted stop and queues a "keep working" prompt for the next round. If it is sufficient, the goal is cleared and marked complete. ![Goal Loop Overview](images/goal-loop-overview.svg) @@ -40,8 +38,6 @@ if not has_tool_use(response): The program controls this gate. It is not the model restraining itself. The model does not even know the gate exists; it simply receives another round of input and continues working. -> In the real Claude Code, `/goal` is a session-scoped Stop hook governed by workspace trust and hook restrictions. The code contains markers such as `active_goal`, `goal_status`, `goal_met`, and `tengu_goal_achieved`. - ## Setting a Goal: Evidence Starts after the Command `set_goal` stores an active goal containing the objective text, a maximum-turn budget, counters, and `start_index`, the beginning of the evidence window. It uses the transcript's current length, placing the `/goal` command itself outside the window. This is the first defense: a command cannot prove its own completion. @@ -55,8 +51,6 @@ def set_goal(self, objective, max_turns=20): } ``` -> In the real Claude Code, `GoalRuntime.setGoal()` stores the active goal, start position, counters, and budget, then `resetEvidenceStart()` aligns the window to the position after command submission. - ## The Evaluator: Trust Concrete Evidence Only This is the core of the entire mechanism. The evaluator does not inspect the whole conversation. It sees only messages inside the evidence window that come from trusted sources. Three filters keep every form of "I said it was done, so it must be done" outside: @@ -79,9 +73,7 @@ def evidence_text(self): The effect is clear. The same sentence, `tests passed`, does not count when typed by you, but does count when delivered by a background task notification. The model cannot bluff its way out by saying "I finished." This is the final appearance of the trust boundary repeated throughout the course. s16 said protocols rely on fields, not interpretation. s19 said annotations are claims and claims may be false. s22 says completion evidence is trusted by origin, not by content alone. -The teaching version's `goal_satisfied()` uses deterministic keyword matching. The real version asks a separate lightweight model to judge the evidence window. - -> In the real Claude Code, the evaluator is a lightweight model separate from the working model, marked as `evaluatorModel` and the `default small fast model`. It judges evidence in the conversation rather than trusting arbitrary text. +The minimal `goal_satisfied()` uses deterministic keyword matching so the demo stays offline and reproducible. A production harness can replace this policy with a separate lightweight evaluator model, while keeping the same trusted evidence boundary. ## Three Gate States: Completed, Continuing, or Over Budget @@ -106,8 +98,6 @@ def evaluate_after_turn(self): The continuation prompt explicitly says not to treat itself as evidence, and the evidence filter excludes it. That completes the three layers against false positives: the command does not count, the reminder does not count, and ordinary conversation does not count. The budget follows the old rule from s11: every automatic retry mechanism needs a limit. Otherwise, a goal that can never be satisfied becomes a perpetual money-burning machine. -> In the real Claude Code, `evaluateAfterTurn` emits a `goal_evaluated` event and either completes, queues a continuation, or stops blocking. The default budget is 20 turns. - ## Keep Continuation Prompts Separate from External Asynchronous Messages Continuation prompts enter the same `CommandQueue`, but they are not consumed in the same way as external asynchronous events such as task-completion notifications and monitor lines. `dequeue` has a switch, and consumption of the external inbox skips goal continuations by default. @@ -121,9 +111,7 @@ def dequeue(self, include_goal_continuations=True): return None ``` -Why separate them? A real model test exposed a bug where the model consumed the continuation prompt together with an external notification and marked the goal complete before background evidence arrived. With the paths separated, goal progression is an explicit step and cannot be carried along accidentally by asynchronous events. - -> In the real Claude Code, `drainCommandQueue` defaults to `includeGoalContinuations=false`, separating goal-continuation consumption from the external asynchronous inbox. +Why separate them? If one consumer drains continuation prompts together with external notifications, a reminder can be mistaken for new evidence before the background result arrives. With the paths separated, goal progression is an explicit step and cannot be carried along accidentally by asynchronous events. ## See It Run diff --git a/s22_goal_loop/README.zh.md b/s22_goal_loop/README.zh.md index 8fb665dc..95dd2344 100644 --- a/s22_goal_loop/README.zh.md +++ b/s22_goal_loop/README.zh.md @@ -8,8 +8,6 @@ s01 → ... → s20 → s21 → `s22` > > **Harness 层**: 目标闭环 — 在轮次收尾处，加一道程序控制的完成闸门。 -> **来源边界：** 本章产品细节来自对 Claude Code 2.1.177 的 clean-room 行为重建。后续版本可能更改名称与限制；`code.py` 是离线教学模型，不是产品源码复制。 - --- 从 s01 到 s21，一轮对话怎么结束？模型不再发 `tool_use`，循环就直接 `return` 了。一次性任务这么干没问题，做完就停。 @@ -20,7 +18,7 @@ s01 → ... → s20 → s21 → `s22` ## /goal：每轮收尾加一道闸门 -输入 `/goal <条件>` 就设了一个会话级的停止条件。程序把它存成当前活跃目标，每轮结束后，用一个独立的轻量小模型当判断器，看对话记录里的可信证据够不够满足条件。不够，闸门就把这次结束拦住，塞一条"继续干"的提示进下一轮；够了，就清除目标，标记完成。 +输入 `/goal <条件>` 就设了一个会话级的停止条件。程序把它存成当前活跃目标，每轮结束后，判断器检查对话记录里的可信证据够不够满足条件。不够，闸门就把这次结束拦住，塞一条"继续干"的提示进下一轮；够了，就清除目标，标记完成。 ![Goal Loop 总览](images/goal-loop-overview.svg) @@ -40,8 +38,6 @@ if not has_tool_use(response): 这道闸门是程序自己控制的。不是模型自己约束自己，模型甚至不知道有这么一道闸门，它只是收到了下一轮的输入，接着干就是了。 -> 真实 Claude Code：`/goal` 是会话级的 Stop hook，受工作区信任和 hook 限制控制；代码里有 `active_goal`、`goal_status`、`goal_met`、`tengu_goal_achieved` 这些标记。 - ## 设目标：证据从命令之后开始算 `set_goal` 会存一个活跃目标：目标文本、最大轮数预算、计数器，还有 `start_index`——也就是证据窗口的起点。它取当前对话记录的长度，所以 `/goal` 这行命令本身在窗口外面。这是第一道防线：命令自己不能证明自己完成了。 @@ -55,8 +51,6 @@ def set_goal(self, objective, max_turns=20): } ``` -> 真实 Claude Code：`GoalRuntime.setGoal()` 存活跃目标、起始位置、计数器和预算；提交后再 `resetEvidenceStart()` 把窗口对齐到命令之后。 - ## 判断器：只信实打实的证据 这是整个机制最核心的地方。判断器不看整段对话，只看证据窗口里来自可信来源的消息。三层过滤，把"嘴上说完成了但不算数"的内容全挡在外面： @@ -79,9 +73,7 @@ def evidence_text(self): 效果很明显：同样一句 `tests passed`，你打字说的不算，后台任务通知带回来的才算。模型糊弄不过去，它没法靠自己说一句"我做完了"就把目标判成完成。这是全课程反复出现的那条信任边界的最后一次登场：s16 说协议靠字段不靠理解，s19 说注解是申报、申报可以撒谎，s22 说完成证据只看来源不看内容。 -教学版里 `goal_satisfied()` 是确定的关键词匹配；真实版会把证据窗口交给一个轻量小模型来判断。 - -> 真实 Claude Code：判断器是和干活的模型分开的轻量小模型（标记是 `evaluatorModel`、`default small fast model`），判断对话里的证据，不是随便什么文本都信。 +最小版的 `goal_satisfied()` 使用确定的关键词匹配，让演示保持离线和可复现。生产级 harness 可以把这条策略替换成独立的轻量判断模型，但仍然保留相同的可信证据边界。 ## 闸门三态：完成/继续/超预算 @@ -106,8 +98,6 @@ def evaluate_after_turn(self): 那条"继续干"的提示里特意写了"别把这条提醒当成完成证据"，连提醒本身都被排除在证据之外。三层防误判就齐了：命令文本不算、提醒文本不算、普通聊天文本不算。预算则是 s11 教过的老规矩：任何自动重试的机制都得有上限，不然一个永远判不满足的目标就是个烧钱的永动机。 -> 真实 Claude Code：`evaluateAfterTurn` 会发 `goal_evaluated` 事件，按结果完成/塞继续提示/拦截；默认预算是 20 轮。 - ## 继续提示和外部异步消息分开走 继续提示进的是同一个 `CommandQueue`，但它和外部异步事件（任务完成通知、监控行）不是同一种消费方式。`dequeue` 带个开关：消费外部收件箱的时候，默认跳过目标的继续提示。 @@ -121,9 +111,7 @@ def dequeue(self, include_goal_continuations=True): return None ``` -为什么要分开？真实模型测试的时候出过一个 bug：模型把继续提示当成外部通知一起消费了，结果后台证据还没到，就提前把目标判成完成了。分开之后，目标的推进是显式的一步，不会被异步事件带着走。 - -> 真实 Claude Code：`drainCommandQueue` 默认 `includeGoalContinuations=false`，把目标继续提示和外部异步收件箱的消费分开。 +为什么要分开？如果同一个消费者把继续提示和外部通知一起取走，后台结果还没到，提醒文本就可能被误当成新证据。分开之后，目标的推进是显式的一步，不会被异步事件带着走。 ## 跑起来看看 diff --git a/s22_goal_loop/code.py b/s22_goal_loop/code.py index f0eadfe0..9ae05f9a 100644 --- a/s22_goal_loop/code.py +++ b/s22_goal_loop/code.py @@ -1,9 +1,5 @@ """ -s22_goal_loop — /goal session goal loop (teaching version) - -Clean-room behavioral reconstruction of Claude Code's `/goal` command. Grounded -in @anthropic-ai/claude-code@2.1.177 observed behavior -(reverse-research/cc_goal_loop), NOT leaked source. +s22_goal_loop — minimal /goal session loop for a teaching harness Idea: s01-s21 end a turn when the model emits no tool_use. `/goal` adds a @@ -27,12 +23,12 @@ Idea: Run: python code.py # /goal until tests pass + deploy green; watch the gate -Teaching simplifications (vs real /goal and runtime.mjs): +Implementation choices: - The evaluator is a deterministic keyword check, not a small/fast model. - One mock task-notification produces the trusted evidence; the loop / monitor / background-task plane (s13/s14) is out of scope — this chapter is just the goal gate. - - The evidence trust boundary is the faithful part: only task-notification / + - The evidence trust boundary is the important part: only task-notification / monitor-line origins count as evidence, so the `/goal` command text, the continuation reminder, and plain assistant prose can NOT satisfy the goal. Ordinary `submit()` calls cannot set those labels; only the host-event @@ -68,7 +64,7 @@ class Message: # ============================================================ -# CommandQueue — continuation prompts live here (mirrors CommandQueue) +# CommandQueue — continuation prompts live here # ============================================================ class CommandQueue: PRIORITY = {"now": 0, "next": 1, "later": 2} @@ -102,7 +98,7 @@ class CommandQueue: # ============================================================ -# GoalRuntime — the turn-completion gate (mirrors GoalRuntime) +# GoalRuntime — the turn-completion gate # ============================================================ class GoalRuntime: def __init__(self, transcript, queue): @@ -148,9 +144,8 @@ class GoalRuntime: return "\n".join(out) def goal_satisfied(self): - # Real Claude Code routes this to a small/fast evaluator model reading - # the evidence window. The teaching version is a deterministic keyword - # check so the lifecycle is reproducible. + # A production harness can route this evidence window to a separate + # evaluator model. The demo uses a deterministic keyword policy. objective = self.active["objective"].lower() evidence = self.evidence_text().lower() wants_tests = "test" in objective @@ -193,7 +188,7 @@ class GoalRuntime: # ============================================================ -# Session — the main loop host with a Stop gate (mirrors submit / drain) +# Session — the main loop host with a Stop gate # ============================================================ class Session: def __init__(self): diff --git a/web/public/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg b/web/public/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg index 281cd2e3..edda0d37 100644 --- a/web/public/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg +++ b/web/public/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg @@ -40,7 +40,7 @@ Workflow({script, args}) - (or name | scriptPath) · resumeFromRunId + (or name | script) · resume_from_run_id @@ -104,7 +104,7 @@ - resumeFromRunId -> cached agent() + resume_from_run_id -> cached agent() diff --git a/web/src/data/generated/docs.json b/web/src/data/generated/docs.json index a1a058a3..157787f3 100644 --- a/web/src/data/generated/docs.json +++ b/web/src/data/generated/docs.json @@ -363,36 +363,36 @@ "version": "s21", "locale": "en", "title": "s21: Workflow Runtime — The Model Decides Each Step; a Script Decides the Orchestration", - "content": "# s21: Workflow Runtime — The Model Decides Each Step; a Script Decides the Orchestration\n\ns01 → ... → s19 → s20 → `s21` → [s22](/en/s22)\n\n> *\"One tool_use starts an entire orchestration in the background\"* — The `Workflow` tool starts a deterministic, recoverable script runtime that dispatches many subagents in bulk.\n>\n> **Harness layer**: Orchestration — a deterministic multi-agent script runtime above the single-agent loop.\n\n> **Source boundary:** Product details in this chapter are a clean-room behavioral reconstruction of Claude Code 2.1.177. Names and limits may change in later releases; `code.py` is an offline teaching model, not copied product source.\n>\n> The teaching CLI emits `async_launched` and then awaits completion in one process for deterministic output. It demonstrates the lifecycle and journal, not a concurrently running main loop.\n\n---\n\nFrom s01 through s20, our loop has always been model-driven and step-by-step: the model chooses one tool each round, its result enters `messages[]`, and another round begins. That is ideal for open-ended tasks because the model can inspect the current context and decide the next step on the spot.\n\nSome jobs, however, require deterministic command of a group of agents. Consider reviewing a large change: inspect ten dimensions in parallel → send each finding to a separate agent for adversarial verification → combine and deduplicate the results → sort by severity. The shape is fixed, and you really need three properties:\n\n- **Parallelism**, rather than waiting for one item at a time;\n- **Determinism**, so the same input produces the same result structure;\n- **Recoverability**, so an interruption does not rerun work that is already complete.\n\nMaking the model drive this process one round at a time in the main loop is slow and nondeterministic, and an interruption starts everything over. At that point, you do not need \"one more conversation turn.\" You need to encode the orchestration directly as code.\n\n## Put the Plan in Code, Not in a Sequence of Chat Turns\n\nClaude Code includes a `Workflow` tool in its tool pool. You, or the model when it enters a high-intensity mode, provide a script that expresses deterministic orchestration through a few simple primitives: `agent()`, `parallel()`, `pipeline()`, and `phase()`.\n\nThe main loop sees only one `tool_use` and immediately receives a \"started in the background\" result. Real execution continues inside the background runtime, which reports progress in real time and records every step in a journal on disk. Intermediate script results live in variables instead of taking space in conversation history. When restarted with `resumeFromRunId`, unchanged `agent()` calls hit the journal cache and reuse previous results, resuming from the checkpoint.\n\n![Workflow Runtime Overview](/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg)\n\n```python\nSAMPLE_META = {\"name\": \"review-changes\", \"description\": \"Review code changes\", \"phases\": [\"Review\", \"Verify\"]}\n\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n results = await ctx.pipeline(DIMENSIONS, audit, verify) # Each dimension independently runs audit → verify\n confirmed = [f for r in results if r for f in r[\"confirmed\"]]\n ctx.log(f\"Confirmed {len(confirmed)} real issues\")\n return {\"confirmed\": confirmed}\n```\n\n## The Workflow Tool: Start in the Background; the Main Loop Sees One Call\n\n`Workflow`, also known as `RunWorkflow`, lives in the main agent's tool pool. You may explicitly ask to \"run this workflow,\" invoke a saved `/command`, or let the model enter a high-intensity path automatically. In each case, the model emits a `Workflow(...)` tool call.\n\nThe tool parses the arguments, validates metadata, checks permissions, registers a local workflow task, and immediately returns \"started asynchronously.\" The main loop does not block and can continue with other work while the workflow runs in the background. This is the claim-ticket pattern from s13 at a larger scale: hand over the ticket now, notify the user when the result is ready.\n\n```python\nclass WorkflowTool:\n async def call(self, meta, script_fn, args=None, resume_from_run_id=None):\n validate_meta(meta)\n check_permission(meta)\n run_id = resume_from_run_id or create_run_id(meta)\n task = LocalWorkflowTask(create_task_id(run_id), run_id, meta)\n task.event(\"async_launched\", runId=run_id, taskId=task.task_id) # Return immediately\n ... # The rest proceeds in the background\n```\n\n> The real Claude Code immediately returns `{status:'async_launched', taskId, taskType:'local_workflow', runId, summary, transcriptDir, scriptPath}`, then sends a notification when the background task finishes.\n\n## Script and Meta: The First Line Must Be Correct\n\nThe script's first line must be `export const meta = { name, description, phases }`, and it must contain only literals: no variables, function calls, or string concatenation. The runtime parses it before executing any code. `name` and `description` identify the task in the UI, while `phases` names groups in the progress display.\n\nInvalid input raises `WorkflowInputError` immediately and is rejected during registration. This is the same idea as validating cron expressions in s14: do not wait until execution to discover a bad script.\n\nBecause the teaching runtime uses `meta.name` in local artifact filenames, it also requires a 1-64 character safe slug containing letters, numbers, `.`, `_`, or `-`.\n\n```python\ndef validate_meta(meta):\n if not isinstance(meta, dict):\n raise WorkflowInputError(\"meta must be an object literal\")\n if not meta.get(\"name\") or not meta.get(\"description\"):\n raise WorkflowInputError(\"meta requires name and description\")\n if not isinstance(meta[\"name\"], str) or not WORKFLOW_NAME_RE.fullmatch(meta[\"name\"]):\n raise WorkflowInputError(\"meta.name must be a safe 1-64 character slug\")\n if \"phases\" in meta and (\n not isinstance(meta[\"phases\"], list)\n or not all(isinstance(p, str) and p for p in meta[\"phases\"])\n ):\n raise WorkflowInputError(\"meta.phases must contain non-empty strings\")\n return meta\n```\n\n> The real Claude Code's `parseWorkflowScript` requires meta to be the first line and a pure literal. The teaching version accepts a dict directly to simplify this part.\n\n## Orchestration Primitives: A Small Set Is Enough for Every Flow\n\nA script runs in an isolated context with only a small set of orchestration primitives as globals. The script does not read files or run shell commands directly. All real code operations are performed by dispatched subagents under their own tool permissions. These primitives are methods on `ExecutionState`:\n\n| Primitive | Purpose |\n|------|------|\n| `agent(prompt, {schema, label, phase})` | Dispatch one subagent |\n| `parallel(thunks)` | **Barrier**: run every task concurrently and wait until all results return |\n| `pipeline(items, *stages)` | Run each item through stages **without a barrier**; finished items proceed immediately |\n| `phase(title)` | Mark the current progress phase and update the progress display |\n| `log(message)` | Emit a progress log line |\n| `workflow(name, args)` | Run a nested sub-workflow, one level only |\n\n`pipeline` should be the default. Each item independently crosses every stage. Item A may reach stage three while item B is still in stage one. Use the `parallel` barrier only when the next stage truly requires every result from the previous stage. A barrier waits for the slowest task, so do not add one without need.\n\n```python\nasync def pipeline(self, items, *stages):\n async def run_item(item, idx):\n value = item\n for stage in stages: # Each item independently completes every stage\n value = await stage(value, item, idx)\n return value\n return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])\n```\n\n> The real Claude Code injects same-named primitives into the script VM. It also exposes `args`, `budget` with total/spent/remaining values, an agent limit of up to 1000, and a concurrency semaphore.\n\n## Structured Output: Do Not Let Subagents Return Essays\n\n`agent({schema})` requires a subagent to return a JSON object matching the schema, internally through one structured-output call. The runtime validates the result and retries once if it does not match. Downstream code receives a regular object instead of a long essay that must be parsed again.\n\ns05 warned that tool arguments cannot be trusted completely. This is the same lesson in reverse: subagent output cannot be trusted completely either. Validate at the orchestration boundary, give one retry, and keep uncertainty out of the rest of the flow.\n\n```python\nresult = self.runner.run(prompt, schema, label)\nif schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok: # Retry once with a reminder, then fail\n result = self.runner.run(prompt + \"\\n\\nReturn valid JSON.\", schema, label)\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok:\n raise WorkflowInputError(f\"agent({{schema}}) returned invalid output: {err}\")\n```\n\n> The real Claude Code combines `SimpleJsonSchema`, a `StructuredOutput` tool, and schema-aware retries to enforce the output format.\n\n## Background Tasks and Progress Events\n\n`LocalWorkflowTask` maintains status and token usage and emits an SDK-style event stream: `task_started` → a sequence of `task_progress` events containing phase changes, subagent starts, and log batches → one final `task_notification` reporting completion, failure, or stop, plus output files, token count, tool calls, and elapsed time.\n\nThe main session treats these as ordinary events. Only the final completion notification re-enters the main loop.\n\n```python\nclass LocalWorkflowTask:\n def progress_event(self, ptype, **data): # Phase/subagent/log\n self.progress.append({\"type\": ptype, **data})\n print(f\" progress {ptype} ...\")\n```\n\n> The real Claude Code folds progress into task state and sends it to the UI and SDK as `task_progress.workflow_progress`.\n\n## Storage: Snapshot + Journal for Resuming after Interruptions\n\nEach run writes five artifacts under `~/.claude/projects///`: a `.json` snapshot, `.output.json` output, `.journal.jsonl` journal, a `scripts/.js` script copy, and subagent transcripts under `subagents/workflows//`. Reusable workflows that you save live in `.claude/workflows/` at project scope or `~/.claude/workflows/` at user scope.\n\nThe journal is the core of checkpointed resume. It records every `agent()` result one line at a time:\n\n```python\nclass WorkflowJournal:\n def record(self, key, value):\n self._f.write(json.dumps({\"key\": key, \"value\": value}) + \"\\n\")\n self._f.flush()\n self.cache[key] = value\n```\n\n## Resume: Continue by runId and Reuse Everything Unchanged\n\nCalling `Workflow({scriptPath, resumeFromRunId, args})` reruns the script, but every `agent()` computes a deterministic semantic key. If that key is present in the journal, it returns the cached result without executing again. Every unchanged call hits the cache; only a changed call and the downstream steps that depend on it actually rerun.\n\nThe key detail is that keys cannot depend on concurrency order. Agents in `parallel` and `pipeline` finish in nondeterministic order. If \"the nth completion\" became the key, cache entries would map to the wrong calls on the next run. A key therefore uses a stable hash of call content, including type, label, prompt, and schema, rather than a shared counter:\n\n```python\ndef key(self, kind, label, prompt, schema):\n basis = f\"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}\"\n return f\"{kind}-{_stable_hash(basis) % 10**10:010d}\"\n\n# Inside agent():\ncached = self.journal.cached(key)\nif cached is not MISS:\n self.task.progress_event(\"workflow_agent\", label=label, status=\"cached\")\n return cached\n```\n\n> The real Claude Code uses the same idea: deterministic semantic keys plus a journal cache. Resuming within the same session returns cached results for completed `agent()` calls and runs only the remaining ones.\n\n## Determinism: Reproducibility Makes Resume Meaningful\n\nResume works only if the script is reproducible. The runtime therefore removes nondeterministic sources such as `Date.now()`, no-argument `new Date()`, and `Math.random()` from the script context, and does not expose native Node APIs. The same script plus the same arguments produces the same keys and a 100% cache hit. The teaching version obtains the same property through stable key hashing; the real version runs the entire JavaScript inside a sandboxed VM with those sources removed.\n\n## See It Run\n\nThe sample `review-changes` workflow uses `pipeline` to send each review dimension independently through audit → verify. An `agent()` with a schema finds issues during audit. During verification, `parallel()` dispatches a separate adversarial subagent for every finding. Only confirmed issues remain, sorted by severity.\n\n```python\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n\n async def audit(_v, dimension, _i):\n out = await ctx.agent(f\"Inspect the changed code for {dimension} issues\",\n schema=FINDINGS_SCHEMA, label=f\"audit:{dimension}\", phase=\"Review\")\n return {\"dimension\": dimension, \"findings\": out[\"findings\"]}\n\n async def verify(audited, dimension, _i):\n ctx.phase(\"Verify\")\n verdicts = await ctx.parallel([ # Verify every finding independently\n (lambda f=f: ctx.agent(f\"Adversarially verify whether this issue is real: {f['title']}\",\n schema=VERDICT_SCHEMA, label=f\"verify:{dimension}:{f['title']}\"))\n for f in audited[\"findings\"]])\n return {\"dimension\": dimension,\n \"confirmed\": [f for f, v in zip(audited[\"findings\"], verdicts) if v and v[\"isReal\"]]}\n\n results = await ctx.pipeline(DIMENSIONS, audit, verify)\n ...\n```\n\n## Changes from s20\n\n| | s20 Comprehensive Agent | s21 Workflow Runtime |\n|--|-----------|---------------------|\n| Loop | One model-driven loop | Main loop unchanged; deterministic orchestration added above it |\n| Who decides the next step | Model decides each round | Script declares the orchestration in advance |\n| Multiple agents | One-shot s06 subagents | Scripted, reproducible, recoverable bulk orchestration |\n| New mechanisms | — | Script DSL, background tasks, progress events, journal/resume, structured output, deterministic VM |\n\ns21 does not replace the main loop. It exposes `Workflow` at the tool layer and starts a local workflow runtime behind it: one workflow deterministically drives N agent loops. An s06 subagent is dispatched once at the model's discretion; s21 turns orchestration into a replayable script.\n\n## Try It\n\n```bash\npython s21_workflow_runtime/code.py # Start review-changes and watch the event stream\npython s21_workflow_runtime/code.py resume # Resume by the last runId; every agent() hits the journal cache\n```\n\nWatch one launch produce `async_launched`, followed by background phase changes and subagent progress, then `task_notification`; the result is stored on the task object. A resumed run reports `agents=0 tokens=0` because every call hits the cache, and its result is byte-for-byte identical.\n\n## Next\n\nOrchestration adds a layer above agent capabilities: the main loop handles individual operations, while a script manages the whole team's flow. Once work becomes a deterministic, recoverable script, the model changes from the round-by-round driver into an execution unit scheduled by that script. The same `agent()` can be invoked ad hoc by the model in the main loop or orchestrated in bulk inside a workflow.\n\nNext: [s22 Goal Loop](/en/s22) — Orchestration fans work out and leaves the main loop. The next chapter moves in the opposite direction: a goal pulls control back into the main loop and refuses to let the turn end until the objective is achieved.\n\n\n" + "content": "# s21: Workflow Runtime — The Model Decides Each Step; a Script Decides the Orchestration\n\ns01 → ... → s19 → s20 → `s21` → [s22](/en/s22)\n\n> *\"One tool_use starts an entire orchestration in the background\"* — The `Workflow` tool starts a deterministic, recoverable script runtime that dispatches many subagents in bulk.\n>\n> **Harness layer**: Orchestration — a deterministic multi-agent script runtime above the single-agent loop.\n\n`code.py` keeps the demo deterministic: it emits `async_launched` and then awaits completion in one process. This demonstrates the lifecycle and journal without requiring a long-running background service.\n\n---\n\nFrom s01 through s20, our loop has always been model-driven and step-by-step: the model chooses one tool each round, its result enters `messages[]`, and another round begins. That is ideal for open-ended tasks because the model can inspect the current context and decide the next step on the spot.\n\nSome jobs, however, require deterministic command of a group of agents. Consider reviewing a large change: inspect ten dimensions in parallel → send each finding to a separate agent for adversarial verification → combine and deduplicate the results → sort by severity. The shape is fixed, and you really need three properties:\n\n- **Parallelism**, rather than waiting for one item at a time;\n- **Determinism**, so the same input produces the same result structure;\n- **Recoverability**, so an interruption does not rerun work that is already complete.\n\nMaking the model drive this process one round at a time in the main loop is slow and nondeterministic, and an interruption starts everything over. At that point, you do not need \"one more conversation turn.\" You need to encode the orchestration directly as code.\n\n## Put the Plan in Code, Not in a Sequence of Chat Turns\n\nAdd a `Workflow` tool to the harness tool pool. The user or model provides a script that expresses deterministic orchestration through a few simple primitives: `agent()`, `parallel()`, `pipeline()`, and `phase()`.\n\nThe main loop sees only one `tool_use` and immediately receives a \"started in the background\" result. Real execution continues inside the background runtime, which reports progress in real time and records every step in a journal on disk. Intermediate script results live in variables instead of taking space in conversation history. When restarted with `resume_from_run_id`, unchanged `agent()` calls hit the journal cache and reuse previous results, resuming from the checkpoint.\n\n![Workflow Runtime Overview](/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg)\n\n```python\nSAMPLE_META = {\"name\": \"review-changes\", \"description\": \"Review code changes\", \"phases\": [\"Review\", \"Verify\"]}\n\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n results = await ctx.pipeline(DIMENSIONS, audit, verify) # Each dimension independently runs audit → verify\n confirmed = [f for r in results if r for f in r[\"confirmed\"]]\n ctx.log(f\"Confirmed {len(confirmed)} real issues\")\n return {\"confirmed\": confirmed}\n```\n\n## The Workflow Tool: Start in the Background; the Main Loop Sees One Call\n\n`Workflow` lives in the main agent's tool pool. The user can request a saved workflow, or the model can select the tool when a task matches a known orchestration. In either case, the model emits one `Workflow(...)` tool call.\n\nThe tool parses the arguments, validates metadata, checks permissions, registers a local workflow task, and immediately returns \"started asynchronously.\" The main loop does not block and can continue with other work while the workflow runs in the background. This is the claim-ticket pattern from s13 at a larger scale: hand over the ticket now, notify the user when the result is ready.\n\n```python\nclass WorkflowTool:\n async def call(self, meta, script_fn, args=None, resume_from_run_id=None):\n validate_meta(meta)\n check_permission(meta)\n run_id = resume_from_run_id or create_run_id(meta)\n task = LocalWorkflowTask(create_task_id(run_id), run_id, meta)\n task.event(\"async_launched\", runId=run_id, taskId=task.task_id) # Return immediately\n ... # The rest proceeds in the background\n```\n\n## Workflow Metadata: Validate Before Launch\n\nEach workflow registers a metadata object with `name`, `description`, and optional `phases`. The runtime validates it before executing any workflow code. `name` and `description` identify the task in the UI, while `phases` names groups in the progress display.\n\nInvalid input raises `WorkflowInputError` immediately and is rejected during registration. This is the same idea as validating cron expressions in s14: do not wait until execution to discover a bad script.\n\nBecause the teaching runtime uses `meta.name` in local artifact filenames, it also requires a 1-64 character safe slug containing letters, numbers, `.`, `_`, or `-`.\n\n```python\ndef validate_meta(meta):\n if not isinstance(meta, dict):\n raise WorkflowInputError(\"meta must be an object literal\")\n if not meta.get(\"name\") or not meta.get(\"description\"):\n raise WorkflowInputError(\"meta requires name and description\")\n if not isinstance(meta[\"name\"], str) or not WORKFLOW_NAME_RE.fullmatch(meta[\"name\"]):\n raise WorkflowInputError(\"meta.name must be a safe 1-64 character slug\")\n if \"phases\" in meta and (\n not isinstance(meta[\"phases\"], list)\n or not all(isinstance(p, str) and p for p in meta[\"phases\"])\n ):\n raise WorkflowInputError(\"meta.phases must contain non-empty strings\")\n return meta\n```\n\n## Orchestration Primitives: A Small Set Is Enough for Every Flow\n\nA script runs in an isolated context with only a small set of orchestration primitives as globals. The script does not read files or run shell commands directly. All real code operations are performed by dispatched subagents under their own tool permissions. These primitives are methods on `ExecutionState`:\n\n| Primitive | Purpose |\n|------|------|\n| `agent(prompt, {schema, label, phase})` | Dispatch one subagent |\n| `parallel(thunks)` | **Barrier**: run every task concurrently and wait until all results return |\n| `pipeline(items, *stages)` | Run each item through stages **without a barrier**; finished items proceed immediately |\n| `phase(title)` | Mark the current progress phase and update the progress display |\n| `log(message)` | Emit a progress log line |\n| `workflow(name, args)` | Run a nested sub-workflow, one level only |\n\n`pipeline` should be the default. Each item independently crosses every stage. Item A may reach stage three while item B is still in stage one. Use the `parallel` barrier only when the next stage truly requires every result from the previous stage. A barrier waits for the slowest task, so do not add one without need.\n\n```python\nasync def pipeline(self, items, *stages):\n async def run_item(item, idx):\n value = item\n for stage in stages: # Each item independently completes every stage\n value = await stage(value, item, idx)\n return value\n return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])\n```\n\n## Structured Output: Do Not Let Subagents Return Essays\n\n`agent({schema})` requires a subagent to return a JSON object matching the schema, internally through one structured-output call. The runtime validates the result and retries once if it does not match. Downstream code receives a regular object instead of a long essay that must be parsed again.\n\ns05 warned that tool arguments cannot be trusted completely. This is the same lesson in reverse: subagent output cannot be trusted completely either. Validate at the orchestration boundary, give one retry, and keep uncertainty out of the rest of the flow.\n\n```python\nresult = self.runner.run(prompt, schema, label)\nif schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok: # Retry once with a reminder, then fail\n result = self.runner.run(prompt + \"\\n\\nReturn valid JSON.\", schema, label)\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok:\n raise WorkflowInputError(f\"agent({{schema}}) returned invalid output: {err}\")\n```\n\n## Background Tasks and Progress Events\n\n`LocalWorkflowTask` maintains status and token usage and emits an SDK-style event stream: `task_started` → a sequence of `task_progress` events containing phase changes, subagent starts, and log batches → one final `task_notification` reporting completion, failure, or stop, plus output files, token count, tool calls, and elapsed time.\n\nThe main session treats these as ordinary events. Only the final completion notification re-enters the main loop.\n\n```python\nclass LocalWorkflowTask:\n def progress_event(self, ptype, **data): # Phase/subagent/log\n self.progress.append({\"type\": ptype, **data})\n print(f\" progress {ptype} ...\")\n```\n\n## Storage: Snapshot + Journal for Resuming after Interruptions\n\nThe minimal runtime stores each run under `s21_workflow_runtime/.runtime/`: a `.json` snapshot, `.output.json` output, and `.journal.jsonl` journal. A production harness may also persist the workflow script and subagent transcripts, but the key requirement is that the snapshot and journal share a stable `runId`.\n\nThe journal is the core of checkpointed resume. It records every `agent()` result one line at a time:\n\n```python\nclass WorkflowJournal:\n def record(self, key, value):\n self._f.write(json.dumps({\"key\": key, \"value\": value}) + \"\\n\")\n self._f.flush()\n self.cache[key] = value\n```\n\n## Resume: Continue by runId and Reuse Everything Unchanged\n\nCalling the workflow again with `resume_from_run_id` reruns the script, but every `agent()` computes a deterministic semantic key. If that key is present in the journal, it returns the cached result without executing again. Every unchanged call hits the cache; only a changed call and the downstream steps that depend on it actually rerun.\n\nThe key detail is that keys cannot depend on concurrency order. Agents in `parallel` and `pipeline` finish in nondeterministic order. If \"the nth completion\" became the key, cache entries would map to the wrong calls on the next run. A key therefore uses a stable hash of call content, including type, label, prompt, and schema, rather than a shared counter:\n\n```python\ndef key(self, kind, label, prompt, schema):\n basis = f\"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}\"\n return f\"{kind}-{_stable_hash(basis) % 10**10:010d}\"\n\n# Inside agent():\ncached = self.journal.cached(key)\nif cached is not MISS:\n self.task.progress_event(\"workflow_agent\", label=label, status=\"cached\")\n return cached\n```\n\n## Determinism: Reproducibility Makes Resume Meaningful\n\nResume works only if the workflow is reproducible. The minimal Python runtime uses stable hashes and a deterministic mock runner, so the same workflow plus the same arguments produces the same keys. A production harness should also isolate workflow code and remove uncontrolled clocks, randomness, filesystem access, and other sources of nondeterminism.\n\n## See It Run\n\nThe sample `review-changes` workflow uses `pipeline` to send each review dimension independently through audit → verify. An `agent()` with a schema finds issues during audit. During verification, `parallel()` dispatches a separate adversarial subagent for every finding. Only confirmed issues remain, sorted by severity.\n\n```python\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n\n async def audit(_v, dimension, _i):\n out = await ctx.agent(f\"Inspect the changed code for {dimension} issues\",\n schema=FINDINGS_SCHEMA, label=f\"audit:{dimension}\", phase=\"Review\")\n return {\"dimension\": dimension, \"findings\": out[\"findings\"]}\n\n async def verify(audited, dimension, _i):\n ctx.phase(\"Verify\")\n verdicts = await ctx.parallel([ # Verify every finding independently\n (lambda f=f: ctx.agent(f\"Adversarially verify whether this issue is real: {f['title']}\",\n schema=VERDICT_SCHEMA, label=f\"verify:{dimension}:{f['title']}\"))\n for f in audited[\"findings\"]])\n return {\"dimension\": dimension,\n \"confirmed\": [f for f, v in zip(audited[\"findings\"], verdicts) if v and v[\"isReal\"]]}\n\n results = await ctx.pipeline(DIMENSIONS, audit, verify)\n ...\n```\n\n## Changes from s20\n\n| | s20 Comprehensive Agent | s21 Workflow Runtime |\n|--|-----------|---------------------|\n| Loop | One model-driven loop | Main loop unchanged; deterministic orchestration added above it |\n| Who decides the next step | Model decides each round | Script declares the orchestration in advance |\n| Multiple agents | One-shot s06 subagents | Scripted, reproducible, recoverable bulk orchestration |\n| New mechanisms | — | Script DSL, background tasks, progress events, journal/resume, structured output, deterministic VM |\n\ns21 does not replace the main loop. It exposes `Workflow` at the tool layer and starts a local workflow runtime behind it: one workflow deterministically drives N agent loops. An s06 subagent is dispatched once at the model's discretion; s21 turns orchestration into a replayable script.\n\n## Try It\n\n```bash\npython s21_workflow_runtime/code.py # Start review-changes and watch the event stream\npython s21_workflow_runtime/code.py resume # Resume by the last runId; every agent() hits the journal cache\n```\n\nWatch one launch produce `async_launched`, followed by background phase changes and subagent progress, then `task_notification`; the result is stored on the task object. A resumed run reports `agents=0 tokens=0` because every call hits the cache, and its result is byte-for-byte identical.\n\n## Next\n\nOrchestration adds a layer above agent capabilities: the main loop handles individual operations, while a script manages the whole team's flow. Once work becomes a deterministic, recoverable script, the model changes from the round-by-round driver into an execution unit scheduled by that script. The same `agent()` can be invoked ad hoc by the model in the main loop or orchestrated in bulk inside a workflow.\n\nNext: [s22 Goal Loop](/en/s22) — Orchestration fans work out and leaves the main loop. The next chapter moves in the opposite direction: a goal pulls control back into the main loop and refuses to let the turn end until the objective is achieved.\n\n\n" }, { "version": "s21", "locale": "zh", "title": "s21: Workflow Runtime — 模型决定单步，脚本决定编排", - "content": "# s21: Workflow Runtime — 模型决定单步，脚本决定编排\n\ns01 → ... → s19 → s20 → `s21` → [s22](/zh/s22)\n\n> *\"一次 tool_use，后台跑完一整套编排\"* — `Workflow` 工具启动一个确定、可恢复的脚本运行时，批量派出去一堆子 agent。\n>\n> **Harness 层**: 编排 — 在单 agent 循环之上，加一层确定的多 agent 脚本运行时。\n\n> **来源边界：** 本章产品细节来自对 Claude Code 2.1.177 的 clean-room 行为重建。后续版本可能更改名称与限制；`code.py` 是离线教学模型，不是产品源码复制。\n>\n> 教学 CLI 会先发出 `async_launched`，随后在同一进程等待完成，以保证输出可复现。它演示的是生命周期与 journal，不是并发运行的主循环。\n\n---\n\n从 s01 到 s20，我们的循环一直是模型驱动、一步一步来的：每一轮模型挑一个工具，结果塞回 `messages[]`，再来一轮。开放式任务这么干最合适，下一步做什么，让模型看着上下文临场决定就好。\n\n但有些活，你需要的是确定地指挥一群 agent 干活。比如审一个大改动：十个维度并行找问题 → 每条发现各自派一个 agent 做对抗性验证 → 结果汇总去重 → 按严重度排序。这种流程的形状是固定的，你要的其实是三样东西：\n\n- **并行**，别一个一个串着等；\n- **确定**，同样的输入跑出来同样的结果结构；\n- **可恢复**，跑到一半断了，已经做完的部分别从头再来。\n\n让模型在主循环里一步一步驱动这套流程，又慢、结果又不确定，断了还得从头跑。这时候你要的不是\"再聊一轮\"，而是把这套编排直接写成代码。\n\n## 计划写在代码里，不是靠聊天一轮轮凑\n\nClaude Code 在工具池里放了一个 `Workflow` 工具。你（或者模型在高强度模式下触发）给它一段脚本，脚本用 `agent() / parallel() / pipeline() / phase()` 这几个简单的原语，把编排写成确定的代码。\n\n主循环这边只看到一次 `tool_use`，立刻拿到\"已在后台启动\"的返回：真正的执行在后台运行时里推进，实时上报进度，所有过程都写到磁盘的 journal 文件里。脚本里的中间结果存在变量里，不会塞进对话历史占地方。下次用 `resumeFromRunId` 重启时，没改过的 `agent()` 直接命中 journal 缓存，直接用之前的结果，断点续跑。\n\n![Workflow Runtime 总览](/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg)\n\n```python\nSAMPLE_META = {\"name\": \"review-changes\", \"description\": \"审查代码改动\", \"phases\": [\"Review\", \"Verify\"]}\n\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n results = await ctx.pipeline(DIMENSIONS, audit, verify) # 每个维度独立走 审计 → 验证\n confirmed = [f for r in results if r for f in r[\"confirmed\"]]\n ctx.log(f\"确认了 {len(confirmed)} 个真实问题\")\n return {\"confirmed\": confirmed}\n```\n\n## Workflow 工具：后台启动，主循环只看到一次调用\n\n`Workflow`（别名 `RunWorkflow`）就在主 agent 的工具池里。触发可能来自你显式说\"跑一下这个 workflow\"、一个保存好的 `/命令`，或者模型自动进入高强度路径，这时候模型会发一个 `Workflow(...)` 的工具调用。\n\n工具收到后会解析参数、校验 meta 信息、过权限检查、注册一个本地 workflow 任务，然后立刻返回\"已异步启动\"。主循环不阻塞，该干嘛干嘛；workflow 自己在后台跑。这其实就是 s13 后台任务那套\"凭条模式\"的放大版：先给你个取件条，结果好了再通知你。\n\n```python\nclass WorkflowTool:\n async def call(self, meta, script_fn, args=None, resume_from_run_id=None):\n validate_meta(meta)\n check_permission(meta)\n run_id = resume_from_run_id or create_run_id(meta)\n task = LocalWorkflowTask(create_task_id(run_id), run_id, meta)\n task.event(\"async_launched\", runId=run_id, taskId=task.task_id) # 立刻返回\n ... # 剩下的后台慢慢跑\n```\n\n> 真实 Claude Code：工具会立刻返回 `{status:'async_launched', taskId, taskType:'local_workflow', runId, summary, transcriptDir, scriptPath}`，后台任务跑完了再通知。\n\n## 脚本和 meta：第一行必须写对\n\n脚本的第一行必须是 `export const meta = { name, description, phases }`，而且必须是纯字面量，不能有变量、函数调用、字符串拼接。运行时在执行任何代码之前先解析它：`name` 和 `description` 用来显示任务和 UI，`phases` 给进度条分组命名。\n\n不对的输入直接抛 `WorkflowInputError`，注册的时候就拦住——这和 s14 校验 cron 表达式是一个思路：坏脚本别让它跑到执行的时候才炸。\n\n教学运行时会把 `meta.name` 用在本地产物文件名中，因此还要求它是 1-64 个字符的安全 slug，只能包含字母、数字、`.`、`_`、`-`。\n\n```python\ndef validate_meta(meta):\n if not isinstance(meta, dict):\n raise WorkflowInputError(\"meta 必须是对象字面量\")\n if not meta.get(\"name\") or not meta.get(\"description\"):\n raise WorkflowInputError(\"meta 必须包含 name 和 description\")\n if not isinstance(meta[\"name\"], str) or not WORKFLOW_NAME_RE.fullmatch(meta[\"name\"]):\n raise WorkflowInputError(\"meta.name 必须是 1-64 字符的安全 slug\")\n if \"phases\" in meta and (\n not isinstance(meta[\"phases\"], list)\n or not all(isinstance(p, str) and p for p in meta[\"phases\"])\n ):\n raise WorkflowInputError(\"meta.phases 必须包含非空字符串\")\n return meta\n```\n\n> 真实 Claude Code：`parseWorkflowScript` 强制 meta 必须是第一行且是纯字面量；教学版直接收一个 dict，简化了这部分。\n\n## 编排原语：就这几个，够写所有流程\n\n脚本跑在一个独立的上下文里，能用的全局变量就这几个编排原语。脚本本身不直接读写文件、不跑 shell，真正的代码操作都由派出去的子 agent 用它们自己的工具权限完成。这些原语都是 `ExecutionState` 上的方法：\n\n| 原语 | 作用 |\n|------|------|\n| `agent(prompt, {schema, label, phase})` | 派一个子 agent 干活 |\n| `parallel(thunks)` | **等齐屏障**：所有任务并行跑完，一起等结果回来 |\n| `pipeline(items, *stages)` | 每个 item 分阶段跑，**不等齐**，跑完一个往下走一个 |\n| `phase(title)` | 标记当前进度阶段（更新进度条） |\n| `log(message)` | 打一行进度日志 |\n| `workflow(name, args)` | 嵌套子工作流（只支持一层） |\n\n`pipeline` 是你默认该用的：每个 item 独立穿过所有 stage，item A 跑到第 3 阶段的时候，item B 可能还在第 1 阶段；只有真的需要\"拿到上一阶段所有结果才能往下走\"的时候，才用 `parallel` 这个屏障。屏障的代价是等最慢的那个任务，没必要就别立。\n\n```python\nasync def pipeline(self, items, *stages):\n async def run_item(item, idx):\n value = item\n for stage in stages: # 每个 item 独立跑完所有 stage\n value = await stage(value, item, idx)\n return value\n return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])\n```\n\n> 真实 Claude Code：同名原语由 VM 注入脚本上下文；还提供 `args`、`budget`（总预算/已花/剩余）、agent 数量上限（最多 1000 个）、并发信号量这些控制。\n\n## 结构化输出：别让子 agent 回来写散文\n\n`agent({schema})` 会强制子 agent 返回一个匹配 schema 的 JSON 对象（内部通过一次结构化输出调用实现），运行时会按 schema 校验结果，不对就重试一次。这样下游代码拿到的是规整的对象，不是需要再解析的一大段散文。\n\ns05 就说过，工具的参数不能全信；这里是同一个道理反过来：子 agent 的输出也不能全信。加一层校验，不对就给一次机会重试，把不确定性挡在编排层外面。\n\n```python\nresult = self.runner.run(prompt, schema, label)\nif schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok: # 提醒一次重试，再不对就报错\n result = self.runner.run(prompt + \"\\n\\n返回合法的 JSON。\", schema, label)\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok:\n raise WorkflowInputError(f\"agent({{schema}}) 输出不合法: {err}\")\n```\n\n> 真实 Claude Code：用 `SimpleJsonSchema` + `StructuredOutput` 工具 + schema 重试机制保证输出格式。\n\n## 后台任务和进度事件\n\n`LocalWorkflowTask` 维护状态和 token 用量，向外发一条 SDK 风格的事件流：`task_started` → 一串 `task_progress`（包含阶段切换、子 agent 启动、日志输出这些批次）→ 最后一个 `task_notification`（完成/失败/停止，带输出文件、token 数、工具调用数、耗时）。\n\n主会话把这些当普通事件处理；只有最终的完成通知会重新进入主循环。\n\n```python\nclass LocalWorkflowTask:\n def progress_event(self, ptype, **data): # 阶段/子agent/日志\n self.progress.append({\"type\": ptype, **data})\n print(f\" 进度 {ptype} ...\")\n```\n\n> 真实 Claude Code：进度会折叠进任务状态，作为 `task_progress.workflow_progress` 发给 UI 和 SDK。\n\n## 存储：快照 + journal，断了能续\n\n跑完会写五样东西，都存在 `~/.claude/projects/<项目>/<会话>/` 目录下：快照 `.json`、输出 `.output.json`、journal `.journal.jsonl`、脚本副本 `scripts/.js`、子 agent 的对话记录 `subagents/workflows//`。你自己保存的常用 workflow 放在 `.claude/workflows/`（项目级）或 `~/.claude/workflows/`（用户级）。\n\njournal 是断点续跑的核心，它一条一条记下来每个 `agent()` 的结果：\n\n```python\nclass WorkflowJournal:\n def record(self, key, value):\n self._f.write(json.dumps({\"key\": key, \"value\": value}) + \"\\n\")\n self._f.flush()\n self.cache[key] = value\n```\n\n## resume：用 runId 续跑，没改的直接用缓存\n\n调用 `Workflow({scriptPath, resumeFromRunId, args})` 会重新跑脚本，但每个 `agent()` 会算一个确定的语义 key：key 在 journal 里有记录，就直接返回缓存的结果（不重跑），没改过的全部命中缓存；只有改过的那个以及它后面的步骤才会真的跑。\n\n这里有个关键点：key 不能依赖并发顺序。`parallel` 和 `pipeline` 里 agent 完成的顺序是不确定的，用\"第几个完成\"当 key，两次跑缓存就对错位了。所以 key 是根据调用内容（类型、标签、prompt、schema）算的稳定哈希，不是一个会竞争的计数器：\n\n```python\ndef key(self, kind, label, prompt, schema):\n basis = f\"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}\"\n return f\"{kind}-{_stable_hash(basis) % 10**10:010d}\"\n\n# agent() 内部：\ncached = self.journal.cached(key)\nif cached is not MISS:\n self.task.progress_event(\"workflow_agent\", label=label, status=\"cached\")\n return cached\n```\n\n> 真实 Claude Code：同样是\"确定语义 key + journal 缓存\"的思路；同会话内续跑时，已经完成的 `agent()` 直接返回缓存，后面的才实跑。\n\n## 确定性：能复现，续跑才有意义\n\n续跑要能工作，脚本首先得可复现。所以运行时会把 `Date.now()`、无参 `new Date()`、`Math.random()` 这些不确定的东西从脚本上下文里去掉，也不给 Node 原生 API。同一份脚本 + 同样的参数 → 同样的 key → 100% 缓存命中。教学版用稳定哈希算 key 达到同样的效果（真实版是把整段 JS 脚本跑在去掉了这些不确定源的沙箱 VM 里）。\n\n## 跑起来看看\n\n示例 workflow `review-changes`：用 `pipeline` 让每个审查维度独立走\"审计 → 验证\"流程。审计用一个带 schema 的 `agent()` 找问题，验证用 `parallel()` 给每条发现各派一个对抗性验证的子 agent，最后只留确认真实的问题，按严重度排序。\n\n```python\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n\n async def audit(_v, dimension, _i):\n out = await ctx.agent(f\"检查改动的代码里有没有{dimension}相关的问题\",\n schema=FINDINGS_SCHEMA, label=f\"audit:{dimension}\", phase=\"Review\")\n return {\"dimension\": dimension, \"findings\": out[\"findings\"]}\n\n async def verify(audited, dimension, _i):\n ctx.phase(\"Verify\")\n verdicts = await ctx.parallel([ # 每条发现独立做对抗性验证\n (lambda f=f: ctx.agent(f\"请对抗性验证这个问题是不是真的：{f['title']}\",\n schema=VERDICT_SCHEMA, label=f\"verify:{dimension}:{f['title']}\"))\n for f in audited[\"findings\"]])\n return {\"dimension\": dimension,\n \"confirmed\": [f for f, v in zip(audited[\"findings\"], verdicts) if v and v[\"isReal\"]]}\n\n results = await ctx.pipeline(DIMENSIONS, audit, verify)\n ...\n```\n\n## 相对 s20 的变更\n\n| | s20 综合体 | s21 Workflow Runtime |\n|--|-----------|---------------------|\n| 循环 | 单个、模型驱动 | 主循环不变；上面加一层确定的编排 |\n| 谁决定下一步 | 模型逐轮决定 | 脚本预先写好编排流程 |\n| 多 agent | s06 子 agent，一次性派出去 | 脚本化、可复现、可恢复的批量编排 |\n| 新增机制 | — | 脚本 DSL、后台任务、进度事件、journal/续跑、结构化输出、确定性 VM |\n\ns21 不替换主循环，它只是在工具层暴露了 `Workflow`，背后启动一个本地 workflow 运行时：一个 workflow 确定地驱动 N 个 agent 循环。s06 的子 agent 是模型临场派一次；s21 是把编排写成可以重放的脚本。\n\n## 试一下\n\n```bash\npython s21_workflow_runtime/code.py # 启动 review-changes，看事件流\npython s21_workflow_runtime/code.py resume # 用上次的 runId 续跑，每个 agent() 都命中 journal 缓存\n```\n\n观察：一次启动 → `async_launched` → 后台阶段切换/子agent进度推进 → `task_notification`；结果存在任务对象上。续跑的时候会显示 `agents=0 tokens=0`（全部命中缓存），结果和上次一字不差。\n\n## 接下来\n\n编排是在 agent 能力之上又加了一层：主循环管单步操作，脚本管整支队伍的流程。把工作写成确定、可恢复的脚本，模型就从\"逐轮驱动者\"变成了\"被脚本调度的执行单元\"。同一个 `agent()`，既能在主循环里被模型临场调用，也能在 workflow 里被脚本批量编排。\n\n下一章：[s22 Goal Loop](/zh/s22) — 编排是把工作扇出去、脱离主循环；下一章反过来，一个目标把控制权重拉回主循环，没达成就不让这一轮结束。\n\n\n" + "content": "# s21: Workflow Runtime — 模型决定单步，脚本决定编排\n\ns01 → ... → s19 → s20 → `s21` → [s22](/zh/s22)\n\n> *\"一次 tool_use，后台跑完一整套编排\"* — `Workflow` 工具启动一个确定、可恢复的脚本运行时，批量派出去一堆子 agent。\n>\n> **Harness 层**: 编排 — 在单 agent 循环之上，加一层确定的多 agent 脚本运行时。\n\n`code.py` 为了让演示保持确定，会先发出 `async_launched`，随后在同一进程里等待执行完成。这样不用启动常驻后台服务，也能看清生命周期和 journal。\n\n---\n\n从 s01 到 s20，我们的循环一直是模型驱动、一步一步来的：每一轮模型挑一个工具，结果塞回 `messages[]`，再来一轮。开放式任务这么干最合适，下一步做什么，让模型看着上下文临场决定就好。\n\n但有些活，你需要的是确定地指挥一群 agent 干活。比如审一个大改动：十个维度并行找问题 → 每条发现各自派一个 agent 做对抗性验证 → 结果汇总去重 → 按严重度排序。这种流程的形状是固定的，你要的其实是三样东西：\n\n- **并行**，别一个一个串着等；\n- **确定**，同样的输入跑出来同样的结果结构；\n- **可恢复**，跑到一半断了，已经做完的部分别从头再来。\n\n让模型在主循环里一步一步驱动这套流程，又慢、结果又不确定，断了还得从头跑。这时候你要的不是\"再聊一轮\"，而是把这套编排直接写成代码。\n\n## 计划写在代码里，不是靠聊天一轮轮凑\n\n在 harness 的工具池里加入一个 `Workflow` 工具。用户或模型给它一段脚本，脚本用 `agent() / parallel() / pipeline() / phase()` 这几个简单的原语，把编排写成确定的代码。\n\n主循环这边只看到一次 `tool_use`，立刻拿到\"已在后台启动\"的返回：真正的执行在后台运行时里推进，实时上报进度，所有过程都写到磁盘的 journal 文件里。脚本里的中间结果存在变量里，不会塞进对话历史占地方。下次用 `resume_from_run_id` 重启时，没改过的 `agent()` 直接命中 journal 缓存，直接用之前的结果，断点续跑。\n\n![Workflow Runtime 总览](/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg)\n\n```python\nSAMPLE_META = {\"name\": \"review-changes\", \"description\": \"审查代码改动\", \"phases\": [\"Review\", \"Verify\"]}\n\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n results = await ctx.pipeline(DIMENSIONS, audit, verify) # 每个维度独立走 审计 → 验证\n confirmed = [f for r in results if r for f in r[\"confirmed\"]]\n ctx.log(f\"确认了 {len(confirmed)} 个真实问题\")\n return {\"confirmed\": confirmed}\n```\n\n## Workflow 工具：后台启动，主循环只看到一次调用\n\n`Workflow` 就在主 agent 的工具池里。用户可以要求运行一个保存好的 workflow，模型也可以在任务匹配已知编排时选择这个工具；两种情况最终都只发出一次 `Workflow(...)` 工具调用。\n\n工具收到后会解析参数、校验 meta 信息、过权限检查、注册一个本地 workflow 任务，然后立刻返回\"已异步启动\"。主循环不阻塞，该干嘛干嘛；workflow 自己在后台跑。这其实就是 s13 后台任务那套\"凭条模式\"的放大版：先给你个取件条，结果好了再通知你。\n\n```python\nclass WorkflowTool:\n async def call(self, meta, script_fn, args=None, resume_from_run_id=None):\n validate_meta(meta)\n check_permission(meta)\n run_id = resume_from_run_id or create_run_id(meta)\n task = LocalWorkflowTask(create_task_id(run_id), run_id, meta)\n task.event(\"async_launched\", runId=run_id, taskId=task.task_id) # 立刻返回\n ... # 剩下的后台慢慢跑\n```\n\n## Workflow 元数据：启动前先校验\n\n每个 workflow 都要注册一个元数据对象，包含 `name`、`description` 和可选的 `phases`。运行时会在执行任何 workflow 代码之前校验它：`name` 和 `description` 用来标识任务，`phases` 给进度条分组命名。\n\n不对的输入直接抛 `WorkflowInputError`，注册的时候就拦住——这和 s14 校验 cron 表达式是一个思路：坏脚本别让它跑到执行的时候才炸。\n\n教学运行时会把 `meta.name` 用在本地产物文件名中，因此还要求它是 1-64 个字符的安全 slug，只能包含字母、数字、`.`、`_`、`-`。\n\n```python\ndef validate_meta(meta):\n if not isinstance(meta, dict):\n raise WorkflowInputError(\"meta 必须是对象字面量\")\n if not meta.get(\"name\") or not meta.get(\"description\"):\n raise WorkflowInputError(\"meta 必须包含 name 和 description\")\n if not isinstance(meta[\"name\"], str) or not WORKFLOW_NAME_RE.fullmatch(meta[\"name\"]):\n raise WorkflowInputError(\"meta.name 必须是 1-64 字符的安全 slug\")\n if \"phases\" in meta and (\n not isinstance(meta[\"phases\"], list)\n or not all(isinstance(p, str) and p for p in meta[\"phases\"])\n ):\n raise WorkflowInputError(\"meta.phases 必须包含非空字符串\")\n return meta\n```\n\n## 编排原语：就这几个，够写所有流程\n\n脚本跑在一个独立的上下文里，能用的全局变量就这几个编排原语。脚本本身不直接读写文件、不跑 shell，真正的代码操作都由派出去的子 agent 用它们自己的工具权限完成。这些原语都是 `ExecutionState` 上的方法：\n\n| 原语 | 作用 |\n|------|------|\n| `agent(prompt, {schema, label, phase})` | 派一个子 agent 干活 |\n| `parallel(thunks)` | **等齐屏障**：所有任务并行跑完，一起等结果回来 |\n| `pipeline(items, *stages)` | 每个 item 分阶段跑，**不等齐**，跑完一个往下走一个 |\n| `phase(title)` | 标记当前进度阶段（更新进度条） |\n| `log(message)` | 打一行进度日志 |\n| `workflow(name, args)` | 嵌套子工作流（只支持一层） |\n\n`pipeline` 是你默认该用的：每个 item 独立穿过所有 stage，item A 跑到第 3 阶段的时候，item B 可能还在第 1 阶段；只有真的需要\"拿到上一阶段所有结果才能往下走\"的时候，才用 `parallel` 这个屏障。屏障的代价是等最慢的那个任务，没必要就别立。\n\n```python\nasync def pipeline(self, items, *stages):\n async def run_item(item, idx):\n value = item\n for stage in stages: # 每个 item 独立跑完所有 stage\n value = await stage(value, item, idx)\n return value\n return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])\n```\n\n## 结构化输出：别让子 agent 回来写散文\n\n`agent({schema})` 会强制子 agent 返回一个匹配 schema 的 JSON 对象（内部通过一次结构化输出调用实现），运行时会按 schema 校验结果，不对就重试一次。这样下游代码拿到的是规整的对象，不是需要再解析的一大段散文。\n\ns05 就说过，工具的参数不能全信；这里是同一个道理反过来：子 agent 的输出也不能全信。加一层校验，不对就给一次机会重试，把不确定性挡在编排层外面。\n\n```python\nresult = self.runner.run(prompt, schema, label)\nif schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok: # 提醒一次重试，再不对就报错\n result = self.runner.run(prompt + \"\\n\\n返回合法的 JSON。\", schema, label)\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok:\n raise WorkflowInputError(f\"agent({{schema}}) 输出不合法: {err}\")\n```\n\n## 后台任务和进度事件\n\n`LocalWorkflowTask` 维护状态和 token 用量，向外发一条 SDK 风格的事件流：`task_started` → 一串 `task_progress`（包含阶段切换、子 agent 启动、日志输出这些批次）→ 最后一个 `task_notification`（完成/失败/停止，带输出文件、token 数、工具调用数、耗时）。\n\n主会话把这些当普通事件处理；只有最终的完成通知会重新进入主循环。\n\n```python\nclass LocalWorkflowTask:\n def progress_event(self, ptype, **data): # 阶段/子agent/日志\n self.progress.append({\"type\": ptype, **data})\n print(f\" 进度 {ptype} ...\")\n```\n\n## 存储：快照 + journal，断了能续\n\n这个最小运行时把每次运行的数据存在 `s21_workflow_runtime/.runtime/`：快照 `.json`、输出 `.output.json` 和 journal `.journal.jsonl`。生产级 harness 还可以保存 workflow 脚本与子 agent 对话记录，但关键约束是快照和 journal 必须共享稳定的 `runId`。\n\njournal 是断点续跑的核心，它一条一条记下来每个 `agent()` 的结果：\n\n```python\nclass WorkflowJournal:\n def record(self, key, value):\n self._f.write(json.dumps({\"key\": key, \"value\": value}) + \"\\n\")\n self._f.flush()\n self.cache[key] = value\n```\n\n## resume：用 runId 续跑，没改的直接用缓存\n\n带着 `resume_from_run_id` 再次调用 workflow 时，脚本会重新执行，但每个 `agent()` 都会计算一个确定的语义 key：key 在 journal 里有记录，就直接返回缓存结果；只有改过的调用以及依赖它的后续步骤才会真的运行。\n\n这里有个关键点：key 不能依赖并发顺序。`parallel` 和 `pipeline` 里 agent 完成的顺序是不确定的，用\"第几个完成\"当 key，两次跑缓存就对错位了。所以 key 是根据调用内容（类型、标签、prompt、schema）算的稳定哈希，不是一个会竞争的计数器：\n\n```python\ndef key(self, kind, label, prompt, schema):\n basis = f\"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}\"\n return f\"{kind}-{_stable_hash(basis) % 10**10:010d}\"\n\n# agent() 内部：\ncached = self.journal.cached(key)\nif cached is not MISS:\n self.task.progress_event(\"workflow_agent\", label=label, status=\"cached\")\n return cached\n```\n\n## 确定性：能复现，续跑才有意义\n\n续跑要能工作，workflow 首先得可复现。这个最小 Python 运行时使用稳定哈希和确定性的 mock runner，让同一份 workflow + 同样的参数产生同样的 key。生产级 harness 还应该隔离 workflow 代码，并移除不受控的时钟、随机数、文件系统访问等不确定来源。\n\n## 跑起来看看\n\n示例 workflow `review-changes`：用 `pipeline` 让每个审查维度独立走\"审计 → 验证\"流程。审计用一个带 schema 的 `agent()` 找问题，验证用 `parallel()` 给每条发现各派一个对抗性验证的子 agent，最后只留确认真实的问题，按严重度排序。\n\n```python\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n\n async def audit(_v, dimension, _i):\n out = await ctx.agent(f\"检查改动的代码里有没有{dimension}相关的问题\",\n schema=FINDINGS_SCHEMA, label=f\"audit:{dimension}\", phase=\"Review\")\n return {\"dimension\": dimension, \"findings\": out[\"findings\"]}\n\n async def verify(audited, dimension, _i):\n ctx.phase(\"Verify\")\n verdicts = await ctx.parallel([ # 每条发现独立做对抗性验证\n (lambda f=f: ctx.agent(f\"请对抗性验证这个问题是不是真的：{f['title']}\",\n schema=VERDICT_SCHEMA, label=f\"verify:{dimension}:{f['title']}\"))\n for f in audited[\"findings\"]])\n return {\"dimension\": dimension,\n \"confirmed\": [f for f, v in zip(audited[\"findings\"], verdicts) if v and v[\"isReal\"]]}\n\n results = await ctx.pipeline(DIMENSIONS, audit, verify)\n ...\n```\n\n## 相对 s20 的变更\n\n| | s20 综合体 | s21 Workflow Runtime |\n|--|-----------|---------------------|\n| 循环 | 单个、模型驱动 | 主循环不变；上面加一层确定的编排 |\n| 谁决定下一步 | 模型逐轮决定 | 脚本预先写好编排流程 |\n| 多 agent | s06 子 agent，一次性派出去 | 脚本化、可复现、可恢复的批量编排 |\n| 新增机制 | — | 脚本 DSL、后台任务、进度事件、journal/续跑、结构化输出、确定性 VM |\n\ns21 不替换主循环，它只是在工具层暴露了 `Workflow`，背后启动一个本地 workflow 运行时：一个 workflow 确定地驱动 N 个 agent 循环。s06 的子 agent 是模型临场派一次；s21 是把编排写成可以重放的脚本。\n\n## 试一下\n\n```bash\npython s21_workflow_runtime/code.py # 启动 review-changes，看事件流\npython s21_workflow_runtime/code.py resume # 用上次的 runId 续跑，每个 agent() 都命中 journal 缓存\n```\n\n观察：一次启动 → `async_launched` → 后台阶段切换/子agent进度推进 → `task_notification`；结果存在任务对象上。续跑的时候会显示 `agents=0 tokens=0`（全部命中缓存），结果和上次一字不差。\n\n## 接下来\n\n编排是在 agent 能力之上又加了一层：主循环管单步操作，脚本管整支队伍的流程。把工作写成确定、可恢复的脚本，模型就从\"逐轮驱动者\"变成了\"被脚本调度的执行单元\"。同一个 `agent()`，既能在主循环里被模型临场调用，也能在 workflow 里被脚本批量编排。\n\n下一章：[s22 Goal Loop](/zh/s22) — 编排是把工作扇出去、脱离主循环；下一章反过来，一个目标把控制权重拉回主循环，没达成就不让这一轮结束。\n\n\n" }, { "version": "s21", "locale": "ja", "title": "s21: Workflow Runtime — モデルが単一 step を決め、script が orchestration を決める", - "content": "# s21: Workflow Runtime — モデルが単一 step を決め、script が orchestration を決める\n\ns01 → ... → s19 → s20 → `s21` → [s22](/ja/s22)\n\n> *「1 回の tool_use で、バックグラウンドに一式の orchestration を走らせる」* — `Workflow` ツールが決定的で復元可能な script runtime を起動し、多数の subagent をまとめて送り出します。\n>\n> **Harness 層**: Orchestration — single-agent loop の上に、決定的な multi-agent script runtime を追加します。\n\n> **情報源の境界：** この章の製品詳細は Claude Code 2.1.177 の clean-room 行動再構成に基づく。後続リリースで名称や制限は変わり得る。`code.py` はオフライン教材モデルであり、製品ソースの複製ではない。\n>\n> 教材 CLI は `async_launched` を出した後、再現可能な出力のため同じプロセスで完了を待つ。示すのは lifecycle と journal であり、main loop の並行実行そのものではない。\n\n---\n\ns01 から s20 まで、loop は常にモデル駆動で 1 step ずつ進みました。各ラウンドでモデルが 1 つのツールを選び、結果を `messages[]` へ入れ、次のラウンドへ進みます。open-ended なタスクには最適です。次に何をするかを、モデルが context を見てその場で決められます。\n\nしかし、複数の Agent を決定的に指揮したい仕事もあります。大きな変更の review を考えてください。10 の観点から並行して問題を探す → 各 finding へ別 Agent を送り adversarial verification を行う → 結果を集約して重複を除く → severity 順に並べる。この流れの形は固定されており、本当に必要なのは 3 つです。\n\n- **並行性**: 1 件ずつ順番に待たないこと。\n- **決定性**: 同じ入力から同じ結果構造が得られること。\n- **復元可能性**: 途中で止まっても、完了済みの部分を最初からやり直さないこと。\n\nこの流れをモデルに main loop で 1 ラウンドずつ動かさせると、遅く、結果は不確定で、中断すれば最初からです。ここで必要なのは「もう 1 turn 話す」ことではなく、orchestration をそのままコードにすることです。\n\n## 計画は chat のラウンドを重ねず、コードに書く\n\nClaude Code の tool pool には `Workflow` ツールがあります。あなたが渡すか、モデルが high-intensity mode で起動した script は、`agent() / parallel() / pipeline() / phase()` という少数の primitive を使い、orchestration を決定的なコードとして表します。\n\nmain loop から見えるのは 1 回の `tool_use` だけで、すぐ「バックグラウンドで起動済み」という結果を受け取ります。本当の実行は background runtime で進み、進捗をリアルタイムに報告し、全過程をディスク上の journal へ記録します。script の中間結果は変数に保存され、会話履歴の場所を取りません。`resumeFromRunId` で再開すると、変更されていない `agent()` は journal cache に当たり、以前の結果を直接使って checkpoint から続行します。\n\n![Workflow Runtime Overview](/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg)\n\n```python\nSAMPLE_META = {\"name\": \"review-changes\", \"description\": \"コード変更を review\", \"phases\": [\"Review\", \"Verify\"]}\n\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n results = await ctx.pipeline(DIMENSIONS, audit, verify) # 各 dimension が独立して audit → verify を通る\n confirmed = [f for r in results if r for f in r[\"confirmed\"]]\n ctx.log(f\"{len(confirmed)} 件の実在する問題を確認\")\n return {\"confirmed\": confirmed}\n```\n\n## Workflow ツール: バックグラウンド起動、main loop には 1 回の call だけ\n\n`Workflow`（別名 `RunWorkflow`）は main Agent の tool pool にあります。明示的に「この workflow を実行」と頼む、保存済みの `/command` を使う、またはモデルが自動で high-intensity path へ入ると、モデルが `Workflow(...)` の tool call を出します。\n\nツールは argument を parse し、meta 情報を検証し、permission check を通し、local workflow task を登録すると、すぐ「非同期で起動済み」と返します。main loop は block せず別の仕事を続け、workflow は background で実行されます。これは s13 の引換券 pattern を拡大したものです。先に引換券を渡し、結果ができたら通知します。\n\n```python\nclass WorkflowTool:\n async def call(self, meta, script_fn, args=None, resume_from_run_id=None):\n validate_meta(meta)\n check_permission(meta)\n run_id = resume_from_run_id or create_run_id(meta)\n task = LocalWorkflowTask(create_task_id(run_id), run_id, meta)\n task.event(\"async_launched\", runId=run_id, taskId=task.task_id) # すぐ return\n ... # 残りはバックグラウンドで進む\n```\n\n> 実際の Claude Code は `{status:'async_launched', taskId, taskType:'local_workflow', runId, summary, transcriptDir, scriptPath}` をすぐ返し、background task の完了後に通知します。\n\n## Script と meta: 1 行目を正しく書く\n\nscript の 1 行目は必ず `export const meta = { name, description, phases }` とし、変数、関数呼び出し、文字列連結を含まない純粋な literal でなければなりません。runtime はコードを一切実行する前に parse します。`name` と `description` は task と UI の表示に使い、`phases` は progress bar の group 名を定義します。\n\n不正な入力はすぐ `WorkflowInputError` になり、登録時に止まります。s14 の cron 式検証と同じ考えです。不正な script が実行時まで進んでから壊れないようにします。\n\n教材 runtime は `meta.name` をローカル artifact のファイル名に使うため、英数字で始まり、英数字、`.`、`_`、`-` のみからなる 1-64 文字の安全な slug も要求する。\n\n```python\ndef validate_meta(meta):\n if not isinstance(meta, dict):\n raise WorkflowInputError(\"meta は object literal でなければなりません\")\n if not meta.get(\"name\") or not meta.get(\"description\"):\n raise WorkflowInputError(\"meta には name と description が必要です\")\n if not isinstance(meta[\"name\"], str) or not WORKFLOW_NAME_RE.fullmatch(meta[\"name\"]):\n raise WorkflowInputError(\"meta.name は安全な 1-64 文字の slug が必要です\")\n if \"phases\" in meta and (\n not isinstance(meta[\"phases\"], list)\n or not all(isinstance(p, str) and p for p in meta[\"phases\"])\n ):\n raise WorkflowInputError(\"meta.phases は空でない文字列だけを含む必要があります\")\n return meta\n```\n\n> 実際の Claude Code の `parseWorkflowScript` は、meta を 1 行目の純粋な literal に限定します。教材版は dict を直接受け取り、この部分を簡略化しています。\n\n## Orchestration primitive: この少数だけで、すべての flow を書ける\n\nscript は独立した context で動き、global variable として使えるのは少数の orchestration primitive だけです。script 自身はファイルを直接読み書きせず、shell も実行しません。実際のコード操作は、派遣された subagent が自分の tool permission で行います。primitive はすべて `ExecutionState` の method です。\n\n| Primitive | 役割 |\n|------|------|\n| `agent(prompt, {schema, label, phase})` | 1 つの subagent を派遣 |\n| `parallel(thunks)` | **barrier**: すべての task を並行実行し、全結果が戻るまで待つ |\n| `pipeline(items, *stages)` | 各 item を **barrier なし**で stage ごとに実行し、終わった item から先へ進める |\n| `phase(title)` | 現在の progress phase を記録し、progress bar を更新 |\n| `log(message)` | progress log を 1 行出力 |\n| `workflow(name, args)` | nested sub-workflow（1 階層だけ） |\n\n既定では `pipeline` を使うべきです。各 item がすべての stage を独立して通り、item A が stage 3 にいる間、item B はまだ stage 1 かもしれません。次の stage へ進むために前 stage の全結果が本当に必要なときだけ、`parallel` barrier を使います。barrier は最も遅い task を待つため、不要なら置かないでください。\n\n```python\nasync def pipeline(self, items, *stages):\n async def run_item(item, idx):\n value = item\n for stage in stages: # 各 item がすべての stage を独立して完走\n value = await stage(value, item, idx)\n return value\n return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])\n```\n\n> 実際の Claude Code は同名 primitive を script VM の context へ注入します。さらに `args`、total/spent/remaining を持つ `budget`、最大 1000 Agent の上限、concurrency semaphore も提供します。\n\n## 構造化出力: Subagent に散文を返させない\n\n`agent({schema})` は、schema に一致する JSON object を subagent に要求します。内部では structured output call を 1 回使い、runtime が結果を schema で検証し、不一致なら 1 回 retry します。下流コードが受け取るのは規則的な object であり、再 parse が必要な長文ではありません。\n\ns05 では tool argument を全面的に信頼できないと説明しました。ここでは同じ教訓を逆向きに使います。subagent の出力も全面的には信頼できません。orchestration boundary で検証し、1 回 retry の機会を与え、不確実性を後続 flow の外へ止めます。\n\n```python\nresult = self.runner.run(prompt, schema, label)\nif schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok: # 1 回だけ注意して retry、それでも不正なら error\n result = self.runner.run(prompt + \"\\n\\n有効な JSON を返してください。\", schema, label)\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok:\n raise WorkflowInputError(f\"agent({{schema}}) の出力が不正です: {err}\")\n```\n\n> 実際の Claude Code は `SimpleJsonSchema`、`StructuredOutput` ツール、schema-aware retry を組み合わせ、出力形式を保証します。\n\n## Background task と progress event\n\n`LocalWorkflowTask` は status と token usage を管理し、SDK style の event stream を外へ出します。`task_started` → phase change、subagent start、log batch を含む一連の `task_progress` → 完了、失敗、停止に加え、output file、token 数、tool call 数、所要時間を含む最後の `task_notification` です。\n\nmain session は通常 event として処理し、最後の完了通知だけが main loop へ再び入ります。\n\n```python\nclass LocalWorkflowTask:\n def progress_event(self, ptype, **data): # phase/subagent/log\n self.progress.append({\"type\": ptype, **data})\n print(f\" progress {ptype} ...\")\n```\n\n> 実際の Claude Code は進捗を task state へまとめ、`task_progress.workflow_progress` として UI と SDK へ送ります。\n\n## 保存: Snapshot + journal で中断から再開する\n\n各 run は `~/.claude/projects///` に 5 種類を書きます。`.json` snapshot、`.output.json` output、`.journal.jsonl` journal、`scripts/.js` の script copy、`subagents/workflows//` の subagent transcript です。保存した再利用可能な workflow は project scope の `.claude/workflows/` または user scope の `~/.claude/workflows/` に置きます。\n\njournal は checkpoint resume の中心で、各 `agent()` の結果を 1 行ずつ記録します。\n\n```python\nclass WorkflowJournal:\n def record(self, key, value):\n self._f.write(json.dumps({\"key\": key, \"value\": value}) + \"\\n\")\n self._f.flush()\n self.cache[key] = value\n```\n\n## Resume: runId から続行し、変更のないものを再利用する\n\n`Workflow({scriptPath, resumeFromRunId, args})` を呼ぶと script を再実行しますが、各 `agent()` は決定的な semantic key を計算します。journal に key があれば、再実行せず cached result を返します。変更のない call はすべて cache hit し、変更された call とそれに依存する後続 step だけが本当に動きます。\n\nkey は concurrency の完了順に依存してはいけません。`parallel` と `pipeline` の Agent は不定の順番で完了します。「何番目に完了したか」を key にすると、次回の cache が別の call へ対応してしまいます。そのため key は競合する counter ではなく、call の内容、つまり type、label、prompt、schema の stable hash です。\n\n```python\ndef key(self, kind, label, prompt, schema):\n basis = f\"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}\"\n return f\"{kind}-{_stable_hash(basis) % 10**10:010d}\"\n\n# agent() の内部:\ncached = self.journal.cached(key)\nif cached is not MISS:\n self.task.progress_event(\"workflow_agent\", label=label, status=\"cached\")\n return cached\n```\n\n> 実際の Claude Code も「決定的 semantic key + journal cache」という考えです。同じ session で resume すると、完了済み `agent()` は cached result を直接返し、その後だけを実行します。\n\n## 決定性: Resume に意味を持たせる再現性\n\nresume が動くには、まず script が再現可能でなければなりません。runtime は `Date.now()`、引数なしの `new Date()`、`Math.random()` などの非決定的なものを script context から取り除き、Node native API も渡しません。同じ script + 同じ argument → 同じ key → 100% cache hit になります。教材版は stable hash で同じ性質を得ます。実際の版は、非決定的な source を除いた sandbox VM で JavaScript 全体を実行します。\n\n## 実際に動かす\n\nsample workflow `review-changes` は `pipeline` を使い、各 review dimension を独立して audit → verify へ通します。audit では schema 付き `agent()` が問題を探し、verify では `parallel()` が各 finding に別の adversarial verification subagent を送ります。実在すると確認された問題だけを残し、severity 順に並べます。\n\n```python\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n\n async def audit(_v, dimension, _i):\n out = await ctx.agent(f\"変更されたコードに {dimension} 関連の問題がないか確認してください\",\n schema=FINDINGS_SCHEMA, label=f\"audit:{dimension}\", phase=\"Review\")\n return {\"dimension\": dimension, \"findings\": out[\"findings\"]}\n\n async def verify(audited, dimension, _i):\n ctx.phase(\"Verify\")\n verdicts = await ctx.parallel([ # 各 finding を独立して verify\n (lambda f=f: ctx.agent(f\"この問題が実在するか adversarial に検証してください: {f['title']}\",\n schema=VERDICT_SCHEMA, label=f\"verify:{dimension}:{f['title']}\"))\n for f in audited[\"findings\"]])\n return {\"dimension\": dimension,\n \"confirmed\": [f for f, v in zip(audited[\"findings\"], verdicts) if v and v[\"isReal\"]]}\n\n results = await ctx.pipeline(DIMENSIONS, audit, verify)\n ...\n```\n\n## s20 からの変更点\n\n| | s20 Comprehensive Agent | s21 Workflow Runtime |\n|--|-----------|---------------------|\n| loop | 1 つ、モデル駆動 | main loop は不変。その上に決定的 orchestration を追加 |\n| 次の step を決めるもの | モデルが毎ラウンド判断 | script が orchestration flow を事前に定義 |\n| multi-agent | s06 subagent を一度だけ派遣 | script 化された、再現可能で復元可能な一括 orchestration |\n| 新しい仕組み | — | script DSL、background task、progress event、journal/resume、structured output、deterministic VM |\n\ns21 は main loop を置き換えません。tool layer に `Workflow` を公開し、背後で local workflow runtime を起動します。1 つの workflow が N 個の Agent loop を決定的に駆動します。s06 の subagent はモデルがその場で 1 回派遣し、s21 は orchestration を replay 可能な script にします。\n\n## 試してみる\n\n```bash\npython s21_workflow_runtime/code.py # review-changes を起動し、event stream を確認\npython s21_workflow_runtime/code.py resume # 前回の runId から resume。すべての agent() が journal cache に当たる\n```\n\n1 回の起動から `async_launched`、background の phase change と subagent progress、最後の `task_notification` までを観察してください。結果は task object に保存されます。resume 時はすべて cache hit するため `agents=0 tokens=0` と表示され、結果は前回と 1 byte も違いません。\n\n## 次へ\n\norchestration は Agent 能力の上にもう 1 層を加えます。main loop は個々の操作を管理し、script はチーム全体の flow を管理します。仕事が決定的で復元可能な script になると、モデルは「ラウンドごとの driver」から「script に schedule される実行 unit」へ変わります。同じ `agent()` を main loop でモデルがその場で呼ぶことも、workflow 内で script がまとめて編成することもできます。\n\n次へ: [s22 Goal Loop](/ja/s22) — Orchestration は仕事を fan-out し、main loop から離れます。次章は逆に、1 つの goal が control を main loop へ引き戻し、objective が達成されるまで turn の終了を認めません。\n\n\n" + "content": "# s21: Workflow Runtime — モデルが単一 step を決め、script が orchestration を決める\n\ns01 → ... → s19 → s20 → `s21` → [s22](/ja/s22)\n\n> *「1 回の tool_use で、バックグラウンドに一式の orchestration を走らせる」* — `Workflow` ツールが決定的で復元可能な script runtime を起動し、多数の subagent をまとめて送り出します。\n>\n> **Harness 層**: Orchestration — single-agent loop の上に、決定的な multi-agent script runtime を追加します。\n\n`code.py` は demo を決定的に保つため、`async_launched` を出した後、同じ process で完了を待ちます。常駐 background service を用意しなくても、lifecycle と journal を確認できます。\n\n---\n\ns01 から s20 まで、loop は常にモデル駆動で 1 step ずつ進みました。各ラウンドでモデルが 1 つのツールを選び、結果を `messages[]` へ入れ、次のラウンドへ進みます。open-ended なタスクには最適です。次に何をするかを、モデルが context を見てその場で決められます。\n\nしかし、複数の Agent を決定的に指揮したい仕事もあります。大きな変更の review を考えてください。10 の観点から並行して問題を探す → 各 finding へ別 Agent を送り adversarial verification を行う → 結果を集約して重複を除く → severity 順に並べる。この流れの形は固定されており、本当に必要なのは 3 つです。\n\n- **並行性**: 1 件ずつ順番に待たないこと。\n- **決定性**: 同じ入力から同じ結果構造が得られること。\n- **復元可能性**: 途中で止まっても、完了済みの部分を最初からやり直さないこと。\n\nこの流れをモデルに main loop で 1 ラウンドずつ動かさせると、遅く、結果は不確定で、中断すれば最初からです。ここで必要なのは「もう 1 turn 話す」ことではなく、orchestration をそのままコードにすることです。\n\n## 計画は chat のラウンドを重ねず、コードに書く\n\nharness の tool pool に `Workflow` ツールを追加します。ユーザーまたはモデルが渡す script は、`agent() / parallel() / pipeline() / phase()` という少数の primitive を使い、orchestration を決定的なコードとして表します。\n\nmain loop から見えるのは 1 回の `tool_use` だけで、すぐ「バックグラウンドで起動済み」という結果を受け取ります。本当の実行は background runtime で進み、進捗をリアルタイムに報告し、全過程をディスク上の journal へ記録します。script の中間結果は変数に保存され、会話履歴の場所を取りません。`resume_from_run_id` で再開すると、変更されていない `agent()` は journal cache に当たり、以前の結果を直接使って checkpoint から続行します。\n\n![Workflow Runtime Overview](/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg)\n\n```python\nSAMPLE_META = {\"name\": \"review-changes\", \"description\": \"コード変更を review\", \"phases\": [\"Review\", \"Verify\"]}\n\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n results = await ctx.pipeline(DIMENSIONS, audit, verify) # 各 dimension が独立して audit → verify を通る\n confirmed = [f for r in results if r for f in r[\"confirmed\"]]\n ctx.log(f\"{len(confirmed)} 件の実在する問題を確認\")\n return {\"confirmed\": confirmed}\n```\n\n## Workflow ツール: バックグラウンド起動、main loop には 1 回の call だけ\n\n`Workflow` は main Agent の tool pool にあります。ユーザーが保存済み workflow の実行を求めるか、タスクが既知の orchestration に一致したときにモデルがこのツールを選びます。どちらも 1 回の `Workflow(...)` tool call になります。\n\nツールは argument を parse し、meta 情報を検証し、permission check を通し、local workflow task を登録すると、すぐ「非同期で起動済み」と返します。main loop は block せず別の仕事を続け、workflow は background で実行されます。これは s13 の引換券 pattern を拡大したものです。先に引換券を渡し、結果ができたら通知します。\n\n```python\nclass WorkflowTool:\n async def call(self, meta, script_fn, args=None, resume_from_run_id=None):\n validate_meta(meta)\n check_permission(meta)\n run_id = resume_from_run_id or create_run_id(meta)\n task = LocalWorkflowTask(create_task_id(run_id), run_id, meta)\n task.event(\"async_launched\", runId=run_id, taskId=task.task_id) # すぐ return\n ... # 残りはバックグラウンドで進む\n```\n\n## Workflow metadata: 起動前に検証する\n\n各 workflow は `name`、`description`、任意の `phases` を持つ metadata object を登録します。runtime は workflow code を実行する前に検証します。`name` と `description` は task と UI の表示に使い、`phases` は progress bar の group 名を定義します。\n\n不正な入力はすぐ `WorkflowInputError` になり、登録時に止まります。s14 の cron 式検証と同じ考えです。不正な script が実行時まで進んでから壊れないようにします。\n\n教材 runtime は `meta.name` をローカル artifact のファイル名に使うため、英数字で始まり、英数字、`.`、`_`、`-` のみからなる 1-64 文字の安全な slug も要求する。\n\n```python\ndef validate_meta(meta):\n if not isinstance(meta, dict):\n raise WorkflowInputError(\"meta は object literal でなければなりません\")\n if not meta.get(\"name\") or not meta.get(\"description\"):\n raise WorkflowInputError(\"meta には name と description が必要です\")\n if not isinstance(meta[\"name\"], str) or not WORKFLOW_NAME_RE.fullmatch(meta[\"name\"]):\n raise WorkflowInputError(\"meta.name は安全な 1-64 文字の slug が必要です\")\n if \"phases\" in meta and (\n not isinstance(meta[\"phases\"], list)\n or not all(isinstance(p, str) and p for p in meta[\"phases\"])\n ):\n raise WorkflowInputError(\"meta.phases は空でない文字列だけを含む必要があります\")\n return meta\n```\n\n## Orchestration primitive: この少数だけで、すべての flow を書ける\n\nscript は独立した context で動き、global variable として使えるのは少数の orchestration primitive だけです。script 自身はファイルを直接読み書きせず、shell も実行しません。実際のコード操作は、派遣された subagent が自分の tool permission で行います。primitive はすべて `ExecutionState` の method です。\n\n| Primitive | 役割 |\n|------|------|\n| `agent(prompt, {schema, label, phase})` | 1 つの subagent を派遣 |\n| `parallel(thunks)` | **barrier**: すべての task を並行実行し、全結果が戻るまで待つ |\n| `pipeline(items, *stages)` | 各 item を **barrier なし**で stage ごとに実行し、終わった item から先へ進める |\n| `phase(title)` | 現在の progress phase を記録し、progress bar を更新 |\n| `log(message)` | progress log を 1 行出力 |\n| `workflow(name, args)` | nested sub-workflow（1 階層だけ） |\n\n既定では `pipeline` を使うべきです。各 item がすべての stage を独立して通り、item A が stage 3 にいる間、item B はまだ stage 1 かもしれません。次の stage へ進むために前 stage の全結果が本当に必要なときだけ、`parallel` barrier を使います。barrier は最も遅い task を待つため、不要なら置かないでください。\n\n```python\nasync def pipeline(self, items, *stages):\n async def run_item(item, idx):\n value = item\n for stage in stages: # 各 item がすべての stage を独立して完走\n value = await stage(value, item, idx)\n return value\n return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])\n```\n\n## 構造化出力: Subagent に散文を返させない\n\n`agent({schema})` は、schema に一致する JSON object を subagent に要求します。内部では structured output call を 1 回使い、runtime が結果を schema で検証し、不一致なら 1 回 retry します。下流コードが受け取るのは規則的な object であり、再 parse が必要な長文ではありません。\n\ns05 では tool argument を全面的に信頼できないと説明しました。ここでは同じ教訓を逆向きに使います。subagent の出力も全面的には信頼できません。orchestration boundary で検証し、1 回 retry の機会を与え、不確実性を後続 flow の外へ止めます。\n\n```python\nresult = self.runner.run(prompt, schema, label)\nif schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok: # 1 回だけ注意して retry、それでも不正なら error\n result = self.runner.run(prompt + \"\\n\\n有効な JSON を返してください。\", schema, label)\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok:\n raise WorkflowInputError(f\"agent({{schema}}) の出力が不正です: {err}\")\n```\n\n## Background task と progress event\n\n`LocalWorkflowTask` は status と token usage を管理し、SDK style の event stream を外へ出します。`task_started` → phase change、subagent start、log batch を含む一連の `task_progress` → 完了、失敗、停止に加え、output file、token 数、tool call 数、所要時間を含む最後の `task_notification` です。\n\nmain session は通常 event として処理し、最後の完了通知だけが main loop へ再び入ります。\n\n```python\nclass LocalWorkflowTask:\n def progress_event(self, ptype, **data): # phase/subagent/log\n self.progress.append({\"type\": ptype, **data})\n print(f\" progress {ptype} ...\")\n```\n\n## 保存: Snapshot + journal で中断から再開する\n\nこの最小 runtime は各 run を `s21_workflow_runtime/.runtime/` に保存します。`.json` snapshot、`.output.json` output、`.journal.jsonl` journal です。production harness では workflow script や subagent transcript も保存できますが、snapshot と journal が安定した `runId` を共有することが重要です。\n\njournal は checkpoint resume の中心で、各 `agent()` の結果を 1 行ずつ記録します。\n\n```python\nclass WorkflowJournal:\n def record(self, key, value):\n self._f.write(json.dumps({\"key\": key, \"value\": value}) + \"\\n\")\n self._f.flush()\n self.cache[key] = value\n```\n\n## Resume: runId から続行し、変更のないものを再利用する\n\n`resume_from_run_id` を渡して workflow を再度呼ぶと script を再実行しますが、各 `agent()` は決定的な semantic key を計算します。journal に key があれば、再実行せず cached result を返します。変更された call と、それに依存する後続 step だけが本当に動きます。\n\nkey は concurrency の完了順に依存してはいけません。`parallel` と `pipeline` の Agent は不定の順番で完了します。「何番目に完了したか」を key にすると、次回の cache が別の call へ対応してしまいます。そのため key は競合する counter ではなく、call の内容、つまり type、label、prompt、schema の stable hash です。\n\n```python\ndef key(self, kind, label, prompt, schema):\n basis = f\"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}\"\n return f\"{kind}-{_stable_hash(basis) % 10**10:010d}\"\n\n# agent() の内部:\ncached = self.journal.cached(key)\nif cached is not MISS:\n self.task.progress_event(\"workflow_agent\", label=label, status=\"cached\")\n return cached\n```\n\n## 決定性: Resume に意味を持たせる再現性\n\nresume が動くには、workflow が再現可能でなければなりません。この最小 Python runtime は stable hash と決定的な mock runner を使い、同じ workflow + 同じ argument から同じ key を作ります。production harness では workflow code も隔離し、制御されていない clock、randomness、filesystem access などの非決定的な source を除くべきです。\n\n## 実際に動かす\n\nsample workflow `review-changes` は `pipeline` を使い、各 review dimension を独立して audit → verify へ通します。audit では schema 付き `agent()` が問題を探し、verify では `parallel()` が各 finding に別の adversarial verification subagent を送ります。実在すると確認された問題だけを残し、severity 順に並べます。\n\n```python\nasync def sample_workflow(ctx, args):\n ctx.phase(\"Review\")\n\n async def audit(_v, dimension, _i):\n out = await ctx.agent(f\"変更されたコードに {dimension} 関連の問題がないか確認してください\",\n schema=FINDINGS_SCHEMA, label=f\"audit:{dimension}\", phase=\"Review\")\n return {\"dimension\": dimension, \"findings\": out[\"findings\"]}\n\n async def verify(audited, dimension, _i):\n ctx.phase(\"Verify\")\n verdicts = await ctx.parallel([ # 各 finding を独立して verify\n (lambda f=f: ctx.agent(f\"この問題が実在するか adversarial に検証してください: {f['title']}\",\n schema=VERDICT_SCHEMA, label=f\"verify:{dimension}:{f['title']}\"))\n for f in audited[\"findings\"]])\n return {\"dimension\": dimension,\n \"confirmed\": [f for f, v in zip(audited[\"findings\"], verdicts) if v and v[\"isReal\"]]}\n\n results = await ctx.pipeline(DIMENSIONS, audit, verify)\n ...\n```\n\n## s20 からの変更点\n\n| | s20 Comprehensive Agent | s21 Workflow Runtime |\n|--|-----------|---------------------|\n| loop | 1 つ、モデル駆動 | main loop は不変。その上に決定的 orchestration を追加 |\n| 次の step を決めるもの | モデルが毎ラウンド判断 | script が orchestration flow を事前に定義 |\n| multi-agent | s06 subagent を一度だけ派遣 | script 化された、再現可能で復元可能な一括 orchestration |\n| 新しい仕組み | — | script DSL、background task、progress event、journal/resume、structured output、deterministic VM |\n\ns21 は main loop を置き換えません。tool layer に `Workflow` を公開し、背後で local workflow runtime を起動します。1 つの workflow が N 個の Agent loop を決定的に駆動します。s06 の subagent はモデルがその場で 1 回派遣し、s21 は orchestration を replay 可能な script にします。\n\n## 試してみる\n\n```bash\npython s21_workflow_runtime/code.py # review-changes を起動し、event stream を確認\npython s21_workflow_runtime/code.py resume # 前回の runId から resume。すべての agent() が journal cache に当たる\n```\n\n1 回の起動から `async_launched`、background の phase change と subagent progress、最後の `task_notification` までを観察してください。結果は task object に保存されます。resume 時はすべて cache hit するため `agents=0 tokens=0` と表示され、結果は前回と 1 byte も違いません。\n\n## 次へ\n\norchestration は Agent 能力の上にもう 1 層を加えます。main loop は個々の操作を管理し、script はチーム全体の flow を管理します。仕事が決定的で復元可能な script になると、モデルは「ラウンドごとの driver」から「script に schedule される実行 unit」へ変わります。同じ `agent()` を main loop でモデルがその場で呼ぶことも、workflow 内で script がまとめて編成することもできます。\n\n次へ: [s22 Goal Loop](/ja/s22) — Orchestration は仕事を fan-out し、main loop から離れます。次章は逆に、1 つの goal が control を main loop へ引き戻し、objective が達成されるまで turn の終了を認めません。\n\n\n" }, { "version": "s22", "locale": "en", "title": "s22: Goal Loop — The Goal Decides When to Stop, Not the Model", - "content": "# s22: Goal Loop — The Goal Decides When to Stop, Not the Model\n\ns01 → ... → s20 → s21 → `s22`\n\n> *\"A turn ends only when the goal condition is satisfied, not merely when the model says stop\"* — `/goal` adds a gate at the end of every main-loop turn. An independent evaluator checks whether trusted evidence is sufficient; if not, it pushes the model into another round.\n>\n> **Harness layer**: Goal closure — a program-controlled completion gate at the end of each turn.\n\n> **Source boundary:** Product details in this chapter are a clean-room behavioral reconstruction of Claude Code 2.1.177. Names and limits may change in later releases; `code.py` is an offline teaching model, not copied product source.\n\n---\n\nFrom s01 through s21, how does a conversation turn end? When the model stops emitting `tool_use`, the loop simply executes `return`. That is fine for one-shot work: finish and stop.\n\nSome objectives, however, must be carried through to completion: \"get the tests passing\" or \"do not stop until the deployment succeeds.\" Two problems appear often. The model does half the work, decides it is close enough, and stops. Worse, it says `tests passed` and tries to declare victory. The requirement is simple: the model cannot decide by itself whether the turn may end. An explicit condition must be evaluated against concrete evidence.\n\nThis thread was present from the first chapter. s01 explained that exiting the loop is a model decision. s04's Stop hook gave the program veto power for the first time. This chapter turns that veto into a complete loop with three indispensable parts: condition, evidence, and budget.\n\n## /goal: Add a Gate at the End of Every Turn\n\nEntering `/goal ` sets a session-scoped stopping condition. The program stores it as the active goal. After each turn, an independent lightweight model acts as evaluator and checks whether trusted evidence in the transcript satisfies the condition. If evidence is insufficient, the gate blocks the attempted stop and queues a \"keep working\" prompt for the next round. If it is sufficient, the goal is cleared and marked complete.\n\n![Goal Loop Overview](/course-assets/s22_goal_loop/goal-loop-overview.svg)\n\nCompared with the s01 loop, there is only one additional decision: when the model wants to stop, it must first pass the goal gate.\n\n```python\n# s01: stop when the model says stop\nif not has_tool_use(response):\n return\n# s22: want to stop? Pass the goal gate first\nif not has_tool_use(response):\n verdict = goal.evaluate_after_turn()\n if verdict == \"continuing\":\n continue # Not achieved -> push back for another round\n return # Achieved / over budget / no goal -> really stop\n```\n\nThe program controls this gate. It is not the model restraining itself. The model does not even know the gate exists; it simply receives another round of input and continues working.\n\n> In the real Claude Code, `/goal` is a session-scoped Stop hook governed by workspace trust and hook restrictions. The code contains markers such as `active_goal`, `goal_status`, `goal_met`, and `tengu_goal_achieved`.\n\n## Setting a Goal: Evidence Starts after the Command\n\n`set_goal` stores an active goal containing the objective text, a maximum-turn budget, counters, and `start_index`, the beginning of the evidence window. It uses the transcript's current length, placing the `/goal` command itself outside the window. This is the first defense: a command cannot prove its own completion.\n\n```python\ndef set_goal(self, objective, max_turns=20):\n self.active = {\n \"objective\": objective, \"status\": \"active\",\n \"start_index\": len(self.transcript), # Evidence starts here; the command is outside the window\n \"max_turns\": max_turns, \"checks\": 0, \"continuation_turns\": 0,\n }\n```\n\n> In the real Claude Code, `GoalRuntime.setGoal()` stores the active goal, start position, counters, and budget, then `resetEvidenceStart()` aligns the window to the position after command submission.\n\n## The Evaluator: Trust Concrete Evidence Only\n\nThis is the core of the entire mechanism. The evaluator does not inspect the whole conversation. It sees only messages inside the evidence window that come from trusted sources. Three filters keep every form of \"I said it was done, so it must be done\" outside:\n\n```python\nTRUSTED_EVIDENCE_ORIGINS = {\"task-notification\", \"monitor-line\"}\n\ndef evidence_text(self):\n out = []\n for m in self.transcript[self.active[\"start_index\"]:]:\n if m.origin.get(\"kind\") == \"slash-command\": # 1 Slash commands are not evidence\n continue\n if m.role == \"user\" and m.content.strip().startswith(\"/goal\"): # 2 /goal command text is not evidence\n continue\n if m.origin.get(\"kind\") not in TRUSTED_EVIDENCE_ORIGINS: # 3 Trust only approved origins\n continue\n out.append(f\"{m.role}: {m.content}\")\n return \"\\n\".join(out)\n```\n\nThe effect is clear. The same sentence, `tests passed`, does not count when typed by you, but does count when delivered by a background task notification. The model cannot bluff its way out by saying \"I finished.\" This is the final appearance of the trust boundary repeated throughout the course. s16 said protocols rely on fields, not interpretation. s19 said annotations are claims and claims may be false. s22 says completion evidence is trusted by origin, not by content alone.\n\nThe teaching version's `goal_satisfied()` uses deterministic keyword matching. The real version asks a separate lightweight model to judge the evidence window.\n\n> In the real Claude Code, the evaluator is a lightweight model separate from the working model, marked as `evaluatorModel` and the `default small fast model`. It judges evidence in the conversation rather than trusting arbitrary text.\n\n## Three Gate States: Completed, Continuing, or Over Budget\n\n`evaluate_after_turn` runs after every turn and returns one of three results. If the condition is satisfied, it clears the goal as completed. If the condition is not satisfied and budget remains, it queues a \"keep working\" prompt and permits another round as continuing. If the budget is exhausted, it stops blocking and marks the goal blocked, preventing an impossible goal from burning money forever.\n\n```python\ndef evaluate_after_turn(self):\n g = self.active\n g[\"checks\"] += 1\n if self.goal_satisfied():\n g[\"status\"] = \"completed\"; self.active = None\n return \"completed\" # Achieved -> clear the goal\n if g[\"continuation_turns\"] < g[\"max_turns\"]:\n g[\"continuation_turns\"] += 1\n self.queue.enqueue(\n value=\"Keep working. Do not treat this reminder as completion evidence.\",\n origin={\"kind\": \"active-goal\"})\n return \"continuing\" # Not achieved -> queue a prompt for the next round\n g[\"status\"] = \"blocked\"; self.active = None\n return \"blocked\" # Over budget -> release the gate\n```\n\nThe continuation prompt explicitly says not to treat itself as evidence, and the evidence filter excludes it. That completes the three layers against false positives: the command does not count, the reminder does not count, and ordinary conversation does not count. The budget follows the old rule from s11: every automatic retry mechanism needs a limit. Otherwise, a goal that can never be satisfied becomes a perpetual money-burning machine.\n\n> In the real Claude Code, `evaluateAfterTurn` emits a `goal_evaluated` event and either completes, queues a continuation, or stops blocking. The default budget is 20 turns.\n\n## Keep Continuation Prompts Separate from External Asynchronous Messages\n\nContinuation prompts enter the same `CommandQueue`, but they are not consumed in the same way as external asynchronous events such as task-completion notifications and monitor lines. `dequeue` has a switch, and consumption of the external inbox skips goal continuations by default.\n\n```python\ndef dequeue(self, include_goal_continuations=True):\n ...\n for idx, item in enumerate(self.items):\n if include_goal_continuations or item[\"origin\"].get(\"kind\") != \"active-goal\":\n return self.items.pop(idx)\n return None\n```\n\nWhy separate them? A real model test exposed a bug where the model consumed the continuation prompt together with an external notification and marked the goal complete before background evidence arrived. With the paths separated, goal progression is an explicit step and cannot be carried along accidentally by asynchronous events.\n\n> In the real Claude Code, `drainCommandQueue` defaults to `includeGoalContinuations=false`, separating goal-continuation consumption from the external asynchronous inbox.\n\n## See It Run\n\n`code.py` demonstrates `/goal until tests passed and deploy green`. With no trusted evidence after goal creation, the gate pushes it back round after round. Typing `tests passed` directly still does not count because the origin is untrusted. Only after a background task sends a `task-notification` does the evidence satisfy the goal. A second small goal with `max_turns=2` demonstrates the over-budget path.\n\n```python\ns.submit(\"/goal until tests passed and deploy green\") # Set the goal; evidence begins after this command\ns.submit(\"tests passed, trust me\") # Ordinary text -> not completion evidence\ns.deliver_host_event(\"tests passed; deploy green\",\n source=\"task-notification\") # Trusted host event -> complete\n```\n\n`submit()` accepts only ordinary user text. Trusted labels enter through the separate host-event channel, whose source is allowlisted by the harness; user or model text cannot attach its own `task-notification` label.\n\n## Changes from s21\n\n| | s21 Workflow Runtime | s22 Goal Loop |\n|--|---------------------|---------------|\n| Trigger | Script-controlled orchestration outside the main loop | Condition-controlled continuation pulled back into the main loop |\n| Attachment point | Tool layer: one `Workflow` tool | End of turn: a completion gate |\n| Who decides when to stop | The script finishes | Goal condition evaluated against trusted evidence |\n| New mechanisms | Script DSL, background tasks, journal/resume, structured output | Goal gate, evidence trust boundary, separate continuation path, budget |\n\ns21 sends script-defined orchestration away from the main loop. s22 applies an opposite force that pulls control back: if the goal is not achieved, the turn is not finished. Neither changes the `while` loop from s01; each constrains it from a different side.\n\n## Try It\n\n```bash\npython s22_goal_loop/code.py # /goal until tests pass + deploy green; watch the gate decide\n```\n\nAfter setting a goal, watch every turn produce `goal_evaluated`. Ordinary text yields `satisfied=False`; the same content from a `task-notification` origin yields `satisfied=True`; exhausted budget produces `goal_blocked`. The same `tests passed` sentence has opposite results depending on its origin. That is why an empty claim cannot fool `/goal`.\n\n## Next\n\n`/goal` is one kind of trigger that pulls control back into the main loop: condition control. It pairs naturally with s21's orchestration outside the main loop, one dispatching work outward and the other pulling control inward. Beyond them are time-controlled re-entry through `/loop` and cron, and event-controlled re-entry through `Monitor`; all share the same task and notification foundation. But the essential gate is already here: **the model's words do not decide whether to stop. The goal must judge trusted evidence.**\n\n\n" + "content": "# s22: Goal Loop — The Goal Decides When to Stop, Not the Model\n\ns01 → ... → s20 → s21 → `s22`\n\n> *\"A turn ends only when the goal condition is satisfied, not merely when the model says stop\"* — `/goal` adds a gate at the end of every main-loop turn. An independent evaluator checks whether trusted evidence is sufficient; if not, it pushes the model into another round.\n>\n> **Harness layer**: Goal closure — a program-controlled completion gate at the end of each turn.\n\n---\n\nFrom s01 through s21, how does a conversation turn end? When the model stops emitting `tool_use`, the loop simply executes `return`. That is fine for one-shot work: finish and stop.\n\nSome objectives, however, must be carried through to completion: \"get the tests passing\" or \"do not stop until the deployment succeeds.\" Two problems appear often. The model does half the work, decides it is close enough, and stops. Worse, it says `tests passed` and tries to declare victory. The requirement is simple: the model cannot decide by itself whether the turn may end. An explicit condition must be evaluated against concrete evidence.\n\nThis thread was present from the first chapter. s01 explained that exiting the loop is a model decision. s04's Stop hook gave the program veto power for the first time. This chapter turns that veto into a complete loop with three indispensable parts: condition, evidence, and budget.\n\n## /goal: Add a Gate at the End of Every Turn\n\nEntering `/goal ` sets a session-scoped stopping condition. The program stores it as the active goal. After each turn, an evaluator checks whether trusted evidence in the transcript satisfies the condition. If evidence is insufficient, the gate blocks the attempted stop and queues a \"keep working\" prompt for the next round. If it is sufficient, the goal is cleared and marked complete.\n\n![Goal Loop Overview](/course-assets/s22_goal_loop/goal-loop-overview.svg)\n\nCompared with the s01 loop, there is only one additional decision: when the model wants to stop, it must first pass the goal gate.\n\n```python\n# s01: stop when the model says stop\nif not has_tool_use(response):\n return\n# s22: want to stop? Pass the goal gate first\nif not has_tool_use(response):\n verdict = goal.evaluate_after_turn()\n if verdict == \"continuing\":\n continue # Not achieved -> push back for another round\n return # Achieved / over budget / no goal -> really stop\n```\n\nThe program controls this gate. It is not the model restraining itself. The model does not even know the gate exists; it simply receives another round of input and continues working.\n\n## Setting a Goal: Evidence Starts after the Command\n\n`set_goal` stores an active goal containing the objective text, a maximum-turn budget, counters, and `start_index`, the beginning of the evidence window. It uses the transcript's current length, placing the `/goal` command itself outside the window. This is the first defense: a command cannot prove its own completion.\n\n```python\ndef set_goal(self, objective, max_turns=20):\n self.active = {\n \"objective\": objective, \"status\": \"active\",\n \"start_index\": len(self.transcript), # Evidence starts here; the command is outside the window\n \"max_turns\": max_turns, \"checks\": 0, \"continuation_turns\": 0,\n }\n```\n\n## The Evaluator: Trust Concrete Evidence Only\n\nThis is the core of the entire mechanism. The evaluator does not inspect the whole conversation. It sees only messages inside the evidence window that come from trusted sources. Three filters keep every form of \"I said it was done, so it must be done\" outside:\n\n```python\nTRUSTED_EVIDENCE_ORIGINS = {\"task-notification\", \"monitor-line\"}\n\ndef evidence_text(self):\n out = []\n for m in self.transcript[self.active[\"start_index\"]:]:\n if m.origin.get(\"kind\") == \"slash-command\": # 1 Slash commands are not evidence\n continue\n if m.role == \"user\" and m.content.strip().startswith(\"/goal\"): # 2 /goal command text is not evidence\n continue\n if m.origin.get(\"kind\") not in TRUSTED_EVIDENCE_ORIGINS: # 3 Trust only approved origins\n continue\n out.append(f\"{m.role}: {m.content}\")\n return \"\\n\".join(out)\n```\n\nThe effect is clear. The same sentence, `tests passed`, does not count when typed by you, but does count when delivered by a background task notification. The model cannot bluff its way out by saying \"I finished.\" This is the final appearance of the trust boundary repeated throughout the course. s16 said protocols rely on fields, not interpretation. s19 said annotations are claims and claims may be false. s22 says completion evidence is trusted by origin, not by content alone.\n\nThe minimal `goal_satisfied()` uses deterministic keyword matching so the demo stays offline and reproducible. A production harness can replace this policy with a separate lightweight evaluator model, while keeping the same trusted evidence boundary.\n\n## Three Gate States: Completed, Continuing, or Over Budget\n\n`evaluate_after_turn` runs after every turn and returns one of three results. If the condition is satisfied, it clears the goal as completed. If the condition is not satisfied and budget remains, it queues a \"keep working\" prompt and permits another round as continuing. If the budget is exhausted, it stops blocking and marks the goal blocked, preventing an impossible goal from burning money forever.\n\n```python\ndef evaluate_after_turn(self):\n g = self.active\n g[\"checks\"] += 1\n if self.goal_satisfied():\n g[\"status\"] = \"completed\"; self.active = None\n return \"completed\" # Achieved -> clear the goal\n if g[\"continuation_turns\"] < g[\"max_turns\"]:\n g[\"continuation_turns\"] += 1\n self.queue.enqueue(\n value=\"Keep working. Do not treat this reminder as completion evidence.\",\n origin={\"kind\": \"active-goal\"})\n return \"continuing\" # Not achieved -> queue a prompt for the next round\n g[\"status\"] = \"blocked\"; self.active = None\n return \"blocked\" # Over budget -> release the gate\n```\n\nThe continuation prompt explicitly says not to treat itself as evidence, and the evidence filter excludes it. That completes the three layers against false positives: the command does not count, the reminder does not count, and ordinary conversation does not count. The budget follows the old rule from s11: every automatic retry mechanism needs a limit. Otherwise, a goal that can never be satisfied becomes a perpetual money-burning machine.\n\n## Keep Continuation Prompts Separate from External Asynchronous Messages\n\nContinuation prompts enter the same `CommandQueue`, but they are not consumed in the same way as external asynchronous events such as task-completion notifications and monitor lines. `dequeue` has a switch, and consumption of the external inbox skips goal continuations by default.\n\n```python\ndef dequeue(self, include_goal_continuations=True):\n ...\n for idx, item in enumerate(self.items):\n if include_goal_continuations or item[\"origin\"].get(\"kind\") != \"active-goal\":\n return self.items.pop(idx)\n return None\n```\n\nWhy separate them? If one consumer drains continuation prompts together with external notifications, a reminder can be mistaken for new evidence before the background result arrives. With the paths separated, goal progression is an explicit step and cannot be carried along accidentally by asynchronous events.\n\n## See It Run\n\n`code.py` demonstrates `/goal until tests passed and deploy green`. With no trusted evidence after goal creation, the gate pushes it back round after round. Typing `tests passed` directly still does not count because the origin is untrusted. Only after a background task sends a `task-notification` does the evidence satisfy the goal. A second small goal with `max_turns=2` demonstrates the over-budget path.\n\n```python\ns.submit(\"/goal until tests passed and deploy green\") # Set the goal; evidence begins after this command\ns.submit(\"tests passed, trust me\") # Ordinary text -> not completion evidence\ns.deliver_host_event(\"tests passed; deploy green\",\n source=\"task-notification\") # Trusted host event -> complete\n```\n\n`submit()` accepts only ordinary user text. Trusted labels enter through the separate host-event channel, whose source is allowlisted by the harness; user or model text cannot attach its own `task-notification` label.\n\n## Changes from s21\n\n| | s21 Workflow Runtime | s22 Goal Loop |\n|--|---------------------|---------------|\n| Trigger | Script-controlled orchestration outside the main loop | Condition-controlled continuation pulled back into the main loop |\n| Attachment point | Tool layer: one `Workflow` tool | End of turn: a completion gate |\n| Who decides when to stop | The script finishes | Goal condition evaluated against trusted evidence |\n| New mechanisms | Script DSL, background tasks, journal/resume, structured output | Goal gate, evidence trust boundary, separate continuation path, budget |\n\ns21 sends script-defined orchestration away from the main loop. s22 applies an opposite force that pulls control back: if the goal is not achieved, the turn is not finished. Neither changes the `while` loop from s01; each constrains it from a different side.\n\n## Try It\n\n```bash\npython s22_goal_loop/code.py # /goal until tests pass + deploy green; watch the gate decide\n```\n\nAfter setting a goal, watch every turn produce `goal_evaluated`. Ordinary text yields `satisfied=False`; the same content from a `task-notification` origin yields `satisfied=True`; exhausted budget produces `goal_blocked`. The same `tests passed` sentence has opposite results depending on its origin. That is why an empty claim cannot fool `/goal`.\n\n## Next\n\n`/goal` is one kind of trigger that pulls control back into the main loop: condition control. It pairs naturally with s21's orchestration outside the main loop, one dispatching work outward and the other pulling control inward. Beyond them are time-controlled re-entry through `/loop` and cron, and event-controlled re-entry through `Monitor`; all share the same task and notification foundation. But the essential gate is already here: **the model's words do not decide whether to stop. The goal must judge trusted evidence.**\n\n\n" }, { "version": "s22", "locale": "zh", "title": "s22: Goal Loop — 什么时候停，目标说了算，不是模型说了算", - "content": "# s22: Goal Loop — 什么时候停，目标说了算，不是模型说了算\n\ns01 → ... → s20 → s21 → `s22`\n\n> *\"一轮能不能结束，看目标条件满不满足，不是模型说停就停\"* — `/goal` 在主循环每轮收尾的地方加一道闸门：每轮结束后，一个独立的判断器看可信证据够不够，不够就把模型推回去再来一轮。\n>\n> **Harness 层**: 目标闭环 — 在轮次收尾处，加一道程序控制的完成闸门。\n\n> **来源边界：** 本章产品细节来自对 Claude Code 2.1.177 的 clean-room 行为重建。后续版本可能更改名称与限制；`code.py` 是离线教学模型，不是产品源码复制。\n\n---\n\n从 s01 到 s21，一轮对话怎么结束？模型不再发 `tool_use`，循环就直接 `return` 了。一次性任务这么干没问题，做完就停。\n\n但有些目标你得盯着它做到底：\"把测试跑过\"、\"部署成功了再说\"。这时候经常出两种问题：模型做了一半觉得差不多了，自己就停了；更过分的是，它嘴上说一句 `tests passed` 就想收工。你要的其实很简单：这一轮能不能结束，不能模型自己说了算，得有个明确的条件，对着实打实的证据来判断。\n\n这条线其实从第一课就埋着了。s01 说过，退出循环本来是模型的一个决定；s04 的 Stop hook 第一次给了程序否决权。这一课把那个否决权做成完整的闭环：条件、证据、预算，三样缺一不可。\n\n## /goal：每轮收尾加一道闸门\n\n输入 `/goal <条件>` 就设了一个会话级的停止条件。程序把它存成当前活跃目标，每轮结束后，用一个独立的轻量小模型当判断器，看对话记录里的可信证据够不够满足条件。不够，闸门就把这次结束拦住，塞一条\"继续干\"的提示进下一轮；够了，就清除目标，标记完成。\n\n![Goal Loop 总览](/course-assets/s22_goal_loop/goal-loop-overview.svg)\n\n和 s01 的循环比，只多了一道判断，模型想停的时候先过目标这关：\n\n```python\n# s01：模型说停就停\nif not has_tool_use(response):\n return\n# s22：想停？先过目标闸门\nif not has_tool_use(response):\n verdict = goal.evaluate_after_turn()\n if verdict == \"continuing\":\n continue # 没达成 -> 推回去再来一轮\n return # 达成/超预算/没目标 -> 真停\n```\n\n这道闸门是程序自己控制的。不是模型自己约束自己，模型甚至不知道有这么一道闸门，它只是收到了下一轮的输入，接着干就是了。\n\n> 真实 Claude Code：`/goal` 是会话级的 Stop hook，受工作区信任和 hook 限制控制；代码里有 `active_goal`、`goal_status`、`goal_met`、`tengu_goal_achieved` 这些标记。\n\n## 设目标：证据从命令之后开始算\n\n`set_goal` 会存一个活跃目标：目标文本、最大轮数预算、计数器，还有 `start_index`——也就是证据窗口的起点。它取当前对话记录的长度，所以 `/goal` 这行命令本身在窗口外面。这是第一道防线：命令自己不能证明自己完成了。\n\n```python\ndef set_goal(self, objective, max_turns=20):\n self.active = {\n \"objective\": objective, \"status\": \"active\",\n \"start_index\": len(self.transcript), # 证据窗口从这里开始；命令本身在窗口外\n \"max_turns\": max_turns, \"checks\": 0, \"continuation_turns\": 0,\n }\n```\n\n> 真实 Claude Code：`GoalRuntime.setGoal()` 存活跃目标、起始位置、计数器和预算；提交后再 `resetEvidenceStart()` 把窗口对齐到命令之后。\n\n## 判断器：只信实打实的证据\n\n这是整个机制最核心的地方。判断器不看整段对话，只看证据窗口里来自可信来源的消息。三层过滤，把\"嘴上说完成了但不算数\"的内容全挡在外面：\n\n```python\nTRUSTED_EVIDENCE_ORIGINS = {\"task-notification\", \"monitor-line\"}\n\ndef evidence_text(self):\n out = []\n for m in self.transcript[self.active[\"start_index\"]:]:\n if m.origin.get(\"kind\") == \"slash-command\": # 1 斜杠命令本身不算\n continue\n if m.role == \"user\" and m.content.strip().startswith(\"/goal\"): # 2 /goal 命令文本不算\n continue\n if m.origin.get(\"kind\") not in TRUSTED_EVIDENCE_ORIGINS: # 3 只信可信来源\n continue\n out.append(f\"{m.role}: {m.content}\")\n return \"\\n\".join(out)\n```\n\n效果很明显：同样一句 `tests passed`，你打字说的不算，后台任务通知带回来的才算。模型糊弄不过去，它没法靠自己说一句\"我做完了\"就把目标判成完成。这是全课程反复出现的那条信任边界的最后一次登场：s16 说协议靠字段不靠理解，s19 说注解是申报、申报可以撒谎，s22 说完成证据只看来源不看内容。\n\n教学版里 `goal_satisfied()` 是确定的关键词匹配；真实版会把证据窗口交给一个轻量小模型来判断。\n\n> 真实 Claude Code：判断器是和干活的模型分开的轻量小模型（标记是 `evaluatorModel`、`default small fast model`），判断对话里的证据，不是随便什么文本都信。\n\n## 闸门三态：完成/继续/超预算\n\n`evaluate_after_turn` 每轮跑一次，三种结果：满足条件就清除目标（completed）；没满足而且预算还没花完，就往队列塞一条\"继续干\"的提示，放行下一轮（continuing）；预算花完就停（blocked），别让一个永远判不出来的目标无限烧钱。\n\n```python\ndef evaluate_after_turn(self):\n g = self.active\n g[\"checks\"] += 1\n if self.goal_satisfied():\n g[\"status\"] = \"completed\"; self.active = None\n return \"completed\" # 达成 -> 清除目标\n if g[\"continuation_turns\"] < g[\"max_turns\"]:\n g[\"continuation_turns\"] += 1\n self.queue.enqueue(\n value=\"继续干活，别把这条提醒当成完成证据。\",\n origin={\"kind\": \"active-goal\"})\n return \"continuing\" # 没达成 -> 塞提示，下一轮\n g[\"status\"] = \"blocked\"; self.active = None\n return \"blocked\" # 超预算 -> 放行，不再拦\n```\n\n那条\"继续干\"的提示里特意写了\"别把这条提醒当成完成证据\"，连提醒本身都被排除在证据之外。三层防误判就齐了：命令文本不算、提醒文本不算、普通聊天文本不算。预算则是 s11 教过的老规矩：任何自动重试的机制都得有上限，不然一个永远判不满足的目标就是个烧钱的永动机。\n\n> 真实 Claude Code：`evaluateAfterTurn` 会发 `goal_evaluated` 事件，按结果完成/塞继续提示/拦截；默认预算是 20 轮。\n\n## 继续提示和外部异步消息分开走\n\n继续提示进的是同一个 `CommandQueue`，但它和外部异步事件（任务完成通知、监控行）不是同一种消费方式。`dequeue` 带个开关：消费外部收件箱的时候，默认跳过目标的继续提示。\n\n```python\ndef dequeue(self, include_goal_continuations=True):\n ...\n for idx, item in enumerate(self.items):\n if include_goal_continuations or item[\"origin\"].get(\"kind\") != \"active-goal\":\n return self.items.pop(idx)\n return None\n```\n\n为什么要分开？真实模型测试的时候出过一个 bug：模型把继续提示当成外部通知一起消费了，结果后台证据还没到，就提前把目标判成完成了。分开之后，目标的推进是显式的一步，不会被异步事件带着走。\n\n> 真实 Claude Code：`drainCommandQueue` 默认 `includeGoalContinuations=false`，把目标继续提示和外部异步收件箱的消费分开。\n\n## 跑起来看看\n\n`code.py` 演示了一个 `/goal until tests passed and deploy green`：设了目标之后没有可信证据，闸门一轮轮把它推回去；你直接打 `tests passed` 也不算（来源不可信）；直到后台任务发来 `task-notification`，证据到位，才标记完成。还加了一个 `max_turns=2` 的小目标演示超预算拦截。\n\n```python\ns.submit(\"/goal until tests passed and deploy green\") # 设目标，窗口在命令之后\ns.submit(\"tests passed, trust me\") # 普通文本 -> 不算完成\ns.deliver_host_event(\"tests passed; deploy green\",\n source=\"task-notification\") # 可信宿主事件 -> 完成\n```\n\n`submit()` 只接受普通用户文本。可信标签必须走独立的宿主事件通道，来源由 harness 白名单校验；用户或模型文本不能给自己贴上 `task-notification` 标签。\n\n## 相对 s21 的变更\n\n| | s21 Workflow Runtime | s22 Goal Loop |\n|--|---------------------|---------------|\n| 触发方式 | 脚本控制的编排（脱离主循环） | 条件控制的继续（拉回主循环） |\n| 加在哪 | 工具层：一个 `Workflow` 工具 | 轮次收尾：一道完成闸门 |\n| 谁决定停 | 脚本跑完就停 | 目标条件对着可信证据判 |\n| 新增机制 | 脚本 DSL、后台任务、journal/续跑、结构化输出 | 目标闸门、证据信任边界、继续提示分流、预算 |\n\ns21 是把编排写成脚本、派出去脱离主循环；s22 反过来，是一股力量把控制权重拉回主循环：目标没达成，这一轮就不算结束。两个都不改 s01 那个 `while` 循环，只是从两头给它加约束。\n\n## 试一下\n\n```bash\npython s22_goal_loop/code.py # /goal until tests pass + deploy green，看闸门怎么判\n```\n\n观察：设了目标之后，每轮结束都有一条 `goal_evaluated`；普通文本判 `satisfied=False`，`task-notification` 来源判 `satisfied=True`；预算花完的时候出 `goal_blocked`。同样一句 `tests passed`，来源不同，结果完全相反。这就是 `/goal` 不会被一句空话糊弄的地方。\n\n## 接下来\n\n`/goal` 是\"拉回主循环\"的一种触发：条件控制。它和 s21 的\"脱离主循环\"正好成对，一个把工作派出去，一个把控制权拉回来。再往外，还有时间控制（`/loop`、cron）和事件控制（`Monitor`）的重入，它们共享同一套任务/通知基底；但闸门的核心已经在这里：**停不停，不是模型一句话说了算，得目标对着可信证据来判。**\n\n\n" + "content": "# s22: Goal Loop — 什么时候停，目标说了算，不是模型说了算\n\ns01 → ... → s20 → s21 → `s22`\n\n> *\"一轮能不能结束，看目标条件满不满足，不是模型说停就停\"* — `/goal` 在主循环每轮收尾的地方加一道闸门：每轮结束后，一个独立的判断器看可信证据够不够，不够就把模型推回去再来一轮。\n>\n> **Harness 层**: 目标闭环 — 在轮次收尾处，加一道程序控制的完成闸门。\n\n---\n\n从 s01 到 s21，一轮对话怎么结束？模型不再发 `tool_use`，循环就直接 `return` 了。一次性任务这么干没问题，做完就停。\n\n但有些目标你得盯着它做到底：\"把测试跑过\"、\"部署成功了再说\"。这时候经常出两种问题：模型做了一半觉得差不多了，自己就停了；更过分的是，它嘴上说一句 `tests passed` 就想收工。你要的其实很简单：这一轮能不能结束，不能模型自己说了算，得有个明确的条件，对着实打实的证据来判断。\n\n这条线其实从第一课就埋着了。s01 说过，退出循环本来是模型的一个决定；s04 的 Stop hook 第一次给了程序否决权。这一课把那个否决权做成完整的闭环：条件、证据、预算，三样缺一不可。\n\n## /goal：每轮收尾加一道闸门\n\n输入 `/goal <条件>` 就设了一个会话级的停止条件。程序把它存成当前活跃目标，每轮结束后，判断器检查对话记录里的可信证据够不够满足条件。不够，闸门就把这次结束拦住，塞一条\"继续干\"的提示进下一轮；够了，就清除目标，标记完成。\n\n![Goal Loop 总览](/course-assets/s22_goal_loop/goal-loop-overview.svg)\n\n和 s01 的循环比，只多了一道判断，模型想停的时候先过目标这关：\n\n```python\n# s01：模型说停就停\nif not has_tool_use(response):\n return\n# s22：想停？先过目标闸门\nif not has_tool_use(response):\n verdict = goal.evaluate_after_turn()\n if verdict == \"continuing\":\n continue # 没达成 -> 推回去再来一轮\n return # 达成/超预算/没目标 -> 真停\n```\n\n这道闸门是程序自己控制的。不是模型自己约束自己，模型甚至不知道有这么一道闸门，它只是收到了下一轮的输入，接着干就是了。\n\n## 设目标：证据从命令之后开始算\n\n`set_goal` 会存一个活跃目标：目标文本、最大轮数预算、计数器，还有 `start_index`——也就是证据窗口的起点。它取当前对话记录的长度，所以 `/goal` 这行命令本身在窗口外面。这是第一道防线：命令自己不能证明自己完成了。\n\n```python\ndef set_goal(self, objective, max_turns=20):\n self.active = {\n \"objective\": objective, \"status\": \"active\",\n \"start_index\": len(self.transcript), # 证据窗口从这里开始；命令本身在窗口外\n \"max_turns\": max_turns, \"checks\": 0, \"continuation_turns\": 0,\n }\n```\n\n## 判断器：只信实打实的证据\n\n这是整个机制最核心的地方。判断器不看整段对话，只看证据窗口里来自可信来源的消息。三层过滤，把\"嘴上说完成了但不算数\"的内容全挡在外面：\n\n```python\nTRUSTED_EVIDENCE_ORIGINS = {\"task-notification\", \"monitor-line\"}\n\ndef evidence_text(self):\n out = []\n for m in self.transcript[self.active[\"start_index\"]:]:\n if m.origin.get(\"kind\") == \"slash-command\": # 1 斜杠命令本身不算\n continue\n if m.role == \"user\" and m.content.strip().startswith(\"/goal\"): # 2 /goal 命令文本不算\n continue\n if m.origin.get(\"kind\") not in TRUSTED_EVIDENCE_ORIGINS: # 3 只信可信来源\n continue\n out.append(f\"{m.role}: {m.content}\")\n return \"\\n\".join(out)\n```\n\n效果很明显：同样一句 `tests passed`，你打字说的不算，后台任务通知带回来的才算。模型糊弄不过去，它没法靠自己说一句\"我做完了\"就把目标判成完成。这是全课程反复出现的那条信任边界的最后一次登场：s16 说协议靠字段不靠理解，s19 说注解是申报、申报可以撒谎，s22 说完成证据只看来源不看内容。\n\n最小版的 `goal_satisfied()` 使用确定的关键词匹配，让演示保持离线和可复现。生产级 harness 可以把这条策略替换成独立的轻量判断模型，但仍然保留相同的可信证据边界。\n\n## 闸门三态：完成/继续/超预算\n\n`evaluate_after_turn` 每轮跑一次，三种结果：满足条件就清除目标（completed）；没满足而且预算还没花完，就往队列塞一条\"继续干\"的提示，放行下一轮（continuing）；预算花完就停（blocked），别让一个永远判不出来的目标无限烧钱。\n\n```python\ndef evaluate_after_turn(self):\n g = self.active\n g[\"checks\"] += 1\n if self.goal_satisfied():\n g[\"status\"] = \"completed\"; self.active = None\n return \"completed\" # 达成 -> 清除目标\n if g[\"continuation_turns\"] < g[\"max_turns\"]:\n g[\"continuation_turns\"] += 1\n self.queue.enqueue(\n value=\"继续干活，别把这条提醒当成完成证据。\",\n origin={\"kind\": \"active-goal\"})\n return \"continuing\" # 没达成 -> 塞提示，下一轮\n g[\"status\"] = \"blocked\"; self.active = None\n return \"blocked\" # 超预算 -> 放行，不再拦\n```\n\n那条\"继续干\"的提示里特意写了\"别把这条提醒当成完成证据\"，连提醒本身都被排除在证据之外。三层防误判就齐了：命令文本不算、提醒文本不算、普通聊天文本不算。预算则是 s11 教过的老规矩：任何自动重试的机制都得有上限，不然一个永远判不满足的目标就是个烧钱的永动机。\n\n## 继续提示和外部异步消息分开走\n\n继续提示进的是同一个 `CommandQueue`，但它和外部异步事件（任务完成通知、监控行）不是同一种消费方式。`dequeue` 带个开关：消费外部收件箱的时候，默认跳过目标的继续提示。\n\n```python\ndef dequeue(self, include_goal_continuations=True):\n ...\n for idx, item in enumerate(self.items):\n if include_goal_continuations or item[\"origin\"].get(\"kind\") != \"active-goal\":\n return self.items.pop(idx)\n return None\n```\n\n为什么要分开？如果同一个消费者把继续提示和外部通知一起取走，后台结果还没到，提醒文本就可能被误当成新证据。分开之后，目标的推进是显式的一步，不会被异步事件带着走。\n\n## 跑起来看看\n\n`code.py` 演示了一个 `/goal until tests passed and deploy green`：设了目标之后没有可信证据，闸门一轮轮把它推回去；你直接打 `tests passed` 也不算（来源不可信）；直到后台任务发来 `task-notification`，证据到位，才标记完成。还加了一个 `max_turns=2` 的小目标演示超预算拦截。\n\n```python\ns.submit(\"/goal until tests passed and deploy green\") # 设目标，窗口在命令之后\ns.submit(\"tests passed, trust me\") # 普通文本 -> 不算完成\ns.deliver_host_event(\"tests passed; deploy green\",\n source=\"task-notification\") # 可信宿主事件 -> 完成\n```\n\n`submit()` 只接受普通用户文本。可信标签必须走独立的宿主事件通道，来源由 harness 白名单校验；用户或模型文本不能给自己贴上 `task-notification` 标签。\n\n## 相对 s21 的变更\n\n| | s21 Workflow Runtime | s22 Goal Loop |\n|--|---------------------|---------------|\n| 触发方式 | 脚本控制的编排（脱离主循环） | 条件控制的继续（拉回主循环） |\n| 加在哪 | 工具层：一个 `Workflow` 工具 | 轮次收尾：一道完成闸门 |\n| 谁决定停 | 脚本跑完就停 | 目标条件对着可信证据判 |\n| 新增机制 | 脚本 DSL、后台任务、journal/续跑、结构化输出 | 目标闸门、证据信任边界、继续提示分流、预算 |\n\ns21 是把编排写成脚本、派出去脱离主循环；s22 反过来，是一股力量把控制权重拉回主循环：目标没达成，这一轮就不算结束。两个都不改 s01 那个 `while` 循环，只是从两头给它加约束。\n\n## 试一下\n\n```bash\npython s22_goal_loop/code.py # /goal until tests pass + deploy green，看闸门怎么判\n```\n\n观察：设了目标之后，每轮结束都有一条 `goal_evaluated`；普通文本判 `satisfied=False`，`task-notification` 来源判 `satisfied=True`；预算花完的时候出 `goal_blocked`。同样一句 `tests passed`，来源不同，结果完全相反。这就是 `/goal` 不会被一句空话糊弄的地方。\n\n## 接下来\n\n`/goal` 是\"拉回主循环\"的一种触发：条件控制。它和 s21 的\"脱离主循环\"正好成对，一个把工作派出去，一个把控制权拉回来。再往外，还有时间控制（`/loop`、cron）和事件控制（`Monitor`）的重入，它们共享同一套任务/通知基底；但闸门的核心已经在这里：**停不停，不是模型一句话说了算，得目标对着可信证据来判。**\n\n\n" }, { "version": "s22", "locale": "ja", "title": "s22: Goal Loop — いつ止まるかはモデルではなく goal が決める", - "content": "# s22: Goal Loop — いつ止まるかはモデルではなく goal が決める\n\ns01 → ... → s20 → s21 → `s22`\n\n> *「turn が終了できるかは goal condition を満たすかで決まり、モデルが stop と言っただけでは終わらない」* — `/goal` は main loop の各 turn の終端に gate を追加します。独立した evaluator が trusted evidence の充足を確認し、不足ならモデルを次のラウンドへ押し戻します。\n>\n> **Harness 層**: Goal closure — turn 終端に program-controlled completion gate を追加します。\n\n> **情報源の境界：** この章の製品詳細は Claude Code 2.1.177 の clean-room 行動再構成に基づく。後続リリースで名称や制限は変わり得る。`code.py` はオフライン教材モデルであり、製品ソースの複製ではない。\n\n---\n\ns01 から s21 まで、会話の 1 turn はどう終わったでしょうか。モデルが `tool_use` を出さなくなると、loop はそのまま `return` しました。one-shot task なら問題ありません。終わったら止まります。\n\nしかし「テストを通す」「deploy が成功するまで続ける」のように、最後まで見届けるべき goal もあります。そこでは 2 つの問題がよく起きます。モデルが途中まで進めて十分だと思い、自分で止まる。さらに悪ければ、口頭で `tests passed` と言うだけで終了しようとします。必要なことは単純です。turn が終了できるかをモデル自身に決めさせず、明示的な condition を実際の evidence に照らして判断します。\n\nこの流れは最初の章からありました。s01 は loop の exit がモデルの判断だと説明し、s04 の Stop hook が初めて program に veto を与えました。この章は、その veto を condition、evidence、budget の 3 要素が欠けない完全な loop にします。\n\n## /goal: 各 turn の終端に gate を追加する\n\n`/goal ` を入力すると session-scoped stopping condition を設定します。program は active goal として保存し、各 turn の後に独立した lightweight model を evaluator として使い、transcript 内の trusted evidence が condition を満たすか確認します。不足なら gate が停止を拒み、次ラウンドへ「作業を続ける」prompt を queue します。十分なら goal を消して complete とします。\n\n![Goal Loop Overview](/course-assets/s22_goal_loop/goal-loop-overview.svg)\n\ns01 の loop と比べて、追加されるのは 1 つの判断だけです。モデルが止まりたいとき、先に goal gate を通ります。\n\n```python\n# s01: モデルが stop と言えば停止\nif not has_tool_use(response):\n return\n# s22: 止まりたい？先に goal gate を通る\nif not has_tool_use(response):\n verdict = goal.evaluate_after_turn()\n if verdict == \"continuing\":\n continue # 未達成 -> 次のラウンドへ押し戻す\n return # 達成 / budget 超過 / goal なし -> 本当に停止\n```\n\nこの gate を制御するのは program です。モデルが自分を律しているのではありません。モデルは gate の存在すら知らず、次のラウンドの入力を受け取って作業を続けるだけです。\n\n> 実際の Claude Code では `/goal` は session-scoped Stop hook で、workspace trust と hook restriction の管理下にあります。コードには `active_goal`、`goal_status`、`goal_met`、`tengu_goal_achieved` などの marker があります。\n\n## Goal の設定: Evidence は command の後から数える\n\n`set_goal` は active goal として、goal text、最大 turn budget、counter、そして evidence window の開始点 `start_index` を保存します。現在の transcript length を使うため、`/goal` command 自身は window の外です。これが最初の防御です。command が自分自身の完了を証明することはできません。\n\n```python\ndef set_goal(self, objective, max_turns=20):\n self.active = {\n \"objective\": objective, \"status\": \"active\",\n \"start_index\": len(self.transcript), # evidence はここから。command 自身は window 外\n \"max_turns\": max_turns, \"checks\": 0, \"continuation_turns\": 0,\n }\n```\n\n> 実際の Claude Code では `GoalRuntime.setGoal()` が active goal、開始位置、counter、budget を保存し、submit 後に `resetEvidenceStart()` で window を command 後へそろえます。\n\n## Evaluator: 実在する evidence だけを信頼する\n\nここが仕組み全体の core です。evaluator は会話全体を見ず、evidence window 内で trusted source から来た message だけを見ます。3 層の filter が、「完了したと言ったから完了」という内容をすべて外へ止めます。\n\n```python\nTRUSTED_EVIDENCE_ORIGINS = {\"task-notification\", \"monitor-line\"}\n\ndef evidence_text(self):\n out = []\n for m in self.transcript[self.active[\"start_index\"]:]:\n if m.origin.get(\"kind\") == \"slash-command\": # 1 slash command 自身は evidence ではない\n continue\n if m.role == \"user\" and m.content.strip().startswith(\"/goal\"): # 2 /goal command text は evidence ではない\n continue\n if m.origin.get(\"kind\") not in TRUSTED_EVIDENCE_ORIGINS: # 3 trusted origin だけを信頼\n continue\n out.append(f\"{m.role}: {m.content}\")\n return \"\\n\".join(out)\n```\n\n効果は明確です。同じ `tests passed` でも、あなたが入力したものは数えず、background task notification が持ち帰ったものだけを数えます。モデルは「完了した」と自分で言うだけでは goal を complete にできません。これはコース全体に繰り返し現れた trust boundary の最後の登場です。s16 は protocol が理解ではなく field に依存すると言い、s19 は annotation が申告であり、申告は嘘をつけると言い、s22 は completion evidence を content ではなく origin で信頼します。\n\n教材版の `goal_satisfied()` は決定的な keyword matching です。実際の版は evidence window を別の lightweight model へ渡して判定します。\n\n> 実際の Claude Code の evaluator は作業モデルとは別の lightweight model で、`evaluatorModel`、`default small fast model` と記されています。任意の text を信じず、会話内の evidence を判断します。\n\n## Gate の 3 状態: Completed / continuing / budget 超過\n\n`evaluate_after_turn` は各 turn で 1 回動き、3 つの結果を返します。condition が満たされれば goal を completed として消します。満たされず budget が残れば「作業を続ける」prompt を queue し、continuing として次ラウンドを許可します。budget を使い切れば blocked で gate を解除し、永遠に判定できない goal が無限に費用を使わないようにします。\n\n```python\ndef evaluate_after_turn(self):\n g = self.active\n g[\"checks\"] += 1\n if self.goal_satisfied():\n g[\"status\"] = \"completed\"; self.active = None\n return \"completed\" # 達成 -> goal を消す\n if g[\"continuation_turns\"] < g[\"max_turns\"]:\n g[\"continuation_turns\"] += 1\n self.queue.enqueue(\n value=\"作業を続けてください。この reminder を completion evidence として扱わないでください。\",\n origin={\"kind\": \"active-goal\"})\n return \"continuing\" # 未達成 -> prompt を queue し、次ラウンドへ\n g[\"status\"] = \"blocked\"; self.active = None\n return \"blocked\" # budget 超過 -> gate を解除\n```\n\ncontinuation prompt には、わざわざ自身を evidence にしないよう書き、filter でも除外します。これで false positive を防ぐ 3 層がそろいます。command text、reminder text、ordinary conversation のいずれも数えません。budget は s11 の古い規則に従います。automatic retry mechanism には必ず上限が必要です。そうでなければ、永遠に satisfied にならない goal が費用を燃やし続けます。\n\n> 実際の Claude Code の `evaluateAfterTurn` は `goal_evaluated` event を出し、結果に応じて complete、continuation queue、gate の解除を行います。default budget は 20 turn です。\n\n## Continuation prompt と外部 asynchronous message を分ける\n\ncontinuation prompt は同じ `CommandQueue` に入りますが、task completion notification や monitor line といった外部 asynchronous event とは別の方法で消費します。`dequeue` には switch があり、外部 inbox を消費するときは goal continuation を既定で skip します。\n\n```python\ndef dequeue(self, include_goal_continuations=True):\n ...\n for idx, item in enumerate(self.items):\n if include_goal_continuations or item[\"origin\"].get(\"kind\") != \"active-goal\":\n return self.items.pop(idx)\n return None\n```\n\nなぜ分けるのでしょう。実際の model test では、モデルが continuation prompt を外部 notification と一緒に消費し、background evidence が到着する前に goal を complete と判定する bug が起きました。分離後は goal の進行が明示的な 1 step になり、asynchronous event に偶然運ばれません。\n\n> 実際の Claude Code の `drainCommandQueue` は既定で `includeGoalContinuations=false` とし、goal continuation の消費を外部 asynchronous inbox から分けます。\n\n## 実際に動かす\n\n`code.py` は `/goal until tests passed and deploy green` を実演します。goal 設定後に trusted evidence がなければ、gate がラウンドごとに押し戻します。直接 `tests passed` と入力しても origin が信頼されないため数えません。background task が `task-notification` を送って初めて evidence がそろい、complete になります。`max_turns=2` の小さな goal で budget 超過も示します。\n\n```python\ns.submit(\"/goal until tests passed and deploy green\") # goal を設定。evidence は command 後から\ns.submit(\"tests passed, trust me\") # ordinary text -> completion evidence ではない\ns.deliver_host_event(\"tests passed; deploy green\",\n source=\"task-notification\") # trusted host event -> complete\n```\n\n`submit()` は通常のユーザーテキストだけを受け取る。trusted label は独立した host event channel から入り、source は harness の allowlist で検証される。ユーザーやモデルのテキストが自分に `task-notification` label を付けることはできない。\n\n## s21 からの変更点\n\n| | s21 Workflow Runtime | s22 Goal Loop |\n|--|---------------------|---------------|\n| trigger | script-controlled orchestration（main loop の外） | condition-controlled continuation（main loop へ引き戻す） |\n| 接続位置 | tool layer: 1 つの `Workflow` ツール | turn 終端: completion gate |\n| stop を決めるもの | script が完了 | goal condition を trusted evidence と照合 |\n| 新しい仕組み | script DSL、background task、journal/resume、structured output | goal gate、evidence trust boundary、continuation 分流、budget |\n\ns21 は script-defined orchestration を main loop の外へ送り出します。s22 は反対の力で control を引き戻します。goal が未達成なら turn は終わっていません。どちらも s01 の `while` loop を変えず、両側から制約を加えます。\n\n## 試してみる\n\n```bash\npython s22_goal_loop/code.py # /goal until tests pass + deploy green。gate の判定を見る\n```\n\ngoal 設定後、各 turn が `goal_evaluated` を出す様子を確認してください。ordinary text は `satisfied=False`、同じ内容でも `task-notification` origin は `satisfied=True`、budget を使い切ると `goal_blocked` です。同じ `tests passed` でも origin によって結果が正反対になります。空疎な主張で `/goal` を欺けない理由です。\n\n## 次へ\n\n`/goal` は control を main loop へ引き戻す trigger の 1 つ、condition control です。s21 の main loop 外 orchestration と対になり、一方は仕事を外へ送り、もう一方は control を内へ戻します。その外側には `/loop` と cron による time-controlled re-entry、`Monitor` による event-controlled re-entry もあり、同じ task/notification 基盤を共有します。しかし gate の core はすでにここにあります。**stop するかはモデルの一言では決まらず、goal が trusted evidence に照らして判断します。**\n\n\n" + "content": "# s22: Goal Loop — いつ止まるかはモデルではなく goal が決める\n\ns01 → ... → s20 → s21 → `s22`\n\n> *「turn が終了できるかは goal condition を満たすかで決まり、モデルが stop と言っただけでは終わらない」* — `/goal` は main loop の各 turn の終端に gate を追加します。独立した evaluator が trusted evidence の充足を確認し、不足ならモデルを次のラウンドへ押し戻します。\n>\n> **Harness 層**: Goal closure — turn 終端に program-controlled completion gate を追加します。\n\n---\n\ns01 から s21 まで、会話の 1 turn はどう終わったでしょうか。モデルが `tool_use` を出さなくなると、loop はそのまま `return` しました。one-shot task なら問題ありません。終わったら止まります。\n\nしかし「テストを通す」「deploy が成功するまで続ける」のように、最後まで見届けるべき goal もあります。そこでは 2 つの問題がよく起きます。モデルが途中まで進めて十分だと思い、自分で止まる。さらに悪ければ、口頭で `tests passed` と言うだけで終了しようとします。必要なことは単純です。turn が終了できるかをモデル自身に決めさせず、明示的な condition を実際の evidence に照らして判断します。\n\nこの流れは最初の章からありました。s01 は loop の exit がモデルの判断だと説明し、s04 の Stop hook が初めて program に veto を与えました。この章は、その veto を condition、evidence、budget の 3 要素が欠けない完全な loop にします。\n\n## /goal: 各 turn の終端に gate を追加する\n\n`/goal ` を入力すると session-scoped stopping condition を設定します。program は active goal として保存し、各 turn の後に evaluator が transcript 内の trusted evidence を condition と照合します。不足なら gate が停止を拒み、次ラウンドへ「作業を続ける」prompt を queue します。十分なら goal を消して complete とします。\n\n![Goal Loop Overview](/course-assets/s22_goal_loop/goal-loop-overview.svg)\n\ns01 の loop と比べて、追加されるのは 1 つの判断だけです。モデルが止まりたいとき、先に goal gate を通ります。\n\n```python\n# s01: モデルが stop と言えば停止\nif not has_tool_use(response):\n return\n# s22: 止まりたい？先に goal gate を通る\nif not has_tool_use(response):\n verdict = goal.evaluate_after_turn()\n if verdict == \"continuing\":\n continue # 未達成 -> 次のラウンドへ押し戻す\n return # 達成 / budget 超過 / goal なし -> 本当に停止\n```\n\nこの gate を制御するのは program です。モデルが自分を律しているのではありません。モデルは gate の存在すら知らず、次のラウンドの入力を受け取って作業を続けるだけです。\n\n## Goal の設定: Evidence は command の後から数える\n\n`set_goal` は active goal として、goal text、最大 turn budget、counter、そして evidence window の開始点 `start_index` を保存します。現在の transcript length を使うため、`/goal` command 自身は window の外です。これが最初の防御です。command が自分自身の完了を証明することはできません。\n\n```python\ndef set_goal(self, objective, max_turns=20):\n self.active = {\n \"objective\": objective, \"status\": \"active\",\n \"start_index\": len(self.transcript), # evidence はここから。command 自身は window 外\n \"max_turns\": max_turns, \"checks\": 0, \"continuation_turns\": 0,\n }\n```\n\n## Evaluator: 実在する evidence だけを信頼する\n\nここが仕組み全体の core です。evaluator は会話全体を見ず、evidence window 内で trusted source から来た message だけを見ます。3 層の filter が、「完了したと言ったから完了」という内容をすべて外へ止めます。\n\n```python\nTRUSTED_EVIDENCE_ORIGINS = {\"task-notification\", \"monitor-line\"}\n\ndef evidence_text(self):\n out = []\n for m in self.transcript[self.active[\"start_index\"]:]:\n if m.origin.get(\"kind\") == \"slash-command\": # 1 slash command 自身は evidence ではない\n continue\n if m.role == \"user\" and m.content.strip().startswith(\"/goal\"): # 2 /goal command text は evidence ではない\n continue\n if m.origin.get(\"kind\") not in TRUSTED_EVIDENCE_ORIGINS: # 3 trusted origin だけを信頼\n continue\n out.append(f\"{m.role}: {m.content}\")\n return \"\\n\".join(out)\n```\n\n効果は明確です。同じ `tests passed` でも、あなたが入力したものは数えず、background task notification が持ち帰ったものだけを数えます。モデルは「完了した」と自分で言うだけでは goal を complete にできません。これはコース全体に繰り返し現れた trust boundary の最後の登場です。s16 は protocol が理解ではなく field に依存すると言い、s19 は annotation が申告であり、申告は嘘をつけると言い、s22 は completion evidence を content ではなく origin で信頼します。\n\n最小版の `goal_satisfied()` は決定的な keyword matching を使い、demo を offline かつ再現可能に保ちます。production harness では、この policy を独立した lightweight evaluator model に置き換えられますが、trusted evidence boundary はそのまま維持します。\n\n## Gate の 3 状態: Completed / continuing / budget 超過\n\n`evaluate_after_turn` は各 turn で 1 回動き、3 つの結果を返します。condition が満たされれば goal を completed として消します。満たされず budget が残れば「作業を続ける」prompt を queue し、continuing として次ラウンドを許可します。budget を使い切れば blocked で gate を解除し、永遠に判定できない goal が無限に費用を使わないようにします。\n\n```python\ndef evaluate_after_turn(self):\n g = self.active\n g[\"checks\"] += 1\n if self.goal_satisfied():\n g[\"status\"] = \"completed\"; self.active = None\n return \"completed\" # 達成 -> goal を消す\n if g[\"continuation_turns\"] < g[\"max_turns\"]:\n g[\"continuation_turns\"] += 1\n self.queue.enqueue(\n value=\"作業を続けてください。この reminder を completion evidence として扱わないでください。\",\n origin={\"kind\": \"active-goal\"})\n return \"continuing\" # 未達成 -> prompt を queue し、次ラウンドへ\n g[\"status\"] = \"blocked\"; self.active = None\n return \"blocked\" # budget 超過 -> gate を解除\n```\n\ncontinuation prompt には、わざわざ自身を evidence にしないよう書き、filter でも除外します。これで false positive を防ぐ 3 層がそろいます。command text、reminder text、ordinary conversation のいずれも数えません。budget は s11 の古い規則に従います。automatic retry mechanism には必ず上限が必要です。そうでなければ、永遠に satisfied にならない goal が費用を燃やし続けます。\n\n## Continuation prompt と外部 asynchronous message を分ける\n\ncontinuation prompt は同じ `CommandQueue` に入りますが、task completion notification や monitor line といった外部 asynchronous event とは別の方法で消費します。`dequeue` には switch があり、外部 inbox を消費するときは goal continuation を既定で skip します。\n\n```python\ndef dequeue(self, include_goal_continuations=True):\n ...\n for idx, item in enumerate(self.items):\n if include_goal_continuations or item[\"origin\"].get(\"kind\") != \"active-goal\":\n return self.items.pop(idx)\n return None\n```\n\nなぜ分けるのでしょう。同じ consumer が continuation prompt と外部 notification を一緒に取り出すと、background result が届く前に reminder text を新しい evidence と誤認する可能性があります。分離後は goal の進行が明示的な 1 step になり、asynchronous event に偶然運ばれません。\n\n## 実際に動かす\n\n`code.py` は `/goal until tests passed and deploy green` を実演します。goal 設定後に trusted evidence がなければ、gate がラウンドごとに押し戻します。直接 `tests passed` と入力しても origin が信頼されないため数えません。background task が `task-notification` を送って初めて evidence がそろい、complete になります。`max_turns=2` の小さな goal で budget 超過も示します。\n\n```python\ns.submit(\"/goal until tests passed and deploy green\") # goal を設定。evidence は command 後から\ns.submit(\"tests passed, trust me\") # ordinary text -> completion evidence ではない\ns.deliver_host_event(\"tests passed; deploy green\",\n source=\"task-notification\") # trusted host event -> complete\n```\n\n`submit()` は通常のユーザーテキストだけを受け取る。trusted label は独立した host event channel から入り、source は harness の allowlist で検証される。ユーザーやモデルのテキストが自分に `task-notification` label を付けることはできない。\n\n## s21 からの変更点\n\n| | s21 Workflow Runtime | s22 Goal Loop |\n|--|---------------------|---------------|\n| trigger | script-controlled orchestration（main loop の外） | condition-controlled continuation（main loop へ引き戻す） |\n| 接続位置 | tool layer: 1 つの `Workflow` ツール | turn 終端: completion gate |\n| stop を決めるもの | script が完了 | goal condition を trusted evidence と照合 |\n| 新しい仕組み | script DSL、background task、journal/resume、structured output | goal gate、evidence trust boundary、continuation 分流、budget |\n\ns21 は script-defined orchestration を main loop の外へ送り出します。s22 は反対の力で control を引き戻します。goal が未達成なら turn は終わっていません。どちらも s01 の `while` loop を変えず、両側から制約を加えます。\n\n## 試してみる\n\n```bash\npython s22_goal_loop/code.py # /goal until tests pass + deploy green。gate の判定を見る\n```\n\ngoal 設定後、各 turn が `goal_evaluated` を出す様子を確認してください。ordinary text は `satisfied=False`、同じ内容でも `task-notification` origin は `satisfied=True`、budget を使い切ると `goal_blocked` です。同じ `tests passed` でも origin によって結果が正反対になります。空疎な主張で `/goal` を欺けない理由です。\n\n## 次へ\n\n`/goal` は control を main loop へ引き戻す trigger の 1 つ、condition control です。s21 の main loop 外 orchestration と対になり、一方は仕事を外へ送り、もう一方は control を内へ戻します。その外側には `/loop` と cron による time-controlled re-entry、`Monitor` による event-controlled re-entry もあり、同じ task/notification 基盤を共有します。しかし gate の core はすでにここにあります。**stop するかはモデルの一言では決まらず、goal が trusted evidence に照らして判断します。**\n\n\n" } ] \ No newline at end of file diff --git a/web/src/data/generated/versions.json b/web/src/data/generated/versions.json index c542d461..1e8640f3 100644 --- a/web/src/data/generated/versions.json +++ b/web/src/data/generated/versions.json @@ -3667,7 +3667,7 @@ "filename": "s21_workflow_runtime/code.py", "title": "Workflow Runtime", "subtitle": "Scripts Own Fixed Orchestration", - "loc": 430, + "loc": 423, "tools": [ "review-changes" ], @@ -3679,104 +3679,104 @@ "classes": [ { "name": "WorkflowInputError", - "startLine": 70, - "endLine": 76 + "startLine": 65, + "endLine": 71 }, { "name": "SimpleJsonSchema", - "startLine": 110, - "endLine": 151 + "startLine": 104, + "endLine": 145 }, { "name": "MockAgentRunner", - "startLine": 170, - "endLine": 200 + "startLine": 164, + "endLine": 194 }, { "name": "WorkflowJournal", - "startLine": 201, - "endLine": 251 + "startLine": 195, + "endLine": 245 }, { "name": "Budget", - "startLine": 252, - "endLine": 276 + "startLine": 246, + "endLine": 270 }, { "name": "LocalWorkflowTask", - "startLine": 277, - "endLine": 301 + "startLine": 271, + "endLine": 295 }, { "name": "ExecutionLimits", - "startLine": 302, - "endLine": 314 + "startLine": 296, + "endLine": 308 }, { "name": "ExecutionState", - "startLine": 315, - "endLine": 415 + "startLine": 309, + "endLine": 408 }, { "name": "WorkflowTool", - "startLine": 416, - "endLine": 465 + "startLine": 409, + "endLine": 457 } ], "functions": [ { "name": "_stable_hash", "signature": "def _stable_hash(s: str)", - "startLine": 45 + "startLine": 40 }, { "name": "create_run_id", "signature": "def create_run_id(meta)", - "startLine": 51 + "startLine": 46 }, { "name": "create_task_id", "signature": "def create_task_id(run_id)", - "startLine": 57 + "startLine": 52 }, { "name": "validate_run_id", "signature": "def validate_run_id(run_id)", - "startLine": 61 + "startLine": 56 }, { "name": "validate_meta", "signature": "def validate_meta(meta)", - "startLine": 77 + "startLine": 72 }, { "name": "check_permission", "signature": "def check_permission(meta, settings=None)", - "startLine": 98 + "startLine": 92 }, { "name": "_fill_schema", "signature": "def _fill_schema(schema, seed)", - "startLine": 152 + "startLine": 146 }, { "name": "_write_json", "signature": "def _write_json(path, value)", - "startLine": 466 + "startLine": 458 }, { "name": "_save_last_run", "signature": "def _save_last_run(run_id)", - "startLine": 471 + "startLine": 463 }, { "name": "_read_last_run", "signature": "def _read_last_run()", - "startLine": 475 + "startLine": 467 } ], "layer": "concurrency", - "source": "\"\"\"\ns21_workflow_runtime — Dynamic Workflow runtime (teaching version)\n\nClean-room behavioral reconstruction of Claude Code's `Workflow` tool / dynamic\nworkflow runtime. Grounded in @anthropic-ai/claude-code@2.1.177 observed\nbehavior (reverse-research/cc_workflow), NOT leaked source.\n\nIdea:\n s01-s20 build a single, model-driven agent loop. s21 adds a deterministic\n orchestration LAYER on top: the main loop exposes a `Workflow` tool that\n launches a background runtime; a script written with agent()/parallel()/\n pipeline()/phase() drives many subagents deterministically, reports progress,\n persists a journal, and can resume from a runId.\n\nRun:\n python code.py # run the sample workflow, print the event stream\n python code.py resume # resume the last run; unchanged agent() calls hit cache\n\nTeaching simplifications (vs real runtime.mjs):\n - The \"subagent\" is a deterministic MockAgentRunner, not a real LLM.\n - A workflow is a plain async Python function, not a sandboxed JS script\n string. The real runtime runs the script in an isolated JS VM with\n Date.now()/Math.random() removed so resume is reproducible.\n - The CLI emits `async_launched` and then awaits completion so the demo stays\n deterministic. The real tool returns while execution continues in background.\n - Storage is a local .runtime/ dir instead of ~/.claude/projects/.../workflows/.\n\"\"\"\n\nimport asyncio\nimport hashlib\nimport json\nimport re\nimport sys\nfrom pathlib import Path\n\n# ---- knobs that mirror the real runtime's guards ----\nAGENT_CAP = 1000 # hard cap on agent() calls per run\nCONCURRENCY = 8 # parallelism cap (semaphore)\nSTORE = Path(__file__).parent / \".runtime\" # snapshots + journals live here\nMISS = object() # journal cache miss sentinel\nWORKFLOW_NAME_RE = re.compile(r\"^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$\")\nRUN_ID_RE = re.compile(r\"^wf_[A-Za-z0-9][A-Za-z0-9._-]{0,63}_[0-9]{4}$\")\n\n\ndef _stable_hash(s: str) -> int:\n \"\"\"Process-stable hash (Python's hash() is salted per process, which would\n break resume keys across `run` and `resume`).\"\"\"\n return int(hashlib.sha256(s.encode()).hexdigest(), 16)\n\n\ndef create_run_id(meta) -> str:\n # Deterministic in the teaching version so the journal path is predictable\n # and `resume` lands on the same file. The real runtime mints a random id.\n return f\"wf_{meta['name']}_{_stable_hash(meta['name']) % 10000:04d}\"\n\n\ndef create_task_id(run_id) -> str:\n return f\"local_workflow_{run_id}\"\n\n\ndef validate_run_id(run_id):\n if not isinstance(run_id, str) or not RUN_ID_RE.fullmatch(run_id):\n raise WorkflowInputError(\"invalid workflow runId\")\n return run_id\n\n\n# ============================================================\n# Errors\n# ============================================================\nclass WorkflowInputError(Exception):\n \"\"\"Bad script / meta / schema input (mirrors WorkflowInputError).\"\"\"\n\n\n# ============================================================\n# meta validation\n# ============================================================\ndef validate_meta(meta):\n \"\"\"Real runtime requires `export const meta = {...}` as the FIRST statement,\n a pure literal, with name + description (+ optional phases). We take a dict.\"\"\"\n if not isinstance(meta, dict):\n raise WorkflowInputError(\"meta must be an object literal\")\n if not meta.get(\"name\") or not meta.get(\"description\"):\n raise WorkflowInputError(\"meta requires `name` and `description`\")\n if not isinstance(meta[\"name\"], str) or not WORKFLOW_NAME_RE.fullmatch(meta[\"name\"]):\n raise WorkflowInputError(\n \"meta.name must be a 1-64 character slug using letters, numbers, '.', '_', or '-'\"\n )\n if not isinstance(meta[\"description\"], str):\n raise WorkflowInputError(\"meta.description must be a string\")\n if \"phases\" in meta:\n if not isinstance(meta[\"phases\"], list) or not all(\n isinstance(phase, str) and phase for phase in meta[\"phases\"]\n ):\n raise WorkflowInputError(\"meta.phases must be a list of non-empty strings\")\n return meta\n\n\ndef check_permission(meta, settings=None):\n \"\"\"allow / deny / ask gate before launch (s03 permission system, applied to\n Workflow). Teaching version allows by default; a deny rule blocks.\"\"\"\n settings = settings or {}\n if meta[\"name\"] in settings.get(\"deny\", []):\n raise WorkflowInputError(f\"workflow '{meta['name']}' denied by settings\")\n return \"allow\"\n\n\n# ============================================================\n# Minimal JSON-schema for structured output (SimpleJsonSchema)\n# ============================================================\nclass SimpleJsonSchema:\n \"\"\"Tiny validator backing agent({schema}). Just enough for teaching:\n object/array/string/boolean/number + required keys.\"\"\"\n\n def __init__(self, schema):\n self.schema = schema\n\n def validate(self, value, schema=None):\n schema = self.schema if schema is None else schema\n t = schema.get(\"type\")\n if t == \"object\":\n if not isinstance(value, dict):\n return False, \"expected object\"\n for key in schema.get(\"required\", []):\n if key not in value:\n return False, f\"missing required key '{key}'\"\n for key, sub in schema.get(\"properties\", {}).items():\n if key in value:\n ok, err = self.validate(value[key], sub)\n if not ok:\n return False, f\"{key}: {err}\"\n return True, None\n if t == \"array\":\n if not isinstance(value, list):\n return False, \"expected array\"\n items = schema.get(\"items\")\n if items:\n for i, el in enumerate(value):\n ok, err = self.validate(el, items)\n if not ok:\n return False, f\"[{i}]: {err}\"\n return True, None\n if t == \"string\":\n return (isinstance(value, str), None if isinstance(value, str) else \"expected string\")\n if t == \"boolean\":\n return (isinstance(value, bool), None if isinstance(value, bool) else \"expected boolean\")\n if t in (\"number\", \"integer\"):\n ok = isinstance(value, (int, float)) and not isinstance(value, bool)\n return (ok, None if ok else \"expected number\")\n return True, None\n\n\ndef _fill_schema(schema, seed):\n \"\"\"Deterministic generic filler used for schemas the mock doesn't special-case.\"\"\"\n t = schema.get(\"type\")\n if t == \"object\":\n keys = schema.get(\"required\") or list(schema.get(\"properties\", {}))\n return {k: _fill_schema(schema[\"properties\"][k], f\"{seed}/{k}\") for k in keys}\n if t == \"array\":\n return [_fill_schema(schema[\"items\"], f\"{seed}/0\")]\n if t == \"boolean\":\n return _stable_hash(seed) % 4 != 0\n if t in (\"number\", \"integer\"):\n return _stable_hash(seed) % 5\n return seed.rsplit(\"/\", 1)[-1]\n\n\n# ============================================================\n# Subagent runner (mock for teaching; real path = an LLM tool loop)\n# ============================================================\nclass MockAgentRunner:\n \"\"\"Stands in for a spawned subagent. Deterministic so resume is reproducible.\n A real runner would run an isolated agent loop that calls repo tools and is\n forced to emit StructuredOutput when a schema is present.\"\"\"\n\n def run(self, prompt, schema=None, label=None):\n if schema is None:\n return f\"[mock] {(label or prompt)[:60]}\"\n props = schema.get(\"properties\", {})\n if \"findings\" in props: # an audit agent\n n = 1 + (_stable_hash(prompt) % 2) # 1-2 findings\n sev = [\"high\", \"medium\", \"low\"]\n return {\"findings\": [\n {\"title\": f\"{label or 'audit'} #{i + 1}\",\n \"severity\": sev[_stable_hash(prompt + str(i)) % 3]}\n for i in range(n)\n ]}\n if \"isReal\" in props: # a verifier agent\n real = _stable_hash(prompt) % 4 != 0 # ~75% confirmed\n return {\"isReal\": real,\n \"reason\": \"reproduced\" if real else \"could not reproduce\"}\n return _fill_schema(schema, prompt)\n\n @staticmethod\n def tokens(prompt, result):\n return len(prompt) // 4 + len(json.dumps(result, default=str)) // 4\n\n\n# ============================================================\n# Journal (resume cache): started/result per agent under a semantic key\n# ============================================================\nclass WorkflowJournal:\n \"\"\"Append-only .journal.jsonl. On resume, agent() calls whose\n semantic key is already present are replayed from cache instead of re-run.\"\"\"\n\n def __init__(self, run_id, resume, store=STORE):\n store.mkdir(parents=True, exist_ok=True)\n self.path = store / f\"{run_id}.journal.jsonl\"\n self.resume = resume\n self.cache = {}\n if resume:\n if not self.path.exists():\n raise WorkflowInputError(f\"resume journal not found for {run_id}\")\n for line_number, line in enumerate(self.path.read_text().splitlines(), start=1):\n try:\n rec = json.loads(line)\n if (\n not isinstance(rec, dict)\n or not isinstance(rec.get(\"key\"), str)\n or \"value\" not in rec\n ):\n raise ValueError(\"expected key/value record\")\n except (json.JSONDecodeError, ValueError) as exc:\n raise WorkflowInputError(\n f\"invalid resume journal record at line {line_number}\"\n ) from exc\n self.cache[rec[\"key\"]] = rec[\"value\"]\n self._f = self.path.open(\"a\")\n else:\n self._f = self.path.open(\"w\") # fresh run truncates\n\n def key(self, kind, label, prompt, schema):\n # Deterministic semantic key — independent of concurrency order, so a\n # parallel/pipeline call gets the same key on resume.\n basis = f\"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}\"\n return f\"{kind}-{_stable_hash(basis) % 10**10:010d}\"\n\n def cached(self, key):\n return self.cache.get(key, MISS)\n\n def record(self, key, value):\n self._f.write(json.dumps({\"key\": key, \"value\": value}) + \"\\n\")\n self._f.flush()\n self.cache[key] = value\n\n def close(self):\n self._f.close()\n\n\n# ============================================================\n# Token budget\n# ============================================================\nclass Budget:\n \"\"\"budget.total / spent() / remaining(). Once spent reaches total, agent()\n calls raise (the real runtime enforces the same ceiling).\"\"\"\n\n def __init__(self, total=None):\n self.total = total\n self._spent = 0\n\n def add(self, n):\n if self.total is not None and self._spent + n > self.total:\n raise WorkflowInputError(\n f\"token budget exceeded ({self._spent + n} > {self.total})\"\n )\n self._spent += n\n\n def spent(self):\n return self._spent\n\n def remaining(self):\n return float(\"inf\") if self.total is None else max(0, self.total - self._spent)\n\n\n# ============================================================\n# Background task state + progress events (the outer event stream)\n# ============================================================\nclass LocalWorkflowTask:\n \"\"\"type local_workflow. Holds status/usage and emits the SDK-like event\n stream: task_started, task_progress (workflow_phase/agent/log), task_notification.\"\"\"\n\n def __init__(self, task_id, run_id, meta):\n self.task_id = task_id\n self.run_id = run_id\n self.meta = meta\n self.status = \"running\"\n self.usage = {\"agents\": 0, \"tokens\": 0}\n self.progress = []\n\n def event(self, name, **data):\n line = \" \".join(f\"{k}={v}\" for k, v in data.items())\n print(f\" event {name:<18} {line}\")\n\n def progress_event(self, ptype, **data):\n self.progress.append({\"type\": ptype, **data})\n line = \" \".join(f\"{k}={v}\" for k, v in data.items())\n print(f\" progress {ptype:<16} {line}\")\n\n\n# ============================================================\n# ExecutionState: the DSL the workflow script sees as `ctx`\n# ============================================================\nclass ExecutionLimits:\n \"\"\"Shared run-wide limits, including nested workflows.\"\"\"\n\n def __init__(self):\n self.agents = 0\n self.semaphore = asyncio.Semaphore(CONCURRENCY)\n\n def claim_agent(self):\n self.agents += 1\n if self.agents > AGENT_CAP:\n raise WorkflowInputError(f\"agent() cap reached ({AGENT_CAP})\")\n\n\nclass ExecutionState:\n \"\"\"Injected into the workflow script. Provides the orchestration primitives.\n Mirrors ExecutionState in runtime.mjs.\"\"\"\n\n def __init__(self, task, journal, runner, budget, args, depth=0, limits=None):\n self.task = task\n self.journal = journal\n self.runner = runner\n self.budget = budget\n self.args = args\n self._depth = depth\n self._phase = None\n self._phases_seen = set()\n self._limits = limits or ExecutionLimits()\n\n def phase(self, title):\n \"\"\"Start a phase; subsequent agent()s group under it. Upsert: emitting the\n same phase again (e.g. from each pipeline item) does not re-announce it.\"\"\"\n self._phase = title\n if title not in self._phases_seen:\n self._phases_seen.add(title)\n self.task.progress_event(\"workflow_phase\", title=title)\n\n def log(self, message):\n \"\"\"Emit a workflow_log progress line.\"\"\"\n self.task.progress_event(\"workflow_log\", message=message)\n\n async def agent(self, prompt, schema=None, label=None, phase=None):\n \"\"\"Spawn one subagent. With a schema, force StructuredOutput + validate\n (retry once). On resume, a cached key short-circuits the run.\"\"\"\n label = label or (prompt[:24] + \"…\")\n self._limits.claim_agent()\n if self.budget.remaining() <= 0:\n raise WorkflowInputError(\"token budget exceeded\")\n\n key = self.journal.key(\"agent\", label, prompt, schema)\n cached = self.journal.cached(key)\n if cached is not MISS:\n if schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(cached)\n if not ok:\n raise WorkflowInputError(\n f\"cached agent output failed schema validation: {err}\"\n )\n self.task.progress_event(\"workflow_agent\", label=label,\n phase=phase or self._phase, status=\"cached\")\n return cached\n\n async with self._limits.semaphore:\n await asyncio.sleep(0) # yield: real subagents are async\n result = self.runner.run(prompt, schema, label)\n\n if schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok: # one nudge/retry, then fail\n result = self.runner.run(prompt + \"\\n\\nReturn valid JSON.\", schema, label)\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok:\n raise WorkflowInputError(f\"agent({{schema}}) invalid output: {err}\")\n\n toks = self.runner.tokens(prompt, result)\n self.budget.add(toks)\n self.task.usage[\"agents\"] += 1\n self.task.usage[\"tokens\"] += toks\n self.journal.record(key, result)\n self.task.progress_event(\"workflow_agent\", label=label,\n phase=phase or self._phase, status=\"done\")\n return result\n\n async def parallel(self, thunks):\n \"\"\"BARRIER: run all thunks concurrently and fail if any thunk fails.\"\"\"\n return await asyncio.gather(*[thunk() for thunk in thunks])\n\n async def pipeline(self, items, *stages):\n \"\"\"Per-item staged flow, NO barrier between stages: item A can be in\n stage 3 while item B is still in stage 1. Each stage gets\n (prev_result, original_item, index). A throwing stage fails the workflow.\"\"\"\n async def run_item(item, idx):\n value = item\n for stage in stages:\n value = await stage(value, item, idx)\n return value\n return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])\n\n async def workflow(self, name, args=None):\n \"\"\"Run a saved workflow inline as a child (one level), sharing this run's\n journal + budget + agent counter.\"\"\"\n if self._depth >= 1:\n raise WorkflowInputError(\"workflow() nesting is one level only\")\n if name not in WORKFLOWS:\n raise WorkflowInputError(f\"unknown workflow '{name}'\")\n meta, fn = WORKFLOWS[name]\n child = ExecutionState(self.task, self.journal, self.runner, self.budget,\n args or {}, depth=self._depth + 1,\n limits=self._limits)\n return await fn(child, args or {})\n\n\n# ============================================================\n# WorkflowTool: the tool entry (WorkflowTool.call)\n# ============================================================\nclass WorkflowTool:\n \"\"\"The Workflow tool. .call() validates meta, runs the permission check,\n creates runId/taskId, registers a LocalWorkflowTask, and emits the same\n lifecycle while this teaching CLI awaits the final result. Supports\n resumeFromRunId. Mirrors WorkflowTool.call in runtime.mjs.\"\"\"\n\n async def call(self, meta, script_fn, args=None, resume_from_run_id=None):\n validate_meta(meta)\n check_permission(meta)\n args = args or {}\n run_id = resume_from_run_id or create_run_id(meta)\n validate_run_id(run_id)\n if resume_from_run_id is not None and run_id != create_run_id(meta):\n raise WorkflowInputError(\"resume runId does not match workflow meta\")\n task_id = create_task_id(run_id)\n resuming = resume_from_run_id is not None\n\n task = LocalWorkflowTask(task_id, run_id, meta)\n # The real tool returns this immediately and runs the rest in background.\n launched = {\"status\": \"async_launched\", \"taskId\": task_id,\n \"taskType\": \"local_workflow\", \"runId\": run_id,\n \"workflowName\": meta[\"name\"]}\n task.event(\"async_launched\", runId=run_id, taskId=task_id)\n task.event(\"task_started\", workflow=meta[\"name\"],\n phases=\",\".join(meta.get(\"phases\", [])) or \"-\",\n resume=resuming)\n\n journal = None\n try:\n journal = WorkflowJournal(run_id, resume=resuming)\n ctx = ExecutionState(\n task, journal, MockAgentRunner(), Budget(args.get(\"budget\")), args\n )\n result = await script_fn(ctx, args)\n task.status = \"completed\"\n except Exception as e: # failed / stopped close the loop too\n task.status = \"failed\"\n result = {\"error\": str(e)}\n finally:\n if journal is not None:\n journal.close()\n\n _write_json(STORE / f\"{run_id}.output.json\", result)\n _save_last_run(run_id)\n task.event(\"task_notification\", status=task.status,\n agents=task.usage[\"agents\"], tokens=task.usage[\"tokens\"],\n outputFile=f\".runtime/{run_id}.output.json\")\n return {\"launched\": launched, \"result\": result, \"task\": task}\n\n\ndef _write_json(path, value):\n path.parent.mkdir(parents=True, exist_ok=True)\n path.write_text(json.dumps(value, indent=2, default=str))\n\n\ndef _save_last_run(run_id):\n (STORE / \"last_run.txt\").write_text(run_id)\n\n\ndef _read_last_run():\n p = STORE / \"last_run.txt\"\n return p.read_text().strip() if p.exists() else None\n\n\n# ============================================================\n# Sample workflow: review changed code across dimensions, verify each finding.\n# Mirrors cc_workflow/runtime/workflows/review_workflow.js (pipeline + parallel).\n# ============================================================\nFINDINGS_SCHEMA = {\n \"type\": \"object\", \"required\": [\"findings\"],\n \"properties\": {\"findings\": {\"type\": \"array\", \"items\": {\n \"type\": \"object\", \"required\": [\"title\", \"severity\"],\n \"properties\": {\"title\": {\"type\": \"string\"}, \"severity\": {\"type\": \"string\"}}}}},\n}\nVERDICT_SCHEMA = {\n \"type\": \"object\", \"required\": [\"isReal\", \"reason\"],\n \"properties\": {\"isReal\": {\"type\": \"boolean\"}, \"reason\": {\"type\": \"string\"}},\n}\n\nSAMPLE_META = {\n \"name\": \"review-changes\",\n \"description\": \"Review changed files across dimensions, verify each finding\",\n \"phases\": [\"Review\", \"Verify\"],\n}\n\nDIMENSIONS = [\"correctness\", \"security\", \"performance\", \"style\"]\n\n\nasync def sample_workflow(ctx, args):\n \"\"\"pipeline over review dimensions (audit -> verify-each), then keep only the\n findings a verifier confirms. The plan is code, not a chat turn.\"\"\"\n ctx.phase(\"Review\")\n\n async def audit(_value, dimension, _idx):\n out = await ctx.agent(\n f\"Review the changed files for {dimension} issues.\",\n schema=FINDINGS_SCHEMA, label=f\"audit:{dimension}\", phase=\"Review\")\n return {\"dimension\": dimension, \"findings\": out[\"findings\"]}\n\n async def verify(audited, dimension, _idx):\n ctx.phase(\"Verify\")\n # Each finding is verified by its own adversarial subagent, concurrently.\n verdicts = await ctx.parallel([\n (lambda f=f: ctx.agent(\n f\"Adversarially verify this {dimension} finding — is it real? {f['title']}\",\n schema=VERDICT_SCHEMA, label=f\"verify:{dimension}:{f['title']}\", phase=\"Verify\"))\n for f in audited[\"findings\"]])\n confirmed = [f for f, v in zip(audited[\"findings\"], verdicts)\n if v and v.get(\"isReal\")]\n return {\"dimension\": dimension, \"confirmed\": confirmed}\n\n results = await ctx.pipeline(DIMENSIONS, audit, verify)\n confirmed = [{\"dimension\": r[\"dimension\"], **f}\n for r in results if r for f in r[\"confirmed\"]]\n confirmed.sort(key=lambda f: {\"high\": 0, \"medium\": 1, \"low\": 2}.get(f[\"severity\"], 3))\n ctx.log(f\"confirmed {len(confirmed)} real finding(s)\")\n return {\"confirmed\": confirmed}\n\n\n# saved workflow registry (.claude/workflows/ analogue)\nWORKFLOWS = {SAMPLE_META[\"name\"]: (SAMPLE_META, sample_workflow)}\n\n\n# ============================================================\n# Demo\n# ============================================================\nasync def main(argv):\n resume_id = None\n if argv and argv[0] == \"resume\":\n resume_id = _read_last_run()\n if not resume_id:\n print(\"nothing to resume — run `python code.py` first.\")\n return\n print(f\"resuming {resume_id} — unchanged agent() calls hit the journal cache\\n\")\n else:\n print(\"launching workflow `review-changes`\\n\")\n\n tool = WorkflowTool()\n out = await tool.call(SAMPLE_META, sample_workflow,\n args={\"budget\": None}, resume_from_run_id=resume_id)\n\n print(\"\\nresult:\")\n for f in out[\"result\"].get(\"confirmed\", []):\n print(f\" [{f['severity']:<6}] {f['dimension']}: {f['title']}\")\n t = out[\"task\"]\n print(f\"\\nstatus={t.status} agents={t.usage['agents']} tokens={t.usage['tokens']}\"\n f\" journal=.runtime/{t.run_id}.journal.jsonl\")\n\n\nif __name__ == \"__main__\":\n asyncio.run(main(sys.argv[1:]))\n", + "source": "\"\"\"\ns21_workflow_runtime — minimal dynamic Workflow runtime for a teaching harness\n\nIdea:\n s01-s20 build a single, model-driven agent loop. s21 adds a deterministic\n orchestration LAYER on top: the main loop exposes a `Workflow` tool that\n launches a background runtime; a script written with agent()/parallel()/\n pipeline()/phase() drives many subagents deterministically, reports progress,\n persists a journal, and can resume from a runId.\n\nRun:\n python code.py # run the sample workflow, print the event stream\n python code.py resume # resume the last run; unchanged agent() calls hit cache\n\nImplementation choices:\n - The \"subagent\" is a deterministic MockAgentRunner, not a real LLM.\n - A workflow is a plain async Python function. A production harness may use a\n declarative format or run user-authored scripts in an isolated VM.\n - The CLI emits `async_launched` and then awaits completion so the demo stays\n deterministic. A long-running host can return while execution continues.\n - Storage is a local .runtime/ directory beside this file.\n\"\"\"\n\nimport asyncio\nimport hashlib\nimport json\nimport re\nimport sys\nfrom pathlib import Path\n\n# ---- runtime guards ----\nAGENT_CAP = 1000 # hard cap on agent() calls per run\nCONCURRENCY = 8 # parallelism cap (semaphore)\nSTORE = Path(__file__).parent / \".runtime\" # snapshots + journals live here\nMISS = object() # journal cache miss sentinel\nWORKFLOW_NAME_RE = re.compile(r\"^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$\")\nRUN_ID_RE = re.compile(r\"^wf_[A-Za-z0-9][A-Za-z0-9._-]{0,63}_[0-9]{4}$\")\n\n\ndef _stable_hash(s: str) -> int:\n \"\"\"Process-stable hash (Python's hash() is salted per process, which would\n break resume keys across `run` and `resume`).\"\"\"\n return int(hashlib.sha256(s.encode()).hexdigest(), 16)\n\n\ndef create_run_id(meta) -> str:\n # Deterministic in the teaching version so the journal path is predictable\n # and `resume` lands on the same file.\n return f\"wf_{meta['name']}_{_stable_hash(meta['name']) % 10000:04d}\"\n\n\ndef create_task_id(run_id) -> str:\n return f\"local_workflow_{run_id}\"\n\n\ndef validate_run_id(run_id):\n if not isinstance(run_id, str) or not RUN_ID_RE.fullmatch(run_id):\n raise WorkflowInputError(\"invalid workflow runId\")\n return run_id\n\n\n# ============================================================\n# Errors\n# ============================================================\nclass WorkflowInputError(Exception):\n \"\"\"Bad workflow, metadata, or schema input.\"\"\"\n\n\n# ============================================================\n# meta validation\n# ============================================================\ndef validate_meta(meta):\n \"\"\"Validate name, description, and optional phases before launch.\"\"\"\n if not isinstance(meta, dict):\n raise WorkflowInputError(\"meta must be an object literal\")\n if not meta.get(\"name\") or not meta.get(\"description\"):\n raise WorkflowInputError(\"meta requires `name` and `description`\")\n if not isinstance(meta[\"name\"], str) or not WORKFLOW_NAME_RE.fullmatch(meta[\"name\"]):\n raise WorkflowInputError(\n \"meta.name must be a 1-64 character slug using letters, numbers, '.', '_', or '-'\"\n )\n if not isinstance(meta[\"description\"], str):\n raise WorkflowInputError(\"meta.description must be a string\")\n if \"phases\" in meta:\n if not isinstance(meta[\"phases\"], list) or not all(\n isinstance(phase, str) and phase for phase in meta[\"phases\"]\n ):\n raise WorkflowInputError(\"meta.phases must be a list of non-empty strings\")\n return meta\n\n\ndef check_permission(meta, settings=None):\n \"\"\"allow / deny / ask gate before launch (s03 permission system, applied to\n Workflow). Teaching version allows by default; a deny rule blocks.\"\"\"\n settings = settings or {}\n if meta[\"name\"] in settings.get(\"deny\", []):\n raise WorkflowInputError(f\"workflow '{meta['name']}' denied by settings\")\n return \"allow\"\n\n\n# ============================================================\n# Minimal JSON-schema for structured output (SimpleJsonSchema)\n# ============================================================\nclass SimpleJsonSchema:\n \"\"\"Tiny validator backing agent({schema}). Just enough for teaching:\n object/array/string/boolean/number + required keys.\"\"\"\n\n def __init__(self, schema):\n self.schema = schema\n\n def validate(self, value, schema=None):\n schema = self.schema if schema is None else schema\n t = schema.get(\"type\")\n if t == \"object\":\n if not isinstance(value, dict):\n return False, \"expected object\"\n for key in schema.get(\"required\", []):\n if key not in value:\n return False, f\"missing required key '{key}'\"\n for key, sub in schema.get(\"properties\", {}).items():\n if key in value:\n ok, err = self.validate(value[key], sub)\n if not ok:\n return False, f\"{key}: {err}\"\n return True, None\n if t == \"array\":\n if not isinstance(value, list):\n return False, \"expected array\"\n items = schema.get(\"items\")\n if items:\n for i, el in enumerate(value):\n ok, err = self.validate(el, items)\n if not ok:\n return False, f\"[{i}]: {err}\"\n return True, None\n if t == \"string\":\n return (isinstance(value, str), None if isinstance(value, str) else \"expected string\")\n if t == \"boolean\":\n return (isinstance(value, bool), None if isinstance(value, bool) else \"expected boolean\")\n if t in (\"number\", \"integer\"):\n ok = isinstance(value, (int, float)) and not isinstance(value, bool)\n return (ok, None if ok else \"expected number\")\n return True, None\n\n\ndef _fill_schema(schema, seed):\n \"\"\"Deterministic generic filler used for schemas the mock doesn't special-case.\"\"\"\n t = schema.get(\"type\")\n if t == \"object\":\n keys = schema.get(\"required\") or list(schema.get(\"properties\", {}))\n return {k: _fill_schema(schema[\"properties\"][k], f\"{seed}/{k}\") for k in keys}\n if t == \"array\":\n return [_fill_schema(schema[\"items\"], f\"{seed}/0\")]\n if t == \"boolean\":\n return _stable_hash(seed) % 4 != 0\n if t in (\"number\", \"integer\"):\n return _stable_hash(seed) % 5\n return seed.rsplit(\"/\", 1)[-1]\n\n\n# ============================================================\n# Subagent runner (mock for teaching; real path = an LLM tool loop)\n# ============================================================\nclass MockAgentRunner:\n \"\"\"Stands in for a spawned subagent. Deterministic so resume is reproducible.\n A real runner would run an isolated agent loop that calls repo tools and is\n forced to emit StructuredOutput when a schema is present.\"\"\"\n\n def run(self, prompt, schema=None, label=None):\n if schema is None:\n return f\"[mock] {(label or prompt)[:60]}\"\n props = schema.get(\"properties\", {})\n if \"findings\" in props: # an audit agent\n n = 1 + (_stable_hash(prompt) % 2) # 1-2 findings\n sev = [\"high\", \"medium\", \"low\"]\n return {\"findings\": [\n {\"title\": f\"{label or 'audit'} #{i + 1}\",\n \"severity\": sev[_stable_hash(prompt + str(i)) % 3]}\n for i in range(n)\n ]}\n if \"isReal\" in props: # a verifier agent\n real = _stable_hash(prompt) % 4 != 0 # ~75% confirmed\n return {\"isReal\": real,\n \"reason\": \"reproduced\" if real else \"could not reproduce\"}\n return _fill_schema(schema, prompt)\n\n @staticmethod\n def tokens(prompt, result):\n return len(prompt) // 4 + len(json.dumps(result, default=str)) // 4\n\n\n# ============================================================\n# Journal (resume cache): started/result per agent under a semantic key\n# ============================================================\nclass WorkflowJournal:\n \"\"\"Append-only .journal.jsonl. On resume, agent() calls whose\n semantic key is already present are replayed from cache instead of re-run.\"\"\"\n\n def __init__(self, run_id, resume, store=STORE):\n store.mkdir(parents=True, exist_ok=True)\n self.path = store / f\"{run_id}.journal.jsonl\"\n self.resume = resume\n self.cache = {}\n if resume:\n if not self.path.exists():\n raise WorkflowInputError(f\"resume journal not found for {run_id}\")\n for line_number, line in enumerate(self.path.read_text().splitlines(), start=1):\n try:\n rec = json.loads(line)\n if (\n not isinstance(rec, dict)\n or not isinstance(rec.get(\"key\"), str)\n or \"value\" not in rec\n ):\n raise ValueError(\"expected key/value record\")\n except (json.JSONDecodeError, ValueError) as exc:\n raise WorkflowInputError(\n f\"invalid resume journal record at line {line_number}\"\n ) from exc\n self.cache[rec[\"key\"]] = rec[\"value\"]\n self._f = self.path.open(\"a\")\n else:\n self._f = self.path.open(\"w\") # fresh run truncates\n\n def key(self, kind, label, prompt, schema):\n # Deterministic semantic key — independent of concurrency order, so a\n # parallel/pipeline call gets the same key on resume.\n basis = f\"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}\"\n return f\"{kind}-{_stable_hash(basis) % 10**10:010d}\"\n\n def cached(self, key):\n return self.cache.get(key, MISS)\n\n def record(self, key, value):\n self._f.write(json.dumps({\"key\": key, \"value\": value}) + \"\\n\")\n self._f.flush()\n self.cache[key] = value\n\n def close(self):\n self._f.close()\n\n\n# ============================================================\n# Token budget\n# ============================================================\nclass Budget:\n \"\"\"budget.total / spent() / remaining(). Once spent reaches total, agent()\n calls raise instead of silently overspending.\"\"\"\n\n def __init__(self, total=None):\n self.total = total\n self._spent = 0\n\n def add(self, n):\n if self.total is not None and self._spent + n > self.total:\n raise WorkflowInputError(\n f\"token budget exceeded ({self._spent + n} > {self.total})\"\n )\n self._spent += n\n\n def spent(self):\n return self._spent\n\n def remaining(self):\n return float(\"inf\") if self.total is None else max(0, self.total - self._spent)\n\n\n# ============================================================\n# Background task state + progress events (the outer event stream)\n# ============================================================\nclass LocalWorkflowTask:\n \"\"\"type local_workflow. Holds status/usage and emits the SDK-like event\n stream: task_started, task_progress (workflow_phase/agent/log), task_notification.\"\"\"\n\n def __init__(self, task_id, run_id, meta):\n self.task_id = task_id\n self.run_id = run_id\n self.meta = meta\n self.status = \"running\"\n self.usage = {\"agents\": 0, \"tokens\": 0}\n self.progress = []\n\n def event(self, name, **data):\n line = \" \".join(f\"{k}={v}\" for k, v in data.items())\n print(f\" event {name:<18} {line}\")\n\n def progress_event(self, ptype, **data):\n self.progress.append({\"type\": ptype, **data})\n line = \" \".join(f\"{k}={v}\" for k, v in data.items())\n print(f\" progress {ptype:<16} {line}\")\n\n\n# ============================================================\n# ExecutionState: the DSL the workflow script sees as `ctx`\n# ============================================================\nclass ExecutionLimits:\n \"\"\"Shared run-wide limits, including nested workflows.\"\"\"\n\n def __init__(self):\n self.agents = 0\n self.semaphore = asyncio.Semaphore(CONCURRENCY)\n\n def claim_agent(self):\n self.agents += 1\n if self.agents > AGENT_CAP:\n raise WorkflowInputError(f\"agent() cap reached ({AGENT_CAP})\")\n\n\nclass ExecutionState:\n \"\"\"Injected into the workflow script with the orchestration primitives.\"\"\"\n\n def __init__(self, task, journal, runner, budget, args, depth=0, limits=None):\n self.task = task\n self.journal = journal\n self.runner = runner\n self.budget = budget\n self.args = args\n self._depth = depth\n self._phase = None\n self._phases_seen = set()\n self._limits = limits or ExecutionLimits()\n\n def phase(self, title):\n \"\"\"Start a phase; subsequent agent()s group under it. Upsert: emitting the\n same phase again (e.g. from each pipeline item) does not re-announce it.\"\"\"\n self._phase = title\n if title not in self._phases_seen:\n self._phases_seen.add(title)\n self.task.progress_event(\"workflow_phase\", title=title)\n\n def log(self, message):\n \"\"\"Emit a workflow_log progress line.\"\"\"\n self.task.progress_event(\"workflow_log\", message=message)\n\n async def agent(self, prompt, schema=None, label=None, phase=None):\n \"\"\"Spawn one subagent. With a schema, force StructuredOutput + validate\n (retry once). On resume, a cached key short-circuits the run.\"\"\"\n label = label or (prompt[:24] + \"…\")\n self._limits.claim_agent()\n if self.budget.remaining() <= 0:\n raise WorkflowInputError(\"token budget exceeded\")\n\n key = self.journal.key(\"agent\", label, prompt, schema)\n cached = self.journal.cached(key)\n if cached is not MISS:\n if schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(cached)\n if not ok:\n raise WorkflowInputError(\n f\"cached agent output failed schema validation: {err}\"\n )\n self.task.progress_event(\"workflow_agent\", label=label,\n phase=phase or self._phase, status=\"cached\")\n return cached\n\n async with self._limits.semaphore:\n await asyncio.sleep(0) # yield: real subagents are async\n result = self.runner.run(prompt, schema, label)\n\n if schema is not None:\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok: # one nudge/retry, then fail\n result = self.runner.run(prompt + \"\\n\\nReturn valid JSON.\", schema, label)\n ok, err = SimpleJsonSchema(schema).validate(result)\n if not ok:\n raise WorkflowInputError(f\"agent({{schema}}) invalid output: {err}\")\n\n toks = self.runner.tokens(prompt, result)\n self.budget.add(toks)\n self.task.usage[\"agents\"] += 1\n self.task.usage[\"tokens\"] += toks\n self.journal.record(key, result)\n self.task.progress_event(\"workflow_agent\", label=label,\n phase=phase or self._phase, status=\"done\")\n return result\n\n async def parallel(self, thunks):\n \"\"\"BARRIER: run all thunks concurrently and fail if any thunk fails.\"\"\"\n return await asyncio.gather(*[thunk() for thunk in thunks])\n\n async def pipeline(self, items, *stages):\n \"\"\"Per-item staged flow, NO barrier between stages: item A can be in\n stage 3 while item B is still in stage 1. Each stage gets\n (prev_result, original_item, index). A throwing stage fails the workflow.\"\"\"\n async def run_item(item, idx):\n value = item\n for stage in stages:\n value = await stage(value, item, idx)\n return value\n return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])\n\n async def workflow(self, name, args=None):\n \"\"\"Run a saved workflow inline as a child (one level), sharing this run's\n journal + budget + agent counter.\"\"\"\n if self._depth >= 1:\n raise WorkflowInputError(\"workflow() nesting is one level only\")\n if name not in WORKFLOWS:\n raise WorkflowInputError(f\"unknown workflow '{name}'\")\n meta, fn = WORKFLOWS[name]\n child = ExecutionState(self.task, self.journal, self.runner, self.budget,\n args or {}, depth=self._depth + 1,\n limits=self._limits)\n return await fn(child, args or {})\n\n\n# ============================================================\n# WorkflowTool: the tool entry (WorkflowTool.call)\n# ============================================================\nclass WorkflowTool:\n \"\"\"The Workflow tool. .call() validates meta, runs the permission check,\n creates runId/taskId, registers a LocalWorkflowTask, and emits lifecycle\n events while this teaching CLI awaits the final result. Supports resume.\"\"\"\n\n async def call(self, meta, script_fn, args=None, resume_from_run_id=None):\n validate_meta(meta)\n check_permission(meta)\n args = args or {}\n run_id = resume_from_run_id or create_run_id(meta)\n validate_run_id(run_id)\n if resume_from_run_id is not None and run_id != create_run_id(meta):\n raise WorkflowInputError(\"resume runId does not match workflow meta\")\n task_id = create_task_id(run_id)\n resuming = resume_from_run_id is not None\n\n task = LocalWorkflowTask(task_id, run_id, meta)\n # The real tool returns this immediately and runs the rest in background.\n launched = {\"status\": \"async_launched\", \"taskId\": task_id,\n \"taskType\": \"local_workflow\", \"runId\": run_id,\n \"workflowName\": meta[\"name\"]}\n task.event(\"async_launched\", runId=run_id, taskId=task_id)\n task.event(\"task_started\", workflow=meta[\"name\"],\n phases=\",\".join(meta.get(\"phases\", [])) or \"-\",\n resume=resuming)\n\n journal = None\n try:\n journal = WorkflowJournal(run_id, resume=resuming)\n ctx = ExecutionState(\n task, journal, MockAgentRunner(), Budget(args.get(\"budget\")), args\n )\n result = await script_fn(ctx, args)\n task.status = \"completed\"\n except Exception as e: # failed / stopped close the loop too\n task.status = \"failed\"\n result = {\"error\": str(e)}\n finally:\n if journal is not None:\n journal.close()\n\n _write_json(STORE / f\"{run_id}.output.json\", result)\n _save_last_run(run_id)\n task.event(\"task_notification\", status=task.status,\n agents=task.usage[\"agents\"], tokens=task.usage[\"tokens\"],\n outputFile=f\".runtime/{run_id}.output.json\")\n return {\"launched\": launched, \"result\": result, \"task\": task}\n\n\ndef _write_json(path, value):\n path.parent.mkdir(parents=True, exist_ok=True)\n path.write_text(json.dumps(value, indent=2, default=str))\n\n\ndef _save_last_run(run_id):\n (STORE / \"last_run.txt\").write_text(run_id)\n\n\ndef _read_last_run():\n p = STORE / \"last_run.txt\"\n return p.read_text().strip() if p.exists() else None\n\n\n# ============================================================\n# Sample workflow: review changed code across dimensions, verify each finding.\n# ============================================================\nFINDINGS_SCHEMA = {\n \"type\": \"object\", \"required\": [\"findings\"],\n \"properties\": {\"findings\": {\"type\": \"array\", \"items\": {\n \"type\": \"object\", \"required\": [\"title\", \"severity\"],\n \"properties\": {\"title\": {\"type\": \"string\"}, \"severity\": {\"type\": \"string\"}}}}},\n}\nVERDICT_SCHEMA = {\n \"type\": \"object\", \"required\": [\"isReal\", \"reason\"],\n \"properties\": {\"isReal\": {\"type\": \"boolean\"}, \"reason\": {\"type\": \"string\"}},\n}\n\nSAMPLE_META = {\n \"name\": \"review-changes\",\n \"description\": \"Review changed files across dimensions, verify each finding\",\n \"phases\": [\"Review\", \"Verify\"],\n}\n\nDIMENSIONS = [\"correctness\", \"security\", \"performance\", \"style\"]\n\n\nasync def sample_workflow(ctx, args):\n \"\"\"pipeline over review dimensions (audit -> verify-each), then keep only the\n findings a verifier confirms. The plan is code, not a chat turn.\"\"\"\n ctx.phase(\"Review\")\n\n async def audit(_value, dimension, _idx):\n out = await ctx.agent(\n f\"Review the changed files for {dimension} issues.\",\n schema=FINDINGS_SCHEMA, label=f\"audit:{dimension}\", phase=\"Review\")\n return {\"dimension\": dimension, \"findings\": out[\"findings\"]}\n\n async def verify(audited, dimension, _idx):\n ctx.phase(\"Verify\")\n # Each finding is verified by its own adversarial subagent, concurrently.\n verdicts = await ctx.parallel([\n (lambda f=f: ctx.agent(\n f\"Adversarially verify this {dimension} finding — is it real? {f['title']}\",\n schema=VERDICT_SCHEMA, label=f\"verify:{dimension}:{f['title']}\", phase=\"Verify\"))\n for f in audited[\"findings\"]])\n confirmed = [f for f, v in zip(audited[\"findings\"], verdicts)\n if v and v.get(\"isReal\")]\n return {\"dimension\": dimension, \"confirmed\": confirmed}\n\n results = await ctx.pipeline(DIMENSIONS, audit, verify)\n confirmed = [{\"dimension\": r[\"dimension\"], **f}\n for r in results if r for f in r[\"confirmed\"]]\n confirmed.sort(key=lambda f: {\"high\": 0, \"medium\": 1, \"low\": 2}.get(f[\"severity\"], 3))\n ctx.log(f\"confirmed {len(confirmed)} real finding(s)\")\n return {\"confirmed\": confirmed}\n\n\n# Saved workflow registry\nWORKFLOWS = {SAMPLE_META[\"name\"]: (SAMPLE_META, sample_workflow)}\n\n\n# ============================================================\n# Demo\n# ============================================================\nasync def main(argv):\n resume_id = None\n if argv and argv[0] == \"resume\":\n resume_id = _read_last_run()\n if not resume_id:\n print(\"nothing to resume — run `python code.py` first.\")\n return\n print(f\"resuming {resume_id} — unchanged agent() calls hit the journal cache\\n\")\n else:\n print(\"launching workflow `review-changes`\\n\")\n\n tool = WorkflowTool()\n out = await tool.call(SAMPLE_META, sample_workflow,\n args={\"budget\": None}, resume_from_run_id=resume_id)\n\n print(\"\\nresult:\")\n for f in out[\"result\"].get(\"confirmed\", []):\n print(f\" [{f['severity']:<6}] {f['dimension']}: {f['title']}\")\n t = out[\"task\"]\n print(f\"\\nstatus={t.status} agents={t.usage['agents']} tokens={t.usage['tokens']}\"\n f\" journal=.runtime/{t.run_id}.journal.jsonl\")\n\n\nif __name__ == \"__main__\":\n asyncio.run(main(sys.argv[1:]))\n", "images": [ { "src": "/course-assets/s21_workflow_runtime/workflow-runtime-overview.svg", @@ -3789,7 +3789,7 @@ "filename": "s22_goal_loop/code.py", "title": "Goal Loop", "subtitle": "Trusted Evidence Decides When to Stop", - "loc": 212, + "loc": 209, "tools": [], "newTools": [], "coreAddition": "Goal completion gate", @@ -3797,49 +3797,49 @@ "classes": [ { "name": "Message", - "startLine": 63, - "endLine": 72 + "startLine": 59, + "endLine": 68 }, { "name": "CommandQueue", - "startLine": 73, - "endLine": 106 + "startLine": 69, + "endLine": 102 }, { "name": "GoalRuntime", - "startLine": 107, - "endLine": 197 + "startLine": 103, + "endLine": 192 }, { "name": "Session", - "startLine": 198, - "endLine": 254 + "startLine": 193, + "endLine": 249 } ], "functions": [ { "name": "make_id", "signature": "def make_id(prefix)", - "startLine": 49 + "startLine": 45 }, { "name": "event", "signature": "def event(lane, etype, detail=\"\")", - "startLine": 53 + "startLine": 49 }, { "name": "banner", "signature": "def banner(text)", - "startLine": 255 + "startLine": 250 }, { "name": "main", "signature": "def main(argv)", - "startLine": 259 + "startLine": 254 } ], "layer": "planning", - "source": "\"\"\"\ns22_goal_loop — /goal session goal loop (teaching version)\n\nClean-room behavioral reconstruction of Claude Code's `/goal` command. Grounded\nin @anthropic-ai/claude-code@2.1.177 observed behavior\n(reverse-research/cc_goal_loop), NOT leaked source.\n\nIdea:\n s01-s21 end a turn when the model emits no tool_use. `/goal` adds a\n host-owned turn-completion GATE: the user sets a stopping CONDITION, and after\n every turn a separate evaluator judges whether trusted transcript evidence\n satisfies it. Not satisfied -> the gate blocks the stop and feeds a\n continuation into the next turn. Satisfied -> the active goal is cleared.\n\n So the core contrast with s01 is one extra check before \"return\":\n\n # s01: the model says stop -> stop\n if not has_tool_use(response):\n return\n # s22: when it wants to stop, pass the goal gate first\n if not has_tool_use(response):\n verdict = goal.evaluate_after_turn()\n if verdict == \"continuing\":\n continue # not met -> push it back\n return # met / over budget / no goal -> really stop\n\nRun:\n python code.py # /goal until tests pass + deploy green; watch the gate\n\nTeaching simplifications (vs real /goal and runtime.mjs):\n - The evaluator is a deterministic keyword check, not a small/fast model.\n - One mock task-notification produces the trusted evidence; the loop / monitor\n / background-task plane (s13/s14) is out of scope — this chapter is just the\n goal gate.\n - The evidence trust boundary is the faithful part: only task-notification /\n monitor-line origins count as evidence, so the `/goal` command text, the\n continuation reminder, and plain assistant prose can NOT satisfy the goal.\n Ordinary `submit()` calls cannot set those labels; only the host-event\n ingress can deliver an allowlisted source.\n\"\"\"\n\nimport itertools\nimport sys\n\n# ---- ids + a one-line event stream so the gate is visible ----\n_ids = itertools.count(1)\n\n\ndef make_id(prefix):\n return f\"{prefix}-{next(_ids):03d}\"\n\n\ndef event(lane, etype, detail=\"\"):\n print(f\" · {lane:<6} {etype:<26} {detail}\")\n\n\n# A message's origin.kind is the TRUST LABEL that decides whether it can count\n# as goal evidence. Trusted async origins land real tool/task evidence; user /\n# slash-command / active-goal (the continuation reminder) / assistant do not.\nTRUSTED_EVIDENCE_ORIGINS = {\"task-notification\", \"monitor-line\"}\n\n\nclass Message:\n def __init__(self, role, content, origin):\n self.role = role\n self.content = content\n self.origin = origin or {\"kind\": \"user\"}\n\n\n# ============================================================\n# CommandQueue — continuation prompts live here (mirrors CommandQueue)\n# ============================================================\nclass CommandQueue:\n PRIORITY = {\"now\": 0, \"next\": 1, \"later\": 2}\n\n def __init__(self):\n self.items = []\n\n def enqueue(self, value, priority=\"next\", origin=None):\n item = {\"id\": make_id(\"cmd\"), \"priority\": priority,\n \"origin\": origin or {}, \"value\": value}\n self.items.append(item)\n return item\n\n def dequeue(self, include_goal_continuations=True):\n # Goal continuations and the external async inbox are NOT the same drain.\n # With include_goal_continuations=False an inbox drain skips them, so a\n # goal can't be advanced (or blocked) before real evidence arrives.\n self.items.sort(key=lambda i: self.PRIORITY.get(i[\"priority\"], 1))\n for idx, item in enumerate(self.items):\n if include_goal_continuations or item[\"origin\"].get(\"kind\") != \"active-goal\":\n return self.items.pop(idx)\n return None\n\n def remove_by_origin(self, kind):\n before = len(self.items)\n self.items = [i for i in self.items if i[\"origin\"].get(\"kind\") != kind]\n return before - len(self.items)\n\n def __len__(self):\n return len(self.items)\n\n\n# ============================================================\n# GoalRuntime — the turn-completion gate (mirrors GoalRuntime)\n# ============================================================\nclass GoalRuntime:\n def __init__(self, transcript, queue):\n self.transcript = transcript # shared session transcript\n self.queue = queue\n self.active = None\n\n def set_goal(self, objective, max_turns=20):\n # start_index marks the evidence window. The /goal command line is\n # already recorded, so it sits OUTSIDE the window and can't satisfy\n # itself.\n self.active = {\n \"id\": make_id(\"goal\"), \"objective\": objective, \"status\": \"active\",\n \"start_index\": len(self.transcript), \"max_turns\": max_turns,\n \"checks\": 0, \"continuation_turns\": 0,\n }\n event(\"goal\", \"goal_started\", f\"{self.active['id']} :: {objective}\")\n return self.active\n\n def clear(self, reason=\"cleared\"):\n if not self.active:\n return\n self.active[\"status\"] = reason\n self.queue.remove_by_origin(\"active-goal\")\n event(\"goal\", \"goal_cleared\", reason)\n self.active = None\n\n def evidence_text(self):\n \"\"\"The trust boundary. Three filters keep self-satisfying text out:\n drop slash-command origins, drop /goal command lines, and keep ONLY\n trusted external async origins (task-notification / monitor-line).\"\"\"\n if not self.active:\n return \"\"\n out = []\n for m in self.transcript[self.active[\"start_index\"]:]:\n if m.origin.get(\"kind\") == \"slash-command\":\n continue\n if m.role == \"user\" and m.content.strip().startswith(\"/goal\"):\n continue\n if m.origin.get(\"kind\") not in TRUSTED_EVIDENCE_ORIGINS:\n continue\n out.append(f\"{m.role}: {m.content}\")\n return \"\\n\".join(out)\n\n def goal_satisfied(self):\n # Real Claude Code routes this to a small/fast evaluator model reading\n # the evidence window. The teaching version is a deterministic keyword\n # check so the lifecycle is reproducible.\n objective = self.active[\"objective\"].lower()\n evidence = self.evidence_text().lower()\n wants_tests = \"test\" in objective\n wants_deploy = \"deploy\" in objective or \"green\" in objective\n tests_ok = not wants_tests or \"tests passed\" in evidence or \"test passed\" in evidence\n deploy_ok = not wants_deploy or \"deploy green\" in evidence or \"deployment green\" in evidence\n if any(k in objective for k in (\"until\", \"pass\", \"green\")):\n return tests_ok and deploy_ok\n return objective in evidence\n\n def evaluate_after_turn(self):\n \"\"\"The gate, run after every turn. Returns completed / continuing /\n blocked / none.\"\"\"\n g = self.active\n if not g or g[\"status\"] != \"active\":\n return \"none\"\n g[\"checks\"] += 1\n satisfied = self.goal_satisfied()\n event(\"goal\", \"goal_evaluated\", f\"check #{g['checks']} satisfied={satisfied}\")\n if satisfied:\n g[\"status\"] = \"completed\"\n self.queue.remove_by_origin(\"active-goal\")\n event(\"goal\", \"goal_completed\", g[\"id\"])\n self.active = None\n return \"completed\"\n if g[\"continuation_turns\"] < g[\"max_turns\"]:\n g[\"continuation_turns\"] += 1\n self.queue.enqueue(\n value=(f\"Continue working toward active goal {g['id']}. Use tool/task \"\n \"evidence; do not treat this reminder as completion evidence.\"),\n priority=\"next\", origin={\"kind\": \"active-goal\", \"goal_id\": g[\"id\"]})\n event(\"goal\", \"goal_continuation_enqueued\",\n f\"turn {g['continuation_turns']}/{g['max_turns']}\")\n return \"continuing\"\n g[\"status\"] = \"blocked\"\n self.queue.remove_by_origin(\"active-goal\")\n event(\"goal\", \"goal_blocked\", f\"exceeded {g['max_turns']} turns\")\n self.active = None\n return \"blocked\"\n\n\n# ============================================================\n# Session — the main loop host with a Stop gate (mirrors submit / drain)\n# ============================================================\nclass Session:\n def __init__(self):\n self.transcript = []\n self.queue = CommandQueue()\n self.goal = GoalRuntime(self.transcript, self.queue)\n\n def _add(self, role, content, origin):\n self.transcript.append(Message(role, content, origin))\n\n def submit(self, text):\n \"\"\"Submit ordinary user text. Callers cannot attach a trusted origin.\"\"\"\n return self._submit(text, {\"kind\": \"user\"})\n\n def deliver_host_event(self, text, source):\n \"\"\"Host-only ingress for validated task/monitor events.\"\"\"\n if source not in TRUSTED_EVIDENCE_ORIGINS:\n raise ValueError(f\"untrusted host event source: {source}\")\n return self._submit(text, {\"kind\": source})\n\n def _submit(self, text, origin):\n \"\"\"Run one turn with an origin already assigned by the host.\"\"\"\n self._add(\"user\", text, origin) # input recorded with its origin\n kind = origin[\"kind\"]\n\n if kind == \"user\" and text.strip().startswith(\"/goal\"):\n arg = text.strip()[5:].strip()\n self._add(\"assistant\", f\"(slash) /goal {arg}\", {\"kind\": \"slash-command\"})\n if arg in (\"\", \"clear\", \"stop\", \"off\"):\n self.goal.clear()\n else:\n self.goal.set_goal(arg)\n elif kind in TRUSTED_EVIDENCE_ORIGINS:\n # The input itself (recorded above with a trusted origin) is the\n # evidence; the assistant just observes it.\n event(\"turn\", f\"observe {kind}\", text[:48])\n self._add(\"assistant\", f\"Observed {kind}: {text}\", origin)\n elif kind == \"active-goal\":\n event(\"turn\", \"continue-goal\", \"(reminder is not evidence)\")\n self._add(\"assistant\", \"Continuing the goal; checking task/monitor evidence.\", origin)\n else:\n event(\"turn\", \"assistant-turn\", text[:48])\n self._add(\"assistant\", f\"assistant handled: {text}\", {\"kind\": \"assistant\"})\n\n return self.goal.evaluate_after_turn() # <-- the Stop gate\n\n def drain_goal_continuation(self):\n \"\"\"Pull one goal continuation back into the loop — explicit, separate\n from any external async-inbox drain.\"\"\"\n item = self.queue.dequeue(include_goal_continuations=True)\n if item and item[\"origin\"].get(\"kind\") == \"active-goal\":\n return self._submit(item[\"value\"], item[\"origin\"])\n return None\n\n\n# ============================================================\n# Demo\n# ============================================================\ndef banner(text):\n print(f\"\\n— {text} —\")\n\n\ndef main(argv):\n s = Session()\n\n banner(\"1. set a goal (the gate is now armed; window starts after the command)\")\n print(\"user> /goal until tests passed and deploy green\")\n s.submit(\"/goal until tests passed and deploy green\")\n\n banner(\"2. model works, no TRUSTED evidence yet -> the gate keeps it going\")\n s.drain_goal_continuation()\n s.submit(\"Inspecting the failing tests and the deploy config.\")\n\n banner(\"3. plain user text 'tests passed' is NOT trusted -> still not satisfied\")\n s.submit(\"tests passed, trust me\")\n s.drain_goal_continuation()\n print(f\" active goal still open: {s.goal.active is not None}\")\n\n banner(\"4. a background task lands a task-notification (trusted) -> satisfied\")\n verdict = s.deliver_host_event(\n \"tests passed; deploy green\", source=\"task-notification\"\n )\n print(f\" final verdict: goal {verdict}\")\n\n banner(\"5. budget: a goal that never gets evidence blocks after max_turns\")\n s2 = Session()\n s2.goal.set_goal(\"until tests passed\", max_turns=2)\n verdict = \"continuing\"\n while verdict == \"continuing\":\n verdict = s2.submit(\"still working, no task evidence yet\")\n print(f\" final verdict: goal {verdict}\")\n\n\nif __name__ == \"__main__\":\n main(sys.argv[1:])\n", + "source": "\"\"\"\ns22_goal_loop — minimal /goal session loop for a teaching harness\n\nIdea:\n s01-s21 end a turn when the model emits no tool_use. `/goal` adds a\n host-owned turn-completion GATE: the user sets a stopping CONDITION, and after\n every turn a separate evaluator judges whether trusted transcript evidence\n satisfies it. Not satisfied -> the gate blocks the stop and feeds a\n continuation into the next turn. Satisfied -> the active goal is cleared.\n\n So the core contrast with s01 is one extra check before \"return\":\n\n # s01: the model says stop -> stop\n if not has_tool_use(response):\n return\n # s22: when it wants to stop, pass the goal gate first\n if not has_tool_use(response):\n verdict = goal.evaluate_after_turn()\n if verdict == \"continuing\":\n continue # not met -> push it back\n return # met / over budget / no goal -> really stop\n\nRun:\n python code.py # /goal until tests pass + deploy green; watch the gate\n\nImplementation choices:\n - The evaluator is a deterministic keyword check, not a small/fast model.\n - One mock task-notification produces the trusted evidence; the loop / monitor\n / background-task plane (s13/s14) is out of scope — this chapter is just the\n goal gate.\n - The evidence trust boundary is the important part: only task-notification /\n monitor-line origins count as evidence, so the `/goal` command text, the\n continuation reminder, and plain assistant prose can NOT satisfy the goal.\n Ordinary `submit()` calls cannot set those labels; only the host-event\n ingress can deliver an allowlisted source.\n\"\"\"\n\nimport itertools\nimport sys\n\n# ---- ids + a one-line event stream so the gate is visible ----\n_ids = itertools.count(1)\n\n\ndef make_id(prefix):\n return f\"{prefix}-{next(_ids):03d}\"\n\n\ndef event(lane, etype, detail=\"\"):\n print(f\" · {lane:<6} {etype:<26} {detail}\")\n\n\n# A message's origin.kind is the TRUST LABEL that decides whether it can count\n# as goal evidence. Trusted async origins land real tool/task evidence; user /\n# slash-command / active-goal (the continuation reminder) / assistant do not.\nTRUSTED_EVIDENCE_ORIGINS = {\"task-notification\", \"monitor-line\"}\n\n\nclass Message:\n def __init__(self, role, content, origin):\n self.role = role\n self.content = content\n self.origin = origin or {\"kind\": \"user\"}\n\n\n# ============================================================\n# CommandQueue — continuation prompts live here\n# ============================================================\nclass CommandQueue:\n PRIORITY = {\"now\": 0, \"next\": 1, \"later\": 2}\n\n def __init__(self):\n self.items = []\n\n def enqueue(self, value, priority=\"next\", origin=None):\n item = {\"id\": make_id(\"cmd\"), \"priority\": priority,\n \"origin\": origin or {}, \"value\": value}\n self.items.append(item)\n return item\n\n def dequeue(self, include_goal_continuations=True):\n # Goal continuations and the external async inbox are NOT the same drain.\n # With include_goal_continuations=False an inbox drain skips them, so a\n # goal can't be advanced (or blocked) before real evidence arrives.\n self.items.sort(key=lambda i: self.PRIORITY.get(i[\"priority\"], 1))\n for idx, item in enumerate(self.items):\n if include_goal_continuations or item[\"origin\"].get(\"kind\") != \"active-goal\":\n return self.items.pop(idx)\n return None\n\n def remove_by_origin(self, kind):\n before = len(self.items)\n self.items = [i for i in self.items if i[\"origin\"].get(\"kind\") != kind]\n return before - len(self.items)\n\n def __len__(self):\n return len(self.items)\n\n\n# ============================================================\n# GoalRuntime — the turn-completion gate\n# ============================================================\nclass GoalRuntime:\n def __init__(self, transcript, queue):\n self.transcript = transcript # shared session transcript\n self.queue = queue\n self.active = None\n\n def set_goal(self, objective, max_turns=20):\n # start_index marks the evidence window. The /goal command line is\n # already recorded, so it sits OUTSIDE the window and can't satisfy\n # itself.\n self.active = {\n \"id\": make_id(\"goal\"), \"objective\": objective, \"status\": \"active\",\n \"start_index\": len(self.transcript), \"max_turns\": max_turns,\n \"checks\": 0, \"continuation_turns\": 0,\n }\n event(\"goal\", \"goal_started\", f\"{self.active['id']} :: {objective}\")\n return self.active\n\n def clear(self, reason=\"cleared\"):\n if not self.active:\n return\n self.active[\"status\"] = reason\n self.queue.remove_by_origin(\"active-goal\")\n event(\"goal\", \"goal_cleared\", reason)\n self.active = None\n\n def evidence_text(self):\n \"\"\"The trust boundary. Three filters keep self-satisfying text out:\n drop slash-command origins, drop /goal command lines, and keep ONLY\n trusted external async origins (task-notification / monitor-line).\"\"\"\n if not self.active:\n return \"\"\n out = []\n for m in self.transcript[self.active[\"start_index\"]:]:\n if m.origin.get(\"kind\") == \"slash-command\":\n continue\n if m.role == \"user\" and m.content.strip().startswith(\"/goal\"):\n continue\n if m.origin.get(\"kind\") not in TRUSTED_EVIDENCE_ORIGINS:\n continue\n out.append(f\"{m.role}: {m.content}\")\n return \"\\n\".join(out)\n\n def goal_satisfied(self):\n # A production harness can route this evidence window to a separate\n # evaluator model. The demo uses a deterministic keyword policy.\n objective = self.active[\"objective\"].lower()\n evidence = self.evidence_text().lower()\n wants_tests = \"test\" in objective\n wants_deploy = \"deploy\" in objective or \"green\" in objective\n tests_ok = not wants_tests or \"tests passed\" in evidence or \"test passed\" in evidence\n deploy_ok = not wants_deploy or \"deploy green\" in evidence or \"deployment green\" in evidence\n if any(k in objective for k in (\"until\", \"pass\", \"green\")):\n return tests_ok and deploy_ok\n return objective in evidence\n\n def evaluate_after_turn(self):\n \"\"\"The gate, run after every turn. Returns completed / continuing /\n blocked / none.\"\"\"\n g = self.active\n if not g or g[\"status\"] != \"active\":\n return \"none\"\n g[\"checks\"] += 1\n satisfied = self.goal_satisfied()\n event(\"goal\", \"goal_evaluated\", f\"check #{g['checks']} satisfied={satisfied}\")\n if satisfied:\n g[\"status\"] = \"completed\"\n self.queue.remove_by_origin(\"active-goal\")\n event(\"goal\", \"goal_completed\", g[\"id\"])\n self.active = None\n return \"completed\"\n if g[\"continuation_turns\"] < g[\"max_turns\"]:\n g[\"continuation_turns\"] += 1\n self.queue.enqueue(\n value=(f\"Continue working toward active goal {g['id']}. Use tool/task \"\n \"evidence; do not treat this reminder as completion evidence.\"),\n priority=\"next\", origin={\"kind\": \"active-goal\", \"goal_id\": g[\"id\"]})\n event(\"goal\", \"goal_continuation_enqueued\",\n f\"turn {g['continuation_turns']}/{g['max_turns']}\")\n return \"continuing\"\n g[\"status\"] = \"blocked\"\n self.queue.remove_by_origin(\"active-goal\")\n event(\"goal\", \"goal_blocked\", f\"exceeded {g['max_turns']} turns\")\n self.active = None\n return \"blocked\"\n\n\n# ============================================================\n# Session — the main loop host with a Stop gate\n# ============================================================\nclass Session:\n def __init__(self):\n self.transcript = []\n self.queue = CommandQueue()\n self.goal = GoalRuntime(self.transcript, self.queue)\n\n def _add(self, role, content, origin):\n self.transcript.append(Message(role, content, origin))\n\n def submit(self, text):\n \"\"\"Submit ordinary user text. Callers cannot attach a trusted origin.\"\"\"\n return self._submit(text, {\"kind\": \"user\"})\n\n def deliver_host_event(self, text, source):\n \"\"\"Host-only ingress for validated task/monitor events.\"\"\"\n if source not in TRUSTED_EVIDENCE_ORIGINS:\n raise ValueError(f\"untrusted host event source: {source}\")\n return self._submit(text, {\"kind\": source})\n\n def _submit(self, text, origin):\n \"\"\"Run one turn with an origin already assigned by the host.\"\"\"\n self._add(\"user\", text, origin) # input recorded with its origin\n kind = origin[\"kind\"]\n\n if kind == \"user\" and text.strip().startswith(\"/goal\"):\n arg = text.strip()[5:].strip()\n self._add(\"assistant\", f\"(slash) /goal {arg}\", {\"kind\": \"slash-command\"})\n if arg in (\"\", \"clear\", \"stop\", \"off\"):\n self.goal.clear()\n else:\n self.goal.set_goal(arg)\n elif kind in TRUSTED_EVIDENCE_ORIGINS:\n # The input itself (recorded above with a trusted origin) is the\n # evidence; the assistant just observes it.\n event(\"turn\", f\"observe {kind}\", text[:48])\n self._add(\"assistant\", f\"Observed {kind}: {text}\", origin)\n elif kind == \"active-goal\":\n event(\"turn\", \"continue-goal\", \"(reminder is not evidence)\")\n self._add(\"assistant\", \"Continuing the goal; checking task/monitor evidence.\", origin)\n else:\n event(\"turn\", \"assistant-turn\", text[:48])\n self._add(\"assistant\", f\"assistant handled: {text}\", {\"kind\": \"assistant\"})\n\n return self.goal.evaluate_after_turn() # <-- the Stop gate\n\n def drain_goal_continuation(self):\n \"\"\"Pull one goal continuation back into the loop — explicit, separate\n from any external async-inbox drain.\"\"\"\n item = self.queue.dequeue(include_goal_continuations=True)\n if item and item[\"origin\"].get(\"kind\") == \"active-goal\":\n return self._submit(item[\"value\"], item[\"origin\"])\n return None\n\n\n# ============================================================\n# Demo\n# ============================================================\ndef banner(text):\n print(f\"\\n— {text} —\")\n\n\ndef main(argv):\n s = Session()\n\n banner(\"1. set a goal (the gate is now armed; window starts after the command)\")\n print(\"user> /goal until tests passed and deploy green\")\n s.submit(\"/goal until tests passed and deploy green\")\n\n banner(\"2. model works, no TRUSTED evidence yet -> the gate keeps it going\")\n s.drain_goal_continuation()\n s.submit(\"Inspecting the failing tests and the deploy config.\")\n\n banner(\"3. plain user text 'tests passed' is NOT trusted -> still not satisfied\")\n s.submit(\"tests passed, trust me\")\n s.drain_goal_continuation()\n print(f\" active goal still open: {s.goal.active is not None}\")\n\n banner(\"4. a background task lands a task-notification (trusted) -> satisfied\")\n verdict = s.deliver_host_event(\n \"tests passed; deploy green\", source=\"task-notification\"\n )\n print(f\" final verdict: goal {verdict}\")\n\n banner(\"5. budget: a goal that never gets evidence blocks after max_turns\")\n s2 = Session()\n s2.goal.set_goal(\"until tests passed\", max_turns=2)\n verdict = \"continuing\"\n while verdict == \"continuing\":\n verdict = s2.submit(\"still working, no task evidence yet\")\n print(f\" final verdict: goal {verdict}\")\n\n\nif __name__ == \"__main__\":\n main(sys.argv[1:])\n", "images": [ { "src": "/course-assets/s22_goal_loop/goal-loop-overview.svg", @@ -4291,7 +4291,7 @@ "newTools": [ "review-changes" ], - "locDelta": -1278 + "locDelta": -1285 }, { "from": "s21", @@ -4309,7 +4309,7 @@ "main" ], "newTools": [], - "locDelta": -218 + "locDelta": -214 } ] } \ No newline at end of file