mirror of
https://github.com/shareAI-lab/analysis_claude_code.git
synced 2026-09-20 12:13:38 +08:00
refactor: streamline the course to 17 lessons
This commit is contained in:
Haoran
245
s16_workflow_runtime/README.ja.md
Normal file
245
s16_workflow_runtime/README.ja.md
Normal file
@@ -0,0 +1,245 @@
|
||||
# s16: Workflow Runtime — モデルが単一 step を決め、script が orchestration を決める
|
||||
|
||||
[English](README.md) · [中文](README.zh.md) · [日本語](README.ja.md)
|
||||
|
||||
s01 → ... → s14 → [s15](../s15_integrated_harness/) → `s16` → [s17](../s17_goal_loop/)
|
||||
|
||||
> *「1 回の tool_use で、一式の orchestration を実行する」* — `Workflow` ツールが復元可能な script runtime を起動し、多数の agent call を協調させます。
|
||||
>
|
||||
> **Harness 層**: Orchestration — single-agent loop の上で保存済み multi-agent script を実行します。
|
||||
|
||||
---
|
||||
|
||||
s01 から s15 まで、各 round で model が呼び出す tools を決めます。tool results が `messages[]` に入ると、model は更新された context から次の step を決めます。次の経路が前の step の発見に依存する task に向いています。
|
||||
|
||||
一方、固定された流れを繰り返す task もあります。code review なら、複数の観点を同時に調べ、各 finding を検証し、重複をまとめて severity 順に並べます。実行前に step と順序が分かっている場合、host には次の 3 つが必要です。
|
||||
|
||||
- **並行性**: 1 件ずつ順番に待たないこと。
|
||||
- **安定した結果構造**: 個々の agent answer が変わっても構造を保つこと。
|
||||
- **復元可能性**: 途中で止まっても、完了済みの部分を最初からやり直さないこと。
|
||||
|
||||
この orchestration が conversation history にしか存在しなければ、順序と checkpoint も history にしか残りません。saved workflow は固定 flow を code に置き、完了した call を journal に記録します。
|
||||
|
||||
## 計画は chat のラウンドを重ねず、コードに書く
|
||||
|
||||
harness の tool pool に `Workflow` ツールを追加します。host は `agent() / parallel() / pipeline() / phase()` で構成した trusted script を登録します。model が渡すのは saved workflow name、argument、任意の resume run ID だけで、実行可能 code や metadata は渡しません。
|
||||
|
||||
workflow は 1 回の `tool_use` として main loop に入ります。script の実行中、runtime は lifecycle event と progress event を出し、各 step を disk journal へ記録します。script が終わると、この call は launch 情報、result、task state を返します。script の中間結果は変数に保存され、conversation history を使いません。`resume_from_run_id` で再開すると、変更されていない `agent()` は journal の結果を再利用します。
|
||||
|
||||
|
||||
|
||||
```python
|
||||
SAMPLE_META = {"name": "review-changes", "description": "コード変更を review", "phases": ["Review", "Verify"]}
|
||||
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify) # 各 dimension が独立して audit → verify を通る
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"{len(confirmed)} 件の実在する問題を確認")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
|
||||
## Workflow ツール: 1 回の call で run 全体を実行する
|
||||
|
||||
`Workflow` は s15 host の既存 tool pool に追加されます。ユーザーが保存済み workflow の実行を求めるか、タスクが既知の orchestration に一致したときにモデルがこのツールを選びます。adapter は name を host-owned `WORKFLOWS` registry で解決し、trusted metadata と function を runtime へ渡します。s15 の他の tools も同じ loop で利用できます。
|
||||
|
||||
model-facing schema が受け取るのは `name`、`args`、`resume_from_run_id` です。unknown name や不正 argument は error tool result として返し、host loop を終了させません。その後 runtime が登録済み metadata を検証し、permission check を通し、local workflow task を登録して、script の実行前に `async_launched` を出します。progress event と最後の `task_notification` が続き、call は JSON-safe な launch 情報、result、task state を返します。
|
||||
|
||||
```python
|
||||
WORKFLOW_TOOL = {
|
||||
"name": "Workflow",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"args": {"type": "object"},
|
||||
"resume_from_run_id": {"type": "string"},
|
||||
},
|
||||
"required": ["name"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
}
|
||||
|
||||
async def run_workflow(name, args=None, resume_from_run_id=None):
|
||||
meta, script_fn = WORKFLOWS[name]
|
||||
out = await WorkflowTool().call(
|
||||
meta, script_fn,
|
||||
args=args,
|
||||
resume_from_run_id=resume_from_run_id,
|
||||
)
|
||||
return {"launched": out["launched"], "result": out["result"],
|
||||
"task": serialize_task(out["task"])}
|
||||
```
|
||||
|
||||
## Workflow metadata: 起動前に検証する
|
||||
|
||||
各 saved workflow は `name`、`description`、任意の `phases` を持つ trusted metadata を登録します。runtime は workflow code を実行する前に検証します。`name` と `description` は task と UI の表示に使い、`phases` は progress 表示の group 名を定義します。これらは model input ではなく host registry に属します。
|
||||
|
||||
不正な登録内容は launch 前に `WorkflowInputError` になります。s12 の cron 式検証と同じ考えです。不正な saved workflow が実行時まで進んでから壊れないようにします。
|
||||
|
||||
runtime は `meta.name` をローカル artifact のファイル名に使うため、英数字で始まり、英数字、`.`、`_`、`-` のみからなる 1-64 文字の安全な slug も要求する。
|
||||
|
||||
```python
|
||||
def validate_meta(meta):
|
||||
if not isinstance(meta, dict):
|
||||
raise WorkflowInputError("meta は object literal でなければなりません")
|
||||
if not meta.get("name") or not meta.get("description"):
|
||||
raise WorkflowInputError("meta には name と description が必要です")
|
||||
if not isinstance(meta["name"], str) or not WORKFLOW_NAME_RE.fullmatch(meta["name"]):
|
||||
raise WorkflowInputError("meta.name は安全な 1-64 文字の slug が必要です")
|
||||
if "phases" in meta and (
|
||||
not isinstance(meta["phases"], list)
|
||||
or not all(isinstance(p, str) and p for p in meta["phases"])
|
||||
):
|
||||
raise WorkflowInputError("meta.phases は空でない文字列だけを含む必要があります")
|
||||
return meta
|
||||
```
|
||||
|
||||
## Orchestration primitive
|
||||
|
||||
script は少数の orchestration primitive だけを公開する `ExecutionState` を受け取り、ファイルを直接読み書きせず、shell も実行しません。default の interactive mode では `agent()` を host と同じ real API client に接続し、各 workflow agent は arguments で渡された内容だけを読みます。`demo` と unit test は `MockAgentRunner` を使い、event と journal replay を繰り返し確認できるようにします。
|
||||
|
||||
| Primitive | 役割 |
|
||||
|------|------|
|
||||
| `agent(prompt, {schema, label, phase})` | 1 つの subagent を派遣 |
|
||||
| `parallel(thunks)` | **barrier**: すべての task を並行実行し、全結果が戻るまで待つ |
|
||||
| `pipeline(items, *stages)` | 各 item を **barrier なし**で stage ごとに実行し、終わった item から先へ進める |
|
||||
| `phase(title)` | 現在の progress phase を記録し、progress bar を更新 |
|
||||
| `log(message)` | progress log を 1 行出力 |
|
||||
| `workflow(name, args)` | nested sub-workflow（1 階層だけ） |
|
||||
|
||||
各 item が同じ stage を独立して通る場合は `pipeline` を使えます。item A が stage 3 にいる間、item B はまだ stage 1 かもしれません。次の処理が前の group の全結果を必要とする場合は `parallel` を使います。
|
||||
|
||||
```python
|
||||
async def pipeline(self, items, *stages):
|
||||
async def run_item(item, idx):
|
||||
value = item
|
||||
for stage in stages: # 各 item がすべての stage を独立して完走
|
||||
value = await stage(value, item, idx)
|
||||
return value
|
||||
return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])
|
||||
```
|
||||
|
||||
## 構造化出力: Subagent に散文を返させない
|
||||
|
||||
`agent({schema})` は、schema に一致する JSON object だけを返すよう workflow agent に要求します。runtime は結果を parse、validate し、不一致なら 1 回 retry します。下流コードは prose から field を取り出さず、object を受け取れます。
|
||||
|
||||
s05 では tool argument を全面的に信頼できないと説明しました。ここでは同じ教訓を逆向きに使います。subagent の出力も全面的には信頼できません。orchestration boundary で検証し、1 回 retry の機会を与え、不確実性を後続 flow の外へ止めます。
|
||||
|
||||
```python
|
||||
run = await asyncio.to_thread(self.runner.run, prompt, schema, label)
|
||||
result = run.value
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok: # 1 回だけ注意して retry、それでも不正なら error
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run, prompt + "\n\n有効な JSON を返してください。", schema, label
|
||||
)
|
||||
result = retry.value
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(f"agent({{schema}}) の出力が不正です: {err}")
|
||||
```
|
||||
|
||||
## Task state と progress event
|
||||
|
||||
`LocalWorkflowTask` は status と token usage を管理し、SDK style の event stream を外へ出します。`task_started` → phase change、subagent start、log を含む一連の `task_progress` → 完了または失敗に加え、output file、agent 数、token 数を含む最後の `task_notification` です。
|
||||
|
||||
demo はこれらの event を順番に表示し、最後の notification の後で task state を返します。
|
||||
|
||||
```python
|
||||
class LocalWorkflowTask:
|
||||
def progress_event(self, ptype, **data): # phase/subagent/log
|
||||
self.progress.append({"type": ptype, **data})
|
||||
print(f" progress {ptype} ...")
|
||||
```
|
||||
|
||||
## 保存: Snapshot + journal で中断から再開する
|
||||
|
||||
runtime は各 run を `s16_workflow_runtime/.runtime/` に保存します。`<runId>.json` snapshot、`<runId>.output.json` output、`<runId>.journal.jsonl` journal、`<runId>.lock` coordination file です。fresh run は journal を開く前に exclusive file creation で新しい `runId` を予約します。run lock は実行と最終永続化が終わるまで保持するため、別 process は同じ run を同時に resume できません。snapshot に workflow name、arguments、task state を記録し、resume は保存済み snapshot と journal を先に検証してから、成功済み artifact を変更します。
|
||||
|
||||
journal は checkpoint resume の中心で、各 `agent()` の結果を 1 行ずつ記録します。
|
||||
|
||||
```python
|
||||
class WorkflowJournal:
|
||||
def record(self, key, value):
|
||||
self._f.write(json.dumps({"key": key, "value": value}) + "\n")
|
||||
self._f.flush()
|
||||
self.cache[key] = value
|
||||
```
|
||||
|
||||
## Resume: runId から続行し、変更のないものを再利用する
|
||||
|
||||
`resume_from_run_id` を渡して workflow を再度呼ぶと script を再実行しますが、各 `agent()` は決定的な semantic key を計算します。journal に key があれば、再実行せず cached result を返します。変更された call と、それに依存する後続 step だけが本当に動きます。
|
||||
|
||||
key は concurrency の完了順に依存してはいけません。`parallel` と `pipeline` の Agent は不定の順番で完了します。「何番目に完了したか」を key にすると、次回の cache が別の call へ対応してしまいます。そのため key は競合する counter ではなく、call の内容、つまり type、label、prompt、schema の stable hash です。
|
||||
|
||||
```python
|
||||
def key(self, kind, label, prompt, schema):
|
||||
basis = f"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}"
|
||||
return f"{kind}-{_stable_hash(basis) % 10**10:010d}"
|
||||
|
||||
# agent() の内部:
|
||||
cached = self.journal.cached(key)
|
||||
if cached is not MISS:
|
||||
self.task.progress_event("workflow_agent", label=label, status="cached")
|
||||
return cached
|
||||
```
|
||||
|
||||
## Stable call key
|
||||
|
||||
resume では、現在の各 `agent()` call を以前の journal record と対応付ける必要があります。stable hash は変更されていない workflow code と arguments に同じ call key を与えます。real model の出力は変化しても、call 内容が同じなら journal に保存済みの result を使います。
|
||||
|
||||
## 実際に動かす
|
||||
|
||||
sample workflow `review-changes` は `pipeline` を使い、各 review dimension を独立して audit → verify へ通します。interactive mode は real API を使い、`args.changes` から review 対象を読みます。`demo` は固定 runner data で pipeline、validation、journal、resume を示します。
|
||||
|
||||
```python
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
changes = args.get("changes", "")
|
||||
|
||||
async def audit(_v, dimension, _i):
|
||||
out = await ctx.agent(f"この変更に {dimension} 関連の問題がないか確認してください:\n{changes}",
|
||||
schema=FINDINGS_SCHEMA, label=f"audit:{dimension}", phase="Review")
|
||||
return {"dimension": dimension, "findings": out["findings"]}
|
||||
|
||||
async def verify(audited, dimension, _i):
|
||||
ctx.phase("Verify")
|
||||
verdicts = await ctx.parallel([ # 各 finding を独立して verify
|
||||
(lambda f=f: ctx.agent(f"変更内容に照らして finding を検証してください:\n{changes}\n\n{f}",
|
||||
schema=VERDICT_SCHEMA, label=f"verify:{dimension}:{f['title']}"))
|
||||
for f in audited["findings"]])
|
||||
return {"dimension": dimension,
|
||||
"confirmed": [f for f, v in zip(audited["findings"], verdicts) if v and v["isReal"]]}
|
||||
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
...
|
||||
```
|
||||
|
||||
## s15 からの変更点
|
||||
|
||||
| | s15 Integrated Harness | s16 Workflow Runtime |
|
||||
|--|-----------|---------------------|
|
||||
| loop | 1 つ、モデル駆動 | main loop は不変。tool の背後で script orchestration を実行 |
|
||||
| 次の step を決めるもの | モデルが毎ラウンド判断 | script が orchestration flow を事前に定義 |
|
||||
| multi-agent | s06 subagent を一度だけ派遣 | agent-runner boundary を通る scripted、resumable call |
|
||||
| 新しい仕組み | — | orchestration primitive、host registry と tool adapter、task lifecycle、progress event、journal/resume、structured output |
|
||||
|
||||
s16 は main loop を置き換えません。tool layer に `Workflow` を公開し、背後で local workflow runtime を起動します。saved script が agent-runner boundary を通じて N 回の call を協調させます。s06 の subagent はモデルがその場で 1 回派遣し、s16 は orchestration を resumable な host code にします。
|
||||
|
||||
## 試してみる
|
||||
|
||||
```bash
|
||||
python s16_workflow_runtime/code.py # main model と Workflow agent の両方が real API を使う
|
||||
python s16_workflow_runtime/code.py demo # deterministic fixture と event stream を確認
|
||||
python s16_workflow_runtime/code.py resume # 前回の runId から resume。すべての agent() が journal cache に当たる
|
||||
```
|
||||
|
||||
default command では、model に changes を読ませ、その text を `args.changes` に入れて保存済み `review-changes` workflow を実行させます。main model と workflow agent の両方が real API を使います。`demo` は固定 runner data で lifecycle と resume を繰り返し観察でき、すべて cache hit した resume は `agents=0 tokens=0` と表示されます。
|
||||
|
||||
## 次へ
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) は、より小さな独立 loop で goal が達成されたかを確認し、次の round が必要かを判断します。
|
||||
|
||||
<!-- translation-sync: zh@v10, en@v10, ja@v10 -->
|
||||
245
s16_workflow_runtime/README.md
Normal file
245
s16_workflow_runtime/README.md
Normal file
@@ -0,0 +1,245 @@
|
||||
# s16: Workflow Runtime — The Model Decides Each Step; a Script Decides the Orchestration
|
||||
|
||||
[English](README.md) · [中文](README.zh.md) · [日本語](README.ja.md)
|
||||
|
||||
s01 → ... → s14 → [s15](../s15_integrated_harness/) → `s16` → [s17](../s17_goal_loop/)
|
||||
|
||||
> *"One tool_use runs an entire orchestration"* — The `Workflow` tool starts a recoverable script runtime that coordinates many agent calls.
|
||||
>
|
||||
> **Harness layer**: Orchestration — run saved multi-agent scripts above the single-agent loop.
|
||||
|
||||
---
|
||||
|
||||
From s01 through s15, the model decides which tools to call in each round. Their results enter `messages[]`, and the model decides the next step from the updated context. This works well when the path depends on what the previous step discovers.
|
||||
|
||||
Some tasks repeat a fixed sequence. A code review may inspect several dimensions concurrently, verify each finding, combine duplicates, and sort the result. The sequence and dependencies are known before execution. Here the host needs three things:
|
||||
|
||||
- **Parallelism**, rather than waiting for one item at a time;
|
||||
- **A stable result structure**, even when individual agent answers vary;
|
||||
- **Recoverability**, so an interruption does not rerun work that is already complete.
|
||||
|
||||
If this orchestration exists only in conversation history, its ordering and checkpoints also exist only in that history. A saved workflow puts the fixed sequence in code and records completed calls in a journal.
|
||||
|
||||
## Put the Plan in Code, Not in a Sequence of Chat Turns
|
||||
|
||||
Add a `Workflow` tool to the harness tool pool. The host registers trusted scripts built from `agent()`, `parallel()`, `pipeline()`, and `phase()`. The model supplies only a saved workflow name, arguments, and an optional run ID to resume; it does not send executable code or metadata.
|
||||
|
||||
The workflow enters the main loop as one `tool_use`. As the script runs, the runtime emits lifecycle and progress events and records every step in a journal on disk. When the script finishes, the call returns the launch envelope, result, and task state. Intermediate script results live in variables instead of taking space in conversation history. When restarted with `resume_from_run_id`, unchanged `agent()` calls hit the journal cache and reuse previous results.
|
||||
|
||||
|
||||
|
||||
```python
|
||||
SAMPLE_META = {"name": "review-changes", "description": "Review code changes", "phases": ["Review", "Verify"]}
|
||||
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify) # Each dimension independently runs audit → verify
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"Confirmed {len(confirmed)} real issues")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
|
||||
## The Workflow Tool: One Call, One Complete Run
|
||||
|
||||
`Workflow` is added to the s15 host's existing tool pool. The user can request a saved workflow, or the model can select it when a task matches a known orchestration. The adapter resolves the name through the host-owned `WORKFLOWS` registry, then passes its trusted metadata and function to the runtime. The other s15 tools remain available in the same loop.
|
||||
|
||||
The model-facing schema accepts `name`, `args`, and `resume_from_run_id`. Unknown names and malformed arguments become an error tool result instead of ending the host loop. The runtime then validates the registered metadata, checks permissions, registers a local workflow task, and emits `async_launched` before running the script. Progress events follow, then the final `task_notification`; the call returns JSON-safe launch information, result, and task state.
|
||||
|
||||
```python
|
||||
WORKFLOW_TOOL = {
|
||||
"name": "Workflow",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"args": {"type": "object"},
|
||||
"resume_from_run_id": {"type": "string"},
|
||||
},
|
||||
"required": ["name"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
}
|
||||
|
||||
async def run_workflow(name, args=None, resume_from_run_id=None):
|
||||
meta, script_fn = WORKFLOWS[name]
|
||||
out = await WorkflowTool().call(
|
||||
meta, script_fn,
|
||||
args=args,
|
||||
resume_from_run_id=resume_from_run_id,
|
||||
)
|
||||
return {"launched": out["launched"], "result": out["result"],
|
||||
"task": serialize_task(out["task"])}
|
||||
```
|
||||
|
||||
## Workflow Metadata: Validate Before Launch
|
||||
|
||||
Each saved workflow registers trusted metadata with `name`, `description`, and optional `phases`. The runtime validates it before executing workflow code. `name` and `description` identify the task in the UI, while `phases` names groups in the progress display. These fields belong to the host registry, not to model input.
|
||||
|
||||
Invalid registration raises `WorkflowInputError` before launch. This is the same idea as validating cron expressions in s12: do not wait until execution to discover a bad saved workflow.
|
||||
|
||||
Because the runtime uses `meta.name` in local artifact filenames, it also requires a 1-64 character safe slug containing letters, numbers, `.`, `_`, or `-`.
|
||||
|
||||
```python
|
||||
def validate_meta(meta):
|
||||
if not isinstance(meta, dict):
|
||||
raise WorkflowInputError("meta must be an object literal")
|
||||
if not meta.get("name") or not meta.get("description"):
|
||||
raise WorkflowInputError("meta requires name and description")
|
||||
if not isinstance(meta["name"], str) or not WORKFLOW_NAME_RE.fullmatch(meta["name"]):
|
||||
raise WorkflowInputError("meta.name must be a safe 1-64 character slug")
|
||||
if "phases" in meta and (
|
||||
not isinstance(meta["phases"], list)
|
||||
or not all(isinstance(p, str) and p for p in meta["phases"])
|
||||
):
|
||||
raise WorkflowInputError("meta.phases must contain non-empty strings")
|
||||
return meta
|
||||
```
|
||||
|
||||
## Orchestration Primitives
|
||||
|
||||
A script receives an `ExecutionState` exposing a small set of orchestration primitives. It does not read files or run shell commands directly. The default interactive mode connects `agent()` to the same real API client as the host, and each workflow agent reads only the content supplied through workflow arguments. `demo` and unit tests use `MockAgentRunner` so events and journal replay are repeatable.
|
||||
|
||||
| Primitive | Purpose |
|
||||
|------|------|
|
||||
| `agent(prompt, {schema, label, phase})` | Dispatch one subagent |
|
||||
| `parallel(thunks)` | **Barrier**: run every task concurrently and wait until all results return |
|
||||
| `pipeline(items, *stages)` | Run each item through stages **without a barrier**; finished items proceed immediately |
|
||||
| `phase(title)` | Mark the current progress phase and update the progress display |
|
||||
| `log(message)` | Emit a progress log line |
|
||||
| `workflow(name, args)` | Run a nested sub-workflow, one level only |
|
||||
|
||||
Use `pipeline` when each item independently crosses the same stages. Item A may reach stage three while item B is still in stage one. Use `parallel` when the next step needs every result from the preceding group.
|
||||
|
||||
```python
|
||||
async def pipeline(self, items, *stages):
|
||||
async def run_item(item, idx):
|
||||
value = item
|
||||
for stage in stages: # Each item independently completes every stage
|
||||
value = await stage(value, item, idx)
|
||||
return value
|
||||
return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])
|
||||
```
|
||||
|
||||
## Structured Output: Do Not Let Subagents Return Essays
|
||||
|
||||
`agent({schema})` asks a workflow agent to return only a JSON object matching the schema. The runtime parses and validates the result, then retries once if it does not match. Downstream code receives an object instead of extracting fields from prose.
|
||||
|
||||
s05 warned that tool arguments cannot be trusted completely. This is the same lesson in reverse: subagent output cannot be trusted completely either. Validate at the orchestration boundary, give one retry, and keep uncertainty out of the rest of the flow.
|
||||
|
||||
```python
|
||||
run = await asyncio.to_thread(self.runner.run, prompt, schema, label)
|
||||
result = run.value
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok: # Retry once with a reminder, then fail
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run, prompt + "\n\nReturn valid JSON.", schema, label
|
||||
)
|
||||
result = retry.value
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(f"agent({{schema}}) returned invalid output: {err}")
|
||||
```
|
||||
|
||||
## Task State and Progress Events
|
||||
|
||||
`LocalWorkflowTask` maintains status and token usage and emits an SDK-style event stream: `task_started` → a sequence of `task_progress` events containing phase changes, subagent starts, and log batches → one final `task_notification` reporting completion or failure, plus the output file and agent and token counts.
|
||||
|
||||
The demo prints these events in order and returns the task state after the final notification.
|
||||
|
||||
```python
|
||||
class LocalWorkflowTask:
|
||||
def progress_event(self, ptype, **data): # Phase/subagent/log
|
||||
self.progress.append({"type": ptype, **data})
|
||||
print(f" progress {ptype} ...")
|
||||
```
|
||||
|
||||
## Storage: Snapshot + Journal for Resuming after Interruptions
|
||||
|
||||
The runtime stores each run under `s16_workflow_runtime/.runtime/`: a `<runId>.json` snapshot, `<runId>.output.json` output, `<runId>.journal.jsonl` journal, and `<runId>.lock` coordination file. Every fresh run reserves a new `runId` with exclusive file creation before opening its journal. The run lock stays held through execution and final persistence, so another process cannot resume the same run at the same time. Its snapshot records the workflow name, arguments, and task state; resume validates the saved snapshot and journal before changing either successful artifact.
|
||||
|
||||
The journal is the core of checkpointed resume. It records every `agent()` result one line at a time:
|
||||
|
||||
```python
|
||||
class WorkflowJournal:
|
||||
def record(self, key, value):
|
||||
self._f.write(json.dumps({"key": key, "value": value}) + "\n")
|
||||
self._f.flush()
|
||||
self.cache[key] = value
|
||||
```
|
||||
|
||||
## Resume: Continue by runId and Reuse Everything Unchanged
|
||||
|
||||
Calling the workflow again with `resume_from_run_id` reruns the script, but every `agent()` computes a deterministic semantic key. If that key is present in the journal, it returns the cached result without executing again. Every unchanged call hits the cache; only a changed call and the downstream steps that depend on it actually rerun.
|
||||
|
||||
The key detail is that keys cannot depend on concurrency order. Agents in `parallel` and `pipeline` finish in nondeterministic order. If "the nth completion" became the key, cache entries would map to the wrong calls on the next run. A key therefore uses a stable hash of call content, including type, label, prompt, and schema, rather than a shared counter:
|
||||
|
||||
```python
|
||||
def key(self, kind, label, prompt, schema):
|
||||
basis = f"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}"
|
||||
return f"{kind}-{_stable_hash(basis) % 10**10:010d}"
|
||||
|
||||
# Inside agent():
|
||||
cached = self.journal.cached(key)
|
||||
if cached is not MISS:
|
||||
self.task.progress_event("workflow_agent", label=label, status="cached")
|
||||
return cached
|
||||
```
|
||||
|
||||
## Stable Call Keys
|
||||
|
||||
On resume, the runtime must match each current `agent()` call with its earlier journal record. A stable hash gives unchanged workflow code and arguments the same call key. Real model output may vary; when the call content has not changed, resume uses the result already saved in the journal.
|
||||
|
||||
## See It Run
|
||||
|
||||
The sample `review-changes` workflow uses `pipeline` to send each review dimension independently through audit → verify. Interactive mode uses the real API and reads the material to review from `args.changes`. `demo` uses fixed runner data to show pipeline, validation, journal, and resume behavior.
|
||||
|
||||
```python
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
changes = args.get("changes", "")
|
||||
|
||||
async def audit(_v, dimension, _i):
|
||||
out = await ctx.agent(f"Inspect this change for {dimension} issues:\n{changes}",
|
||||
schema=FINDINGS_SCHEMA, label=f"audit:{dimension}", phase="Review")
|
||||
return {"dimension": dimension, "findings": out["findings"]}
|
||||
|
||||
async def verify(audited, dimension, _i):
|
||||
ctx.phase("Verify")
|
||||
verdicts = await ctx.parallel([ # Verify every finding independently
|
||||
(lambda f=f: ctx.agent(f"Verify this finding against the change:\n{changes}\n\n{f}",
|
||||
schema=VERDICT_SCHEMA, label=f"verify:{dimension}:{f['title']}"))
|
||||
for f in audited["findings"]])
|
||||
return {"dimension": dimension,
|
||||
"confirmed": [f for f, v in zip(audited["findings"], verdicts) if v and v["isReal"]]}
|
||||
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
...
|
||||
```
|
||||
|
||||
## Changes from s15
|
||||
|
||||
| | s15 Integrated Harness | s16 Workflow Runtime |
|
||||
|--|-----------|---------------------|
|
||||
| Loop | One model-driven loop | Main loop unchanged; a tool runs scripted orchestration |
|
||||
| Who decides the next step | Model decides each round | Script declares the orchestration in advance |
|
||||
| Multiple agents | One-shot s06 subagents | Scripted, resumable calls through an agent-runner boundary |
|
||||
| New mechanisms | — | Script primitives, host registry and tool adapter, task lifecycle, progress events, journal/resume, structured output |
|
||||
|
||||
s16 does not replace the main loop. It exposes `Workflow` at the tool layer and starts a local workflow runtime behind it: one saved script coordinates N calls through an agent-runner boundary. An s06 subagent is dispatched once at the model's discretion; s16 turns the orchestration into resumable host code.
|
||||
|
||||
## Try It
|
||||
|
||||
```bash
|
||||
python s16_workflow_runtime/code.py # Both the main model and Workflow agents use the real API
|
||||
python s16_workflow_runtime/code.py demo # Deterministic review-changes fixture and event stream
|
||||
python s16_workflow_runtime/code.py resume # Resume by the last runId; every agent() hits the journal cache
|
||||
```
|
||||
|
||||
In the default command, ask the model to read the changes, place that text in `args.changes`, and run the saved `review-changes` workflow. Both the main model and workflow agents use the real API. The `demo` command uses fixed runner data so lifecycle and resume behavior can be observed repeatedly. A resumed demo reports `agents=0 tokens=0` when every call hits the cache.
|
||||
|
||||
## Next
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) uses a smaller, independent loop to check whether a stated goal has been reached and decide whether another turn is needed.
|
||||
|
||||
<!-- translation-sync: zh@v10, en@v10, ja@v10 -->
|
||||
245
s16_workflow_runtime/README.zh.md
Normal file
245
s16_workflow_runtime/README.zh.md
Normal file
@@ -0,0 +1,245 @@
|
||||
# s16: Workflow Runtime — 模型决定单步，脚本决定编排
|
||||
|
||||
[English](README.md) · [中文](README.zh.md) · [日本語](README.ja.md)
|
||||
|
||||
s01 → ... → s14 → [s15](../s15_integrated_harness/) → `s16` → [s17](../s17_goal_loop/)
|
||||
|
||||
> *"一次 tool_use，跑完一整套编排"* — `Workflow` 工具启动一个可恢复的脚本运行时，协调多次 agent 调用。
|
||||
>
|
||||
> **Harness 层**: 编排 — 在单 agent 循环之上，执行保存好的多 agent 脚本。
|
||||
|
||||
---
|
||||
|
||||
从 s01 到 s15，每一轮都由模型决定调用哪些工具。工具结果进入 `messages[]` 后，模型再根据更新后的上下文决定下一步。当后续路径取决于上一步发现了什么时，这种方式很合适。
|
||||
|
||||
有些任务会重复一套固定流程。例如代码审查可以同时检查多个维度，再逐条验证发现、合并重复项并按严重程度排序。执行前已经知道步骤及其先后关系，这时宿主需要三样东西：
|
||||
|
||||
- **并行**，别一个一个串着等；
|
||||
- **稳定的结果结构**，即使每个 agent 的回答会变化；
|
||||
- **可恢复**，跑到一半断了，已经做完的部分别从头再来。
|
||||
|
||||
如果这套编排只存在于对话历史里，步骤顺序和检查点也只存在于历史里。保存好的 workflow 把固定流程写进代码，并在 journal 中记录已经完成的调用。
|
||||
|
||||
## 计划写在代码里，不是靠聊天一轮轮凑
|
||||
|
||||
在 harness 的工具池里加入一个 `Workflow` 工具。宿主注册由 `agent() / parallel() / pipeline() / phase()` 组成的可信脚本。模型只提供保存好的 workflow 名称、参数和可选的续跑 run ID，不会提交可执行代码或元数据。
|
||||
|
||||
workflow 以一次 `tool_use` 进入主循环。脚本运行时，runtime 会发出生命周期和进度事件，并把每一步写进磁盘上的 journal。脚本结束后，这次调用返回启动信息、结果和任务状态。脚本里的中间结果存在变量里，不会塞进对话历史。下次用 `resume_from_run_id` 重启时，没改过的 `agent()` 会直接使用 journal 中的结果。
|
||||
|
||||
|
||||
|
||||
```python
|
||||
SAMPLE_META = {"name": "review-changes", "description": "审查代码改动", "phases": ["Review", "Verify"]}
|
||||
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify) # 每个维度独立走 审计 → 验证
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"确认了 {len(confirmed)} 个真实问题")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
|
||||
## Workflow 工具：一次调用，完成整次运行
|
||||
|
||||
`Workflow` 会加入 s15 宿主已有的工具池。用户可以要求运行一个保存好的 workflow，模型也可以在任务匹配已知编排时选择这个工具。适配器会用名称查询宿主管理的 `WORKFLOWS` registry，再把可信的元数据和函数交给运行时；s15 的其他工具仍在同一个循环里可用。
|
||||
|
||||
模型可见的 schema 只接受 `name`、`args` 和 `resume_from_run_id`。名称未知或参数格式错误时，适配器会返回错误工具结果，不会让宿主循环退出。随后运行时校验已经注册的元数据、经过权限检查、注册本地 workflow 任务，并在执行脚本前发出 `async_launched`。进度事件和最终的 `task_notification` 随后到达；调用返回可写入 JSON 的启动信息、结果和任务状态。
|
||||
|
||||
```python
|
||||
WORKFLOW_TOOL = {
|
||||
"name": "Workflow",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"args": {"type": "object"},
|
||||
"resume_from_run_id": {"type": "string"},
|
||||
},
|
||||
"required": ["name"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
}
|
||||
|
||||
async def run_workflow(name, args=None, resume_from_run_id=None):
|
||||
meta, script_fn = WORKFLOWS[name]
|
||||
out = await WorkflowTool().call(
|
||||
meta, script_fn,
|
||||
args=args,
|
||||
resume_from_run_id=resume_from_run_id,
|
||||
)
|
||||
return {"launched": out["launched"], "result": out["result"],
|
||||
"task": serialize_task(out["task"])}
|
||||
```
|
||||
|
||||
## Workflow 元数据：启动前先校验
|
||||
|
||||
每个保存好的 workflow 都会注册一份可信元数据，包含 `name`、`description` 和可选的 `phases`。运行时会在执行 workflow 代码前校验它：`name` 和 `description` 用来标识任务，`phases` 给进度显示分组命名。这些字段属于宿主 registry，不是模型输入。
|
||||
|
||||
注册内容不合法时，运行时会在启动前抛出 `WorkflowInputError`。这和 s12 校验 cron 表达式是一个思路：保存好的 workflow 有问题，就不要等到执行时才发现。
|
||||
|
||||
运行时会把 `meta.name` 用在本地产物文件名中，因此还要求它是 1-64 个字符的安全 slug，只能包含字母、数字、`.`、`_`、`-`。
|
||||
|
||||
```python
|
||||
def validate_meta(meta):
|
||||
if not isinstance(meta, dict):
|
||||
raise WorkflowInputError("meta 必须是对象字面量")
|
||||
if not meta.get("name") or not meta.get("description"):
|
||||
raise WorkflowInputError("meta 必须包含 name 和 description")
|
||||
if not isinstance(meta["name"], str) or not WORKFLOW_NAME_RE.fullmatch(meta["name"]):
|
||||
raise WorkflowInputError("meta.name 必须是 1-64 字符的安全 slug")
|
||||
if "phases" in meta and (
|
||||
not isinstance(meta["phases"], list)
|
||||
or not all(isinstance(p, str) and p for p in meta["phases"])
|
||||
):
|
||||
raise WorkflowInputError("meta.phases 必须包含非空字符串")
|
||||
return meta
|
||||
```
|
||||
|
||||
## 编排原语
|
||||
|
||||
脚本收到一个只暴露少量编排原语的 `ExecutionState`，本身不直接读写文件，也不运行 shell。默认交互模式把 `agent()` 接到与宿主相同的真实 API client；每个子 agent 只读取 workflow 参数中提供的内容。`demo` 和单元测试使用 `MockAgentRunner`，便于重复观察事件和 journal。
|
||||
|
||||
| 原语 | 作用 |
|
||||
|------|------|
|
||||
| `agent(prompt, {schema, label, phase})` | 派一个子 agent 干活 |
|
||||
| `parallel(thunks)` | **等齐屏障**：所有任务并行跑完，一起等结果回来 |
|
||||
| `pipeline(items, *stages)` | 每个 item 分阶段跑，**不等齐**，跑完一个往下走一个 |
|
||||
| `phase(title)` | 标记当前进度阶段（更新进度条） |
|
||||
| `log(message)` | 打一行进度日志 |
|
||||
| `workflow(name, args)` | 嵌套子工作流（只支持一层） |
|
||||
|
||||
每个 item 都要独立经过相同步骤时，可以使用 `pipeline`。item A 跑到第 3 阶段时，item B 可能还在第 1 阶段；下一步必须同时使用上一阶段全部结果时，再使用 `parallel` 等待所有调用完成。
|
||||
|
||||
```python
|
||||
async def pipeline(self, items, *stages):
|
||||
async def run_item(item, idx):
|
||||
value = item
|
||||
for stage in stages: # 每个 item 独立跑完所有 stage
|
||||
value = await stage(value, item, idx)
|
||||
return value
|
||||
return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])
|
||||
```
|
||||
|
||||
## 结构化输出：别让子 agent 回来写散文
|
||||
|
||||
`agent({schema})` 会要求子 agent 只返回匹配 schema 的 JSON 对象。运行时解析并校验结果，不符合时重试一次。这样下游代码拿到的是对象，不必再从自然语言中提取字段。
|
||||
|
||||
s05 就说过，工具的参数不能全信；这里是同一个道理反过来：子 agent 的输出也不能全信。加一层校验，不对就给一次机会重试，把不确定性挡在编排层外面。
|
||||
|
||||
```python
|
||||
run = await asyncio.to_thread(self.runner.run, prompt, schema, label)
|
||||
result = run.value
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok: # 提醒一次重试，再不对就报错
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run, prompt + "\n\n返回合法的 JSON。", schema, label
|
||||
)
|
||||
result = retry.value
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(f"agent({{schema}}) 输出不合法: {err}")
|
||||
```
|
||||
|
||||
## 任务状态和进度事件
|
||||
|
||||
`LocalWorkflowTask` 维护状态和 token 用量，向外发一条 SDK 风格的事件流：`task_started` → 一串 `task_progress`（包含阶段切换、子 agent 启动和日志输出）→ 最后一个 `task_notification`（完成或失败，带输出文件、agent 数和 token 数）。
|
||||
|
||||
演示会按顺序打印这些事件，并在最终通知后返回任务状态。
|
||||
|
||||
```python
|
||||
class LocalWorkflowTask:
|
||||
def progress_event(self, ptype, **data): # 阶段/子agent/日志
|
||||
self.progress.append({"type": ptype, **data})
|
||||
print(f" 进度 {ptype} ...")
|
||||
```
|
||||
|
||||
## 存储：快照 + journal，断了能续
|
||||
|
||||
运行时把每次运行的数据存在 `s16_workflow_runtime/.runtime/`：快照 `<runId>.json`、输出 `<runId>.output.json`、journal `<runId>.journal.jsonl` 和协调文件 `<runId>.lock`。每次新运行都会在打开 journal 前，用排他式文件创建预留新的 `runId`。整次执行和最终持久化期间都持有 run lock，另一个进程不能同时 resume 同一次运行。快照记录 workflow 名称、参数和任务状态；resume 会先验证已保存的快照和 journal，再改动原有的成功产物。
|
||||
|
||||
journal 是断点续跑的核心，它一条一条记下来每个 `agent()` 的结果：
|
||||
|
||||
```python
|
||||
class WorkflowJournal:
|
||||
def record(self, key, value):
|
||||
self._f.write(json.dumps({"key": key, "value": value}) + "\n")
|
||||
self._f.flush()
|
||||
self.cache[key] = value
|
||||
```
|
||||
|
||||
## resume：用 runId 续跑，没改的直接用缓存
|
||||
|
||||
带着 `resume_from_run_id` 再次调用 workflow 时，脚本会重新执行，但每个 `agent()` 都会计算一个确定的语义 key：key 在 journal 里有记录，就直接返回缓存结果；只有改过的调用以及依赖它的后续步骤才会真的运行。
|
||||
|
||||
这里有个关键点：key 不能依赖并发顺序。`parallel` 和 `pipeline` 里 agent 完成的顺序是不确定的，用"第几个完成"当 key，两次跑缓存就对错位了。所以 key 是根据调用内容（类型、标签、prompt、schema）算的稳定哈希，不是一个会竞争的计数器：
|
||||
|
||||
```python
|
||||
def key(self, kind, label, prompt, schema):
|
||||
basis = f"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}"
|
||||
return f"{kind}-{_stable_hash(basis) % 10**10:010d}"
|
||||
|
||||
# agent() 内部：
|
||||
cached = self.journal.cached(key)
|
||||
if cached is not MISS:
|
||||
self.task.progress_event("workflow_agent", label=label, status="cached")
|
||||
return cached
|
||||
```
|
||||
|
||||
## 稳定调用键
|
||||
|
||||
续跑时，运行时需要把当前 `agent()` 与 journal 中的旧调用对应起来。稳定哈希让同一份 workflow 和同样的参数产生相同的调用 key。真实模型的回答可以变化；只要调用内容没有变化，resume 就直接使用 journal 中已经保存的结果。
|
||||
|
||||
## 跑起来看看
|
||||
|
||||
示例 workflow `review-changes` 用 `pipeline` 让每个审查维度独立走“审计 → 验证”。默认交互模式使用真实 API，并从 `args.changes` 读取待审查内容；`demo` 使用固定 runner 数据来展示 pipeline、结构校验、journal 和续跑。
|
||||
|
||||
```python
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
changes = args.get("changes", "")
|
||||
|
||||
async def audit(_v, dimension, _i):
|
||||
out = await ctx.agent(f"检查这段变更里有没有{dimension}相关的问题：\n{changes}",
|
||||
schema=FINDINGS_SCHEMA, label=f"audit:{dimension}", phase="Review")
|
||||
return {"dimension": dimension, "findings": out["findings"]}
|
||||
|
||||
async def verify(audited, dimension, _i):
|
||||
ctx.phase("Verify")
|
||||
verdicts = await ctx.parallel([ # 每条发现独立做对抗性验证
|
||||
(lambda f=f: ctx.agent(f"根据变更内容验证这条 finding：\n{changes}\n\n{f}",
|
||||
schema=VERDICT_SCHEMA, label=f"verify:{dimension}:{f['title']}"))
|
||||
for f in audited["findings"]])
|
||||
return {"dimension": dimension,
|
||||
"confirmed": [f for f, v in zip(audited["findings"], verdicts) if v and v["isReal"]]}
|
||||
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
...
|
||||
```
|
||||
|
||||
## 相对 s15 的变更
|
||||
|
||||
| | s15 Agent Harness 集成 | s16 Workflow Runtime |
|
||||
|--|-----------|---------------------|
|
||||
| 循环 | 单个、模型驱动 | 主循环不变；工具背后执行脚本编排 |
|
||||
| 谁决定下一步 | 模型逐轮决定 | 脚本预先写好编排流程 |
|
||||
| 多 agent | s06 子 agent，一次性派出去 | 通过 agent-runner 边界执行脚本化、可续跑的调用 |
|
||||
| 新增机制 | — | 编排原语、宿主 registry 与工具适配器、任务生命周期、进度事件、journal/续跑、结构化输出 |
|
||||
|
||||
s16 不替换主循环，它只是在工具层暴露 `Workflow`，背后启动一个本地 workflow 运行时：一份保存好的脚本通过 agent-runner 边界协调 N 次调用。s06 的子 agent 是模型临场派一次；s16 把编排写成可续跑的宿主代码。
|
||||
|
||||
## 试一下
|
||||
|
||||
```bash
|
||||
python s16_workflow_runtime/code.py # 主模型和 Workflow 子 agent 都使用真实 API
|
||||
python s16_workflow_runtime/code.py demo # 运行确定性的 review-changes 测试数据并观察事件流
|
||||
python s16_workflow_runtime/code.py resume # 用上次的 runId 续跑，每个 agent() 都命中 journal 缓存
|
||||
```
|
||||
|
||||
默认命令里，可以先让模型读取改动，再把内容放进 `args.changes` 并运行保存好的 `review-changes` workflow。主模型和 workflow 子 agent 都使用真实 API。`demo` 命令使用固定 runner 数据，便于重复观察生命周期和续跑；续跑命中全部缓存时显示 `agents=0 tokens=0`。
|
||||
|
||||
## 接下来
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) 会使用一个更小、独立的循环检查既定目标是否已经达成，并据此决定是否还需要下一轮。
|
||||
|
||||
<!-- translation-sync: zh@v10, en@v10, ja@v10 -->
|
||||
874
s16_workflow_runtime/code.py
Normal file
874
s16_workflow_runtime/code.py
Normal file
@@ -0,0 +1,874 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
s16: Workflow Runtime - run a saved orchestration through one tool call.
|
||||
|
||||
Run:
|
||||
python s16_workflow_runtime/code.py
|
||||
python s16_workflow_runtime/code.py demo
|
||||
python s16_workflow_runtime/code.py resume
|
||||
|
||||
+-------------+ +--------------------------------+
|
||||
| Agent loop | ----> | Workflow(name, args, run_id) |
|
||||
+-------------+ +---------------+----------------+
|
||||
|
|
||||
+--------------+--------------+
|
||||
| agent | parallel | pipeline |
|
||||
+--------------+--------------+
|
||||
|
|
||||
journal + result
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import fcntl
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import sys
|
||||
import threading
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
# -- Runtime Guards --
|
||||
AGENT_CAP = 1000 # hard cap on agent() calls per run
|
||||
CONCURRENCY = 8 # parallelism cap (semaphore)
|
||||
STORE = Path(__file__).parent / ".runtime" # snapshots + journals live here
|
||||
MISS = object() # journal cache miss sentinel
|
||||
WORKFLOW_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$")
|
||||
RUN_ID_RE = re.compile(r"^wf_[A-Za-z0-9][A-Za-z0-9._-]{0,63}_[0-9a-f]{16}$")
|
||||
|
||||
|
||||
def _stable_hash(s: str) -> int:
|
||||
"""Process-stable hash (Python's hash() is salted per process, which would
|
||||
break resume keys across `run` and `resume`)."""
|
||||
return int(hashlib.sha256(s.encode()).hexdigest(), 16)
|
||||
|
||||
|
||||
def create_run_id(meta) -> str:
|
||||
return f"wf_{meta['name']}_{secrets.token_hex(8)}"
|
||||
|
||||
|
||||
def reserve_run_id(meta) -> str:
|
||||
"""Reserve a fresh run identity before any journal can be truncated."""
|
||||
STORE.mkdir(parents=True, exist_ok=True)
|
||||
for _ in range(32):
|
||||
run_id = validate_run_id(create_run_id(meta))
|
||||
snapshot_path = STORE / f"{run_id}.json"
|
||||
try:
|
||||
fd = os.open(snapshot_path, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600)
|
||||
except FileExistsError:
|
||||
continue
|
||||
os.close(fd)
|
||||
return run_id
|
||||
raise WorkflowInputError("could not allocate a unique workflow runId")
|
||||
|
||||
|
||||
def create_task_id(run_id) -> str:
|
||||
return f"local_workflow_{run_id}"
|
||||
|
||||
|
||||
def validate_run_id(run_id):
|
||||
if not isinstance(run_id, str) or not RUN_ID_RE.fullmatch(run_id):
|
||||
raise WorkflowInputError("invalid workflow runId")
|
||||
return run_id
|
||||
|
||||
|
||||
# -- Errors --
|
||||
class WorkflowInputError(Exception):
|
||||
"""Bad workflow, metadata, or schema input."""
|
||||
|
||||
|
||||
_run_locks_guard = threading.Lock()
|
||||
_run_locks: dict[str, threading.Lock] = {}
|
||||
|
||||
|
||||
@contextmanager
|
||||
def workflow_run_lock(run_id: str):
|
||||
"""Hold one run across threads and host processes for its full lifecycle."""
|
||||
with _run_locks_guard:
|
||||
local_lock = _run_locks.setdefault(run_id, threading.Lock())
|
||||
if not local_lock.acquire(blocking=False):
|
||||
raise WorkflowInputError(f"workflow run {run_id} is already active")
|
||||
|
||||
handle = None
|
||||
try:
|
||||
STORE.mkdir(parents=True, exist_ok=True)
|
||||
handle = (STORE / f"{run_id}.lock").open("a+")
|
||||
try:
|
||||
fcntl.flock(handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except BlockingIOError as exc:
|
||||
raise WorkflowInputError(
|
||||
f"workflow run {run_id} is already active"
|
||||
) from exc
|
||||
yield
|
||||
finally:
|
||||
if handle is not None:
|
||||
try:
|
||||
fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
|
||||
finally:
|
||||
handle.close()
|
||||
local_lock.release()
|
||||
with _run_locks_guard:
|
||||
if not local_lock.locked() and _run_locks.get(run_id) is local_lock:
|
||||
_run_locks.pop(run_id, None)
|
||||
|
||||
|
||||
# -- Metadata Validation --
|
||||
def validate_meta(meta):
|
||||
"""Validate name, description, and optional phases before launch."""
|
||||
if not isinstance(meta, dict):
|
||||
raise WorkflowInputError("meta must be an object literal")
|
||||
if not meta.get("name") or not meta.get("description"):
|
||||
raise WorkflowInputError("meta requires `name` and `description`")
|
||||
if not isinstance(meta["name"], str) or not WORKFLOW_NAME_RE.fullmatch(meta["name"]):
|
||||
raise WorkflowInputError(
|
||||
"meta.name must be a 1-64 character slug using letters, numbers, '.', '_', or '-'"
|
||||
)
|
||||
if not isinstance(meta["description"], str):
|
||||
raise WorkflowInputError("meta.description must be a string")
|
||||
if "phases" in meta:
|
||||
if not isinstance(meta["phases"], list) or not all(
|
||||
isinstance(phase, str) and phase for phase in meta["phases"]
|
||||
):
|
||||
raise WorkflowInputError("meta.phases must be a list of non-empty strings")
|
||||
return meta
|
||||
|
||||
|
||||
def check_permission(meta, settings=None):
|
||||
"""Apply the s03 allow/deny gate before launching a workflow."""
|
||||
settings = settings or {}
|
||||
if meta["name"] in settings.get("deny", []):
|
||||
raise WorkflowInputError(f"workflow '{meta['name']}' denied by settings")
|
||||
return "allow"
|
||||
|
||||
|
||||
# -- Minimal JSON Schema --
|
||||
class SimpleJsonSchema:
|
||||
"""Tiny validator backing agent({schema}):
|
||||
object/array/string/boolean/number + required keys."""
|
||||
|
||||
def __init__(self, schema):
|
||||
self.schema = schema
|
||||
|
||||
def validate(self, value, schema=None):
|
||||
schema = self.schema if schema is None else schema
|
||||
if "enum" in schema and value not in schema["enum"]:
|
||||
return False, f"expected one of {schema['enum']}"
|
||||
t = schema.get("type")
|
||||
if t == "object":
|
||||
if not isinstance(value, dict):
|
||||
return False, "expected object"
|
||||
for key in schema.get("required", []):
|
||||
if key not in value:
|
||||
return False, f"missing required key '{key}'"
|
||||
for key, sub in schema.get("properties", {}).items():
|
||||
if key in value:
|
||||
ok, err = self.validate(value[key], sub)
|
||||
if not ok:
|
||||
return False, f"{key}: {err}"
|
||||
return True, None
|
||||
if t == "array":
|
||||
if not isinstance(value, list):
|
||||
return False, "expected array"
|
||||
items = schema.get("items")
|
||||
if items:
|
||||
for i, el in enumerate(value):
|
||||
ok, err = self.validate(el, items)
|
||||
if not ok:
|
||||
return False, f"[{i}]: {err}"
|
||||
return True, None
|
||||
if t == "string":
|
||||
return (isinstance(value, str), None if isinstance(value, str) else "expected string")
|
||||
if t == "boolean":
|
||||
return (isinstance(value, bool), None if isinstance(value, bool) else "expected boolean")
|
||||
if t in ("number", "integer"):
|
||||
ok = isinstance(value, (int, float)) and not isinstance(value, bool)
|
||||
return (ok, None if ok else "expected number")
|
||||
return True, None
|
||||
|
||||
|
||||
def _fill_schema(schema, seed):
|
||||
"""Deterministic generic filler used for schemas the mock doesn't special-case."""
|
||||
t = schema.get("type")
|
||||
if t == "object":
|
||||
keys = schema.get("required") or list(schema.get("properties", {}))
|
||||
return {k: _fill_schema(schema["properties"][k], f"{seed}/{k}") for k in keys}
|
||||
if t == "array":
|
||||
return [_fill_schema(schema["items"], f"{seed}/0")]
|
||||
if t == "boolean":
|
||||
return _stable_hash(seed) % 4 != 0
|
||||
if t in ("number", "integer"):
|
||||
return _stable_hash(seed) % 5
|
||||
return seed.rsplit("/", 1)[-1]
|
||||
|
||||
|
||||
# -- Agent Runners --
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RunnerOutput:
|
||||
value: object
|
||||
tokens: int
|
||||
|
||||
|
||||
class MockAgentRunner:
|
||||
"""Deterministic runner used by demo mode and unit tests."""
|
||||
|
||||
def run(self, prompt, schema=None, label=None):
|
||||
if schema is None:
|
||||
value = f"[mock] {(label or prompt)[:60]}"
|
||||
return RunnerOutput(value, self._tokens(prompt, value))
|
||||
props = schema.get("properties", {})
|
||||
if "findings" in props:
|
||||
n = 1 + (_stable_hash(prompt) % 2)
|
||||
sev = ["high", "medium", "low"]
|
||||
value = {"findings": [
|
||||
{"title": f"{label or 'audit'} #{i + 1}",
|
||||
"severity": sev[_stable_hash(prompt + str(i)) % 3]}
|
||||
for i in range(n)
|
||||
]}
|
||||
elif "isReal" in props:
|
||||
real = _stable_hash(prompt) % 4 != 0
|
||||
value = {"isReal": real,
|
||||
"reason": "reproduced" if real else "could not reproduce"}
|
||||
else:
|
||||
value = _fill_schema(schema, prompt)
|
||||
return RunnerOutput(value, self._tokens(prompt, value))
|
||||
|
||||
@staticmethod
|
||||
def _tokens(prompt, result):
|
||||
return len(prompt) // 4 + len(json.dumps(result, default=str)) // 4
|
||||
|
||||
|
||||
def _response_text(response) -> str:
|
||||
return "\n".join(
|
||||
str(getattr(block, "text", ""))
|
||||
for block in getattr(response, "content", [])
|
||||
if getattr(block, "type", None) == "text"
|
||||
).strip()
|
||||
|
||||
|
||||
def _parse_runner_json(text: str) -> object:
|
||||
stripped = text.strip()
|
||||
if stripped.startswith("```"):
|
||||
lines = stripped.splitlines()
|
||||
lines = lines[1:] if lines else lines
|
||||
if lines and lines[-1].strip() == "```":
|
||||
lines = lines[:-1]
|
||||
stripped = "\n".join(lines).strip()
|
||||
try:
|
||||
return json.loads(stripped)
|
||||
except json.JSONDecodeError:
|
||||
decoder = json.JSONDecoder()
|
||||
for position, character in enumerate(stripped):
|
||||
if character != "{":
|
||||
continue
|
||||
try:
|
||||
value, _ = decoder.raw_decode(stripped[position:])
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
return value
|
||||
raise WorkflowInputError("workflow agent returned invalid JSON")
|
||||
|
||||
|
||||
class AnthropicAgentRunner:
|
||||
"""Run workflow agents through the same API client as the host."""
|
||||
|
||||
def __init__(self, client, model):
|
||||
self.client = client
|
||||
self.model = model
|
||||
|
||||
def run(self, prompt, schema=None, label=None):
|
||||
request = prompt
|
||||
if schema is not None:
|
||||
request += (
|
||||
"\n\nReturn only one JSON object matching this schema:\n"
|
||||
+ json.dumps(schema, ensure_ascii=True, sort_keys=True)
|
||||
)
|
||||
response = self.client.messages.create(
|
||||
model=self.model,
|
||||
system=(
|
||||
"You are a focused workflow agent. Complete only the supplied "
|
||||
"step. Do not claim access to files or results not included in "
|
||||
"the prompt."
|
||||
),
|
||||
messages=[{"role": "user", "content": request}],
|
||||
max_tokens=2000,
|
||||
)
|
||||
text = _response_text(response)
|
||||
if schema is None:
|
||||
value = text
|
||||
else:
|
||||
try:
|
||||
value = _parse_runner_json(text)
|
||||
except WorkflowInputError:
|
||||
# Let ExecutionState's schema check trigger its single retry.
|
||||
value = text
|
||||
usage = getattr(response, "usage", None)
|
||||
tokens = int(getattr(usage, "input_tokens", 0) or 0) + int(
|
||||
getattr(usage, "output_tokens", 0) or 0
|
||||
)
|
||||
return RunnerOutput(value, tokens)
|
||||
|
||||
|
||||
RUNNER_FACTORY = MockAgentRunner
|
||||
|
||||
|
||||
# -- Journal --
|
||||
class WorkflowJournal:
|
||||
"""Append-only <runId>.journal.jsonl. On resume, agent() calls whose
|
||||
semantic key is already present are replayed from cache instead of re-run."""
|
||||
|
||||
def __init__(self, run_id, resume, store=None):
|
||||
store = STORE if store is None else store
|
||||
store.mkdir(parents=True, exist_ok=True)
|
||||
self.path = store / f"{run_id}.journal.jsonl"
|
||||
self.resume = resume
|
||||
self.cache = {}
|
||||
if resume:
|
||||
if not self.path.exists():
|
||||
raise WorkflowInputError(f"resume journal not found for {run_id}")
|
||||
for line_number, line in enumerate(self.path.read_text().splitlines(), start=1):
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
if (
|
||||
not isinstance(rec, dict)
|
||||
or not isinstance(rec.get("key"), str)
|
||||
or "value" not in rec
|
||||
):
|
||||
raise ValueError("expected key/value record")
|
||||
except (json.JSONDecodeError, ValueError) as exc:
|
||||
raise WorkflowInputError(
|
||||
f"invalid resume journal record at line {line_number}"
|
||||
) from exc
|
||||
self.cache[rec["key"]] = rec["value"]
|
||||
self._f = self.path.open("a")
|
||||
else:
|
||||
self._f = self.path.open("w") # fresh run truncates
|
||||
|
||||
def key(self, kind, label, prompt, schema):
|
||||
# Deterministic semantic key, independent of concurrency order, so a
|
||||
# parallel/pipeline call gets the same key on resume.
|
||||
basis = f"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}"
|
||||
return f"{kind}-{_stable_hash(basis) % 10**10:010d}"
|
||||
|
||||
def cached(self, key):
|
||||
return self.cache.get(key, MISS)
|
||||
|
||||
def record(self, key, value):
|
||||
self._f.write(json.dumps({"key": key, "value": value}) + "\n")
|
||||
self._f.flush()
|
||||
self.cache[key] = value
|
||||
|
||||
def close(self):
|
||||
self._f.close()
|
||||
|
||||
|
||||
# -- Token Budget --
|
||||
class Budget:
|
||||
"""budget.total / spent() / remaining(). Once spent reaches total, agent()
|
||||
calls raise instead of silently overspending."""
|
||||
|
||||
def __init__(self, total=None):
|
||||
self.total = total
|
||||
self._spent = 0
|
||||
|
||||
def add(self, n):
|
||||
if self.total is not None and self._spent + n > self.total:
|
||||
raise WorkflowInputError(
|
||||
f"token budget exceeded ({self._spent + n} > {self.total})"
|
||||
)
|
||||
self._spent += n
|
||||
|
||||
def spent(self):
|
||||
return self._spent
|
||||
|
||||
def remaining(self):
|
||||
return float("inf") if self.total is None else max(0, self.total - self._spent)
|
||||
|
||||
|
||||
# -- Workflow Task Lifecycle --
|
||||
class LocalWorkflowTask:
|
||||
"""Hold workflow status, usage, and progress events."""
|
||||
|
||||
def __init__(self, task_id, run_id, meta):
|
||||
self.task_id = task_id
|
||||
self.run_id = run_id
|
||||
self.meta = meta
|
||||
self.status = "running"
|
||||
self.usage = {"agents": 0, "tokens": 0}
|
||||
self.progress = []
|
||||
|
||||
def event(self, name, **data):
|
||||
line = " ".join(f"{k}={v}" for k, v in data.items())
|
||||
print(f" event {name:<18} {line}")
|
||||
|
||||
def progress_event(self, ptype, **data):
|
||||
self.progress.append({"type": ptype, **data})
|
||||
line = " ".join(f"{k}={v}" for k, v in data.items())
|
||||
print(f" progress {ptype:<16} {line}")
|
||||
|
||||
|
||||
# -- Workflow Primitives --
|
||||
class ExecutionLimits:
|
||||
"""Shared run-wide limits, including nested workflows."""
|
||||
|
||||
def __init__(self):
|
||||
self.agents = 0
|
||||
self.semaphore = asyncio.Semaphore(CONCURRENCY)
|
||||
|
||||
def claim_agent(self):
|
||||
self.agents += 1
|
||||
if self.agents > AGENT_CAP:
|
||||
raise WorkflowInputError(f"agent() cap reached ({AGENT_CAP})")
|
||||
|
||||
|
||||
class ExecutionState:
|
||||
"""Injected into the workflow script with the orchestration primitives."""
|
||||
|
||||
def __init__(self, task, journal, runner, budget, args, depth=0, limits=None):
|
||||
self.task = task
|
||||
self.journal = journal
|
||||
self.runner = runner
|
||||
self.budget = budget
|
||||
self.args = args
|
||||
self._depth = depth
|
||||
self._phase = None
|
||||
self._phases_seen = set()
|
||||
self._limits = limits or ExecutionLimits()
|
||||
|
||||
def phase(self, title):
|
||||
"""Start a phase; subsequent agent()s group under it. Upsert: emitting the
|
||||
same phase again (e.g. from each pipeline item) does not re-announce it."""
|
||||
self._phase = title
|
||||
if title not in self._phases_seen:
|
||||
self._phases_seen.add(title)
|
||||
self.task.progress_event("workflow_phase", title=title)
|
||||
|
||||
def log(self, message):
|
||||
"""Emit a workflow_log progress line."""
|
||||
self.task.progress_event("workflow_log", message=message)
|
||||
|
||||
async def agent(self, prompt, schema=None, label=None, phase=None):
|
||||
"""Spawn one subagent. With a schema, force StructuredOutput + validate
|
||||
(retry once). On resume, a cached key short-circuits the run."""
|
||||
label = label or (prompt[:24] + "...")
|
||||
self._limits.claim_agent()
|
||||
if self.budget.remaining() <= 0:
|
||||
raise WorkflowInputError("token budget exceeded")
|
||||
|
||||
key = self.journal.key("agent", label, prompt, schema)
|
||||
cached = self.journal.cached(key)
|
||||
if cached is not MISS:
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(cached)
|
||||
if not ok:
|
||||
raise WorkflowInputError(
|
||||
f"cached agent output failed schema validation: {err}"
|
||||
)
|
||||
self.task.progress_event("workflow_agent", label=label,
|
||||
phase=phase or self._phase, status="cached")
|
||||
return cached
|
||||
|
||||
async with self._limits.semaphore:
|
||||
run = await asyncio.to_thread(
|
||||
self.runner.run, prompt, schema, label
|
||||
)
|
||||
result = run.value
|
||||
tokens = run.tokens
|
||||
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run,
|
||||
prompt + "\n\nReturn valid JSON.",
|
||||
schema,
|
||||
label,
|
||||
)
|
||||
result = retry.value
|
||||
tokens += retry.tokens
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(f"agent({{schema}}) invalid output: {err}")
|
||||
|
||||
self.budget.add(tokens)
|
||||
self.task.usage["agents"] += 1
|
||||
self.task.usage["tokens"] += tokens
|
||||
self.journal.record(key, result)
|
||||
self.task.progress_event("workflow_agent", label=label,
|
||||
phase=phase or self._phase, status="done")
|
||||
return result
|
||||
|
||||
async def parallel(self, thunks):
|
||||
"""BARRIER: run all thunks concurrently and fail if any thunk fails."""
|
||||
return await asyncio.gather(*[thunk() for thunk in thunks])
|
||||
|
||||
async def pipeline(self, items, *stages):
|
||||
"""Per-item staged flow, NO barrier between stages: item A can be in
|
||||
stage 3 while item B is still in stage 1. Each stage gets
|
||||
(prev_result, original_item, index). A throwing stage fails the workflow."""
|
||||
async def run_item(item, idx):
|
||||
value = item
|
||||
for stage in stages:
|
||||
value = await stage(value, item, idx)
|
||||
return value
|
||||
return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])
|
||||
|
||||
async def workflow(self, name, args=None):
|
||||
"""Run a saved workflow inline as a child (one level), sharing this run's
|
||||
journal + budget + agent counter."""
|
||||
if self._depth >= 1:
|
||||
raise WorkflowInputError("workflow() nesting is one level only")
|
||||
if name not in WORKFLOWS:
|
||||
raise WorkflowInputError(f"unknown workflow '{name}'")
|
||||
meta, fn = WORKFLOWS[name]
|
||||
child = ExecutionState(self.task, self.journal, self.runner, self.budget,
|
||||
args or {}, depth=self._depth + 1,
|
||||
limits=self._limits)
|
||||
return await fn(child, args or {})
|
||||
|
||||
|
||||
# -- Workflow Tool --
|
||||
class WorkflowTool:
|
||||
"""The Workflow tool. .call() validates meta, runs the permission check,
|
||||
creates runId/taskId, registers a LocalWorkflowTask, and emits lifecycle
|
||||
events while executing the script. It returns the result and task state and
|
||||
supports resume."""
|
||||
|
||||
async def call(self, meta, script_fn, args=None, resume_from_run_id=None):
|
||||
validate_meta(meta)
|
||||
check_permission(meta)
|
||||
resuming = resume_from_run_id is not None
|
||||
if resuming:
|
||||
run_id = validate_run_id(resume_from_run_id)
|
||||
else:
|
||||
run_id = reserve_run_id(meta)
|
||||
with workflow_run_lock(run_id):
|
||||
return await self._call_locked(
|
||||
meta, script_fn, args, run_id, resuming
|
||||
)
|
||||
|
||||
async def _call_locked(self, meta, script_fn, args, run_id, resuming):
|
||||
if resuming:
|
||||
snapshot = _read_snapshot(run_id)
|
||||
if snapshot.get("workflowName") != meta["name"]:
|
||||
raise WorkflowInputError("resume runId does not match workflow meta")
|
||||
saved_args = snapshot.get("args", {})
|
||||
if args is None:
|
||||
args = saved_args
|
||||
elif args != saved_args:
|
||||
raise WorkflowInputError("resume args do not match the original run")
|
||||
journal = WorkflowJournal(run_id, resume=True)
|
||||
else:
|
||||
args = args or {}
|
||||
journal = WorkflowJournal(run_id, resume=False)
|
||||
task_id = create_task_id(run_id)
|
||||
|
||||
task = LocalWorkflowTask(task_id, run_id, meta)
|
||||
# Record the launch envelope before workflow execution starts.
|
||||
launched = {"status": "async_launched", "taskId": task_id,
|
||||
"taskType": "local_workflow", "runId": run_id,
|
||||
"workflowName": meta["name"]}
|
||||
task.event("async_launched", runId=run_id, taskId=task_id)
|
||||
task.event("task_started", workflow=meta["name"],
|
||||
phases=",".join(meta.get("phases", [])) or "-",
|
||||
resume=resuming)
|
||||
_write_json(STORE / f"{run_id}.json", {
|
||||
"runId": run_id,
|
||||
"workflowName": meta["name"],
|
||||
"args": args,
|
||||
"task": serialize_task(task),
|
||||
})
|
||||
|
||||
try:
|
||||
ctx = ExecutionState(
|
||||
task, journal, RUNNER_FACTORY(), Budget(args.get("budget")), args
|
||||
)
|
||||
result = await script_fn(ctx, args)
|
||||
task.status = "completed"
|
||||
except Exception as e: # failed / stopped close the loop too
|
||||
task.status = "failed"
|
||||
result = {"error": str(e)}
|
||||
finally:
|
||||
journal.close()
|
||||
|
||||
_write_json(STORE / f"{run_id}.output.json", result)
|
||||
_write_json(STORE / f"{run_id}.json", {
|
||||
"runId": run_id,
|
||||
"workflowName": meta["name"],
|
||||
"args": args,
|
||||
"task": serialize_task(task),
|
||||
})
|
||||
_save_last_run(run_id)
|
||||
task.event("task_notification", status=task.status,
|
||||
agents=task.usage["agents"], tokens=task.usage["tokens"],
|
||||
outputFile=f".runtime/{run_id}.output.json")
|
||||
return {"launched": launched, "result": result, "task": task}
|
||||
|
||||
|
||||
def _write_json(path, value):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_suffix(path.suffix + ".tmp")
|
||||
temporary.write_text(json.dumps(value, indent=2, default=str))
|
||||
os.replace(temporary, path)
|
||||
|
||||
|
||||
def _read_snapshot(run_id):
|
||||
path = STORE / f"{run_id}.json"
|
||||
if not path.exists():
|
||||
raise WorkflowInputError(f"resume snapshot not found for {run_id}")
|
||||
try:
|
||||
snapshot = json.loads(path.read_text())
|
||||
except json.JSONDecodeError as exc:
|
||||
raise WorkflowInputError(f"invalid resume snapshot for {run_id}") from exc
|
||||
if not isinstance(snapshot, dict):
|
||||
raise WorkflowInputError(f"invalid resume snapshot for {run_id}")
|
||||
return snapshot
|
||||
|
||||
|
||||
def _save_last_run(run_id):
|
||||
(STORE / "last_run.txt").write_text(run_id)
|
||||
|
||||
|
||||
def _read_last_run():
|
||||
p = STORE / "last_run.txt"
|
||||
return p.read_text().strip() if p.exists() else None
|
||||
|
||||
|
||||
# -- Sample Workflow --
|
||||
FINDINGS_SCHEMA = {
|
||||
"type": "object", "required": ["findings"],
|
||||
"properties": {"findings": {"type": "array", "items": {
|
||||
"type": "object", "required": ["title", "severity"],
|
||||
"properties": {
|
||||
"title": {"type": "string"},
|
||||
"severity": {
|
||||
"type": "string", "enum": ["high", "medium", "low"]
|
||||
},
|
||||
}}}},
|
||||
}
|
||||
VERDICT_SCHEMA = {
|
||||
"type": "object", "required": ["isReal", "reason"],
|
||||
"properties": {"isReal": {"type": "boolean"}, "reason": {"type": "string"}},
|
||||
}
|
||||
|
||||
SAMPLE_META = {
|
||||
"name": "review-changes",
|
||||
"description": "Review changed files across dimensions, verify each finding",
|
||||
"phases": ["Review", "Verify"],
|
||||
}
|
||||
|
||||
DIMENSIONS = ["correctness", "security", "performance", "style"]
|
||||
DEMO_CHANGES = (
|
||||
"def load_user(user_id):\n"
|
||||
" query = f\"SELECT * FROM users WHERE id = {user_id}\"\n"
|
||||
" return db.execute(query).fetchone()\n"
|
||||
)
|
||||
|
||||
|
||||
async def sample_workflow(ctx, args):
|
||||
"""pipeline over review dimensions (audit -> verify-each), then keep only the
|
||||
findings a verifier confirms. The plan is code, not a chat turn."""
|
||||
ctx.phase("Review")
|
||||
changes = args.get("changes", "")
|
||||
if not isinstance(changes, str):
|
||||
raise WorkflowInputError("args.changes must be a string")
|
||||
review_input = changes.strip() or "No change context was supplied."
|
||||
|
||||
async def audit(_value, dimension, _idx):
|
||||
out = await ctx.agent(
|
||||
f"Review this change context for {dimension} issues. "
|
||||
"Report only issues supported by the supplied text.\n\n"
|
||||
f"{review_input}",
|
||||
schema=FINDINGS_SCHEMA, label=f"audit:{dimension}", phase="Review")
|
||||
return {"dimension": dimension, "findings": out["findings"]}
|
||||
|
||||
async def verify(audited, dimension, _idx):
|
||||
ctx.phase("Verify")
|
||||
# Each finding is verified by its own adversarial subagent, concurrently.
|
||||
verdicts = await ctx.parallel([
|
||||
(lambda f=f: ctx.agent(
|
||||
f"Adversarially verify this {dimension} finding against the "
|
||||
"supplied change context.\n\n"
|
||||
f"Change context:\n{review_input}\n\n"
|
||||
f"Finding:\n{json.dumps(f, ensure_ascii=True)}",
|
||||
schema=VERDICT_SCHEMA, label=f"verify:{dimension}:{f['title']}", phase="Verify"))
|
||||
for f in audited["findings"]])
|
||||
confirmed = [f for f, v in zip(audited["findings"], verdicts)
|
||||
if v and v.get("isReal")]
|
||||
return {"dimension": dimension, "confirmed": confirmed}
|
||||
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
confirmed = [{"dimension": r["dimension"], **f}
|
||||
for r in results if r for f in r["confirmed"]]
|
||||
confirmed.sort(key=lambda f: {"high": 0, "medium": 1, "low": 2}.get(f["severity"], 3))
|
||||
ctx.log(f"confirmed {len(confirmed)} real finding(s)")
|
||||
return {"confirmed": confirmed}
|
||||
|
||||
|
||||
# Saved workflow registry
|
||||
WORKFLOWS = {SAMPLE_META["name"]: (SAMPLE_META, sample_workflow)}
|
||||
|
||||
WORKFLOW_TOOL = {
|
||||
"name": "Workflow",
|
||||
"description": "Run a saved workflow by name. Pass input in args.",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"args": {"type": "object"},
|
||||
"resume_from_run_id": {"type": "string"},
|
||||
},
|
||||
"required": ["name"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def serialize_task(task):
|
||||
return {
|
||||
"taskId": task.task_id,
|
||||
"taskType": "local_workflow",
|
||||
"runId": task.run_id,
|
||||
"workflowName": task.meta["name"],
|
||||
"status": task.status,
|
||||
"usage": dict(task.usage),
|
||||
"progress": list(task.progress),
|
||||
}
|
||||
|
||||
|
||||
async def run_workflow(name, args=None, resume_from_run_id=None):
|
||||
"""Model-facing adapter: resolve trusted code from the host registry."""
|
||||
if not isinstance(name, str):
|
||||
raise WorkflowInputError("workflow name must be a string")
|
||||
if name not in WORKFLOWS:
|
||||
raise WorkflowInputError(f"unknown workflow '{name}'")
|
||||
if args is not None and not isinstance(args, dict):
|
||||
raise WorkflowInputError("workflow args must be an object")
|
||||
meta, script_fn = WORKFLOWS[name]
|
||||
out = await WorkflowTool().call(
|
||||
meta,
|
||||
script_fn,
|
||||
args=args,
|
||||
resume_from_run_id=resume_from_run_id,
|
||||
)
|
||||
return {
|
||||
"launched": out["launched"],
|
||||
"result": out["result"],
|
||||
"task": serialize_task(out["task"]),
|
||||
}
|
||||
|
||||
|
||||
WORKFLOW_HANDLERS = {"Workflow": run_workflow}
|
||||
INHERITS_TOOLS_FROM = "s15"
|
||||
|
||||
|
||||
def run_workflow_sync(**tool_input):
|
||||
"""Bridge the synchronous host dispatcher to the async workflow runtime."""
|
||||
try:
|
||||
return json.dumps(asyncio.run(run_workflow(**tool_input)), default=str)
|
||||
except WorkflowInputError as exc:
|
||||
return f"Error: {exc}"
|
||||
|
||||
|
||||
def install_workflow_tool(host):
|
||||
"""Extend the s15 host tool pool without changing its dispatch loop."""
|
||||
global RUNNER_FACTORY
|
||||
RUNNER_FACTORY = lambda: AnthropicAgentRunner(host.client, host.MODEL)
|
||||
if getattr(host, "_workflow_tool_installed", False):
|
||||
return
|
||||
base_assemble = host.assemble_tool_pool
|
||||
|
||||
def assemble_with_workflow():
|
||||
tools, handlers = base_assemble()
|
||||
if not any(tool.get("name") == "Workflow" for tool in tools):
|
||||
tools.append(WORKFLOW_TOOL)
|
||||
handlers["Workflow"] = run_workflow_sync
|
||||
return tools, handlers
|
||||
|
||||
host.assemble_tool_pool = assemble_with_workflow
|
||||
host._workflow_tool_installed = True
|
||||
|
||||
|
||||
def load_integrated_host():
|
||||
"""Load s15 lazily so deterministic workflow tests need no API key."""
|
||||
path = Path(__file__).resolve().parents[1] / "s15_integrated_harness" / "code.py"
|
||||
spec = importlib.util.spec_from_file_location("integrated_host", path)
|
||||
if spec is None or spec.loader is None:
|
||||
raise RuntimeError(f"unable to load integrated host from {path}")
|
||||
host = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = host
|
||||
spec.loader.exec_module(host)
|
||||
return host
|
||||
|
||||
|
||||
# -- CLI --
|
||||
async def run_demo(argv):
|
||||
resume_id = None
|
||||
if argv and argv[0] == "resume":
|
||||
resume_id = _read_last_run()
|
||||
if not resume_id:
|
||||
print("nothing to resume; run `python code.py demo` first.")
|
||||
return
|
||||
print(f"resuming {resume_id}; unchanged agent() calls use the journal cache\n")
|
||||
else:
|
||||
print("launching workflow `review-changes`\n")
|
||||
|
||||
out = await WORKFLOW_HANDLERS["Workflow"](
|
||||
name="review-changes",
|
||||
args={"budget": None, "changes": DEMO_CHANGES},
|
||||
resume_from_run_id=resume_id,
|
||||
)
|
||||
|
||||
print("\nresult:")
|
||||
for f in out["result"].get("confirmed", []):
|
||||
print(f" [{f['severity']:<6}] {f['dimension']}: {f['title']}")
|
||||
task = out["task"]
|
||||
usage = task["usage"]
|
||||
print(f"\nstatus={task['status']} agents={usage['agents']} "
|
||||
f"tokens={usage['tokens']} journal=.runtime/{task['runId']}.journal.jsonl")
|
||||
|
||||
|
||||
def run_cli():
|
||||
"""Run the cumulative s15 host with Workflow added to its tool pool."""
|
||||
host = load_integrated_host()
|
||||
install_workflow_tool(host)
|
||||
host.CLI_ACTIVE = True
|
||||
host.start_runtime_services()
|
||||
print("s16: workflow runtime")
|
||||
print("Enter a question, press Enter to send. Type q to quit.\n")
|
||||
history = []
|
||||
context = host.update_context({}, history)
|
||||
session_state = {"active_user_request": "(no active user request)"}
|
||||
threading.Thread(
|
||||
target=host.async_event_loop,
|
||||
args=(history, context, session_state),
|
||||
daemon=True,
|
||||
).start()
|
||||
while True:
|
||||
try:
|
||||
query = host.CONSOLE.ask("\033[36ms16 >> \033[0m")
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
break
|
||||
if query.strip().lower() in ("q", "exit", ""):
|
||||
break
|
||||
with host.agent_lock:
|
||||
host.trigger_hooks("UserPromptSubmit", query)
|
||||
turn_start = len(history)
|
||||
session_state["active_user_request"] = query
|
||||
history.append({"role": "user", "content": query})
|
||||
host.agent_loop(history, context, query)
|
||||
context = host.update_context(context, history)
|
||||
host.print_turn_assistants(history, turn_start)
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if sys.argv[1:] and sys.argv[1] in {"demo", "resume"}:
|
||||
asyncio.run(run_demo(sys.argv[1:]))
|
||||
else:
|
||||
run_cli()
|
||||
115
s16_workflow_runtime/images/workflow-runtime-overview.svg
Normal file
115
s16_workflow_runtime/images/workflow-runtime-overview.svg
Normal file
@@ -0,0 +1,115 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 640" font-family="system-ui, -apple-system, sans-serif" role="img" aria-labelledby="title description">
|
||||
<title id="title">Workflow Runtime execution flow</title>
|
||||
<desc id="description">One Workflow tool call executes a complete workflow run. Lifecycle and progress events remain inside the call, which returns one tool result containing launch metadata, the result, and task state.</desc>
|
||||
<defs>
|
||||
<marker id="arrow-green" viewBox="0 0 10 10" refX="10" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#22c55e"/>
|
||||
</marker>
|
||||
<marker id="arrow-gray" viewBox="0 0 10 10" refX="10" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#888888"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<rect width="960" height="640" rx="8" fill="#ffffff"/>
|
||||
|
||||
<text x="480" y="30" text-anchor="middle" fill="#1a1a1a" font-size="19" font-weight="700">Workflow Runtime — one Workflow call executes one complete run</text>
|
||||
<text x="480" y="51" text-anchor="middle" fill="#888888" font-size="12">lifecycle and progress events are emitted during the call; one final tool_result returns to messages[]</text>
|
||||
|
||||
<!-- Main session loop -->
|
||||
<rect x="20" y="68" width="920" height="132" rx="8" fill="#ffffff" stroke="#d0d0d0" stroke-width="1.5" stroke-dasharray="6,3"/>
|
||||
<text x="40" y="88" fill="#1a1a1a" font-size="13" font-weight="700">Main session loop</text>
|
||||
|
||||
<path d="M 818 118 L 818 100 L 110 100 L 110 118" fill="none" stroke="#22c55e" stroke-width="1.5" stroke-dasharray="6,3" marker-end="url(#arrow-green)"/>
|
||||
<text x="464" y="95" text-anchor="middle" fill="#22c55e" font-size="10" font-weight="600">append one tool_result to messages[]</text>
|
||||
|
||||
<rect x="42" y="118" width="136" height="54" rx="6" fill="#ffffff" stroke="#1a1a1a" stroke-width="1.5" stroke-dasharray="6,3"/>
|
||||
<text x="110" y="141" text-anchor="middle" fill="#1a1a1a" font-size="12" font-weight="700" font-family="monospace">messages[]</text>
|
||||
<text x="110" y="159" text-anchor="middle" fill="#888888" font-size="9">message history</text>
|
||||
|
||||
<line x1="178" y1="145" x2="218" y2="145" stroke="#22c55e" stroke-width="2" marker-end="url(#arrow-green)"/>
|
||||
|
||||
<rect x="220" y="118" width="92" height="54" rx="6" fill="#ffffff" stroke="#1a1a1a" stroke-width="1.5"/>
|
||||
<text x="266" y="141" text-anchor="middle" fill="#1a1a1a" font-size="13" font-weight="700">LLM</text>
|
||||
<text x="266" y="159" text-anchor="middle" fill="#888888" font-size="9">tool_use?</text>
|
||||
|
||||
<line x1="312" y1="145" x2="352" y2="145" stroke="#22c55e" stroke-width="2" marker-end="url(#arrow-green)"/>
|
||||
|
||||
<rect x="354" y="118" width="220" height="54" rx="6" fill="#ffffff" stroke="#1a1a1a" stroke-width="1.5"/>
|
||||
<text x="464" y="140" text-anchor="middle" fill="#1a1a1a" font-size="11" font-weight="700" font-family="monospace">Workflow({name, args})</text>
|
||||
<text x="464" y="159" text-anchor="middle" fill="#888888" font-size="8.5" font-family="monospace">resume_from_run_id?</text>
|
||||
|
||||
<rect x="715" y="110" width="205" height="70" rx="6" fill="#fafafa" stroke="#1a1a1a" stroke-width="1.5"/>
|
||||
<text x="818" y="134" text-anchor="middle" fill="#1a1a1a" font-size="12" font-weight="700">tool_result</text>
|
||||
<text x="818" y="153" text-anchor="middle" fill="#22c55e" font-size="9.5" font-weight="600" font-family="monospace">launched + result + task</text>
|
||||
<text x="818" y="168" text-anchor="middle" fill="#888888" font-size="8.5">one return after the run</text>
|
||||
|
||||
<!-- Complete WorkflowTool.call lifecycle -->
|
||||
<rect x="20" y="232" width="920" height="350" rx="8" fill="#ffffff" stroke="#d0d0d0" stroke-width="1.5" stroke-dasharray="6,3"/>
|
||||
<text x="40" y="253" fill="#1a1a1a" font-size="13" font-weight="700">WorkflowTool.call — complete workflow task lifecycle</text>
|
||||
|
||||
<rect x="45" y="272" width="180" height="76" rx="6" fill="#ffffff" stroke="#1a1a1a" stroke-width="1.5"/>
|
||||
<text x="135" y="294" text-anchor="middle" fill="#1a1a1a" font-size="11" font-weight="700" font-family="monospace">WorkflowTool.call</text>
|
||||
<text x="135" y="314" text-anchor="middle" fill="#888888" font-size="9">validate meta · permission</text>
|
||||
<text x="135" y="331" text-anchor="middle" fill="#888888" font-size="9" font-family="monospace">runId · taskId · envelope</text>
|
||||
|
||||
<line x1="225" y1="310" x2="263" y2="310" stroke="#22c55e" stroke-width="2" marker-end="url(#arrow-green)"/>
|
||||
|
||||
<rect x="265" y="272" width="180" height="76" rx="6" fill="#fafafa" stroke="#d0d0d0" stroke-width="1.5"/>
|
||||
<text x="355" y="294" text-anchor="middle" fill="#1a1a1a" font-size="11" font-weight="700">Emit lifecycle</text>
|
||||
<text x="355" y="314" text-anchor="middle" fill="#22c55e" font-size="9.5" font-weight="600" font-family="monospace">async_launched</text>
|
||||
<text x="355" y="331" text-anchor="middle" fill="#888888" font-size="9.5" font-family="monospace">task_started</text>
|
||||
|
||||
<line x1="445" y1="310" x2="483" y2="310" stroke="#22c55e" stroke-width="2" marker-end="url(#arrow-green)"/>
|
||||
|
||||
<rect x="485" y="272" width="180" height="76" rx="6" fill="#ffffff" stroke="#1a1a1a" stroke-width="1.5" stroke-dasharray="6,3"/>
|
||||
<text x="575" y="294" text-anchor="middle" fill="#1a1a1a" font-size="11" font-weight="700">Execute script</text>
|
||||
<text x="575" y="314" text-anchor="middle" fill="#888888" font-size="9" font-family="monospace">phase · agent()</text>
|
||||
<text x="575" y="331" text-anchor="middle" fill="#888888" font-size="9" font-family="monospace">parallel · pipeline</text>
|
||||
|
||||
<line x1="665" y1="310" x2="703" y2="310" stroke="#22c55e" stroke-width="2" marker-end="url(#arrow-green)"/>
|
||||
|
||||
<rect x="705" y="272" width="210" height="76" rx="6" fill="#fafafa" stroke="#d0d0d0" stroke-width="1.5"/>
|
||||
<text x="810" y="294" text-anchor="middle" fill="#1a1a1a" font-size="11" font-weight="700">Finalize task</text>
|
||||
<text x="810" y="314" text-anchor="middle" fill="#888888" font-size="9">write output · save last run</text>
|
||||
<text x="810" y="331" text-anchor="middle" fill="#22c55e" font-size="9.5" font-weight="600" font-family="monospace">task_notification</text>
|
||||
|
||||
<!-- Script execution details -->
|
||||
<line x1="575" y1="348" x2="575" y2="388" stroke="#22c55e" stroke-width="2" marker-end="url(#arrow-green)"/>
|
||||
<text x="588" y="371" fill="#22c55e" font-size="9" font-weight="600">agent()</text>
|
||||
|
||||
<rect x="475" y="390" width="190" height="64" rx="6" fill="#ffffff" stroke="#1a1a1a" stroke-width="1.5"/>
|
||||
<text x="570" y="413" text-anchor="middle" fill="#1a1a1a" font-size="11" font-weight="700">Agent runner calls × N</text>
|
||||
<text x="570" y="432" text-anchor="middle" fill="#888888" font-size="9">schema validation · token budget</text>
|
||||
<text x="570" y="447" text-anchor="middle" fill="#888888" font-size="8.5">parallel work, structured results</text>
|
||||
|
||||
<line x1="665" y1="422" x2="703" y2="422" stroke="#22c55e" stroke-width="2" marker-end="url(#arrow-green)"/>
|
||||
<text x="684" y="414" text-anchor="middle" fill="#22c55e" font-size="8.5" font-weight="600">record</text>
|
||||
|
||||
<rect x="705" y="390" width="190" height="64" rx="6" fill="#fafafa" stroke="#d0d0d0" stroke-width="1.5"/>
|
||||
<text x="800" y="413" text-anchor="middle" fill="#1a1a1a" font-size="11" font-weight="700">Journal</text>
|
||||
<text x="800" y="432" text-anchor="middle" fill="#888888" font-size="9" font-family="monospace">semantic key → result</text>
|
||||
<text x="800" y="447" text-anchor="middle" fill="#888888" font-size="8.5">resume returns cached calls</text>
|
||||
|
||||
<path d="M 705 441 L 680 441 L 680 372 L 635 372 L 635 348" fill="none" stroke="#888888" stroke-width="1.5" stroke-dasharray="5,3" marker-end="url(#arrow-gray)"/>
|
||||
<text x="670" y="365" text-anchor="end" fill="#888888" font-size="8.5">cached</text>
|
||||
|
||||
<!-- Events remain within the call; the call returns once -->
|
||||
<rect x="45" y="490" width="470" height="58" rx="6" fill="#fafafa" stroke="#d0d0d0" stroke-width="1.5" stroke-dasharray="6,3"/>
|
||||
<text x="280" y="512" text-anchor="middle" fill="#1a1a1a" font-size="10.5" font-weight="700">Lifecycle + progress events emitted during the call</text>
|
||||
<text x="280" y="532" text-anchor="middle" fill="#888888" font-size="8.8" font-family="monospace">async_launched · task_started · workflow_phase / agent / log · task_notification</text>
|
||||
|
||||
<path d="M 810 348 L 810 472 L 738 472 L 738 488" fill="none" stroke="#22c55e" stroke-width="2" marker-end="url(#arrow-green)"/>
|
||||
|
||||
<rect x="560" y="490" width="355" height="58" rx="6" fill="#ffffff" stroke="#1a1a1a" stroke-width="1.5"/>
|
||||
<text x="738" y="513" text-anchor="middle" fill="#1a1a1a" font-size="11" font-weight="700" font-family="monospace">return { launched, result, task }</text>
|
||||
<text x="738" y="532" text-anchor="middle" fill="#888888" font-size="9">after task_notification</text>
|
||||
|
||||
<!-- Cross-lane call and return -->
|
||||
<path d="M 464 172 L 464 215 L 135 215 L 135 270" fill="none" stroke="#22c55e" stroke-width="1.5" stroke-dasharray="6,3" marker-end="url(#arrow-green)"/>
|
||||
<text x="250" y="209" text-anchor="middle" fill="#22c55e" font-size="9" font-weight="600">execute complete run</text>
|
||||
|
||||
<path d="M 915 519 L 934 519 L 934 211 L 818 211 L 818 180" fill="none" stroke="#22c55e" stroke-width="1.8" marker-end="url(#arrow-green)"/>
|
||||
<text x="928" y="380" text-anchor="middle" fill="#22c55e" font-size="9" font-weight="600" transform="rotate(-90 928 380)">return once</text>
|
||||
|
||||
<text x="480" y="616" text-anchor="middle" fill="#888888" font-size="10">One return boundary: async_launched is a lifecycle event; launched + result + task return together.</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 9.4 KiB |
Reference in New Issue
Block a user