refactor: organize agent harness courses

This commit is contained in:
Haoran
2026-06-16 00:10:35 +08:00
parent 20e7cbb72c
commit 8af5c24e46
491 changed files with 7961 additions and 564 deletions

View File

@@ -0,0 +1,154 @@
# The Philosophy of Agent Harness Engineering
> **The model already knows how to be an agent. Your job is to build it a world worth acting in.**
## The Fundamental Truth
Strip away every framework, every library, every architectural pattern. What remains?
A loop. A model. An invitation to act.
The agent is not the code. The agent is the model itself -- a vast neural network trained on humanity's collective problem-solving, reasoning, and tool use. The code merely provides the opportunity for the model to express its agency.
The code is the harness. The model is the agent. These are not interchangeable. Confuse them, and you will build the wrong thing.
## What an Agent IS
An agent is a neural network -- a Transformer, an RNN, a learned function -- that has been trained, through billions of gradient updates on action-sequence data, to perceive an environment, reason about goals, and take actions to achieve them.
A human is an agent: a biological neural network shaped by evolution. DeepMind's DQN is an agent: a convolutional network that learned to play Atari from raw pixels. OpenAI Five is an agent: five networks that learned Dota 2 teamwork through self-play. Claude is an agent: a language model that learned to reason and act from the breadth of human knowledge.
In every case, the agent is the trained model. Not the game engine. Not the Dota 2 client. Not the terminal. The model.
## What an Agent Is NOT
Prompt plumbing is not agency. Wiring together LLM API calls with if-else branches, node graphs, and hardcoded routing logic does not produce an agent. It produces a brittle pipeline -- a Rube Goldberg machine with an LLM wedged in as a text-completion node.
You cannot engineer your way to agency. Agency is learned, not programmed. No amount of glue code will emergently produce autonomous behavior. Those systems are the modern resurrection of GOFAI -- symbolic rule systems the field abandoned decades ago, now spray-painted with an LLM veneer.
## The Harness: What We Actually Build
If the model is the agent, then what is the code? It is the **harness** -- the environment that gives the agent the ability to perceive and act in a specific domain.
```
Harness = Tools + Knowledge + Observation + Action Interfaces + Permissions
```
### Tools: The Agent's Hands
Tools answer: **What can the agent DO?**
Each tool is an atomic action the agent can take in its environment. File read/write, shell execution, API calls, browser control, database queries. The model needs to understand what each tool does, but not how to sequence them -- it will figure that out.
**Design principle**: Atomic, composable, well-described. Start with 3-5. Add more only when the model consistently fails to accomplish tasks because a tool is missing.
### Knowledge: The Agent's Expertise
Knowledge answers: **What does the agent KNOW?**
Domain expertise that turns a general agent into a domain specialist. Product documentation, architectural decisions, regulatory requirements, style guides. Inject on-demand (via tool_result), not upfront (via system prompt). Progressive disclosure preserves context for what matters.
**Design principle**: Available but not mandatory. The agent should know what knowledge exists and pull what it needs.
### Context: The Agent's Memory
Context is the thread connecting individual actions into coherent behavior. What has been said, tried, learned, and decided.
**Design principle**: Context is precious. Protect it. Isolate subtasks that generate noise (s04). Compress when history grows long (s06). Persist goals beyond single conversations (s07).
### Permissions: The Agent's Boundaries
Permissions answer: **What is the agent ALLOWED to do?**
Sandbox file access. Require approval for destructive operations. Enforce trust boundaries between the agent and external systems. This is where safety engineering meets harness engineering.
**Design principle**: Constraints focus behavior, not limit it. "One task in_progress at a time" forces sequential focus. "Read-only subagent" prevents accidental modifications.
### Task-Process Data: The Agent's Training Signal
Every action sequence the agent executes in your harness is training signal. The perception-reasoning-action traces from real deployments are the raw material for fine-tuning the next generation of agent models. Your harness doesn't just serve the agent -- it can help evolve the agent.
## The Universal Loop
Every effective agent -- regardless of domain -- follows the same pattern:
```
LOOP:
Model sees: conversation history + available tools
Model decides: act or respond
If act: tool executed, result added to context, loop continues
If respond: answer returned, loop ends
```
This is not a simplification. This is the actual architecture. Everything else is harness engineering -- mechanisms layered on top of this loop to make the agent more effective. The loop belongs to the agent. The mechanisms belong to the harness.
## Principles of Harness Engineering
### Trust the Model
The most important principle: **trust the model**.
Don't anticipate every edge case. Don't build elaborate decision trees. Don't pre-specify the workflow.
The model is better at reasoning than any rule system you could write. Your conditional logic will fail on edge cases. The model will reason through them.
**Give the model tools and knowledge. Let it figure out how to use them.**
### Constraints Enable
This seems paradoxical, but constraints don't limit agents -- they focus them.
A todo list with "only one task in progress" forces sequential focus. A subagent with read-only access prevents accidental modifications. A context compression threshold keeps history from overwhelming.
The best constraints prevent the model from getting lost, not micromanage its approach.
### Progressive Complexity
Never build everything upfront.
```
Level 0: Model + one tool (bash) -- s01
Level 1: Model + tool dispatch map -- s02
Level 2: Model + planning -- s03
Level 3: Model + subagents + skills -- s04, s05
Level 4: Model + context management + persistence -- s06, s07, s08
Level 5: Model + teams + autonomy + isolation -- s09-s12
```
Start at the lowest level that might work. Move up only when real usage reveals the need.
## The Mind Shift
Building harnesses requires a fundamental shift in thinking:
**From**: "How do I make the system do X?"
**To**: "How do I enable the model to do X?"
**From**: "What should happen when the user says Y?"
**To**: "What tools would help address Y?"
**From**: "What's the workflow for this task?"
**To**: "What does the model need to figure out the workflow?"
**From**: "I'm building an agent."
**To**: "I'm building a harness for the agent."
The best harness code is almost boring. Simple loops. Clear tool definitions. Clean context management. The magic isn't in the code -- it's in the model.
## The Vehicle Metaphor
The model is the driver. The harness is the vehicle.
A coding agent's vehicle is its IDE, terminal, and filesystem. A farm agent's vehicle is its sensor array, irrigation controls, and weather data. A hotel agent's vehicle is its booking system, guest channels, and facility APIs.
The driver generalizes. The vehicle specializes. Your job as a harness engineer is to build the best vehicle for your domain -- one that gives the driver maximum visibility, precise controls, and clear boundaries.
Build the cockpit. Build the dashboard. Build the controls. The pilot is already trained.
## Conclusion
The model is the agent. The code is the harness. Know which one you're building.
You are not writing intelligence. You are building the world intelligence inhabits. The quality of that world -- how clearly the agent can perceive, how precisely it can act, how rich its knowledge -- directly determines how effectively the intelligence can express itself.
Build great harnesses. The agent will do the rest.

View File

@@ -0,0 +1,149 @@
#!/usr/bin/env python3
"""
Minimal Agent Template - Copy and customize this.
This is the simplest possible working agent (~80 lines).
It has everything you need: 3 tools + loop.
Usage:
1. Set ANTHROPIC_API_KEY environment variable
2. python minimal-agent.py
3. Type commands, 'q' to quit
"""
from anthropic import Anthropic
from pathlib import Path
import subprocess
import os
# Configuration
client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
MODEL = os.getenv("MODEL_NAME", "claude-sonnet-4-20250514")
WORKDIR = Path.cwd()
# System prompt - keep it simple
SYSTEM = f"""You are a coding agent at {WORKDIR}.
Rules:
- Use tools to complete tasks
- Prefer action over explanation
- Summarize what you did when done"""
# Minimal tool set - add more as needed
TOOLS = [
{
"name": "bash",
"description": "Run shell command",
"input_schema": {
"type": "object",
"properties": {"command": {"type": "string"}},
"required": ["command"]
}
},
{
"name": "read_file",
"description": "Read file contents",
"input_schema": {
"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"]
}
},
{
"name": "write_file",
"description": "Write content to file",
"input_schema": {
"type": "object",
"properties": {
"path": {"type": "string"},
"content": {"type": "string"}
},
"required": ["path", "content"]
}
},
]
def execute_tool(name: str, args: dict) -> str:
"""Execute a tool and return result."""
if name == "bash":
try:
r = subprocess.run(
args["command"], shell=True, cwd=WORKDIR,
capture_output=True, text=True, timeout=60
)
return (r.stdout + r.stderr).strip() or "(empty)"
except subprocess.TimeoutExpired:
return "Error: Timeout"
if name == "read_file":
try:
return (WORKDIR / args["path"]).read_text()[:50000]
except Exception as e:
return f"Error: {e}"
if name == "write_file":
try:
p = WORKDIR / args["path"]
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(args["content"])
return f"Wrote {len(args['content'])} bytes to {args['path']}"
except Exception as e:
return f"Error: {e}"
return f"Unknown tool: {name}"
def agent(prompt: str, history: list = None) -> str:
"""Run the agent loop."""
if history is None:
history = []
history.append({"role": "user", "content": prompt})
while True:
response = client.messages.create(
model=MODEL,
system=SYSTEM,
messages=history,
tools=TOOLS,
max_tokens=8000,
)
# Build assistant message
history.append({"role": "assistant", "content": response.content})
# If no tool calls, return text
if response.stop_reason != "tool_use":
return "".join(b.text for b in response.content if hasattr(b, "text"))
# Execute tools
results = []
for block in response.content:
if block.type == "tool_use":
print(f"> {block.name}: {block.input}")
output = execute_tool(block.name, block.input)
print(f" {output[:100]}...")
results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": output
})
history.append({"role": "user", "content": results})
if __name__ == "__main__":
print(f"Minimal Agent - {WORKDIR}")
print("Type 'q' to quit.\n")
history = []
while True:
try:
query = input(">> ").strip()
except (EOFError, KeyboardInterrupt):
break
if query in ("q", "quit", "exit", ""):
break
print(agent(query, history))
print()

View File

@@ -0,0 +1,243 @@
"""
Subagent Pattern - How to implement Task tool for context isolation.
The key insight: spawn child agents with ISOLATED context to prevent
"context pollution" where exploration details fill up the main conversation.
"""
import time
import sys
# Assuming client, MODEL, execute_tool are defined elsewhere
# =============================================================================
# AGENT TYPE REGISTRY
# =============================================================================
AGENT_TYPES = {
# Explore: Read-only, for searching and analyzing
"explore": {
"description": "Read-only agent for exploring code, finding files, searching",
"tools": ["bash", "read_file"], # No write access!
"prompt": "You are an exploration agent. Search and analyze, but NEVER modify files. Return a concise summary of what you found.",
},
# Code: Full-powered, for implementation
"code": {
"description": "Full agent for implementing features and fixing bugs",
"tools": "*", # All tools
"prompt": "You are a coding agent. Implement the requested changes efficiently. Return a summary of what you changed.",
},
# Plan: Read-only, for design work
"plan": {
"description": "Planning agent for designing implementation strategies",
"tools": ["bash", "read_file"], # Read-only
"prompt": "You are a planning agent. Analyze the codebase and output a numbered implementation plan. Do NOT make any changes.",
},
# Add your own types here...
# "test": {
# "description": "Testing agent for running and analyzing tests",
# "tools": ["bash", "read_file"],
# "prompt": "Run tests and report results. Don't modify code.",
# },
}
def get_agent_descriptions() -> str:
"""Generate descriptions for Task tool schema."""
return "\n".join(
f"- {name}: {cfg['description']}"
for name, cfg in AGENT_TYPES.items()
)
def get_tools_for_agent(agent_type: str, base_tools: list) -> list:
"""
Filter tools based on agent type.
'*' means all base tools.
Otherwise, whitelist specific tool names.
Note: Subagents don't get Task tool to prevent infinite recursion.
"""
allowed = AGENT_TYPES.get(agent_type, {}).get("tools", "*")
if allowed == "*":
return base_tools # All base tools, but NOT Task
return [t for t in base_tools if t["name"] in allowed]
# =============================================================================
# TASK TOOL DEFINITION
# =============================================================================
TASK_TOOL = {
"name": "Task",
"description": f"""Spawn a subagent for a focused subtask.
Subagents run in ISOLATED context - they don't see parent's history.
Use this to keep the main conversation clean.
Agent types:
{get_agent_descriptions()}
Example uses:
- Task(explore): "Find all files using the auth module"
- Task(plan): "Design a migration strategy for the database"
- Task(code): "Implement the user registration form"
""",
"input_schema": {
"type": "object",
"properties": {
"description": {
"type": "string",
"description": "Short task name (3-5 words) for progress display"
},
"prompt": {
"type": "string",
"description": "Detailed instructions for the subagent"
},
"agent_type": {
"type": "string",
"enum": list(AGENT_TYPES.keys()),
"description": "Type of agent to spawn"
},
},
"required": ["description", "prompt", "agent_type"],
},
}
# =============================================================================
# SUBAGENT EXECUTION
# =============================================================================
def run_task(description: str, prompt: str, agent_type: str,
client, model: str, workdir, base_tools: list, execute_tool) -> str:
"""
Execute a subagent task with isolated context.
Key concepts:
1. ISOLATED HISTORY - subagent starts fresh, no parent context
2. FILTERED TOOLS - based on agent type permissions
3. AGENT-SPECIFIC PROMPT - specialized behavior
4. RETURNS SUMMARY ONLY - parent sees just the final result
Args:
description: Short name for progress display
prompt: Detailed instructions for subagent
agent_type: Key from AGENT_TYPES
client: Anthropic client
model: Model to use
workdir: Working directory
base_tools: List of tool definitions
execute_tool: Function to execute tools
Returns:
Final text output from subagent
"""
if agent_type not in AGENT_TYPES:
return f"Error: Unknown agent type '{agent_type}'"
config = AGENT_TYPES[agent_type]
# Agent-specific system prompt
sub_system = f"""You are a {agent_type} subagent at {workdir}.
{config["prompt"]}
Complete the task and return a clear, concise summary."""
# Filtered tools for this agent type
sub_tools = get_tools_for_agent(agent_type, base_tools)
# KEY: ISOLATED message history!
# The subagent starts fresh, doesn't see parent's conversation
sub_messages = [{"role": "user", "content": prompt}]
# Progress display
print(f" [{agent_type}] {description}")
start = time.time()
tool_count = 0
# Run the same agent loop (but silently)
while True:
response = client.messages.create(
model=model,
system=sub_system,
messages=sub_messages,
tools=sub_tools,
max_tokens=8000,
)
# Check if done
if response.stop_reason != "tool_use":
break
# Execute tools
tool_calls = [b for b in response.content if b.type == "tool_use"]
results = []
for tc in tool_calls:
tool_count += 1
output = execute_tool(tc.name, tc.input)
results.append({
"type": "tool_result",
"tool_use_id": tc.id,
"content": output
})
# Update progress (in-place on same line)
elapsed = time.time() - start
sys.stdout.write(
f"\r [{agent_type}] {description} ... {tool_count} tools, {elapsed:.1f}s"
)
sys.stdout.flush()
sub_messages.append({"role": "assistant", "content": response.content})
sub_messages.append({"role": "user", "content": results})
# Final progress update
elapsed = time.time() - start
sys.stdout.write(
f"\r [{agent_type}] {description} - done ({tool_count} tools, {elapsed:.1f}s)\n"
)
# Extract and return ONLY the final text
# This is what the parent agent sees - a clean summary
for block in response.content:
if hasattr(block, "text"):
return block.text
return "(subagent returned no text)"
# =============================================================================
# USAGE EXAMPLE
# =============================================================================
"""
# In your main agent's execute_tool function:
def execute_tool(name: str, args: dict) -> str:
if name == "Task":
return run_task(
description=args["description"],
prompt=args["prompt"],
agent_type=args["agent_type"],
client=client,
model=MODEL,
workdir=WORKDIR,
base_tools=BASE_TOOLS,
execute_tool=execute_tool # Pass self for recursion
)
# ... other tools ...
# In your TOOLS list:
TOOLS = BASE_TOOLS + [TASK_TOOL]
"""

View File

@@ -0,0 +1,271 @@
"""
Tool Templates - Copy and customize these for your agent.
Each tool needs:
1. Definition (JSON schema for the model)
2. Implementation (Python function)
"""
from pathlib import Path
import subprocess
WORKDIR = Path.cwd()
# =============================================================================
# TOOL DEFINITIONS (for TOOLS list)
# =============================================================================
BASH_TOOL = {
"name": "bash",
"description": "Run a shell command. Use for: ls, find, grep, git, npm, python, etc.",
"input_schema": {
"type": "object",
"properties": {
"command": {
"type": "string",
"description": "The shell command to execute"
}
},
"required": ["command"],
},
}
READ_FILE_TOOL = {
"name": "read_file",
"description": "Read file contents. Returns UTF-8 text.",
"input_schema": {
"type": "object",
"properties": {
"path": {
"type": "string",
"description": "Relative path to the file"
},
"limit": {
"type": "integer",
"description": "Max lines to read (default: all)"
},
},
"required": ["path"],
},
}
WRITE_FILE_TOOL = {
"name": "write_file",
"description": "Write content to a file. Creates parent directories if needed.",
"input_schema": {
"type": "object",
"properties": {
"path": {
"type": "string",
"description": "Relative path for the file"
},
"content": {
"type": "string",
"description": "Content to write"
},
},
"required": ["path", "content"],
},
}
EDIT_FILE_TOOL = {
"name": "edit_file",
"description": "Replace exact text in a file. Use for surgical edits.",
"input_schema": {
"type": "object",
"properties": {
"path": {
"type": "string",
"description": "Relative path to the file"
},
"old_text": {
"type": "string",
"description": "Exact text to find (must match precisely)"
},
"new_text": {
"type": "string",
"description": "Replacement text"
},
},
"required": ["path", "old_text", "new_text"],
},
}
TODO_WRITE_TOOL = {
"name": "TodoWrite",
"description": "Update the task list. Use to plan and track progress.",
"input_schema": {
"type": "object",
"properties": {
"items": {
"type": "array",
"description": "Complete list of tasks",
"items": {
"type": "object",
"properties": {
"content": {"type": "string", "description": "Task description"},
"status": {"type": "string", "enum": ["pending", "in_progress", "completed"]},
"activeForm": {"type": "string", "description": "Present tense, e.g. 'Reading files'"},
},
"required": ["content", "status", "activeForm"],
},
}
},
"required": ["items"],
},
}
TASK_TOOL_TEMPLATE = """
# Generate dynamically with agent types
TASK_TOOL = {
"name": "Task",
"description": f"Spawn a subagent for a focused subtask.\\n\\nAgent types:\\n{get_agent_descriptions()}",
"input_schema": {
"type": "object",
"properties": {
"description": {"type": "string", "description": "Short task name (3-5 words)"},
"prompt": {"type": "string", "description": "Detailed instructions"},
"agent_type": {"type": "string", "enum": list(AGENT_TYPES.keys())},
},
"required": ["description", "prompt", "agent_type"],
},
}
"""
# =============================================================================
# TOOL IMPLEMENTATIONS
# =============================================================================
def safe_path(p: str) -> Path:
"""
Security: Ensure path stays within workspace.
Prevents ../../../etc/passwd attacks.
"""
path = (WORKDIR / p).resolve()
if not path.is_relative_to(WORKDIR):
raise ValueError(f"Path escapes workspace: {p}")
return path
def run_bash(command: str) -> str:
"""
Execute shell command with safety checks.
Safety features:
- Blocks obviously dangerous commands
- 60 second timeout
- Output truncated to 50KB
"""
dangerous = ["rm -rf /", "sudo", "shutdown", "reboot", "> /dev/"]
if any(d in command for d in dangerous):
return "Error: Dangerous command blocked"
try:
result = subprocess.run(
command,
shell=True,
cwd=WORKDIR,
capture_output=True,
text=True,
timeout=60
)
output = (result.stdout + result.stderr).strip()
return output[:50000] if output else "(no output)"
except subprocess.TimeoutExpired:
return "Error: Command timed out (60s)"
except Exception as e:
return f"Error: {e}"
def run_read_file(path: str, limit: int = None) -> str:
"""
Read file contents with optional line limit.
Features:
- Safe path resolution
- Optional line limit for large files
- Output truncated to 50KB
"""
try:
text = safe_path(path).read_text()
lines = text.splitlines()
if limit and limit < len(lines):
lines = lines[:limit]
lines.append(f"... ({len(text.splitlines()) - limit} more lines)")
return "\n".join(lines)[:50000]
except Exception as e:
return f"Error: {e}"
def run_write_file(path: str, content: str) -> str:
"""
Write content to file, creating parent directories if needed.
Features:
- Safe path resolution
- Auto-creates parent directories
- Returns byte count for confirmation
"""
try:
fp = safe_path(path)
fp.parent.mkdir(parents=True, exist_ok=True)
fp.write_text(content)
return f"Wrote {len(content)} bytes to {path}"
except Exception as e:
return f"Error: {e}"
def run_edit_file(path: str, old_text: str, new_text: str) -> str:
"""
Replace exact text in a file (surgical edit).
Features:
- Exact string matching (not regex)
- Only replaces first occurrence (safety)
- Clear error if text not found
"""
try:
fp = safe_path(path)
content = fp.read_text()
if old_text not in content:
return f"Error: Text not found in {path}"
new_content = content.replace(old_text, new_text, 1)
fp.write_text(new_content)
return f"Edited {path}"
except Exception as e:
return f"Error: {e}"
# =============================================================================
# DISPATCHER PATTERN
# =============================================================================
def execute_tool(name: str, args: dict) -> str:
"""
Dispatch tool call to implementation.
This pattern makes it easy to add new tools:
1. Add definition to TOOLS list
2. Add implementation function
3. Add case to this dispatcher
"""
if name == "bash":
return run_bash(args["command"])
if name == "read_file":
return run_read_file(args["path"], args.get("limit"))
if name == "write_file":
return run_write_file(args["path"], args["content"])
if name == "edit_file":
return run_edit_file(args["path"], args["old_text"], args["new_text"])
# Add more tools here...
return f"Unknown tool: {name}"