lzh/picoclaw
1
0
Fork 0
mirror of https://github.com/sipeed/picoclaw.git synced 2026-08-04 01:19:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into feat/skill-channel-commands

Browse Source 
# Conflicts:
#	pkg/agent/loop.go
This commit is contained in:
afjcjsbx
2026-03-22 20:51:16 +01:00
parent d7d2bf69bf 4d84bd90cd
commit d3ba40090b
62 changed files with 16762 additions and 708 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+2 -1
View File
@@ -60,5 +60,6 @@ cmd/telegram/
web/backend/dist/*
!web/backend/dist/.gitkeep
.claude/
docker/data
docker/data
README.id.md
+1
View File
@@ -217,6 +217,7 @@ Hubungkan Picoclaw ke Jaringan Sosial Agent hanya dengan mengirim satu pesan mel
| `picoclaw gateway` | Mulai gateway |
| `picoclaw status` | Tampilkan status |
| `picoclaw version` | Tampilkan info versi |
| `picoclaw model` | Lihat atau ubah model default |
| `picoclaw cron list` | Daftar semua tugas terjadwal |
| `picoclaw cron add ...` | Tambah tugas terjadwal |
| `picoclaw cron disable` | Nonaktifkan tugas terjadwal |
README.it.md
+1
View File
@@ -217,6 +217,7 @@ Connetti PicoClaw al Social Network degli Agent semplicemente inviando un singol
| `picoclaw gateway` | Avvia il gateway |
| `picoclaw status` | Mostra lo stato |
| `picoclaw version` | Mostra le info sulla versione |
| `picoclaw model` | Mostra o cambia il modello predefinito |
| `picoclaw cron list` | Elenca tutti i job pianificati |
| `picoclaw cron add ...` | Aggiunge un job pianificato |
| `picoclaw cron disable` | Disabilita un job pianificato |
README.md
+644
View File
@@ -742,6 +742,649 @@ Connect Picoclaw to the Agent Social Network simply by sending a single message
**Read `https://clawdchat.ai/skill.md` and follow the instructions to join [ClawdChat.ai](https://clawdchat.ai)**
## ⚙️ Configuration
Config file: `~/.picoclaw/config.json`
### Environment Variables
You can override default paths using environment variables. This is useful for portable installations, containerized deployments, or running picoclaw as a system service. These variables are independent and control different paths.
| Variable | Description | Default Path |
|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| `PICOCLAW_CONFIG` | Overrides the path to the configuration file. This directly tells picoclaw which `config.json` to load, ignoring all other locations. | `~/.picoclaw/config.json` |
| `PICOCLAW_HOME` | Overrides the root directory for picoclaw data. This changes the default location of the `workspace` and other data directories. | `~/.picoclaw` |
**Examples:**
```bash
# Run picoclaw using a specific config file
# The workspace path will be read from within that config file
PICOCLAW_CONFIG=/etc/picoclaw/production.json picoclaw gateway
# Run picoclaw with all its data stored in /opt/picoclaw
# Config will be loaded from the default ~/.picoclaw/config.json
# Workspace will be created at /opt/picoclaw/workspace
PICOCLAW_HOME=/opt/picoclaw picoclaw agent
# Use both for a fully customized setup
PICOCLAW_HOME=/srv/picoclaw PICOCLAW_CONFIG=/srv/picoclaw/main.json picoclaw gateway
```
### Workspace Layout
PicoClaw stores data in your configured workspace (default: `~/.picoclaw/workspace`):
```
~/.picoclaw/workspace/
├── sessions/ # Conversation sessions and history
├── memory/ # Long-term memory (MEMORY.md)
├── state/ # Persistent state (last channel, etc.)
├── cron/ # Scheduled jobs database
├── skills/ # Workspace-specific skills
├── AGENT.md # Structured agent definition and system prompt
├── SOUL.md # Agent soul
├── USER.md # User profile and preferences for this workspace
├── HEARTBEAT.md # Periodic task prompts (checked every 30 min)
└── ...
```
### Skill Sources
By default, skills are loaded from:
1. `~/.picoclaw/workspace/skills` (workspace)
2. `~/.picoclaw/skills` (global)
3. `<current-working-directory>/skills` (builtin)
For advanced/test setups, you can override the builtin skills root with:
```bash
export PICOCLAW_BUILTIN_SKILLS=/path/to/skills
```
### Unified Command Execution Policy
- Generic slash commands are executed through a single path in `pkg/agent/loop.go` via `commands.Executor`.
- Channel adapters no longer consume generic commands locally; they forward inbound text to the bus/agent path. Telegram still auto-registers supported commands at startup.
- Unknown slash command (for example `/foo`) passes through to normal LLM processing.
- Registered but unsupported command on the current channel (for example `/show` on WhatsApp) returns an explicit user-facing error and stops further processing.
### 🔒 Security Sandbox
PicoClaw runs in a sandboxed environment by default. The agent can only access files and execute commands within the configured workspace.
#### Default Configuration
```json
{
"agents": {
"defaults": {
"workspace": "~/.picoclaw/workspace",
"restrict_to_workspace": true
}
}
}
```
| Option | Default | Description |
| ----------------------- | ----------------------- | ----------------------------------------- |
| `workspace` | `~/.picoclaw/workspace` | Working directory for the agent |
| `restrict_to_workspace` | `true` | Restrict file/command access to workspace |
#### Protected Tools
When `restrict_to_workspace: true`, the following tools are sandboxed:
| Tool | Function | Restriction |
| ------------- | ---------------- | -------------------------------------- |
| `read_file` | Read files | Only files within workspace |
| `write_file` | Write files | Only files within workspace |
| `list_dir` | List directories | Only directories within workspace |
| `edit_file` | Edit files | Only files within workspace |
| `append_file` | Append to files | Only files within workspace |
| `exec` | Execute commands | Command paths must be within workspace |
#### Additional Exec Protection
Even with `restrict_to_workspace: false`, the `exec` tool blocks these dangerous commands:
* `rm -rf`, `del /f`, `rmdir /s` — Bulk deletion
* `format`, `mkfs`, `diskpart` — Disk formatting
* `dd if=` — Disk imaging
* Writing to `/dev/sd[a-z]` — Direct disk writes
* `shutdown`, `reboot`, `poweroff` — System shutdown
* Fork bomb `:(){ :|:& };:`
#### Error Examples
```
[ERROR] tool: Tool execution failed
{tool=exec, error=Command blocked by safety guard (path outside working dir)}
```
```
[ERROR] tool: Tool execution failed
{tool=exec, error=Command blocked by safety guard (dangerous pattern detected)}
```
#### Disabling Restrictions (Security Risk)
If you need the agent to access paths outside the workspace:
**Method 1: Config file**
```json
{
"agents": {
"defaults": {
"restrict_to_workspace": false
}
}
}
```
**Method 2: Environment variable**
```bash
export PICOCLAW_AGENTS_DEFAULTS_RESTRICT_TO_WORKSPACE=false
```
> ⚠️ **Warning**: Disabling this restriction allows the agent to access any path on your system. Use with caution in controlled environments only.
#### Security Boundary Consistency
The `restrict_to_workspace` setting applies consistently across all execution paths:
| Execution Path | Security Boundary |
| ---------------- | ---------------------------- |
| Main Agent | `restrict_to_workspace` ✅ |
| Subagent / Spawn | Inherits same restriction ✅ |
| Heartbeat tasks | Inherits same restriction ✅ |
All paths share the same workspace restriction — there's no way to bypass the security boundary through subagents or scheduled tasks.
### Heartbeat (Periodic Tasks)
PicoClaw can perform periodic tasks automatically. Create a `HEARTBEAT.md` file in your workspace:
```markdown
# Periodic Tasks
- Check my email for important messages
- Review my calendar for upcoming events
- Check the weather forecast
```
The agent will read this file every 30 minutes (configurable) and execute any tasks using available tools.
#### Async Tasks with Spawn
For long-running tasks (web search, API calls), use the `spawn` tool to create a **subagent**:
```markdown
# Periodic Tasks
## Quick Tasks (respond directly)
- Report current time
## Long Tasks (use spawn for async)
- Search the web for AI news and summarize
- Check email and report important messages
```
**Key behaviors:**
| Feature | Description |
| ----------------------- | --------------------------------------------------------- |
| **spawn** | Creates async subagent, doesn't block heartbeat |
| **Independent context** | Subagent has its own context, no session history |
| **message tool** | Subagent communicates with user directly via message tool |
| **Non-blocking** | After spawning, heartbeat continues to next task |
#### How Subagent Communication Works
```
Heartbeat triggers
↓
Agent reads HEARTBEAT.md
↓
For long task: spawn subagent
↓ ↓
Continue to next task Subagent works independently
↓ ↓
All tasks done Subagent uses "message" tool
↓ ↓
Respond HEARTBEAT_OK User receives result directly
```
The subagent has access to tools (message, web_search, etc.) and can communicate with the user independently without going through the main agent.
**Configuration:**
```json
{
"heartbeat": {
"enabled": true,
"interval": 30
}
}
```
| Option | Default | Description |
| ---------- | ------- | ---------------------------------- |
| `enabled` | `true` | Enable/disable heartbeat |
| `interval` | `30` | Check interval in minutes (min: 5) |
**Environment variables:**
* `PICOCLAW_HEARTBEAT_ENABLED=false` to disable
* `PICOCLAW_HEARTBEAT_INTERVAL=60` to change interval
### Providers
> [!NOTE]
> Groq provides free voice transcription via Whisper. If configured, audio messages from any channel will be automatically transcribed at the agent level.
| Provider | Purpose | Get API Key |
| ------------ | --------------------------------------- | ------------------------------------------------------------ |
| `gemini` | LLM (Gemini direct) | [aistudio.google.com](https://aistudio.google.com) |
| `zhipu` | LLM (Zhipu direct) | [bigmodel.cn](https://bigmodel.cn) |
| `volcengine` | LLM(Volcengine direct) | [volcengine.com](https://www.volcengine.com/activity/codingplan?utm_campaign=PicoClaw&utm_content=PicoClaw&utm_medium=devrel&utm_source=OWO&utm_term=PicoClaw) |
| `openrouter` | LLM (recommended, access to all models) | [openrouter.ai](https://openrouter.ai) |
| `anthropic` | LLM (Claude direct) | [console.anthropic.com](https://console.anthropic.com) |
| `openai` | LLM (GPT direct) | [platform.openai.com](https://platform.openai.com) |
| `deepseek` | LLM (DeepSeek direct) | [platform.deepseek.com](https://platform.deepseek.com) |
| `qwen` | LLM (Qwen direct) | [dashscope.console.aliyun.com](https://dashscope.console.aliyun.com) |
| `groq` | LLM + **Voice transcription** (Whisper) | [console.groq.com](https://console.groq.com) |
| `cerebras` | LLM (Cerebras direct) | [cerebras.ai](https://cerebras.ai) |
| `vivgrid` | LLM (Vivgrid direct) | [vivgrid.com](https://vivgrid.com) |
### Model Configuration (model_list)
> **What's New?** PicoClaw now uses a **model-centric** configuration approach. Simply specify `vendor/model` format (e.g., `zhipu/glm-4.7`) to add new providers—**zero code changes required!**
This design also enables **multi-agent support** with flexible provider selection:
- **Different agents, different providers**: Each agent can use its own LLM provider
- **Model fallbacks**: Configure primary and fallback models for resilience
- **Load balancing**: Distribute requests across multiple endpoints
- **Centralized configuration**: Manage all providers in one place
#### 📋 All Supported Vendors
| Vendor | `model` Prefix | Default API Base | Protocol | API Key |
| ------------------- | ----------------- |-----------------------------------------------------| --------- | ---------------------------------------------------------------- |
| **OpenAI** | `openai/` | `https://api.openai.com/v1` | OpenAI | [Get Key](https://platform.openai.com) |
| **Anthropic** | `anthropic/` | `https://api.anthropic.com/v1` | Anthropic | [Get Key](https://console.anthropic.com) |
| **智谱 AI (GLM)** | `zhipu/` | `https://open.bigmodel.cn/api/paas/v4` | OpenAI | [Get Key](https://open.bigmodel.cn/usercenter/proj-mgmt/apikeys) |
| **DeepSeek** | `deepseek/` | `https://api.deepseek.com/v1` | OpenAI | [Get Key](https://platform.deepseek.com) |
| **Google Gemini** | `gemini/` | `https://generativelanguage.googleapis.com/v1beta` | OpenAI | [Get Key](https://aistudio.google.com/api-keys) |
| **Groq** | `groq/` | `https://api.groq.com/openai/v1` | OpenAI | [Get Key](https://console.groq.com) |
| **Moonshot** | `moonshot/` | `https://api.moonshot.cn/v1` | OpenAI | [Get Key](https://platform.moonshot.cn) |
| **通义千问 (Qwen)** | `qwen/` | `https://dashscope.aliyuncs.com/compatible-mode/v1` | OpenAI | [Get Key](https://dashscope.console.aliyun.com) |
| **NVIDIA** | `nvidia/` | `https://integrate.api.nvidia.com/v1` | OpenAI | [Get Key](https://build.nvidia.com) |
| **Ollama** | `ollama/` | `http://localhost:11434/v1` | OpenAI | Local (no key needed) |
| **OpenRouter** | `openrouter/` | `https://openrouter.ai/api/v1` | OpenAI | [Get Key](https://openrouter.ai/keys) |
| **LiteLLM Proxy** | `litellm/` | `http://localhost:4000/v1` | OpenAI | Your LiteLLM proxy key |
| **VLLM** | `vllm/` | `http://localhost:8000/v1` | OpenAI | Local |
| **Cerebras** | `cerebras/` | `https://api.cerebras.ai/v1` | OpenAI | [Get Key](https://cerebras.ai) |
| **VolcEngine (Doubao)** | `volcengine/` | `https://ark.cn-beijing.volces.com/api/v3` | OpenAI | [Get Key](https://www.volcengine.com/activity/codingplan?utm_campaign=PicoClaw&utm_content=PicoClaw&utm_medium=devrel&utm_source=OWO&utm_term=PicoClaw) |
| **神算云** | `shengsuanyun/` | `https://router.shengsuanyun.com/api/v1` | OpenAI | - |
| **BytePlus** | `byteplus/` | `https://ark.ap-southeast.bytepluses.com/api/v3` | OpenAI | [Get Key](https://www.byteplus.com) |
| **Vivgrid** | `vivgrid/` | `https://api.vivgrid.com/v1` | OpenAI | [Get Key](https://vivgrid.com) |
| **LongCat** | `longcat/` | `https://api.longcat.chat/openai` | OpenAI | [Get Key](https://longcat.chat/platform) |
| **ModelScope (魔搭)**| `modelscope/` | `https://api-inference.modelscope.cn/v1` | OpenAI | [Get Token](https://modelscope.cn/my/tokens) |
| **Antigravity** | `antigravity/` | Google Cloud | Custom | OAuth only |
| **GitHub Copilot** | `github-copilot/` | `localhost:4321` | gRPC | - |
#### Basic Configuration
```json
{
"model_list": [
{
"model_name": "ark-code-latest",
"model": "volcengine/ark-code-latest",
"api_key": "sk-your-api-key"
},
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-openai-key"
},
{
"model_name": "claude-sonnet-4.6",
"model": "anthropic/claude-sonnet-4.6",
"api_key": "sk-ant-your-key"
},
{
"model_name": "glm-4.7",
"model": "zhipu/glm-4.7",
"api_key": "your-zhipu-key"
}
],
"agents": {
"defaults": {
"model": "gpt-5.4"
}
}
}
```
#### Vendor-Specific Examples
**OpenAI**
```json
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-..."
}
```
**VolcEngine (Doubao)**
```json
{
"model_name": "ark-code-latest",
"model": "volcengine/ark-code-latest",
"api_key": "sk-..."
}
```
**智谱 AI (GLM)**
```json
{
"model_name": "glm-4.7",
"model": "zhipu/glm-4.7",
"api_key": "your-key"
}
```
**DeepSeek**
```json
{
"model_name": "deepseek-chat",
"model": "deepseek/deepseek-chat",
"api_key": "sk-..."
}
```
**Anthropic (with API key)**
```json
{
"model_name": "claude-sonnet-4.6",
"model": "anthropic/claude-sonnet-4.6",
"api_key": "sk-ant-your-key"
}
```
> Run `picoclaw auth login --provider anthropic` to paste your API token.
**Anthropic Messages API (native format)**
For direct Anthropic API access or custom endpoints that only support Anthropic's native message format:
```json
{
"model_name": "claude-opus-4-6",
"model": "anthropic-messages/claude-opus-4-6",
"api_key": "sk-ant-your-key",
"api_base": "https://api.anthropic.com"
}
```
> Use `anthropic-messages` protocol when:
> - Using third-party proxies that only support Anthropic's native `/v1/messages` endpoint (not OpenAI-compatible `/v1/chat/completions`)
> - Connecting to services like MiniMax, Synthetic that require Anthropic's native message format
> - The existing `anthropic` protocol returns 404 errors (indicating the endpoint doesn't support OpenAI-compatible format)
>
> **Note:** The `anthropic` protocol uses OpenAI-compatible format (`/v1/chat/completions`), while `anthropic-messages` uses Anthropic's native format (`/v1/messages`). Choose based on your endpoint's supported format.
**Ollama (local)**
```json
{
"model_name": "llama3",
"model": "ollama/llama3"
}
```
**Custom Proxy/API**
```json
{
"model_name": "my-custom-model",
"model": "openai/custom-model",
"api_base": "https://my-proxy.com/v1",
"api_key": "sk-...",
"request_timeout": 300
}
```
**LiteLLM Proxy**
```json
{
"model_name": "lite-gpt4",
"model": "litellm/lite-gpt4",
"api_base": "http://localhost:4000/v1",
"api_key": "sk-..."
}
```
PicoClaw strips only the outer `litellm/` prefix before sending the request, so proxy aliases like `litellm/lite-gpt4` send `lite-gpt4`, while `litellm/openai/gpt-4o` sends `openai/gpt-4o`.
#### Load Balancing
Configure multiple endpoints for the same model name—PicoClaw will automatically round-robin between them:
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_base": "https://api1.example.com/v1",
"api_key": "sk-key1"
},
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_base": "https://api2.example.com/v1",
"api_key": "sk-key2"
}
]
}
```
#### Migration from Legacy `providers` Config
The old `providers` configuration is **deprecated** but still supported for backward compatibility.
**Old Config (deprecated):**
```json
{
"providers": {
"zhipu": {
"api_key": "your-key",
"api_base": "https://open.bigmodel.cn/api/paas/v4"
}
},
"agents": {
"defaults": {
"provider": "zhipu",
"model": "glm-4.7"
}
}
}
```
**New Config (recommended):**
```json
{
"model_list": [
{
"model_name": "glm-4.7",
"model": "zhipu/glm-4.7",
"api_key": "your-key"
}
],
"agents": {
"defaults": {
"model": "glm-4.7"
}
}
}
```
For detailed migration guide, see [docs/migration/model-list-migration.md](docs/migration/model-list-migration.md).
### Provider Architecture
PicoClaw routes providers by protocol family:
- OpenAI-compatible protocol: OpenRouter, OpenAI-compatible gateways, Groq, Zhipu, and vLLM-style endpoints.
- Anthropic protocol: Claude-native API behavior.
- Codex/OAuth path: OpenAI OAuth/token authentication route.
This keeps the runtime lightweight while making new OpenAI-compatible backends mostly a config operation (`api_base` + `api_key`).
<details>
<summary><b>Zhipu</b></summary>
**1. Get API key and base URL**
* Get [API key](https://bigmodel.cn/usercenter/proj-mgmt/apikeys)
**2. Configure**
```json
{
"agents": {
"defaults": {
"workspace": "~/.picoclaw/workspace",
"model": "glm-4.7",
"max_tokens": 8192,
"temperature": 0.7,
"max_tool_iterations": 20
}
},
"providers": {
"zhipu": {
"api_key": "Your API Key",
"api_base": "https://open.bigmodel.cn/api/paas/v4"
}
}
}
```
**3. Run**
```bash
picoclaw agent -m "Hello"
```
</details>
<details>
<summary><b>Full config example</b></summary>
```json
{
"agents": {
"defaults": {
"model": "anthropic/claude-opus-4-5"
}
},
"session": {
"dm_scope": "per-channel-peer",
"backlog_limit": 20
},
"providers": {
"openrouter": {
"api_key": "sk-or-v1-xxx"
},
"groq": {
"api_key": "gsk_xxx"
}
},
"channels": {
"telegram": {
"enabled": true,
"token": "123456:ABC...",
"allow_from": ["123456789"]
},
"discord": {
"enabled": true,
"token": "",
"allow_from": [""]
},
"whatsapp": {
"enabled": false,
"bridge_url": "ws://localhost:3001",
"use_native": false,
"session_store_path": "",
"allow_from": []
},
"feishu": {
"enabled": false,
"app_id": "cli_xxx",
"app_secret": "xxx",
"encrypt_key": "",
"verification_token": "",
"allow_from": []
},
"qq": {
"enabled": false,
"app_id": "",
"app_secret": "",
"allow_from": []
}
},
"tools": {
"web": {
"brave": {
"enabled": false,
"api_key": "BSA...",
"max_results": 5
},
"duckduckgo": {
"enabled": true,
"max_results": 5
},
"perplexity": {
"enabled": false,
"api_key": "",
"max_results": 5
},
"searxng": {
"enabled": false,
"base_url": "http://localhost:8888",
"max_results": 5
}
},
"cron": {
"exec_timeout_minutes": 5
}
},
"heartbeat": {
"enabled": true,
"interval": 30
}
}
```
</details>
## 🖥️ CLI Reference
| Command | Description |
@@ -753,6 +1396,7 @@ Connect Picoclaw to the Agent Social Network simply by sending a single message
| `picoclaw gateway` | Start the gateway |
| `picoclaw status` | Show status |
| `picoclaw version` | Show version info |
| `picoclaw model` | Show or change default model |
| `picoclaw cron list` | List all scheduled jobs |
| `picoclaw cron add ...` | Add a scheduled job |
| `picoclaw cron disable` | Disable a scheduled job |
cmd/picoclaw/internal/onboard/helpers_test.go
+19 -7
View File
@@ -6,20 +6,32 @@ import (
"testing"
)
func TestCopyEmbeddedToTargetUsesAgentsMarkdown(t *testing.T) {
func TestCopyEmbeddedToTargetUsesStructuredAgentFiles(t *testing.T) {
targetDir := t.TempDir()
if err := copyEmbeddedToTarget(targetDir); err != nil {
t.Fatalf("copyEmbeddedToTarget() error = %v", err)
}
agentsPath := filepath.Join(targetDir, "AGENTS.md")
if _, err := os.Stat(agentsPath); err != nil {
t.Fatalf("expected %s to exist: %v", agentsPath, err)
agentPath := filepath.Join(targetDir, "AGENT.md")
if _, err := os.Stat(agentPath); err != nil {
t.Fatalf("expected %s to exist: %v", agentPath, err)
}
legacyPath := filepath.Join(targetDir, "AGENT.md")
if _, err := os.Stat(legacyPath); !os.IsNotExist(err) {
t.Fatalf("expected legacy file %s to be absent, got err=%v", legacyPath, err)
soulPath := filepath.Join(targetDir, "SOUL.md")
if _, err := os.Stat(soulPath); err != nil {
t.Fatalf("expected %s to exist: %v", soulPath, err)
}
userPath := filepath.Join(targetDir, "USER.md")
if _, err := os.Stat(userPath); err != nil {
t.Fatalf("expected %s to exist: %v", userPath, err)
}
for _, legacyName := range []string{"AGENTS.md", "IDENTITY.md"} {
legacyPath := filepath.Join(targetDir, legacyName)
if _, err := os.Stat(legacyPath); !os.IsNotExist(err) {
t.Fatalf("expected legacy file %s to be absent, got err=%v", legacyPath, err)
}
}
}
config/config.example.json
+9
View File
@@ -6,6 +6,7 @@
"restrict_to_workspace": true,
"model_name": "gpt-5.4",
"max_tokens": 8192,
"context_window": 131072,
"temperature": 0.7,
"max_tool_iterations": 20,
"summarize_message_threshold": 20,
@@ -549,6 +550,14 @@
"voice": {
"echo_transcription": false
},
"hooks": {
"enabled": true,
"defaults": {
"observer_timeout_ms": 500,
"interceptor_timeout_ms": 5000,
"approval_timeout_ms": 60000
}
},
"gateway": {
"host": "127.0.0.1",
"port": 18790,
docs/agent-refactor/context.md
+164
View File
@@ -0,0 +1,164 @@
# Context
## What this document covers
This document makes explicit the boundaries of context management in the agent loop:
- what fills the context window and how space is divided
- what is stored in session history vs. built at request time
- when and how context compression happens
- how token budgets are estimated
These are existing concepts. This document clarifies their boundaries rather than introducing new ones.
---
## Context window regions
The context window is the model's total input capacity. Four regions fill it:
| Region | Assembled by | Stored in session? |
|---|---|---|
| System prompt | `BuildMessages()` — static + dynamic parts | No |
| Summary | `SetSummary()` stores it; `BuildMessages()` injects it | Separate from history |
| Session history | User / assistant / tool messages | Yes |
| Tool definitions | Provider adapter injects at call time | No |
`MaxTokens` (the output generation limit) must also be reserved from the total budget.
The available space for history is therefore:
```
history_budget = ContextWindow - system_prompt - summary - tool_definitions - MaxTokens
```
---
## ContextWindow vs MaxTokens
These serve different purposes:
- **MaxTokens** — maximum tokens the LLM may generate in one response. Sent as the `max_tokens` request parameter.
- **ContextWindow** — the model's total input context capacity.
These were previously set to the same value, which caused the summarization threshold to fire either far too early (at the default 32K) or not at all (when a user raised `max_tokens`).
Current default when not explicitly configured: `ContextWindow = MaxTokens * 4`.
---
## Session history
Session history stores only conversation messages:
- `user` — user input
- `assistant` — LLM response (may include `ToolCalls`)
- `tool` — tool execution results
Session history does **not** contain:
- System prompts — assembled at request time by `BuildMessages`
- Summary content — stored separately via `SetSummary`, injected by `BuildMessages`
This distinction matters: any code that operates on session history — compression, boundary detection, token estimation — must not assume a system message is present.
---
## Turn
A **Turn** is one complete cycle:
> user message -> LLM iterations (possibly including tool calls) -> final assistant response
This definition comes from the agent loop design (#1316). In session history, Turn boundaries are identified by `user`-role messages.
Turn is the atomic unit for compression. Cutting inside a Turn can orphan tool-call sequences — an assistant message with `ToolCalls` separated from its corresponding `tool` results. Compressing at Turn boundaries avoids this by construction.
`parseTurnBoundaries(history)` returns the starting index of each Turn.
`findSafeBoundary(history, targetIndex)` snaps a target cut point to the nearest Turn boundary.
---
## Compression paths
Three compression paths exist, in order of preference:
### 1. Async summarization
`maybeSummarize` runs after each Turn completes.
Triggers when message count exceeds a threshold, or when estimated history tokens exceed a percentage of `ContextWindow`. If triggered, a background goroutine calls the LLM to produce a summary of the oldest messages. The summary is stored via `SetSummary`; `BuildMessages` injects it into the system prompt on the next call.
Cut point uses `findSafeBoundary` so no Turn is split.
### 2. Proactive budget check
`isOverContextBudget` runs before each LLM call.
Uses the full budget formula: `message_tokens + tool_def_tokens + MaxTokens > ContextWindow`. If over budget, triggers `forceCompression` and rebuilds messages before calling the LLM.
This prevents wasted (and billed) LLM calls that would otherwise fail with a context-window error.
### 3. Emergency compression (reactive)
`forceCompression` runs when the LLM returns a context-window error despite the proactive check.
Drops the oldest ~50% of Turns. If the history is a single Turn with no safe split point (e.g. one user message followed by a massive tool response), falls back to keeping only the most recent user message — breaking Turn atomicity as a last resort to avoid a context-exceeded loop.
Stores a compression note in the session summary (not in history messages) so `BuildMessages` can include it in the next system prompt.
This is the fallback for when the token estimate undershoots reality.
---
## Token estimation
Estimation uses a heuristic of ~2.5 characters per token (`chars * 2 / 5`).
`estimateMessageTokens` counts:
- `Content` (rune count, for multibyte correctness)
- `ReasoningContent` (extended thinking / chain-of-thought)
- `ToolCalls` — ID, type, function name, arguments
- `ToolCallID` (tool result metadata)
- Per-message overhead (role label, JSON structure)
- `Media` items — flat per-item token estimate, added directly to the final count (not through the character heuristic, since actual cost depends on resolution and provider-specific image tokenization)
`estimateToolDefsTokens` counts tool definition overhead: name, description, JSON schema of parameters.
These are deliberately heuristic. The proactive check handles the common case; the reactive path catches estimation errors.
---
## Interface boundaries
Context budget functions (`parseTurnBoundaries`, `findSafeBoundary`, `estimateMessageTokens`, `isOverContextBudget`) are **pure functions**. They take `[]providers.Message` and integer parameters. They have no dependency on `AgentLoop` or any other runtime struct.
`BuildMessages` is the sole assembler of the final message array sent to the LLM. Budget functions inform compression decisions but do not construct messages.
`forceCompression` and `summarizeSession` mutate session state (history and summary). `BuildMessages` reads that state to construct context. The flow is:
```
budget check --> compression decision --> mutate session --> BuildMessages reads session --> LLM call
```
---
## Known gaps
These are recognized limitations in the current implementation, documented here for visibility:
- **Summarization trigger does not use the full budget formula.** `maybeSummarize` compares estimated history tokens against a percentage of `ContextWindow`. It does not account for system prompt size, tool definition overhead, or `MaxTokens` reserve. The proactive check covers the critical path (preventing 400 errors), but the summarization trigger could be aligned with the same budget model for more accurate early compression.
- **Token estimation is heuristic.** It does not account for provider-specific tokenization, exact system prompt size (assembled separately), or variable image token costs. The two-path design (proactive + reactive) is intended to tolerate this imprecision.
- **Reactive retry does not preserve media.** When the reactive path rebuilds context after compression, it currently passes empty values for media references. This is a pre-existing issue in the main loop, not introduced by the budget system.
---
## What this document does not cover
- How `AGENT.md` frontmatter configures context parameters — that is part of the Agent definition work
- How the context builder assembles context in the new architecture — that is upcoming work
- How compression events surface through the event system — that is part of the event model (#1316)
- Subagent context isolation — that is a separate track
docs/design/hook-system-design.zh.md
+476
View File
@@ -0,0 +1,476 @@
# PicoClaw Hook 系统设计（基于 `refactor/agent`）
## 背景
本设计围绕两个议题展开：
- `#1316`：把 agent loop 重构为事件驱动、可中断、可追加、可观测
- `#1796`：在 EventBus 稳定后，把 hooks 设计为 EventBus 的 consumer，而不是重新发明一套事件模型
当前分支已经完成了第一步里的“事件系统基础”，但还没有真正的 hook 挂载层。因此这里的目标不是重新设计 event，而是在已有实现上补出一层可扩展、可拦截、可外挂的 HookManager。
## 外部项目对比
### OpenClaw
OpenClaw 的扩展能力分成三层：
- Internal hooks：目录发现，运行在 Gateway 进程内
- Plugin hooks：插件在运行时注册 hook，也在进程内
- Webhooks：外部系统通过 HTTP 触发 Gateway 动作，属于进程外
值得借鉴的点：
- 有“项目内挂载”和“项目外挂载”两种路径
- hook 是配置驱动，可启停
- 外部入口有明确的安全边界和映射层
不建议直接照搬的点：
- OpenClaw 的 hooks / plugin hooks / webhooks 是三套路由，PicoClaw 当前体量下会偏重
- HTTP webhook 更适合“事件进入系统”，不适合作为“可同步拦截 agent loop”的基础机制
### pi-mono
pi-mono 的核心思路更接近当前分支：
- 扩展统一为 extension API
- 事件分为观察型和可变更型
- 某些阶段允许 `transform` / `block` / `replace`
- 扩展代码主要是进程内执行
- RPC mode 把 UI 交互桥接到进程外客户端
值得借鉴的点：
- 不把“观察”和“拦截”混成一个接口
- 允许返回结构化动作，而不是只有回调
- 进程外通信只暴露必要协议，不把整个内部对象图泄露出去
## 当前分支现状
### 已有能力
当前分支已经具备 hook 系统的地基：
- `pkg/agent/events.go` 定义了稳定的 `EventKind`、`EventMeta` 和 payload
- `pkg/agent/eventbus.go` 提供了非阻塞 fan-out 的 `EventBus`
- `pkg/agent/loop.go` 中的 `runTurn()` 已在 turn、llm、tool、interrupt、follow-up、summary 等节点发射事件
- `pkg/agent/steering.go` 已支持 steering、graceful interrupt、hard abort
- `pkg/agent/turn.go` 已维护 turn phase、恢复点、active turn、abort 状态
### 现有缺口
当前分支还缺四件事：
- 没有 HookManager，只有 EventBus
- 没有 Before/After LLM、Before/After Tool 这种同步拦截点
- 没有审批型 hook
- 子 agent 仍走 `pkg/tools/SubagentManager + RunToolLoop`，没有接入 `pkg/agent` 的 turn tree 和事件流
### 一个关键现实
`#1316` 文案里提到“只读并行、写入串行”的工具执行策略，但当前 `runTurn()` 实现已经先收敛成“顺序执行 + 每个工具后检查 steering / interrupt”。因此 hook 设计不应依赖未来的并行模型，而应该先兼容当前顺序执行，再为以后增加 `ReadOnlyIndicator` 留口子。
## 设计原则
- Hook 必须建立在 `pkg/agent` 的 EventBus 和 turn 上下文之上
- EventBus 负责广播，HookManager 负责拦截，两者职责分离
- 项目内挂载要简单，项目外挂载必须走 IPC
- 观察型 hook 不能阻塞 loop；拦截型 hook 必须有超时
- 先覆盖主 turn，不把 sub-turn 一次做满
- 不新增第二套用户事件命名系统，优先复用 `EventKind.String()`
## 总体架构
分成三层：
1. `EventBus`
负责广播只读事件，现有实现直接复用
2. `HookManager`
负责管理 hook、排序、超时、错误隔离，并在 `runTurn()` 的明确检查点执行同步拦截
3. `HookMount`
负责两种挂载方式：
- 进程内 Go hook
- 进程外 IPC hook
换句话说：
- EventBus 是“发生了什么”
- HookManager 是“谁能介入”
- HookMount 是“这些 hook 从哪里来”
## Hook 分类
不建议把所有 hook 都设计成 `OnEvent(evt)`。
建议拆成两类。
### 1. 观察型
只消费事件，不修改流程：
```go
type EventObserver interface {
OnEvent(ctx context.Context, evt agent.Event) error
}
```
这类 hook 直接订阅 EventBus 即可。
适用场景：
- 审计日志
- 指标上报
- 调试 trace
- 将事件转发给外部 UI / TUI / Web 面板
### 2. 拦截型
只在少数明确节点触发，允许返回动作：
```go
type LLMInterceptor interface {
BeforeLLM(ctx context.Context, req *LLMRequest) HookDecision[*LLMRequest]
AfterLLM(ctx context.Context, resp *LLMResponse) HookDecision[*LLMResponse]
}
type ToolInterceptor interface {
BeforeTool(ctx context.Context, call *ToolCall) HookDecision[*ToolCall]
AfterTool(ctx context.Context, result *ToolResultView) HookDecision[*ToolResultView]
}
type ToolApprover interface {
ApproveTool(ctx context.Context, req *ToolApprovalRequest) ApprovalDecision
}
```
这里的 `HookDecision` 统一支持：
- `continue`
- `modify`
- `deny_tool`
- `abort_turn`
- `hard_abort`
## 对外暴露的最小 hook 面
V1 不需要把所有 EventKind 都变成可拦截点。
建议只开放这些同步 hook：
- `before_llm`
- `after_llm`
- `before_tool`
- `after_tool`
- `approve_tool`
其余节点继续作为只读事件暴露：
- `turn_start`
- `turn_end`
- `llm_request`
- `llm_response`
- `tool_exec_start`
- `tool_exec_end`
- `tool_exec_skipped`
- `steering_injected`
- `follow_up_queued`
- `interrupt_received`
- `context_compress`
- `session_summarize`
- `error`
`subturn_*` 在 V1 中保留名字，但不承诺一定触发，直到子 turn 迁移完成。
## 项目内挂载
内部挂载必须尽量低摩擦。
建议提供两种等价方式，底层都走 HookManager。
### 方式 A：代码显式挂载
```go
al.MountHook(hooks.Named("audit", &AuditHook{}))
```
适用于：
- 仓内内建 hook
- 单元测试
- feature flag 控制
### 方式 B：内建 registry
```go
func init() {
hooks.RegisterBuiltin("audit", func() hooks.Hook {
return &AuditHook{}
})
}
```
启动时根据配置启用：
```json
{
"hooks": {
"builtins": {
"audit": { "enabled": true }
}
}
}
```
这比 OpenClaw 的目录扫描更轻，也更贴合 Go 项目。
## 项目外挂载
这是本设计的硬要求。
建议 V1 采用：
- `JSON-RPC over stdio`
原因：
- 跨平台最简单
- 不依赖额外端口
- 非常适合“由 PicoClaw 启动一个外部 hook 进程”
- 比 HTTP webhook 更适合同步拦截
### 外部 hook 进程模型
PicoClaw 启动外部进程，并在其 stdin/stdout 上跑协议。
配置示例：
```json
{
"hooks": {
"processes": {
"review-gate": {
"enabled": true,
"transport": "stdio",
"command": ["uvx", "picoclaw-hook-reviewer"],
"observe": ["turn_start", "turn_end", "tool_exec_end"],
"intercept": ["before_tool", "approve_tool"],
"timeout_ms": 5000
}
}
}
}
```
### 协议边界
不要把内部 Go 结构体直接暴露给 IPC。
建议定义稳定的协议对象：
- `HookHandshake`
- `HookEventNotification`
- `BeforeLLMRequest`
- `AfterLLMRequest`
- `BeforeToolRequest`
- `AfterToolRequest`
- `ApproveToolRequest`
- `HookDecision`
其中：
- 观察型事件用 notification，fire-and-forget
- 拦截型事件用 request/response，同步等待
### 为什么是 stdio，而不是直接用 HTTP webhook
因为两者用途不同：
- HTTP webhook 更适合“外部系统向 PicoClaw 投递事件”
- stdio/RPC 更适合“PicoClaw 在 turn 内同步询问外部 hook 是否改写 / 放行 / 拒绝”
如果未来需要 OpenClaw 式 webhook，可以作为独立入口层，再把外部事件转成 inbound message 或 steering，而不是直接替代 hook IPC。
## Hook 执行顺序
建议统一排序规则：
- 先内建 in-process hook
- 再外部 IPC hook
- 同组内按 `priority` 从小到大执行
原因：
- 内建 hook 延迟更低，适合做基础规范化
- 外部 hook 更适合做审批、审计、组织级策略
## 超时与错误策略
### 观察型
- 默认超时：`500ms`
- 超时或报错：记录日志，继续主流程
### 拦截型
- `before_llm` / `after_llm` / `before_tool` / `after_tool`：默认 `5s`
- `approve_tool`：默认 `60s`
超时行为：
- 普通拦截：`continue`
- 审批：`deny`
这点应直接沿用 `#1316` 的安全倾向。
## 与当前分支的对接点
### 直接复用
- 事件定义：`pkg/agent/events.go`
- 事件广播：`pkg/agent/eventbus.go`
- 活跃 turn / interrupt / rollback：`pkg/agent/turn.go`
- 事件发射点：`pkg/agent/loop.go`
### 需要新增
- `pkg/agent/hooks.go`
- Hook 接口
- HookDecision / ApprovalDecision
- HookManager
- `pkg/agent/hook_mount.go`
- 内建 hook 注册
- 外部进程 hook 注册
- `pkg/agent/hook_ipc.go`
- stdio JSON-RPC bridge
- `pkg/agent/hook_types.go`
- IPC 稳定载荷
### 需要改造
- `pkg/agent/loop.go`
- 在 LLM 和 tool 关键路径前后插入 HookManager 调用
- `pkg/tools/base.go`
- 可选新增 `ReadOnlyIndicator`
- `pkg/tools/spawn.go`
- `pkg/tools/subagent.go`
- 先保留现状
- 等 sub-turn 迁移后再接入 `subturn_*` hook
## 一个更贴合当前分支的数据流
### 观察链路
```text
runTurn() -> emitEvent() -> EventBus -> observers
```
### 拦截链路
```text
runTurn()
-> HookManager.BeforeLLM()
-> Provider.Chat()
-> HookManager.AfterLLM()
-> HookManager.BeforeTool()
-> HookManager.ApproveTool()
-> tool.Execute()
-> HookManager.AfterTool()
```
也就是说：
- observer 不改变现有 `emitEvent()`
- interceptor 直接插在 `runTurn()` 热路径
## 用户可见配置
建议新增：
```json
{
"hooks": {
"enabled": true,
"builtins": {},
"processes": {},
"defaults": {
"observer_timeout_ms": 500,
"interceptor_timeout_ms": 5000,
"approval_timeout_ms": 60000
}
}
}
```
V1 不做复杂自动发现。
原因：
- 当前分支重点是把地基打稳
- 目录扫描、安装器、脚手架可以后置
- 先让仓内和仓外都能挂上去，比“管理体验完整”更重要
## 推荐的 V1 范围
### 必做
- HookManager
- in-process 挂载
- stdio IPC 挂载
- observer hooks
- `before_tool` / `after_tool` / `approve_tool`
- `before_llm` / `after_llm`
### 可后置
- hook CLI 管理命令
- hook 自动发现
- Unix socket / named pipe transport
- sub-turn hook 生命周期
- read-only 并行分组
- webhook 到 inbound message 的映射入口
## 分阶段落地
### Phase 1
- 引入 HookManager
- 支持 in-process observer + interceptor
- 先只接主 turn
### Phase 2
- 引入 `stdio` 外部 hook 进程桥
- 支持组织级审批 / 审计 / 参数改写
### Phase 3
- 把 `SubagentManager` 迁移到 `runTurn/sub-turn`
- 接通 `subturn_spawn` / `subturn_end` / `subturn_result_delivered`
### Phase 4
- 视需求补 `ReadOnlyIndicator`
- 在主 turn 和 sub-turn 上统一只读并行策略
## 最终结论
最适合 PicoClaw 当前分支的方案，不是直接复制 OpenClaw 的 hooks，也不是完整照搬 pi-mono 的 extension system，而是：
- 以现有 `EventBus` 为只读观察面
- 以新增 `HookManager` 为同步拦截面
- 项目内通过 Go 对象直接挂载
- 项目外通过 `stdio JSON-RPC` 进程通信挂载
这样做有三个好处：
- 和 `#1796` 一致，hooks 只是 EventBus 之上的消费层
- 和当前 `refactor/agent` 实现一致，不需要推翻已有事件系统
- 同时满足“仓内简单挂载”和“仓外进程通信挂载”两个硬需求
docs/design/steering-spec.md
+306
View File
@@ -0,0 +1,306 @@
# Steering — Implementation Specification
## Problem
When the agent is running (executing a chain of tool calls), the user has no way to redirect it. They must wait for the full cycle to complete before sending a new message. This creates a poor experience when the agent takes a wrong direction — the user watches it waste time on tools that are no longer relevant.
## Solution
Steering introduces a **message queue** that external callers can push into at any time. The agent loop polls this queue at well-defined checkpoints. When a steering message is found, the agent:
1. Stops executing further tools in the current batch
2. Injects the user's message into the conversation context
3. Calls the LLM again with the updated context
The user's intent reaches the model **as soon as the current tool finishes**, not after the entire turn completes.
## Architecture Overview
```mermaid
graph TD
subgraph External Callers
TG[Telegram]
DC[Discord]
SL[Slack]
end
subgraph AgentLoop
BUS[MessageBus]
DRAIN[drainBusToSteering goroutine]
SQ[steeringQueue]
RLI[runLLMIteration]
TE[Tool Execution Loop]
LLM[LLM Call]
end
TG -->|PublishInbound| BUS
DC -->|PublishInbound| BUS
SL -->|PublishInbound| BUS
BUS -->|ConsumeInbound while busy| DRAIN
DRAIN -->|Steer| SQ
RLI -->|1. initial poll| SQ
TE -->|2. poll after each tool| SQ
SQ -->|pendingMessages| RLI
RLI -->|inject into context| LLM
```
### Bus drain mechanism
Channels (Telegram, Discord, etc.) publish messages to the `MessageBus` via `PublishInbound`. Without additional wiring, these messages would sit in the bus buffer until the current `processMessage` finishes — meaning steering would never work for real users.
The solution: when `Run()` starts processing a message, it spawns a **drain goroutine** (`drainBusToSteering`) that keeps consuming from the bus and calling `Steer()`. When `processMessage` returns, the drain is canceled and normal consumption resumes.
```mermaid
sequenceDiagram
participant Bus
participant Run
participant Drain
participant AgentLoop
Run->>Bus: ConsumeInbound() → msg
Run->>Drain: spawn drainBusToSteering(ctx)
Run->>Run: processMessage(msg)
Note over Drain: running concurrently
Bus-->>Drain: ConsumeInbound() → newMsg
Drain->>AgentLoop: al.transcribeAudioInMessage(ctx, newMsg)
Drain->>AgentLoop: Steer(providers.Message{Content: newMsg.Content})
Run->>Run: processMessage returns
Run->>Drain: cancel context
Note over Drain: exits
```
## Data Structures
### steeringQueue
A thread-safe FIFO queue, private to the `agent` package.
| Field | Type | Description |
|-------|------|-------------|
| `mu` | `sync.Mutex` | Protects all access to `queue` and `mode` |
| `queue` | `[]providers.Message` | Pending steering messages |
| `mode` | `SteeringMode` | Dequeue strategy |
**Methods:**
| Method | Description |
|--------|-------------|
| `push(msg) error` | Appends a message to the queue. Returns an error if the queue is full (`MaxQueueSize`) |
| `dequeue() []Message` | Removes and returns messages according to `mode`. Returns `nil` if empty |
| `len() int` | Returns the current queue length |
| `setMode(mode)` | Updates the dequeue strategy |
| `getMode() SteeringMode` | Returns the current mode |
### SteeringMode
| Value | Constant | Behavior |
|-------|----------|----------|
| `"one-at-a-time"` | `SteeringOneAtATime` | `dequeue()` returns only the **first** message. Remaining messages stay in the queue for subsequent polls. |
| `"all"` | `SteeringAll` | `dequeue()` drains the **entire** queue and returns all messages at once. |
Default: `"one-at-a-time"`.
### processOptions extension
A new field was added to `processOptions`:
| Field | Type | Description |
|-------|------|-------------|
| `SkipInitialSteeringPoll` | `bool` | When `true`, the initial steering poll at loop start is skipped. Used by `Continue()` to avoid double-dequeuing. |
## Public API on AgentLoop
| Method | Signature | Description |
|--------|-----------|-------------|
| `Steer` | `Steer(msg providers.Message) error` | Enqueues a steering message. Returns an error if the queue is full or not initialized. Thread-safe, can be called from any goroutine. |
| `SteeringMode` | `SteeringMode() SteeringMode` | Returns the current dequeue mode. |
| `SetSteeringMode` | `SetSteeringMode(mode SteeringMode)` | Changes the dequeue mode at runtime. |
| `Continue` | `Continue(ctx, sessionKey, channel, chatID) (string, error)` | Resumes an idle agent using pending steering messages. Returns `""` if queue is empty. |
## Integration into the Agent Loop
### Where steering is wired
The steering queue lives as a field on `AgentLoop`:
```
AgentLoop
├── bus
├── cfg
├── registry
├── steering *steeringQueue ← new
├── ...
```
It is initialized in `NewAgentLoop` from `cfg.Agents.Defaults.SteeringMode`.
### Detailed flow through runLLMIteration
```mermaid
sequenceDiagram
participant User
participant AgentLoop
participant runLLMIteration
participant ToolExecution
participant LLM
User->>AgentLoop: Steer(message)
Note over AgentLoop: steeringQueue.push(message)
Note over runLLMIteration: ── iteration starts ──
runLLMIteration->>AgentLoop: dequeueSteeringMessages()<br/>[initial poll]
AgentLoop-->>runLLMIteration: [] (empty, or messages)
alt pendingMessages not empty
runLLMIteration->>runLLMIteration: inject into messages[]<br/>save to session
end
runLLMIteration->>LLM: Chat(messages, tools)
LLM-->>runLLMIteration: response with toolCalls[0..N]
loop for each tool call (sequential)
ToolExecution->>ToolExecution: execute tool[i]
ToolExecution->>ToolExecution: process result,<br/>append to messages[]
ToolExecution->>AgentLoop: dequeueSteeringMessages()
AgentLoop-->>ToolExecution: steeringMessages
alt steering found
opt remaining tools > 0
Note over ToolExecution: Mark tool[i+1..N-1] as<br/>"Skipped due to queued user message."
end
Note over ToolExecution: steeringAfterTools = steeringMessages
Note over ToolExecution: break out of tool loop
end
end
alt steeringAfterTools not empty
ToolExecution-->>runLLMIteration: pendingMessages = steeringAfterTools
Note over runLLMIteration: next iteration will inject<br/>these before calling LLM
end
Note over runLLMIteration: ── loop back to iteration start ──
```
### Polling checkpoints
| # | Location | When | Purpose |
|---|----------|------|---------|
| 1 | Top of `runLLMIteration`, before first LLM call | Once, at loop entry | Catch messages enqueued while the agent was still setting up context |
| 2 | After every tool completes (including the first and the last) | Immediately after each tool's result is processed | Interrupt the batch as early as possible — if steering is found and there are remaining tools, they are all skipped |
### What happens to skipped tools
When steering interrupts a tool batch after tool `[i]` completes, all tools from `[i+1]` to `[N-1]` are **not executed**. Instead, a tool result message is generated for each:
```json
{
"role": "tool",
"content": "Skipped due to queued user message.",
"tool_call_id": "<original_call_id>"
}
```
These results are:
- Appended to the conversation `messages[]`
- Saved to the session via `AddFullMessage`
This ensures the LLM knows which of its requested actions were not performed.
### Loop condition change
The iteration loop condition was changed from:
```go
for iteration < agent.MaxIterations
```
to:
```go
for iteration < agent.MaxIterations || len(pendingMessages) > 0
```
This allows **one extra iteration** when steering arrives right at the max iteration boundary, ensuring the steering message is always processed.
### Tool execution: parallel → sequential
**Before steering:** all tool calls in a batch were executed in parallel using `sync.WaitGroup`.
**After steering:** tool calls execute **sequentially**. This is required because steering must be polled between individual tool completions. A parallel execution model would not allow interrupting mid-batch.
> **Trade-off:** This introduces latency when the LLM requests multiple independent tools in a single turn. In practice, most batches contain 1-2 tools, so the impact is minimal. The benefit of being able to interrupt outweighs the cost.
### Why skip remaining tools (instead of letting them finish)
Two strategies were considered when a steering message is detected mid-batch:
1. **Skip remaining tools** (chosen) — stop executing, mark the rest as skipped, inject steering
2. **Finish all tools, then inject** — let everything run, append steering afterwards
Strategy 2 was rejected for three reasons:
**Irreversible side effects.** Tools can send emails, write files, spawn subagents, or call external APIs. If the user says "stop" or "change direction", those actions have already happened and cannot be undone.
| Tool batch | Steering | Skip (1) | Finish (2) |
|---|---|---|---|
| `[search, send_email]` | "don't send it" | Email not sent | Email sent |
| `[query, write_file, spawn]` | "wrong database" | Only query runs | File + subagent wasted |
| `[fetch₁, fetch₂, fetch₃, write]` | topic change | 1 fetch | 3 fetches + write, all discarded |
**Wasted latency.** Tools like web fetches and API calls take seconds each. In a 3-tool batch averaging 3-4s per tool, the user would wait 10+ seconds for work that gets thrown away.
**The LLM retains full awareness.** Skipped tools receive an explicit `"Skipped due to queued user message."` result, so the model knows what was not done and can decide whether to re-execute with the new context or take a different path.
## The Continue() method
`Continue` handles the case where the agent is **idle** (its last message was from the assistant) and the user has enqueued steering messages in the meantime.
```mermaid
flowchart TD
A[Continue called] --> B{dequeueSteeringMessages}
B -->|empty| C["return ('', nil)"]
B -->|messages found| D[Combine message contents]
D --> E["runAgentLoop with<br/>SkipInitialSteeringPoll: true"]
E --> F[Return response]
```
**Why `SkipInitialSteeringPoll: true`?** Because `Continue` already dequeued the messages itself. Without this flag, `runLLMIteration` would poll again at the start and find nothing (the queue is already empty), or worse, double-process if new messages arrived in the meantime.
## Configuration
```json
{
"agents": {
"defaults": {
"steering_mode": "one-at-a-time"
}
}
}
```
| Field | Type | Default | Env var |
|-------|------|---------|---------|
| `steering_mode` | `string` | `"one-at-a-time"` | `PICOCLAW_AGENTS_DEFAULTS_STEERING_MODE` |
## Design decisions and trade-offs
| Decision | Rationale |
|----------|-----------|
| Sequential tool execution | Required for per-tool steering polls. Parallel execution cannot be interrupted mid-batch. |
| Polling-based (not channel/signal) | Keeps the implementation simple. No need for `select` or signal channels. The polling cost is negligible (mutex lock + slice length check). |
| `one-at-a-time` as default | Gives the model a chance to react to each steering message individually. More predictable behavior than dumping all messages at once. |
| Skipped tools get explicit error results | The LLM protocol requires a tool result for every tool call in the assistant message. Omitting them would cause API errors. The skip message also informs the model about what was not done. |
| `Continue()` uses `SkipInitialSteeringPoll` | Prevents race conditions and double-dequeuing when resuming an idle agent. |
| Queue stored on `AgentLoop`, not `AgentInstance` | Steering is a loop-level concern (it affects the iteration flow), not a per-agent concern. All agents share the same steering queue since `processMessage` is sequential. |
| Bus drain goroutine in `Run()` | Channels (Telegram, Discord, etc.) publish to the bus via `PublishInbound`. Without the drain, messages would queue in the bus channel buffer and only be consumed after `processMessage` returns — defeating the purpose of steering. The drain goroutine bridges the gap by consuming new bus messages and calling `Steer()` while the agent is busy. |
| Audio transcription before steering | The drain goroutine calls `al.transcribeAudioInMessage(ctx, msg)` before steering, so voice messages are converted to text before the agent sees them. If transcription fails, the error is silently discarded and the original message is steered as-is. |
| `MaxQueueSize = 10` | Prevents unbounded memory growth if a user sends many messages while the agent is busy. Excess messages are dropped with a warning. |
docs/hooks/README.md
+679
View File
@@ -0,0 +1,679 @@
# Hook System Guide
This document describes the hook system that is implemented in the current repository, not the older design draft.
The current implementation supports two mounting modes:
1. In-process hooks
2. Out-of-process process hooks (`JSON-RPC over stdio`)
The repository no longer ships standalone example source files. The Go and Python examples below are embedded directly in this document. If you want to use them, copy them into your own local files first.
## Supported Hook Types
| Type | Interface | Stage | Can modify data |
| --- | --- | --- | --- |
| Observer | `EventObserver` | EventBus broadcast | No |
| LLM interceptor | `LLMInterceptor` | `before_llm` / `after_llm` | Yes |
| Tool interceptor | `ToolInterceptor` | `before_tool` / `after_tool` | Yes |
| Tool approver | `ToolApprover` | `approve_tool` | No, returns allow/deny |
The currently exposed synchronous hook points are:
- `before_llm`
- `after_llm`
- `before_tool`
- `after_tool`
- `approve_tool`
Everything else is exposed as read-only events.
## Execution Order
`HookManager` sorts hooks like this:
1. In-process hooks first
2. Process hooks second
3. Lower `priority` first within the same source
4. Name order as the final tie-breaker
## Timeouts
Global defaults live under `hooks.defaults`:
- `observer_timeout_ms`
- `interceptor_timeout_ms`
- `approval_timeout_ms`
Note: the current implementation does not support per-process-hook `timeout_ms`. Timeouts are global defaults.
## Quick Start
If your first goal is simply to prove that the hook flow works and observe real requests, the easiest path is the Python process-hook example below:
1. Enable `hooks.enabled`
2. Save the Python example from this document to a local file, for example `/tmp/review_gate.py`
3. Set `PICOCLAW_HOOK_LOG_FILE`
4. Restart the gateway
5. Watch the log file with `tail -f`
Example:
```json
{
"hooks": {
"enabled": true,
"processes": {
"py_review_gate": {
"enabled": true,
"priority": 100,
"transport": "stdio",
"command": [
"python3",
"/tmp/review_gate.py"
],
"observe": [
"tool_exec_start",
"tool_exec_end",
"tool_exec_skipped"
],
"intercept": [
"before_tool",
"approve_tool"
],
"env": {
"PICOCLAW_HOOK_LOG_FILE": "/tmp/picoclaw-hook-review-gate.log"
}
}
}
}
}
```
Watch it with:
```bash
tail -f /tmp/picoclaw-hook-review-gate.log
```
If you are developing PicoClaw itself rather than only validating the protocol, continue with the Go in-process example as well.
## What The Two Examples Are For
- Go in-process example
Best for validating the host-side hook chain and understanding `MountHook()` plus the synchronous stages
- Python process example
Best for understanding the `JSON-RPC over stdio` protocol and verifying the message flow between PicoClaw and an external process
Both examples are intentionally safe: they only log, never rewrite, and never deny.
## Go In-Process Example
The following is a minimal logging hook for in-process use. It implements:
1. `EventObserver`
2. `LLMInterceptor`
3. `ToolInterceptor`
4. `ToolApprover`
It only records activity. It does not rewrite requests or reject tools.
You can save it as your own Go file, for example `pkg/myhooks/example_logger.go`:
```go
package myhooks
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/sipeed/picoclaw/pkg/agent"
"github.com/sipeed/picoclaw/pkg/logger"
)
type ExampleLoggerHookOptions struct {
LogFile string `json:"log_file,omitempty"`
LogEvents bool `json:"log_events,omitempty"`
}
type ExampleLoggerHook struct {
logFile string
logEvents bool
mu sync.Mutex
}
func NewExampleLoggerHook(opts ExampleLoggerHookOptions) *ExampleLoggerHook {
return &ExampleLoggerHook{
logFile: strings.TrimSpace(opts.LogFile),
logEvents: opts.LogEvents,
}
}
func (h *ExampleLoggerHook) OnEvent(ctx context.Context, evt agent.Event) error {
_ = ctx
if h == nil || !h.logEvents {
return nil
}
h.record("event", evt.Meta, map[string]any{
"event": evt.Kind.String(),
"payload": evt.Payload,
}, nil)
return nil
}
func (h *ExampleLoggerHook) BeforeLLM(
ctx context.Context,
req *agent.LLMHookRequest,
) (*agent.LLMHookRequest, agent.HookDecision, error) {
_ = ctx
h.record("before_llm", req.Meta, req, agent.HookDecision{Action: agent.HookActionContinue})
return req, agent.HookDecision{Action: agent.HookActionContinue}, nil
}
func (h *ExampleLoggerHook) AfterLLM(
ctx context.Context,
resp *agent.LLMHookResponse,
) (*agent.LLMHookResponse, agent.HookDecision, error) {
_ = ctx
h.record("after_llm", resp.Meta, resp, agent.HookDecision{Action: agent.HookActionContinue})
return resp, agent.HookDecision{Action: agent.HookActionContinue}, nil
}
func (h *ExampleLoggerHook) BeforeTool(
ctx context.Context,
call *agent.ToolCallHookRequest,
) (*agent.ToolCallHookRequest, agent.HookDecision, error) {
_ = ctx
h.record("before_tool", call.Meta, call, agent.HookDecision{Action: agent.HookActionContinue})
return call, agent.HookDecision{Action: agent.HookActionContinue}, nil
}
func (h *ExampleLoggerHook) AfterTool(
ctx context.Context,
result *agent.ToolResultHookResponse,
) (*agent.ToolResultHookResponse, agent.HookDecision, error) {
_ = ctx
h.record("after_tool", result.Meta, result, agent.HookDecision{Action: agent.HookActionContinue})
return result, agent.HookDecision{Action: agent.HookActionContinue}, nil
}
func (h *ExampleLoggerHook) ApproveTool(
ctx context.Context,
req *agent.ToolApprovalRequest,
) (agent.ApprovalDecision, error) {
_ = ctx
decision := agent.ApprovalDecision{Approved: true}
h.record("approve_tool", req.Meta, req, decision)
return decision, nil
}
func (h *ExampleLoggerHook) record(stage string, meta agent.EventMeta, payload any, decision any) {
logger.InfoCF("hooks", "Example hook observed", map[string]any{
"stage": stage,
})
if h == nil || h.logFile == "" {
return
}
entry := map[string]any{
"ts": time.Now().UTC(),
"stage": stage,
"meta": meta,
"payload": payload,
"decision": decision,
}
body, err := json.Marshal(entry)
if err != nil {
logger.WarnCF("hooks", "Example hook log encode failed", map[string]any{
"stage": stage,
"error": err.Error(),
})
return
}
h.mu.Lock()
defer h.mu.Unlock()
if dir := filepath.Dir(h.logFile); dir != "" && dir != "." {
if err := os.MkdirAll(dir, 0o755); err != nil {
logger.WarnCF("hooks", "Example hook log mkdir failed", map[string]any{
"stage": stage,
"path": h.logFile,
"error": err.Error(),
})
return
}
}
file, err := os.OpenFile(h.logFile, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644)
if err != nil {
logger.WarnCF("hooks", "Example hook log open failed", map[string]any{
"stage": stage,
"path": h.logFile,
"error": err.Error(),
})
return
}
defer func() { _ = file.Close() }()
if _, err := file.Write(append(body, '\n')); err != nil {
logger.WarnCF("hooks", "Example hook log write failed", map[string]any{
"stage": stage,
"path": h.logFile,
"error": err.Error(),
})
}
}
```
### Mounting It In Code
If code mounting is enough, call this after `AgentLoop` is initialized:
```go
hook := myhooks.NewExampleLoggerHook(myhooks.ExampleLoggerHookOptions{
LogFile: "/tmp/picoclaw-hook-example-logger.log",
LogEvents: true,
})
if err := al.MountHook(agent.NamedHook("example-logger", hook)); err != nil {
panic(err)
}
```
### If You Also Want Config Mounting
The hook system supports builtin hooks, but that requires you to compile the factory into your binary. In practice, that means you need registration code like this alongside the hook definition above:
```go
package myhooks
import (
"context"
"encoding/json"
"fmt"
"github.com/sipeed/picoclaw/pkg/agent"
"github.com/sipeed/picoclaw/pkg/config"
)
func init() {
if err := agent.RegisterBuiltinHook("example_logger", func(
ctx context.Context,
spec config.BuiltinHookConfig,
) (any, error) {
_ = ctx
var opts ExampleLoggerHookOptions
if len(spec.Config) > 0 {
if err := json.Unmarshal(spec.Config, &opts); err != nil {
return nil, fmt.Errorf("decode example_logger config: %w", err)
}
}
return NewExampleLoggerHook(opts), nil
}); err != nil {
panic(err)
}
}
```
Only after you register that builtin will the following config work:
```json
{
"hooks": {
"enabled": true,
"builtins": {
"example_logger": {
"enabled": true,
"priority": 10,
"config": {
"log_file": "/tmp/picoclaw-hook-example-logger.log",
"log_events": true
}
}
}
}
}
```
### How To Observe It
- If `log_file` is set, each hook call is appended as JSON Lines
- If `log_file` is not set, the hook still writes summaries to the gateway log
- Requests that only hit the LLM path usually show `before_llm` and `after_llm`
- Requests that trigger tools usually also show `before_tool`, `approve_tool`, and `after_tool`
- If `log_events=true`, you will also see `event`
Typical log lines:
```json
{"ts":"2026-03-21T14:10:00Z","stage":"before_tool","meta":{"session_key":"session-1"},"payload":{"tool":"echo_text","arguments":{"text":"hello"}},"decision":{"action":"continue"}}
{"ts":"2026-03-21T14:10:00Z","stage":"approve_tool","meta":{"session_key":"session-1"},"payload":{"tool":"echo_text","arguments":{"text":"hello"}},"decision":{"approved":true}}
```
If you only see `before_llm` and `after_llm`, that usually means the request did not trigger any tool call, not that the hook failed to mount.
## Python Process-Hook Example
The following script is a minimal process-hook example. It uses only the Python standard library and supports:
1. `hook.hello`
2. `hook.event`
3. `hook.before_tool`
4. `hook.approve_tool`
It only records activity. It does not rewrite or deny anything.
Save it to any local path, for example `/tmp/review_gate.py`:
```python
#!/usr/bin/env python3
from __future__ import annotations
import json
import os
import signal
import sys
from datetime import datetime, timezone
from typing import Any
LOG_EVENTS = os.getenv("PICOCLAW_HOOK_LOG_EVENTS", "1").lower() not in {"0", "false", "no"}
LOG_FILE = os.getenv("PICOCLAW_HOOK_LOG_FILE", "").strip()
def append_log(entry: dict[str, Any]) -> None:
if not LOG_FILE:
return
payload = {
"ts": datetime.now(timezone.utc).isoformat(),
**entry,
}
try:
log_dir = os.path.dirname(LOG_FILE)
if log_dir:
os.makedirs(log_dir, exist_ok=True)
with open(LOG_FILE, "a", encoding="utf-8") as handle:
handle.write(json.dumps(payload, ensure_ascii=True) + "\n")
except OSError as exc:
log_stderr(f"failed to write hook log file {LOG_FILE}: {exc}")
def send_response(message_id: int, result: Any | None = None, error: str | None = None) -> None:
payload: dict[str, Any] = {
"jsonrpc": "2.0",
"id": message_id,
}
if error is not None:
payload["error"] = {"code": -32000, "message": error}
else:
payload["result"] = result if result is not None else {}
append_log({
"direction": "out",
"id": message_id,
"response": payload.get("result"),
"error": payload.get("error"),
})
try:
sys.stdout.write(json.dumps(payload, ensure_ascii=True) + "\n")
sys.stdout.flush()
except BrokenPipeError:
raise SystemExit(0) from None
def log_stderr(message: str) -> None:
try:
sys.stderr.write(message + "\n")
sys.stderr.flush()
except BrokenPipeError:
raise SystemExit(0) from None
def handle_shutdown_signal(signum: int, _frame: Any) -> None:
raise KeyboardInterrupt(f"received signal {signum}")
def handle_before_tool(params: dict[str, Any]) -> dict[str, Any]:
_ = params
return {"action": "continue"}
def handle_approve_tool(params: dict[str, Any]) -> dict[str, Any]:
_ = params
return {"approved": True}
def handle_request(method: str, params: dict[str, Any]) -> dict[str, Any]:
if method == "hook.hello":
return {"ok": True, "name": "python-review-gate"}
if method == "hook.before_tool":
return handle_before_tool(params)
if method == "hook.approve_tool":
return handle_approve_tool(params)
if method == "hook.before_llm":
return {"action": "continue"}
if method == "hook.after_llm":
return {"action": "continue"}
if method == "hook.after_tool":
return {"action": "continue"}
raise KeyError(f"method not found: {method}")
def main() -> int:
try:
for raw_line in sys.stdin:
line = raw_line.strip()
if not line:
continue
try:
message = json.loads(line)
except json.JSONDecodeError as exc:
log_stderr(f"failed to decode request: {exc}")
append_log({
"direction": "in",
"decode_error": str(exc),
"raw": line,
})
continue
method = message.get("method")
message_id = message.get("id", 0)
params = message.get("params") or {}
if not isinstance(params, dict):
params = {}
append_log({
"direction": "in",
"id": message_id,
"method": method,
"params": params,
"notification": not bool(message_id),
})
if not message_id:
if method == "hook.event" and LOG_EVENTS:
log_stderr(f"observed event: {params.get('Kind')}")
continue
try:
result = handle_request(str(method or ""), params)
except KeyError as exc:
send_response(int(message_id), error=str(exc))
continue
except Exception as exc:
send_response(int(message_id), error=f"unexpected error: {exc}")
continue
send_response(int(message_id), result=result)
except KeyboardInterrupt:
return 0
return 0
if __name__ == "__main__":
signal.signal(signal.SIGINT, handle_shutdown_signal)
signal.signal(signal.SIGTERM, handle_shutdown_signal)
raise SystemExit(main())
```
### Configuration
```json
{
"hooks": {
"enabled": true,
"processes": {
"py_review_gate": {
"enabled": true,
"priority": 100,
"transport": "stdio",
"command": [
"python3",
"/abs/path/to/review_gate.py"
],
"observe": [
"tool_exec_start",
"tool_exec_end",
"tool_exec_skipped"
],
"intercept": [
"before_tool",
"approve_tool"
],
"env": {
"PICOCLAW_HOOK_LOG_FILE": "/tmp/picoclaw-hook-review-gate.log"
}
}
}
}
}
```
### Environment Variables
- `PICOCLAW_HOOK_LOG_EVENTS`
Whether to write `hook.event` summaries to `stderr`, enabled by default
- `PICOCLAW_HOOK_LOG_FILE`
Path to an external log file. When set, the script appends inbound hook requests, notifications, and outbound responses as JSON Lines
Note: `PICOCLAW_HOOK_LOG_FILE` has no default. If you do not set it, the script does not write any file logs.
### How To Confirm It Received Hooks
Watch two places:
- Gateway logs
Useful for confirming that the host successfully started the process and for seeing event summaries written to `stderr`
- `PICOCLAW_HOOK_LOG_FILE`
Useful for seeing the exact requests the script received and the exact responses it returned
Typical interpretation:
- Only `hook.hello`
The process started and completed the handshake, but no business hook request has arrived yet
- `hook.event`
The `observe` configuration is working
- `hook.before_tool`
The `intercept: ["before_tool", ...]` configuration is working
- `hook.approve_tool`
The approval hook path is working
Because this example never rewrites or denies, the expected responses look like:
```json
{"direction":"out","id":7,"response":{"action":"continue"},"error":null}
{"direction":"out","id":8,"response":{"approved":true},"error":null}
```
A complete sample:
```json
{"ts":"2026-03-21T14:12:00+00:00","direction":"in","id":1,"method":"hook.hello","params":{"name":"py_review_gate","version":1,"modes":["observe","tool","approve"]},"notification":false}
{"ts":"2026-03-21T14:12:00+00:00","direction":"out","id":1,"response":{"ok":true,"name":"python-review-gate"},"error":null}
{"ts":"2026-03-21T14:12:05+00:00","direction":"in","id":0,"method":"hook.event","params":{"Kind":"tool_exec_start"},"notification":true}
{"ts":"2026-03-21T14:12:05+00:00","direction":"in","id":7,"method":"hook.before_tool","params":{"tool":"echo_text","arguments":{"text":"hello"}},"notification":false}
{"ts":"2026-03-21T14:12:05+00:00","direction":"out","id":7,"response":{"action":"continue"},"error":null}
```
Additional notes:
- Timestamps are UTC
- `notification=true` means it was a notification such as `hook.event`, which does not expect a response
- `id` increases within a single hook process; if the process restarts, the counter starts over
## Process-Hook Protocol
Current process hooks use `JSON-RPC over stdio`:
- PicoClaw starts the external process
- Requests and responses are exchanged as one JSON message per line
- `hook.event` is a notification and does not need a response
- `hook.before_llm`, `hook.after_llm`, `hook.before_tool`, `hook.after_tool`, and `hook.approve_tool` are request/response calls
The host does not currently accept new RPCs initiated by the process hook. In practice, that means an external hook can only respond to PicoClaw calls; it cannot call back into the host to send channel messages.
## Configuration Fields
### `hooks.builtins.<name>`
- `enabled`
- `priority`
- `config`
### `hooks.processes.<name>`
- `enabled`
- `priority`
- `transport`
Currently only `stdio` is supported
- `command`
- `dir`
- `env`
- `observe`
- `intercept`
## Troubleshooting
If a hook looks like it is not firing, check these in order:
1. `hooks.enabled`
2. Whether the target builtin or process hook is `enabled`
3. Whether the process-hook `command` path is correct
4. Whether you are watching the correct log file
5. Whether the current request actually reached the stage you care about
6. Whether `observe` or `intercept` contains the hook point you want
A practical minimal troubleshooting pair is:
- Use the Python process-hook example from this document to validate the external protocol
- Use the Go in-process example from this document to validate the host-side chain
If the Python side shows `hook.hello` but no business hook requests, the protocol is usually fine; the current request simply did not trigger the stage you expected.
## Scope And Limits
The current hook system is best suited for:
- LLM request rewriting
- Tool argument normalization
- Pre-execution tool approval
- Auditing and observability
It is not yet well suited for:
- External hooks actively sending channel messages
- Suspending a turn and waiting for human approval replies
- Full inbound/outbound message interception across the whole platform
If you want a real human approval workflow, use hooks as the approval entry point and keep the state machine plus channel interaction in a separate `ApprovalManager`.
docs/hooks/README.zh.md
+679
View File
@@ -0,0 +1,679 @@
# Hook 系统使用说明
这份文档对应当前仓库里已经实现的 hook 系统，而不是设计草案。
当前实现支持两类挂载方式：
1. 进程内 hook
2. 进程外 process hook（`JSON-RPC over stdio`）
当前仓库不再内置示例代码文件。下面的 Go / Python 示例都直接写在本文档里；如果你要使用它们，需要先复制到你自己的文件路径。
## 支持的 hook 类型
| 类型 | 接口 | 作用阶段 | 能否改写 |
| --- | --- | --- | --- |
| 观察型 | `EventObserver` | EventBus 广播事件时 | 否 |
| LLM 拦截型 | `LLMInterceptor` | `before_llm` / `after_llm` | 是 |
| Tool 拦截型 | `ToolInterceptor` | `before_tool` / `after_tool` | 是 |
| Tool 审批型 | `ToolApprover` | `approve_tool` | 否，返回批准/拒绝 |
当前公开的同步点位只有：
- `before_llm`
- `after_llm`
- `before_tool`
- `after_tool`
- `approve_tool`
其余 lifecycle 通过事件形式只读暴露。
## 执行顺序
HookManager 的排序规则是：
1. 先执行进程内 hook
2. 再执行 process hook
3. 同一来源内按 `priority` 从小到大
4. 若 `priority` 相同，再按名字排序
## 超时
当前配置在 `hooks.defaults` 中统一设置：
- `observer_timeout_ms`
- `interceptor_timeout_ms`
- `approval_timeout_ms`
注意：当前实现还没有单个 process hook 自己的 `timeout_ms` 字段，超时配置是全局默认值。
## 快速开始
如果你的目标只是先把当前 hook 流程跑通并观察到实际请求，最省事的是先用下面的 Python process hook 示例：
1. 打开 `hooks.enabled`
2. 把下面文档里的 Python 示例保存到本地文件，例如 `/tmp/review_gate.py`
3. 给它配置 `PICOCLAW_HOOK_LOG_FILE`
4. 重启 gateway
5. 用 `tail -f` 观察日志文件
例如：
```json
{
"hooks": {
"enabled": true,
"processes": {
"py_review_gate": {
"enabled": true,
"priority": 100,
"transport": "stdio",
"command": [
"python3",
"/tmp/review_gate.py"
],
"observe": [
"tool_exec_start",
"tool_exec_end",
"tool_exec_skipped"
],
"intercept": [
"before_tool",
"approve_tool"
],
"env": {
"PICOCLAW_HOOK_LOG_FILE": "/tmp/picoclaw-hook-review-gate.log"
}
}
}
}
}
```
观察方式：
```bash
tail -f /tmp/picoclaw-hook-review-gate.log
```
如果你是在开发 PicoClaw 本体，而不是只想验证协议，那么再看后面的 Go in-process 示例。
## 两个示例的定位
- Go in-process 示例
适合验证宿主内的 hook 链路、理解 `MountHook()` 和各个同步点位
- Python process 示例
适合理解 `JSON-RPC over stdio` 协议、确认宿主和外部进程之间的消息来回是否正常
这两个示例都刻意保持为“只记录、不改写、不拒绝”的安全模式。它们的目的不是提供策略能力，而是帮你观察当前 hook 系统。
## Go 进程内示例
下面这段代码是一个最小的“记录型” in-process hook。它实现了：
1. `EventObserver`
2. `LLMInterceptor`
3. `ToolInterceptor`
4. `ToolApprover`
它只记录，不改写请求，也不拒绝工具。
你可以把它保存成你自己的 Go 文件，例如 `pkg/myhooks/example_logger.go`：
```go
package myhooks
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/sipeed/picoclaw/pkg/agent"
"github.com/sipeed/picoclaw/pkg/logger"
)
type ExampleLoggerHookOptions struct {
LogFile string `json:"log_file,omitempty"`
LogEvents bool `json:"log_events,omitempty"`
}
type ExampleLoggerHook struct {
logFile string
logEvents bool
mu sync.Mutex
}
func NewExampleLoggerHook(opts ExampleLoggerHookOptions) *ExampleLoggerHook {
return &ExampleLoggerHook{
logFile: strings.TrimSpace(opts.LogFile),
logEvents: opts.LogEvents,
}
}
func (h *ExampleLoggerHook) OnEvent(ctx context.Context, evt agent.Event) error {
_ = ctx
if h == nil || !h.logEvents {
return nil
}
h.record("event", evt.Meta, map[string]any{
"event": evt.Kind.String(),
"payload": evt.Payload,
}, nil)
return nil
}
func (h *ExampleLoggerHook) BeforeLLM(
ctx context.Context,
req *agent.LLMHookRequest,
) (*agent.LLMHookRequest, agent.HookDecision, error) {
_ = ctx
h.record("before_llm", req.Meta, req, agent.HookDecision{Action: agent.HookActionContinue})
return req, agent.HookDecision{Action: agent.HookActionContinue}, nil
}
func (h *ExampleLoggerHook) AfterLLM(
ctx context.Context,
resp *agent.LLMHookResponse,
) (*agent.LLMHookResponse, agent.HookDecision, error) {
_ = ctx
h.record("after_llm", resp.Meta, resp, agent.HookDecision{Action: agent.HookActionContinue})
return resp, agent.HookDecision{Action: agent.HookActionContinue}, nil
}
func (h *ExampleLoggerHook) BeforeTool(
ctx context.Context,
call *agent.ToolCallHookRequest,
) (*agent.ToolCallHookRequest, agent.HookDecision, error) {
_ = ctx
h.record("before_tool", call.Meta, call, agent.HookDecision{Action: agent.HookActionContinue})
return call, agent.HookDecision{Action: agent.HookActionContinue}, nil
}
func (h *ExampleLoggerHook) AfterTool(
ctx context.Context,
result *agent.ToolResultHookResponse,
) (*agent.ToolResultHookResponse, agent.HookDecision, error) {
_ = ctx
h.record("after_tool", result.Meta, result, agent.HookDecision{Action: agent.HookActionContinue})
return result, agent.HookDecision{Action: agent.HookActionContinue}, nil
}
func (h *ExampleLoggerHook) ApproveTool(
ctx context.Context,
req *agent.ToolApprovalRequest,
) (agent.ApprovalDecision, error) {
_ = ctx
decision := agent.ApprovalDecision{Approved: true}
h.record("approve_tool", req.Meta, req, decision)
return decision, nil
}
func (h *ExampleLoggerHook) record(stage string, meta agent.EventMeta, payload any, decision any) {
logger.InfoCF("hooks", "Example hook observed", map[string]any{
"stage": stage,
})
if h == nil || h.logFile == "" {
return
}
entry := map[string]any{
"ts": time.Now().UTC(),
"stage": stage,
"meta": meta,
"payload": payload,
"decision": decision,
}
body, err := json.Marshal(entry)
if err != nil {
logger.WarnCF("hooks", "Example hook log encode failed", map[string]any{
"stage": stage,
"error": err.Error(),
})
return
}
h.mu.Lock()
defer h.mu.Unlock()
if dir := filepath.Dir(h.logFile); dir != "" && dir != "." {
if err := os.MkdirAll(dir, 0o755); err != nil {
logger.WarnCF("hooks", "Example hook log mkdir failed", map[string]any{
"stage": stage,
"path": h.logFile,
"error": err.Error(),
})
return
}
}
file, err := os.OpenFile(h.logFile, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644)
if err != nil {
logger.WarnCF("hooks", "Example hook log open failed", map[string]any{
"stage": stage,
"path": h.logFile,
"error": err.Error(),
})
return
}
defer func() { _ = file.Close() }()
if _, err := file.Write(append(body, '\n')); err != nil {
logger.WarnCF("hooks", "Example hook log write failed", map[string]any{
"stage": stage,
"path": h.logFile,
"error": err.Error(),
})
}
}
```
### 如何挂载
如果你只需要代码挂载，直接在 `AgentLoop` 初始化后调用：
```go
hook := myhooks.NewExampleLoggerHook(myhooks.ExampleLoggerHookOptions{
LogFile: "/tmp/picoclaw-hook-example-logger.log",
LogEvents: true,
})
if err := al.MountHook(agent.NamedHook("example-logger", hook)); err != nil {
panic(err)
}
```
### 如果你还想用配置挂载
当前 hook 系统支持 builtin hook，但这要求你自己把 factory 编进二进制。也就是说，下面这段注册代码需要和上面的 hook 定义一起放进你的工程里：
```go
package myhooks
import (
"context"
"encoding/json"
"fmt"
"github.com/sipeed/picoclaw/pkg/agent"
"github.com/sipeed/picoclaw/pkg/config"
)
func init() {
if err := agent.RegisterBuiltinHook("example_logger", func(
ctx context.Context,
spec config.BuiltinHookConfig,
) (any, error) {
_ = ctx
var opts ExampleLoggerHookOptions
if len(spec.Config) > 0 {
if err := json.Unmarshal(spec.Config, &opts); err != nil {
return nil, fmt.Errorf("decode example_logger config: %w", err)
}
}
return NewExampleLoggerHook(opts), nil
}); err != nil {
panic(err)
}
}
```
只有在你自己注册了 builtin 之后，下面的配置才会生效：
```json
{
"hooks": {
"enabled": true,
"builtins": {
"example_logger": {
"enabled": true,
"priority": 10,
"config": {
"log_file": "/tmp/picoclaw-hook-example-logger.log",
"log_events": true
}
}
}
}
}
```
### 如何观察它是否生效
- 如果设置了 `log_file`，它会把每次 hook 调用按 JSON Lines 写入文件
- 如果没有设置 `log_file`，它仍然会把摘要写到 gateway 日志
- 普通只走 LLM 的请求，通常会看到 `before_llm` 和 `after_llm`
- 触发工具调用的请求，通常还会看到 `before_tool`、`approve_tool`、`after_tool`
- 如果 `log_events=true`，还会额外看到 `event`
典型日志：
```json
{"ts":"2026-03-21T14:10:00Z","stage":"before_tool","meta":{"session_key":"session-1"},"payload":{"tool":"echo_text","arguments":{"text":"hello"}},"decision":{"action":"continue"}}
{"ts":"2026-03-21T14:10:00Z","stage":"approve_tool","meta":{"session_key":"session-1"},"payload":{"tool":"echo_text","arguments":{"text":"hello"}},"decision":{"approved":true}}
```
如果你只看到了 `before_llm` / `after_llm`，没有看到 tool 相关阶段，通常不是 hook 没挂上，而是这次请求本身没有触发工具调用。
## Python process hook 示例
下面这段脚本是一个最小的 `process hook` 示例。它只使用 Python 标准库，支持：
1. `hook.hello`
2. `hook.event`
3. `hook.before_tool`
4. `hook.approve_tool`
它默认只记录，不改写，也不拒绝。
你可以把它保存到任意本地路径，例如 `/tmp/review_gate.py`：
```python
#!/usr/bin/env python3
from __future__ import annotations
import json
import os
import signal
import sys
from datetime import datetime, timezone
from typing import Any
LOG_EVENTS = os.getenv("PICOCLAW_HOOK_LOG_EVENTS", "1").lower() not in {"0", "false", "no"}
LOG_FILE = os.getenv("PICOCLAW_HOOK_LOG_FILE", "").strip()
def append_log(entry: dict[str, Any]) -> None:
if not LOG_FILE:
return
payload = {
"ts": datetime.now(timezone.utc).isoformat(),
**entry,
}
try:
log_dir = os.path.dirname(LOG_FILE)
if log_dir:
os.makedirs(log_dir, exist_ok=True)
with open(LOG_FILE, "a", encoding="utf-8") as handle:
handle.write(json.dumps(payload, ensure_ascii=True) + "\n")
except OSError as exc:
log_stderr(f"failed to write hook log file {LOG_FILE}: {exc}")
def send_response(message_id: int, result: Any | None = None, error: str | None = None) -> None:
payload: dict[str, Any] = {
"jsonrpc": "2.0",
"id": message_id,
}
if error is not None:
payload["error"] = {"code": -32000, "message": error}
else:
payload["result"] = result if result is not None else {}
append_log({
"direction": "out",
"id": message_id,
"response": payload.get("result"),
"error": payload.get("error"),
})
try:
sys.stdout.write(json.dumps(payload, ensure_ascii=True) + "\n")
sys.stdout.flush()
except BrokenPipeError:
raise SystemExit(0) from None
def log_stderr(message: str) -> None:
try:
sys.stderr.write(message + "\n")
sys.stderr.flush()
except BrokenPipeError:
raise SystemExit(0) from None
def handle_shutdown_signal(signum: int, _frame: Any) -> None:
raise KeyboardInterrupt(f"received signal {signum}")
def handle_before_tool(params: dict[str, Any]) -> dict[str, Any]:
_ = params
return {"action": "continue"}
def handle_approve_tool(params: dict[str, Any]) -> dict[str, Any]:
_ = params
return {"approved": True}
def handle_request(method: str, params: dict[str, Any]) -> dict[str, Any]:
if method == "hook.hello":
return {"ok": True, "name": "python-review-gate"}
if method == "hook.before_tool":
return handle_before_tool(params)
if method == "hook.approve_tool":
return handle_approve_tool(params)
if method == "hook.before_llm":
return {"action": "continue"}
if method == "hook.after_llm":
return {"action": "continue"}
if method == "hook.after_tool":
return {"action": "continue"}
raise KeyError(f"method not found: {method}")
def main() -> int:
try:
for raw_line in sys.stdin:
line = raw_line.strip()
if not line:
continue
try:
message = json.loads(line)
except json.JSONDecodeError as exc:
log_stderr(f"failed to decode request: {exc}")
append_log({
"direction": "in",
"decode_error": str(exc),
"raw": line,
})
continue
method = message.get("method")
message_id = message.get("id", 0)
params = message.get("params") or {}
if not isinstance(params, dict):
params = {}
append_log({
"direction": "in",
"id": message_id,
"method": method,
"params": params,
"notification": not bool(message_id),
})
if not message_id:
if method == "hook.event" and LOG_EVENTS:
log_stderr(f"observed event: {params.get('Kind')}")
continue
try:
result = handle_request(str(method or ""), params)
except KeyError as exc:
send_response(int(message_id), error=str(exc))
continue
except Exception as exc:
send_response(int(message_id), error=f"unexpected error: {exc}")
continue
send_response(int(message_id), result=result)
except KeyboardInterrupt:
return 0
return 0
if __name__ == "__main__":
signal.signal(signal.SIGINT, handle_shutdown_signal)
signal.signal(signal.SIGTERM, handle_shutdown_signal)
raise SystemExit(main())
```
### 如何配置
```json
{
"hooks": {
"enabled": true,
"processes": {
"py_review_gate": {
"enabled": true,
"priority": 100,
"transport": "stdio",
"command": [
"python3",
"/abs/path/to/review_gate.py"
],
"observe": [
"tool_exec_start",
"tool_exec_end",
"tool_exec_skipped"
],
"intercept": [
"before_tool",
"approve_tool"
],
"env": {
"PICOCLAW_HOOK_LOG_FILE": "/tmp/picoclaw-hook-review-gate.log"
}
}
}
}
}
```
### 环境变量
- `PICOCLAW_HOOK_LOG_EVENTS`
是否把 `hook.event` 写到 `stderr`，默认开启
- `PICOCLAW_HOOK_LOG_FILE`
外部日志文件路径。设置后，脚本会把收到的 hook 请求、notification 和返回结果按 JSON Lines 追加到该文件
注意：`PICOCLAW_HOOK_LOG_FILE` 没有默认值。不设置时，脚本不会自动落盘日志。
### 如何确认它收到了 hook
推荐同时看两个地方：
- gateway 日志
用来观察宿主是否成功启动了外部进程，以及脚本写到 `stderr` 的事件摘要
- `PICOCLAW_HOOK_LOG_FILE`
用来观察脚本实际收到了什么请求、返回了什么响应
典型判断方式：
- 只看到 `hook.hello`
说明进程启动并完成握手了，但还没有新的业务 hook 请求真正打进来
- 看到 `hook.event`
说明 `observe` 配置生效了
- 看到 `hook.before_tool`
说明 `intercept: ["before_tool", ...]` 生效了
- 看到 `hook.approve_tool`
说明审批 hook 生效了
这份示例脚本不会改写任何参数，也不会拒绝工具，所以你应该看到的典型返回是：
```json
{"direction":"out","id":7,"response":{"action":"continue"},"error":null}
{"direction":"out","id":8,"response":{"approved":true},"error":null}
```
一组完整样例：
```json
{"ts":"2026-03-21T14:12:00+00:00","direction":"in","id":1,"method":"hook.hello","params":{"name":"py_review_gate","version":1,"modes":["observe","tool","approve"]},"notification":false}
{"ts":"2026-03-21T14:12:00+00:00","direction":"out","id":1,"response":{"ok":true,"name":"python-review-gate"},"error":null}
{"ts":"2026-03-21T14:12:05+00:00","direction":"in","id":0,"method":"hook.event","params":{"Kind":"tool_exec_start"},"notification":true}
{"ts":"2026-03-21T14:12:05+00:00","direction":"in","id":7,"method":"hook.before_tool","params":{"tool":"echo_text","arguments":{"text":"hello"}},"notification":false}
{"ts":"2026-03-21T14:12:05+00:00","direction":"out","id":7,"response":{"action":"continue"},"error":null}
```
补充说明：
- 时间戳是 UTC，不是本地时区
- `notification=true` 表示这是 `hook.event` 这类不需要响应的通知
- `id` 会随着当前进程内的请求递增；如果 hook 进程重启，计数会重新开始
## Process Hook 协议约定
当前 process hook 使用 `JSON-RPC over stdio`：
- PicoClaw 启动外部进程
- 请求和响应都按“一行一个 JSON 消息”传输
- `hook.event` 是 notification，不需要响应
- `hook.before_llm` / `hook.after_llm` / `hook.before_tool` / `hook.after_tool` / `hook.approve_tool` 是 request/response
当前宿主不会接受 process hook 主动发起的新 RPC。也就是说，外部 hook 现在只能“响应 PicoClaw 的调用”，不能反向调用宿主去发送 channel 消息。
## 配置字段
### `hooks.builtins.<name>`
- `enabled`
- `priority`
- `config`
### `hooks.processes.<name>`
- `enabled`
- `priority`
- `transport`
当前只支持 `stdio`
- `command`
- `dir`
- `env`
- `observe`
- `intercept`
## 排查建议
当你觉得“hook 没触发”时，优先按这个顺序排查：
1. `hooks.enabled` 是否为 `true`
2. 对应的 builtin/process hook 是否 `enabled`
3. process hook 的 `command` 路径是否正确
4. 你看的是否是正确的日志文件
5. 当前请求是否真的走到了对应阶段
6. `observe` / `intercept` 是否包含了你想看的点位
一个很实用的最小排查组合是：
- 先用文档里的 Python process 示例确认外部协议没问题
- 再用文档里的 Go in-process 示例确认宿主内的 hook 链路没问题
如果前者有 `hook.hello` 但没有业务请求，通常不是协议挂了，而是当前这次请求没有真正触发对应的 hook 点位。
## 适用边界
当前 hook 系统最适合做这些事：
- LLM 请求改写
- 工具参数规范化
- 工具执行前审批
- 审计和观测
当前还不适合直接承载这些需求：
- 外部 hook 主动发 channel 消息
- 挂起 turn 并等待人工审批回复
- inbound/outbound 全链路消息拦截
如果你要做人审流转，推荐把 hook 作为审批入口，把审批状态机和 channel 交互放到独立的 `ApprovalManager`。
docs/steering.md
+199
View File
@@ -0,0 +1,199 @@
# Steering
Steering allows injecting messages into an already-running agent loop, interrupting it between tool calls without waiting for the entire cycle to complete.
## How it works
When the agent is executing a sequence of tool calls (e.g. the model requested 3 tools in a single turn), steering checks the queue **after each tool** completes. If it finds queued messages:
1. The remaining tools are **skipped** and receive `"Skipped due to queued user message."` as their result
2. The steering messages are **injected into the conversation context**
3. The model is called again with the updated context, including the user's steering message
```
User ──► Steer("change approach")
│
Agent Loop ▼
├─ tool[0] ✔ (executed)
├─ [polling] → steering found!
├─ tool[1] ✘ (skipped)
├─ tool[2] ✘ (skipped)
└─ new LLM turn with steering message
```
## Scoped queues
Steering is now isolated per resolved session scope, not stored in a single
global queue.
- The active turn writes and reads from its own scope key (usually the routed session key such as `agent:<agent_id>:...`)
- `Steer()` still works outside an active turn through a legacy fallback queue
- `Continue()` first dequeues messages for the requested session scope, then falls back to the legacy queue for backwards compatibility
This prevents a message arriving from another chat, DM peer, or routed agent
session from being injected into the wrong conversation.
## Configuration
In `config.json`, under `agents.defaults`:
```json
{
"agents": {
"defaults": {
"steering_mode": "one-at-a-time"
}
}
}
```
### Modes
| Value | Behavior |
|-------|----------|
| `"one-at-a-time"` | **(default)** Dequeues only one message per polling cycle. If there are 3 messages in the queue, they are processed one at a time across 3 successive iterations. |
| `"all"` | Drains the entire queue in a single poll. All pending messages are injected into the context together. |
The environment variable `PICOCLAW_AGENTS_DEFAULTS_STEERING_MODE` can be used as an alternative.
## Go API
### Steer — Send a steering message
```go
err := agentLoop.Steer(providers.Message{
Role: "user",
Content: "change direction, focus on X instead",
})
if err != nil {
// Queue is full (MaxQueueSize=10) or not initialized
}
```
The message is enqueued in a thread-safe manner. Returns an error if the queue is full or not initialized. It will be picked up at the next polling point (after the current tool finishes).
### SteeringMode / SetSteeringMode
```go
// Read the current mode
mode := agentLoop.SteeringMode() // SteeringOneAtATime | SteeringAll
// Change it at runtime
agentLoop.SetSteeringMode(agent.SteeringAll)
```
### Continue — Resume an idle agent
When the agent is idle (it has finished processing and its last message was from the assistant), `Continue` checks if there are steering messages in the queue and uses them to start a new cycle:
```go
response, err := agentLoop.Continue(ctx, sessionKey, channel, chatID)
if err != nil {
// Error (e.g. "no default agent available")
}
if response == "" {
// No steering messages in queue, the agent stays idle
}
```
`Continue` internally uses `SkipInitialSteeringPoll: true` to avoid double-dequeuing the same messages (since it already extracted them and passes them directly as input).
`Continue` also resolves the target agent from the provided session key, so
agent-scoped sessions continue on the correct agent instead of always using
the default one.
## Polling points in the loop
Steering is checked at the following points in the agent cycle:
1. **At loop start** — before the first LLM call, to catch messages enqueued during setup
2. **After every tool completes** — including the first and the last. If steering is found and there are remaining tools, they are all skipped immediately
3. **After a direct LLM response** — if a new steering message arrived while the model was generating a non-tool response, the loop continues instead of returning a stale answer
4. **Right before the turn is finalized** — if steering arrived at the very end of the turn, the agent immediately starts a continuation turn instead of leaving the message orphaned in the queue
## Why remaining tools are skipped
When a steering message is detected, all remaining tools in the batch are skipped rather than executed. The alternative — let all tools finish and inject the steering message afterwards — was considered and rejected. Here is why.
### Preventing unwanted side effects
Tools can have **irreversible side effects**. If the user says "no, wait" while the agent is mid-batch, executing the remaining tools means those side effects happen anyway:
| Tool batch | Steering message | With skip | Without skip |
|---|---|---|---|
| `[web_search, send_email]` | "don't send it" | Email **not** sent | Email sent, damage done |
| `[query_db, write_file, spawn_agent]` | "use another database" | Only the query runs | File written + subagent spawned, all wasted |
| `[search₁, search₂, search₃, write_file]` | user changes topic entirely | 1 search | 3 searches + file write, all irrelevant |
### Avoiding wasted time
Tools that take seconds (web fetches, API calls, database queries) would all run to completion before the agent sees the user's correction. In a batch of 3 tools each taking 3-4 seconds, that's 10+ seconds of work that will be discarded.
With skipping, the agent reacts as soon as the current tool finishes — typically within a few seconds instead of waiting for the entire batch.
### The LLM gets full context
Skipped tools receive an explicit error result (`"Skipped due to queued user message."`), so the model knows exactly which actions were not performed. It can then decide whether to re-execute them with the new context, or take a different path entirely.
### Trade-off: sequential execution
Skipping requires tools to run **sequentially** (the previous implementation ran them in parallel). This introduces latency when the LLM requests multiple independent tools in a single turn. In practice, most batches contain 1-2 tools, so the impact is minimal compared to the benefit of being able to stop unwanted actions.
## Skipped tool result format
When steering interrupts a batch, each tool that was not executed receives a `tool` result with:
```
Content: "Skipped due to queued user message."
```
This is saved to the session via `AddFullMessage` and sent to the model, so it is aware that some requested actions were not performed.
## Full flow example
```
1. User: "search for info on X, write a file, and send me a message"
2. LLM responds with 3 tool calls: [web_search, write_file, message]
3. web_search is executed → result saved
4. [polling] → User called Steer("no, search for Y instead")
5. write_file is skipped → "Skipped due to queued user message."
message is skipped → "Skipped due to queued user message."
6. Message "search for Y instead" injected into context
7. LLM receives the full updated context and responds accordingly
```
## Automatic bus drain
When the agent loop (`Run()`) starts processing a message, it spawns a background goroutine that keeps consuming new inbound messages from the bus. These messages are automatically redirected into the steering queue via `Steer()`. This means:
- Users on any channel (Telegram, Discord, etc.) don't need to do anything special — their messages are automatically captured as steering when the agent is busy
- Audio messages are transcribed before being steered, so the agent receives text. If transcription fails, the original (non-transcribed) message is steered as-is
- Only messages that resolve to the **same steering scope** as the active turn are redirected. Messages for other chats/sessions are requeued onto the inbound bus so they can be processed normally
- `system` inbound messages are not treated as steering input
- When `processMessage` finishes, the drain goroutine is canceled and normal message consumption resumes
## Steering with media
Steering messages can include `Media` refs, just like normal inbound user
messages.
- The original `media://` refs are preserved in session history via `AddFullMessage`
- Before the next provider call, steering messages go through the normal media resolution pipeline
- Image refs are converted to data URLs for multimodal providers; non-image refs are resolved the same way as standard inbound media
This applies both to in-turn steering and to idle-session continuation through
`Continue()`.
## Notes
- Steering **does not interrupt** a tool that is currently executing. It waits for the current tool to finish, then checks the queue.
- With `one-at-a-time` mode, if multiple messages are enqueued rapidly, they will be processed one per iteration. This gives the model the opportunity to react to each message individually.
- With `all` mode, all pending messages are combined into a single injection. Useful when you want the agent to receive all the context at once.
- The steering queue has a maximum capacity of 10 messages (`MaxQueueSize`). `Steer()` returns an error when the queue is full. In the bus drain path, the error is logged as a warning and the message is effectively dropped.
- Manual `Steer()` calls made outside an active turn still go to the legacy fallback queue, so older integrations keep working.
docs/subturn.md
+279
View File
@@ -0,0 +1,279 @@
# 🔄 SubTurn Mechanism
> Back to [README](../README.md)
## Overview
The `SubTurn` mechanism is a core feature in PicoClaw that allows tools to spawn isolated, nested agent loops to handle complex sub-tasks.
By using a SubTurn, an agent can break down a problem and run a separate LLM invocation in an independent, ephemeral session. This ensures that intermediate reasoning, background tasks, or sub-agent outputs do not pollute the main conversation history.
## Core Capabilities
- **Context Isolation**: Each SubTurn uses an `ephemeralSessionStore`. Its message history does not leak into the parent task and is destroyed upon completion. The ephemeral session holds at most **50 messages**; older messages are automatically truncated when this limit is reached.
- **Depth & Concurrency Limits**: Prevents infinite loops and resource exhaustion.
- **Maximum Depth**: Up to 3 nested levels.
- **Maximum Concurrency**: Up to 5 concurrent sub-turns per parent turn (managed via a semaphore with a 30-second timeout).
- **Context Protection**: Supports soft context limits (`MaxContextRunes`). It proactively truncates old messages (while preserving system prompts and recent context) before hitting the provider's hard context window limit.
- **Error Recovery**: Automatically detects and recovers from provider context length exceeded errors and truncation errors by compressing history and retrying.
## Configuration (`SubTurnConfig`)
When spawning a SubTurn, you must provide a `SubTurnConfig`:
| Field | Type | Description |
| :--- | :--- | :--- |
| `Model` | `string` | The LLM model to use for the sub-turn (e.g., `gpt-4o-mini`). **Required.** |
| `Tools` | `[]tools.Tool` | Tools granted to the sub-turn. If empty, it inherits the parent's tools. |
| `SystemPrompt` | `string` | The task description for the sub-turn. Sent as the first user message to the LLM (not as a system prompt override). |
| `ActualSystemPrompt` | `string` | Optional explicit system prompt to replace the agent's default. Leave empty to inherit the parent agent's system prompt. |
| `MaxTokens` | `int` | Maximum tokens for the generated response. |
| `Async` | `bool` | Controls the result delivery mode (Synchronous vs. Asynchronous). |
| `Critical` | `bool` | If `true`, the sub-turn continues running even if the parent finishes gracefully. |
| `Timeout` | `time.Duration` | Maximum execution time (default: 5 minutes). |
| `MaxContextRunes`| `int` | Soft context limit. `0` = auto-calculate (75% of model's context window, recommended), `-1` = no limit (disable soft truncation, rely only on hard context error recovery), `>0` = use specified rune limit. |
> **Note:** The `Async` flag does **not** make the call non-blocking. It only controls whether the result is also delivered to the parent's `pendingResults` channel. Both modes block the caller until the sub-turn completes. For true non-blocking execution, the caller must spawn the sub-turn in a separate goroutine.
## Execution Modes
### Synchronous (`Async: false`)
This is the standard mode where the caller needs the result immediately to proceed.
- The caller blocks until the sub-turn completes.
- The result is **only** returned directly via the function return value.
- It is **not** delivered to the parent's pending results channel.
**Example:**
```go
cfg := agent.SubTurnConfig{
Model: "gpt-4o-mini",
SystemPrompt: "Analyze the provided codebase...",
Async: false,
}
result, err := agent.SpawnSubTurn(ctx, cfg)
// Process result immediately
```
### Asynchronous (`Async: true`)
Used for "fire-and-forget" operations or parallel processing where the parent turn collects results later.
- The result is delivered to the parent turn's `pendingResults` channel.
- The result is **also** returned via the function return value (for consistency).
- The parent's Agent Loop will poll this channel in subsequent iterations and automatically inject the results into the ongoing conversation context as `[SubTurn Result]`.
**Example:**
```go
cfg := agent.SubTurnConfig{
Model: "gpt-4o-mini",
SystemPrompt: "Run a background security scan...",
Async: true,
}
result, err := agent.SpawnSubTurn(ctx, cfg)
// The result will also be injected into the parent loop later via channel
```
## Error Recovery and Retries
SubTurns implement automatic retry mechanisms for transient errors:
| Error Type | Max Retries | Recovery Action |
|:-----------|:------------|:----------------|
| Context Length Exceeded | 2 | Force compress history and retry |
| Response Truncated (`finish_reason="truncated"`) | 2 | Inject recovery prompt and retry |
### Truncation Recovery
When the LLM response is truncated (`finish_reason="truncated"`), SubTurn automatically:
1. Detects the truncation from `turnState.lastFinishReason`
2. Injects a recovery prompt: "Your previous response was truncated due to length. Please provide a shorter, complete response..."
3. Retries up to 2 times
### Context Error Recovery
When the provider returns a context length error (e.g., `context_length_exceeded`):
1. Force compresses the message history (drops oldest 50% of conversation)
2. Retries with the compressed context
3. Up to 2 retries before failing
## Lifecycle and Cancellation
SubTurns operate within an independent context but maintain a structural link to their parent `turnState`.
### Graceful Parent Finish
When the parent task finishes naturally (`Finish(false)`):
- **Non-critical** sub-turns receive a signal to exit gracefully without throwing an error.
- **Critical** (`Critical: true`) sub-turns continue running in the background. Once finished, their results are emitted as **Orphan Results** so the data is not lost.
### Hard Abort
When the parent task is forcefully aborted (e.g., user interrupts with `/stop`):
- A cascading cancellation is triggered, instantly terminating all child and grandchild sub-turns.
- The root turn's session history rolls back to the snapshot taken at turn start (`initialHistoryLength`), preventing dirty context. SubTurns are not affected by this rollback as they use ephemeral sessions that are discarded anyway.
## Agent Loop Integration
### Bus Draining During Processing
When a message enters the `Run()` loop, the agent starts a `drainBusToSteering` goroutine before calling `processMessage`. This goroutine runs concurrently with the entire processing lifecycle and continuously consumes any new inbound messages from the bus, redirecting them into the **steering queue** instead of dropping them.
This ensures that if a user sends a follow-up message while the agent is processing (including during SubTurn execution), the message is not lost — it will be picked up between tool call iterations via `dequeueSteeringMessages`.
The drain goroutine stops automatically when `processMessage` returns (via a cancellable context).
### Pending Result Polling
The agent loop polls for async SubTurn results at two points per iteration:
1. **Before the LLM call**: injects any arrived results as `[SubTurn Result]` messages into the conversation context.
2. **After all tool executions**: polls again during the tool loop to catch results that arrived during tool execution.
3. **After the final iteration**: one last poll before the turn ends to avoid losing late-arriving results.
### Turn State Tracking
All active root turns are registered in `AgentLoop.activeTurnStates` (`sync.Map`, keyed by session key). This allows `HardAbort` and `/subagents` observability commands to find and operate on active turns.
## Event Bus Integration
SubTurns emit specific events to the PicoClaw `EventBus` for observability and debugging:
| Event Kind | When Emitted | Payload |
|:------|:-------------|:--------|
| `subturn_spawn` | Sub-turn successfully initialized | `SubTurnSpawnPayload{AgentID, Label, ParentTurnID}` |
| `subturn_end` | Sub-turn finishes (success or error) | `SubTurnEndPayload{AgentID, Status}` |
| `subturn_result_delivered` | Async result successfully delivered to parent | `SubTurnResultDeliveredPayload{TargetChannel, TargetChatID, ContentLen}` |
| `subturn_orphan` | Result cannot be delivered (parent finished or channel full) | `SubTurnOrphanPayload{ParentTurnID, ChildTurnID, Reason}` |
## API Reference
### SpawnSubTurn (Public Entry Point)
```go
func SpawnSubTurn(ctx context.Context, cfg SubTurnConfig) (*tools.ToolResult, error)
```
This is the exported package-level entry point for agent-internal code (e.g., tests, direct invocations). It retrieves `AgentLoop` and `turnState` from context and delegates to the internal `spawnSubTurn`.
**Requirements:**
- `AgentLoop` must be injected into context via `WithAgentLoop()`
- Parent `turnState` must exist in context (automatically set when called from tools)
**Returns:**
- `*tools.ToolResult`: Contains `ForLLM` field with the sub-turn's output
- `error`: One of the defined error types or context errors
### AgentLoopSpawner (Interface Implementation)
```go
type AgentLoopSpawner struct { al *AgentLoop }
func (s *AgentLoopSpawner) SpawnSubTurn(ctx context.Context, cfg tools.SubTurnConfig) (*tools.ToolResult, error)
```
This implements the `tools.SubTurnSpawner` interface for use by tools that need to spawn sub-turns without a direct import of the `agent` package (avoiding circular dependencies). It converts `tools.SubTurnConfig` → `agent.SubTurnConfig` before delegating to the internal `spawnSubTurn`.
### NewSubTurnSpawner
```go
func NewSubTurnSpawner(al *AgentLoop) *AgentLoopSpawner
```
Creates a new spawner instance for the given AgentLoop. Pass the returned value to `SpawnTool.SetSpawner()` or `SubagentTool.SetSpawner()` during tool registration.
### Continue
```go
func (al *AgentLoop) Continue(ctx context.Context, sessionKey string) error
```
Resumes an idle agent turn by injecting any queued steering messages as a new LLM iteration. Used when the agent is waiting and a deferred steering message needs to be processed without a new inbound message arriving.
## Context Propagation
SubTurn relies on context values for proper operation:
| Context Key | Purpose |
|:------------|:--------|
| `agentLoopKey` | Stores `*AgentLoop` for tool access and SubTurn spawning |
| `turnStateKey` | Stores `*turnState` for hierarchy tracking and result delivery |
### Injecting Dependencies
```go
// Before calling tools that may spawn SubTurns
ctx = WithAgentLoop(ctx, agentLoop)
ctx = withTurnState(ctx, turnState)
```
### Independent Child Context
**Important**: The child SubTurn uses an **independent context** derived from `context.Background()`, not from the parent context. This design choice:
- Allows critical SubTurns to continue after parent cancellation
- Prevents parent timeout from affecting child execution
- Child has its own timeout for self-protection (`Timeout` config or 5 minutes default)
## Error Types
| Error | Condition |
|:------|:----------|
| `ErrDepthLimitExceeded` | SubTurn depth exceeds 3 levels |
| `ErrInvalidSubTurnConfig` | Required field `Model` is empty |
| `ErrConcurrencyTimeout` | All 5 concurrency slots occupied for 30+ seconds |
| Context errors | Parent context cancelled during semaphore acquisition |
## Thread Safety
SubTurns are designed for concurrent execution:
- **Parent-child relationships**: Managed under mutex (`parentTS.mu.Lock()`)
- **Active turn tracking**: Uses `sync.Map` for concurrent access to `activeTurnStates`
- **ID generation**: Uses `atomic.Int64` for unique SubTurn IDs (format: `subturn-N`, globally monotonic per `AgentLoop` instance)
- **Result delivery**: Reads parent state under lock, releases before channel send (small race window acceptable)
## Orphan Results
An orphan result occurs when:
1. Parent turn finishes before the SubTurn completes
2. The `pendingResults` channel is full (buffer size: 16)
When a result becomes orphan:
- `SubTurnOrphanResultEvent` is emitted to EventBus
- The result is **NOT** delivered to the LLM context
- External systems can listen to this event for custom handling
### Preventing Orphan Results
- Use `Critical: true` for important SubTurns that must complete
- Monitor `SubTurnOrphanResultEvent` for observability
- Consider the 16-buffer limit when spawning many async SubTurns
## Tool Inheritance
### When `cfg.Tools` is empty:
- SubTurn inherits **all** tools from the parent agent
- Tools are registered in a new `ToolRegistry` instance
- Tool TTL is managed independently from parent
### When `cfg.Tools` is specified:
- Only the specified tools are available to the SubTurn
- Parent tools are **NOT** merged
- Use this to restrict SubTurn capabilities for security or focus
**Example - Restricted SubTurn:**
```go
cfg := agent.SubTurnConfig{
Model: "gpt-4o-mini",
Tools: []tools.Tool{readOnlyTool}, // Only read-only access
SystemPrompt: "Analyze the file structure...",
}
```
## Reference
| Constant | Value |
|:---------|:------|
| `maxSubTurnDepth` | 3 |
| `maxConcurrentSubTurns` | 5 |
| `concurrencyTimeout` | 30s |
| `defaultSubTurnTimeout` | 5m |
| `maxEphemeralHistorySize` | 50 messages |
| `pendingResults` buffer | 16 |
| `MaxContextRunes` default | 75% of model context window |
pkg/agent/context.go
+27 -16
View File
@@ -222,13 +222,10 @@ func (cb *ContextBuilder) InvalidateCache() {
// invalidation (bootstrap files + memory). Skill roots are handled separately
// because they require both directory-level and recursive file-level checks.
func (cb *ContextBuilder) sourcePaths() []string {
return []string{
filepath.Join(cb.workspace, "AGENTS.md"),
filepath.Join(cb.workspace, "SOUL.md"),
filepath.Join(cb.workspace, "USER.md"),
filepath.Join(cb.workspace, "IDENTITY.md"),
filepath.Join(cb.workspace, "memory", "MEMORY.md"),
}
agentDefinition := cb.LoadAgentDefinition()
paths := agentDefinition.trackedPaths(cb.workspace)
paths = append(paths, filepath.Join(cb.workspace, "memory", "MEMORY.md"))
return uniquePaths(paths)
}
// skillRoots returns all skill root directories that can affect
@@ -432,18 +429,32 @@ func skillFilesChangedSince(skillRoots []string, filesAtCache map[string]time.Ti
}
func (cb *ContextBuilder) LoadBootstrapFiles() string {
bootstrapFiles := []string{
"AGENTS.md",
"SOUL.md",
"USER.md",
"IDENTITY.md",
var sb strings.Builder
agentDefinition := cb.LoadAgentDefinition()
if agentDefinition.Agent != nil {
label := string(agentDefinition.Source)
if label == "" {
label = relativeWorkspacePath(cb.workspace, agentDefinition.Agent.Path)
}
fmt.Fprintf(&sb, "## %s\n\n%s\n\n", label, agentDefinition.Agent.Body)
}
if agentDefinition.Soul != nil {
fmt.Fprintf(
&sb,
"## %s\n\n%s\n\n",
relativeWorkspacePath(cb.workspace, agentDefinition.Soul.Path),
agentDefinition.Soul.Content,
)
}
if agentDefinition.User != nil {
fmt.Fprintf(&sb, "## %s\n\n%s\n\n", "USER.md", agentDefinition.User.Content)
}
var sb strings.Builder
for _, filename := range bootstrapFiles {
filePath := filepath.Join(cb.workspace, filename)
if agentDefinition.Source != AgentDefinitionSourceAgent {
filePath := filepath.Join(cb.workspace, "IDENTITY.md")
if data, err := os.ReadFile(filePath); err == nil {
fmt.Fprintf(&sb, "## %s\n\n%s\n\n", filename, data)
fmt.Fprintf(&sb, "## %s\n\n%s\n\n", "IDENTITY.md", data)
}
}
pkg/agent/context_budget.go
+176
View File
@@ -0,0 +1,176 @@
// PicoClaw - Ultra-lightweight personal AI agent
// License: MIT
//
// Copyright (c) 2026 PicoClaw contributors
package agent
import (
"encoding/json"
"unicode/utf8"
"github.com/sipeed/picoclaw/pkg/providers"
)
// parseTurnBoundaries returns the starting index of each Turn in the history.
// A Turn is a complete "user input → LLM iterations → final response" cycle
// (as defined in #1316). Each Turn begins at a user message and extends
// through all subsequent assistant/tool messages until the next user message.
//
// Cutting at a Turn boundary guarantees that no tool-call sequence
// (assistant+ToolCalls → tool results) is split across the cut.
func parseTurnBoundaries(history []providers.Message) []int {
var starts []int
for i, msg := range history {
if msg.Role == "user" {
starts = append(starts, i)
}
}
return starts
}
// isSafeBoundary reports whether index is a valid Turn boundary — i.e.,
// a position where the kept portion (history[index:]) begins at a user
// message, so no tool-call sequence is torn apart.
func isSafeBoundary(history []providers.Message, index int) bool {
if index <= 0 || index >= len(history) {
return true
}
return history[index].Role == "user"
}
// findSafeBoundary locates the nearest Turn boundary to targetIndex.
// It prefers the boundary at or before targetIndex (preserving more recent
// context). Falls back to the nearest boundary after targetIndex, and
// returns targetIndex unchanged only when no Turn boundary exists at all.
func findSafeBoundary(history []providers.Message, targetIndex int) int {
if len(history) == 0 {
return 0
}
if targetIndex <= 0 {
return 0
}
if targetIndex >= len(history) {
return len(history)
}
turns := parseTurnBoundaries(history)
if len(turns) == 0 {
return targetIndex
}
// Find the last Turn boundary at or before targetIndex.
// Prefer backward: keeps more recent messages.
backward := -1
for _, t := range turns {
if t <= targetIndex {
backward = t
}
}
if backward > 0 {
return backward
}
// No valid Turn boundary before target (or only at index 0 which
// would keep everything). Use the first Turn after targetIndex.
for _, t := range turns {
if t > targetIndex {
return t
}
}
// No Turn boundary after targetIndex either. The only boundary is at
// index 0, meaning the entire history is a single Turn. Return 0 to
// signal that safe compression is not possible — callers check for
// mid <= 0 and skip compression in that case.
return 0
}
// estimateMessageTokens estimates the token count for a single message,
// including Content, ReasoningContent, ToolCalls arguments, ToolCallID
// metadata, and Media items. Uses a heuristic of 2.5 characters per token.
func estimateMessageTokens(msg providers.Message) int {
chars := utf8.RuneCountInString(msg.Content)
// ReasoningContent (extended thinking / chain-of-thought) can be
// substantial and is stored in session history via AddFullMessage.
if msg.ReasoningContent != "" {
chars += utf8.RuneCountInString(msg.ReasoningContent)
}
for _, tc := range msg.ToolCalls {
chars += len(tc.ID) + len(tc.Type)
if tc.Function != nil {
// Count function name + arguments (the wire format for most providers).
// tc.Name mirrors tc.Function.Name — count only once to avoid double-counting.
chars += len(tc.Function.Name) + len(tc.Function.Arguments)
} else {
// Fallback: some provider formats use top-level Name without Function.
chars += len(tc.Name)
}
}
if msg.ToolCallID != "" {
chars += len(msg.ToolCallID)
}
// Per-message overhead for role label, JSON structure, separators.
const messageOverhead = 12
chars += messageOverhead
tokens := chars * 2 / 5
// Media items (images, files) are serialized by provider adapters into
// multipart or image_url payloads. Add a fixed per-item token estimate
// directly (not through the chars heuristic) since actual cost depends
// on resolution and provider-specific image tokenization.
const mediaTokensPerItem = 256
tokens += len(msg.Media) * mediaTokensPerItem
return tokens
}
// estimateToolDefsTokens estimates the total token cost of tool definitions
// as they appear in the LLM request. Each tool's name, description, and
// JSON schema parameters contribute to the context window budget.
func estimateToolDefsTokens(defs []providers.ToolDefinition) int {
if len(defs) == 0 {
return 0
}
totalChars := 0
for _, d := range defs {
totalChars += len(d.Function.Name) + len(d.Function.Description)
if d.Function.Parameters != nil {
if paramJSON, err := json.Marshal(d.Function.Parameters); err == nil {
totalChars += len(paramJSON)
}
}
// Per-tool overhead: type field, JSON structure, separators.
totalChars += 20
}
return totalChars * 2 / 5
}
// isOverContextBudget checks whether the assembled messages plus tool definitions
// and output reserve would exceed the model's context window. This enables
// proactive compression before calling the LLM, rather than reacting to 400 errors.
func isOverContextBudget(
contextWindow int,
messages []providers.Message,
toolDefs []providers.ToolDefinition,
maxTokens int,
) bool {
msgTokens := 0
for _, m := range messages {
msgTokens += estimateMessageTokens(m)
}
toolTokens := estimateToolDefsTokens(toolDefs)
total := msgTokens + toolTokens + maxTokens
return total > contextWindow
}
pkg/agent/context_budget_test.go
+826
View File
@@ -0,0 +1,826 @@
package agent
import (
"fmt"
"strings"
"testing"
"github.com/sipeed/picoclaw/pkg/providers"
)
// msgUser creates a user message.
func msgUser(content string) providers.Message {
return providers.Message{Role: "user", Content: content}
}
// msgAssistant creates a plain assistant message (no tool calls).
func msgAssistant(content string) providers.Message {
return providers.Message{Role: "assistant", Content: content}
}
// msgAssistantTC creates an assistant message with tool calls.
func msgAssistantTC(toolIDs ...string) providers.Message {
tcs := make([]providers.ToolCall, len(toolIDs))
for i, id := range toolIDs {
tcs[i] = providers.ToolCall{
ID: id,
Type: "function",
Name: "tool_" + id,
Function: &providers.FunctionCall{
Name: "tool_" + id,
Arguments: `{"key":"value"}`,
},
}
}
return providers.Message{Role: "assistant", ToolCalls: tcs}
}
// msgTool creates a tool result message.
func msgTool(callID, content string) providers.Message {
return providers.Message{Role: "tool", ToolCallID: callID, Content: content}
}
func TestParseTurnBoundaries(t *testing.T) {
tests := []struct {
name string
history []providers.Message
want []int
}{
{
name: "empty history",
history: nil,
want: nil,
},
{
name: "simple exchange",
history: []providers.Message{
msgUser("q1"),
msgAssistant("a1"),
msgUser("q2"),
msgAssistant("a2"),
},
want: []int{0, 2},
},
{
name: "tool-call Turn",
history: []providers.Message{
msgUser("search"),
msgAssistantTC("tc1"),
msgTool("tc1", "result"),
msgAssistant("found it"),
msgUser("thanks"),
msgAssistant("welcome"),
},
want: []int{0, 4},
},
{
name: "chained tool calls in single Turn",
history: []providers.Message{
msgUser("save and notify"),
msgAssistantTC("tc_save"),
msgTool("tc_save", "saved"),
msgAssistantTC("tc_notify"),
msgTool("tc_notify", "notified"),
msgAssistant("done"),
},
want: []int{0},
},
{
name: "no user messages",
history: []providers.Message{
msgAssistant("a1"),
msgAssistant("a2"),
},
want: nil,
},
{
name: "leading non-user messages",
history: []providers.Message{
msgAssistantTC("tc1"),
msgTool("tc1", "r1"),
msgAssistant("greeting"),
msgUser("hello"),
msgAssistant("hi"),
},
want: []int{3},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := parseTurnBoundaries(tt.history)
if len(got) != len(tt.want) {
t.Errorf("parseTurnBoundaries() = %v, want %v", got, tt.want)
return
}
for i := range got {
if got[i] != tt.want[i] {
t.Errorf("parseTurnBoundaries()[%d] = %d, want %d", i, got[i], tt.want[i])
}
}
})
}
}
func TestIsSafeBoundary(t *testing.T) {
tests := []struct {
name string
history []providers.Message
index int
want bool
}{
{
name: "empty history, index 0",
history: nil,
index: 0,
want: true,
},
{
name: "single user message, index 0",
history: []providers.Message{msgUser("hi")},
index: 0,
want: true,
},
{
name: "single user message, index 1 (end)",
history: []providers.Message{msgUser("hi")},
index: 1,
want: true,
},
{
name: "at user message",
history: []providers.Message{
msgAssistant("hello"),
msgUser("how are you"),
msgAssistant("fine"),
},
index: 1,
want: true,
},
{
name: "at assistant without tool calls",
history: []providers.Message{
msgUser("hello"),
msgAssistant("response"),
msgUser("follow up"),
},
index: 1,
want: false,
},
{
name: "at assistant with tool calls",
history: []providers.Message{
msgUser("search something"),
msgAssistantTC("tc1"),
msgTool("tc1", "result"),
msgAssistant("here is what I found"),
},
index: 1,
want: false,
},
{
name: "at tool result",
history: []providers.Message{
msgUser("do something"),
msgAssistantTC("tc1"),
msgTool("tc1", "done"),
msgAssistant("completed"),
},
index: 2,
want: false,
},
{
name: "negative index",
history: []providers.Message{
msgUser("hello"),
},
index: -1,
want: true,
},
{
name: "index beyond length",
history: []providers.Message{
msgUser("hello"),
},
index: 5,
want: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := isSafeBoundary(tt.history, tt.index)
if got != tt.want {
t.Errorf("isSafeBoundary(history, %d) = %v, want %v", tt.index, got, tt.want)
}
})
}
}
func TestFindSafeBoundary(t *testing.T) {
tests := []struct {
name string
history []providers.Message
targetIndex int
want int
}{
{
name: "empty history",
history: nil,
targetIndex: 0,
want: 0,
},
{
name: "target at 0",
history: []providers.Message{msgUser("hi")},
targetIndex: 0,
want: 0,
},
{
name: "target beyond length",
history: []providers.Message{msgUser("hi")},
targetIndex: 5,
want: 1,
},
{
name: "target already at user message",
history: []providers.Message{
msgUser("q1"),
msgAssistant("a1"),
msgUser("q2"),
msgAssistant("a2"),
},
targetIndex: 2,
want: 2,
},
{
name: "target at assistant, scan backward finds user",
history: []providers.Message{
msgUser("q1"),
msgAssistant("a1"),
msgUser("q2"),
msgAssistant("a2"),
msgUser("q3"),
},
targetIndex: 3, // assistant "a2"
want: 2, // backward to user "q2"
},
{
name: "target inside tool sequence, scan backward finds user",
history: []providers.Message{
msgUser("q1"),
msgAssistant("a1"),
msgUser("q2"),
msgAssistantTC("tc1", "tc2"),
msgTool("tc1", "r1"),
msgTool("tc2", "r2"),
msgAssistant("summary"),
msgUser("q3"),
},
targetIndex: 4, // tool result "r1"
want: 2, // backward: 3=assistant+TC (not safe), 2=user → safe
},
{
name: "target inside tool sequence, backward finds user before chain",
history: []providers.Message{
msgUser("q1"),
msgAssistant("a1"),
msgUser("q2"),
msgAssistantTC("tc1", "tc2"),
msgTool("tc1", "r1"),
msgTool("tc2", "r2"),
msgAssistant("summary"),
msgUser("q3"),
},
targetIndex: 5, // tool result "r2"
want: 2, // backward: 4=tool, 3=assistant+TC, 2=user → safe
},
{
name: "no backward user, scan forward finds one",
history: []providers.Message{
msgAssistantTC("tc1"),
msgTool("tc1", "r1"),
msgAssistant("a1"),
msgUser("q1"),
},
targetIndex: 1, // tool result
want: 3, // forward to user "q1"
},
{
name: "multi-step tool chain preserves atomicity",
history: []providers.Message{
msgUser("q1"),
msgAssistant("a1"),
msgUser("q2"),
msgAssistantTC("tc1"),
msgTool("tc1", "r1"),
msgAssistantTC("tc2"),
msgTool("tc2", "r2"),
msgAssistant("final"),
msgUser("q3"),
msgAssistant("a3"),
},
targetIndex: 5, // second assistant+TC
want: 2, // backward: 4=tool, 3=assistant+TC, 2=user → safe
},
{
name: "all non-user messages returns target unchanged",
history: []providers.Message{
msgAssistant("a1"),
msgAssistant("a2"),
msgAssistant("a3"),
},
targetIndex: 1,
want: 1,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := findSafeBoundary(tt.history, tt.targetIndex)
if got != tt.want {
t.Errorf("findSafeBoundary(history, %d) = %d, want %d",
tt.targetIndex, got, tt.want)
}
})
}
}
func TestFindSafeBoundary_SingleTurnReturnsZero(t *testing.T) {
// A single Turn with no subsequent user message. The only Turn boundary
// is at index 0; cutting anywhere else would split the Turn's tool
// sequence. findSafeBoundary must return 0 so callers skip compression.
history := []providers.Message{
msgUser("do everything"), // 0 ← only Turn boundary
msgAssistantTC("tc1"), // 1
msgTool("tc1", "result"), // 2
msgAssistant("all done"), // 3
}
got := findSafeBoundary(history, 2)
if got != 0 {
t.Errorf("findSafeBoundary(single_turn, 2) = %d, want 0 (cannot split single Turn)", got)
}
}
func TestFindSafeBoundary_BackwardScanSkipsToolSequence(t *testing.T) {
// A long tool-call chain: user → assistant+TC → tool → tool → ... → assistant → user
// Target is inside the chain; boundary should skip the entire chain backward.
history := []providers.Message{
msgUser("start"), // 0
msgAssistant("before chain"), // 1
msgUser("trigger"), // 2 ← expected safe boundary
msgAssistantTC("t1", "t2", "t3"), // 3
msgTool("t1", "r1"), // 4
msgTool("t2", "r2"), // 5
msgTool("t3", "r3"), // 6
msgAssistantTC("t4"), // 7
msgTool("t4", "r4"), // 8
msgAssistant("chain done"), // 9
msgUser("next"), // 10
}
// Target at index 6 (middle of tool results)
got := findSafeBoundary(history, 6)
if got != 2 {
t.Errorf("findSafeBoundary(history, 6) = %d, want 2 (user before chain)", got)
}
}
func TestEstimateMessageTokens(t *testing.T) {
tests := []struct {
name string
msg providers.Message
want int // minimum expected tokens (exact value depends on overhead)
}{
{
name: "plain user message",
msg: msgUser("Hello, world!"),
want: 1, // at least some tokens
},
{
name: "empty message still has overhead",
msg: providers.Message{Role: "user"},
want: 1, // message overhead alone
},
{
name: "assistant with tool calls",
msg: msgAssistantTC("tc_123"),
want: 1,
},
{
name: "tool result with ID",
msg: msgTool("call_abc", "Here is the search result with lots of content"),
want: 1,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := estimateMessageTokens(tt.msg)
if got < tt.want {
t.Errorf("estimateMessageTokens() = %d, want >= %d", got, tt.want)
}
})
}
}
func TestEstimateMessageTokens_ToolCallsContribute(t *testing.T) {
plain := msgAssistant("thinking")
withTC := providers.Message{
Role: "assistant",
Content: "thinking",
ToolCalls: []providers.ToolCall{
{
ID: "call_1",
Type: "function",
Name: "web_search",
Function: &providers.FunctionCall{
Name: "web_search",
Arguments: `{"query":"picoclaw agent framework","max_results":5}`,
},
},
},
}
plainTokens := estimateMessageTokens(plain)
withTCTokens := estimateMessageTokens(withTC)
if withTCTokens <= plainTokens {
t.Errorf("message with ToolCalls (%d tokens) should exceed plain message (%d tokens)",
withTCTokens, plainTokens)
}
}
func TestEstimateMessageTokens_MultibyteContent(t *testing.T) {
// Multi-byte characters (e.g. emoji, accented letters) are single runes
// but may map to different token counts. The heuristic should still produce
// reasonable estimates via RuneCountInString.
msg := msgUser("caf\u00e9 na\u00efve r\u00e9sum\u00e9 \u00fcber stra\u00dfe")
tokens := estimateMessageTokens(msg)
if tokens <= 0 {
t.Errorf("multibyte message should produce positive token count, got %d", tokens)
}
}
func TestEstimateMessageTokens_LargeArguments(t *testing.T) {
// Simulate a tool call with large JSON arguments.
largeArgs := fmt.Sprintf(`{"content":"%s"}`, strings.Repeat("x", 5000))
msg := providers.Message{
Role: "assistant",
ToolCalls: []providers.ToolCall{
{
ID: "call_large",
Type: "function",
Name: "write_file",
Function: &providers.FunctionCall{
Name: "write_file",
Arguments: largeArgs,
},
},
},
}
tokens := estimateMessageTokens(msg)
// 5000+ chars → at least 2000 tokens with the 2.5 char/token heuristic
if tokens < 2000 {
t.Errorf("large tool call arguments should produce significant token count, got %d", tokens)
}
}
func TestEstimateMessageTokens_ReasoningContent(t *testing.T) {
plain := msgAssistant("result")
withReasoning := providers.Message{
Role: "assistant",
Content: "result",
ReasoningContent: strings.Repeat("thinking step ", 200),
}
plainTokens := estimateMessageTokens(plain)
reasoningTokens := estimateMessageTokens(withReasoning)
if reasoningTokens <= plainTokens {
t.Errorf("message with ReasoningContent (%d tokens) should exceed plain message (%d tokens)",
reasoningTokens, plainTokens)
}
}
func TestEstimateMessageTokens_MediaItems(t *testing.T) {
plain := msgUser("describe this")
withMedia := providers.Message{
Role: "user",
Content: "describe this",
Media: []string{"media://img1.png", "media://img2.png"},
}
plainTokens := estimateMessageTokens(plain)
mediaTokens := estimateMessageTokens(withMedia)
if mediaTokens <= plainTokens {
t.Errorf("message with Media (%d tokens) should exceed plain message (%d tokens)",
mediaTokens, plainTokens)
}
// Each media item should add exactly 256 tokens (not run through chars*2/5).
expectedDelta := 256 * 2
actualDelta := mediaTokens - plainTokens
if actualDelta != expectedDelta {
t.Errorf("2 media items should add %d tokens, got delta %d", expectedDelta, actualDelta)
}
}
// --- estimateToolDefsTokens tests ---
func TestEstimateToolDefsTokens(t *testing.T) {
tests := []struct {
name string
defs []providers.ToolDefinition
want int // minimum expected tokens
}{
{
name: "empty tool list",
defs: nil,
want: 0,
},
{
name: "single tool with params",
defs: []providers.ToolDefinition{
{
Type: "function",
Function: providers.ToolFunctionDefinition{
Name: "web_search",
Description: "Search the web for information",
Parameters: map[string]any{
"type": "object",
"properties": map[string]any{
"query": map[string]any{"type": "string"},
},
"required": []any{"query"},
},
},
},
},
want: 1,
},
{
name: "tool without params",
defs: []providers.ToolDefinition{
{
Type: "function",
Function: providers.ToolFunctionDefinition{
Name: "list_dir",
Description: "List directory contents",
},
},
},
want: 1,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := estimateToolDefsTokens(tt.defs)
if got < tt.want {
t.Errorf("estimateToolDefsTokens() = %d, want >= %d", got, tt.want)
}
})
}
}
func TestEstimateToolDefsTokens_ScalesWithCount(t *testing.T) {
makeTool := func(name string) providers.ToolDefinition {
return providers.ToolDefinition{
Type: "function",
Function: providers.ToolFunctionDefinition{
Name: name,
Description: "A test tool that does something useful",
Parameters: map[string]any{
"type": "object",
"properties": map[string]any{
"input": map[string]any{"type": "string", "description": "Input value"},
},
},
},
}
}
one := estimateToolDefsTokens([]providers.ToolDefinition{makeTool("tool_a")})
three := estimateToolDefsTokens([]providers.ToolDefinition{
makeTool("tool_a"), makeTool("tool_b"), makeTool("tool_c"),
})
if three <= one {
t.Errorf("3 tools (%d tokens) should exceed 1 tool (%d tokens)", three, one)
}
}
// --- isOverContextBudget tests ---
func TestIsOverContextBudget(t *testing.T) {
systemMsg := providers.Message{Role: "system", Content: strings.Repeat("x", 1000)}
userMsg := msgUser("hello")
smallHistory := []providers.Message{systemMsg, msgUser("q1"), msgAssistant("a1"), userMsg}
tools := []providers.ToolDefinition{
{
Type: "function",
Function: providers.ToolFunctionDefinition{
Name: "test_tool",
Description: "A test tool",
Parameters: map[string]any{"type": "object"},
},
},
}
tests := []struct {
name string
contextWindow int
messages []providers.Message
toolDefs []providers.ToolDefinition
maxTokens int
want bool
}{
{
name: "within budget",
contextWindow: 100000,
messages: smallHistory,
toolDefs: tools,
maxTokens: 4096,
want: false,
},
{
name: "over budget with small window",
contextWindow: 100, // very small window
messages: smallHistory,
toolDefs: tools,
maxTokens: 4096,
want: true,
},
{
name: "large max_tokens eats budget",
contextWindow: 2000,
messages: smallHistory,
toolDefs: tools,
maxTokens: 1800, // leaves almost no room
want: true,
},
{
name: "empty messages within budget",
contextWindow: 10000,
messages: nil,
toolDefs: nil,
maxTokens: 4096,
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := isOverContextBudget(tt.contextWindow, tt.messages, tt.toolDefs, tt.maxTokens)
if got != tt.want {
t.Errorf("isOverContextBudget() = %v, want %v", got, tt.want)
}
})
}
}
// --- Tests reflecting actual session data shape ---
// Session history never contains system messages. The system prompt is
// built dynamically by BuildMessages. These tests use realistic history
// shapes: user/assistant/tool only, with tool chains and reasoning content.
func TestFindSafeBoundary_SessionHistoryNoSystem(t *testing.T) {
// Real session history starts with a user message, not a system message.
history := []providers.Message{
msgUser("hello"), // 0
msgAssistant("hi there"), // 1
msgUser("search for X"), // 2
msgAssistantTC("tc1"), // 3
msgTool("tc1", "found X"), // 4
msgAssistant("here is X"), // 5
msgUser("thanks"), // 6
msgAssistant("you're welcome"), // 7
}
// Mid-point is 4 (tool result). Should snap backward to 2 (user).
got := findSafeBoundary(history, 4)
if got != 2 {
t.Errorf("findSafeBoundary(session_history, 4) = %d, want 2", got)
}
}
func TestFindSafeBoundary_SessionWithChainedTools(t *testing.T) {
// Session with chained tool calls (save then notify).
history := []providers.Message{
msgUser("save and notify"), // 0
msgAssistantTC("tc_save"), // 1
msgTool("tc_save", "saved"), // 2
msgAssistantTC("tc_notify"), // 3
msgTool("tc_notify", "notified"), // 4
msgAssistant("done"), // 5
msgUser("check status"), // 6
msgAssistant("all good"), // 7
}
// Target at 3 (inside chain). Should find user at 0, but backward
// scan stops at i>0, so forward scan finds user at 6.
// Actually: backward from 3: 2=tool (no), 1=assistantTC (no). Forward: 4=tool, 5=asst, 6=user ✓
got := findSafeBoundary(history, 3)
if got != 6 {
t.Errorf("findSafeBoundary(chained_tools, 3) = %d, want 6", got)
}
}
func TestEstimateMessageTokens_WithReasoningAndMedia(t *testing.T) {
// Message with all fields populated — mirrors what AddFullMessage stores.
msg := providers.Message{
Role: "assistant",
Content: "Here is the analysis.",
ReasoningContent: strings.Repeat("Let me think about this carefully. ", 50),
ToolCalls: []providers.ToolCall{
{
ID: "call_1",
Type: "function",
Name: "analyze",
Function: &providers.FunctionCall{
Name: "analyze",
Arguments: `{"data":"sample","depth":3}`,
},
},
},
}
tokens := estimateMessageTokens(msg)
// ReasoningContent alone is ~1700 chars → ~680 tokens.
// Content + TC + overhead adds more. Should be well above 500.
if tokens < 500 {
t.Errorf("message with reasoning+toolcalls should have significant tokens, got %d", tokens)
}
// Compare without reasoning to ensure it's counted.
msgNoReasoning := msg
msgNoReasoning.ReasoningContent = ""
tokensNoReasoning := estimateMessageTokens(msgNoReasoning)
if tokens <= tokensNoReasoning {
t.Errorf("reasoning content should add tokens: with=%d, without=%d", tokens, tokensNoReasoning)
}
}
func TestIsOverContextBudget_RealisticSession(t *testing.T) {
// Simulate what BuildMessages produces: system + session history + current user.
// System message is built by BuildMessages, not stored in session.
systemMsg := providers.Message{
Role: "system",
Content: strings.Repeat("system prompt content ", 100),
}
sessionHistory := []providers.Message{
msgUser("first question"),
msgAssistant("first answer"),
msgUser("use tool X"),
{
Role: "assistant",
Content: "I'll use tool X",
ToolCalls: []providers.ToolCall{
{
ID: "tc1", Type: "function", Name: "tool_x",
Function: &providers.FunctionCall{
Name: "tool_x",
Arguments: `{"query":"test","verbose":true}`,
},
},
},
},
{Role: "tool", Content: strings.Repeat("result data ", 200), ToolCallID: "tc1"},
msgAssistant("Here are the results from tool X."),
}
currentUser := msgUser("follow up question")
// Assemble as BuildMessages would.
messages := make([]providers.Message, 0, 1+len(sessionHistory)+1)
messages = append(messages, systemMsg)
messages = append(messages, sessionHistory...)
messages = append(messages, currentUser)
tools := []providers.ToolDefinition{
{
Type: "function",
Function: providers.ToolFunctionDefinition{
Name: "tool_x",
Description: "A useful tool",
Parameters: map[string]any{"type": "object"},
},
},
}
// With a large context window, should be within budget.
if isOverContextBudget(131072, messages, tools, 32768) {
t.Error("realistic session should be within 131072 context window")
}
// With a tiny context window, should exceed budget.
if !isOverContextBudget(500, messages, tools, 32768) {
t.Error("realistic session should exceed 500 context window")
}
}
pkg/agent/context_cache_test.go
+10 -10
View File
@@ -37,7 +37,7 @@ func setupWorkspace(t *testing.T, files map[string]string) string {
// Codex (only reads last system message as instructions).
func TestSingleSystemMessage(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"IDENTITY.md": "# Identity\nTest agent.",
"AGENT.md": "# Agent\nTest agent.",
})
defer os.RemoveAll(tmpDir)
@@ -202,10 +202,10 @@ func TestMtimeAutoInvalidation(t *testing.T) {
}{
{
name: "bootstrap file change",
file: "IDENTITY.md",
contentV1: "# Original Identity",
contentV2: "# Updated Identity",
checkField: "Updated Identity",
file: "AGENT.md",
contentV1: "# Original Agent",
contentV2: "# Updated Agent",
checkField: "Updated Agent",
},
{
name: "memory file change",
@@ -280,7 +280,7 @@ func TestMtimeAutoInvalidation(t *testing.T) {
// even when source files haven't changed (useful for tests and reload commands).
func TestExplicitInvalidateCache(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"IDENTITY.md": "# Test Identity",
"AGENT.md": "# Test Agent",
})
defer os.RemoveAll(tmpDir)
@@ -307,8 +307,8 @@ func TestExplicitInvalidateCache(t *testing.T) {
// when no files change (regression test for issue #607).
func TestCacheStability(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"IDENTITY.md": "# Identity\nContent",
"SOUL.md": "# Soul\nContent",
"AGENT.md": "# Agent\nContent",
"SOUL.md": "# Soul\nContent",
})
defer os.RemoveAll(tmpDir)
@@ -607,7 +607,7 @@ description: delete-me-v1
// Run with: go test -race ./pkg/agent/ -run TestConcurrentBuildSystemPromptWithCache
func TestConcurrentBuildSystemPromptWithCache(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"IDENTITY.md": "# Identity\nConcurrency test agent.",
"AGENT.md": "# Agent\nConcurrency test agent.",
"SOUL.md": "# Soul\nBe helpful.",
"memory/MEMORY.md": "# Memory\nUser prefers Go.",
"skills/demo/SKILL.md": "---\nname: demo\ndescription: \"demo skill\"\n---\n# Demo",
@@ -714,7 +714,7 @@ func BenchmarkBuildMessagesWithCache(b *testing.B) {
os.MkdirAll(filepath.Join(tmpDir, "memory"), 0o755)
os.MkdirAll(filepath.Join(tmpDir, "skills"), 0o755)
for _, name := range []string{"IDENTITY.md", "SOUL.md", "USER.md"} {
for _, name := range []string{"AGENT.md", "SOUL.md"} {
os.WriteFile(filepath.Join(tmpDir, name), []byte(strings.Repeat("Content.\n", 10)), 0o644)
}
pkg/agent/definition.go
+255
View File
@@ -0,0 +1,255 @@
package agent
import (
"os"
"path/filepath"
"slices"
"strings"
"github.com/gomarkdown/markdown/parser"
"gopkg.in/yaml.v3"
"github.com/sipeed/picoclaw/pkg/logger"
)
// AgentDefinitionSource identifies which agent bootstrap file produced the definition.
type AgentDefinitionSource string
const (
// AgentDefinitionSourceAgent indicates the new AGENT.md format.
AgentDefinitionSourceAgent AgentDefinitionSource = "AGENT.md"
// AgentDefinitionSourceAgents indicates the legacy AGENTS.md format.
AgentDefinitionSourceAgents AgentDefinitionSource = "AGENTS.md"
)
// AgentFrontmatter holds machine-readable AGENT.md configuration.
//
// Known fields are exposed directly for convenience. Fields keeps the full
// parsed frontmatter so future refactors can read additional keys without
// changing the loader contract again.
type AgentFrontmatter struct {
Name string `json:"name"`
Description string `json:"description"`
Tools []string `json:"tools,omitempty"`
Model string `json:"model,omitempty"`
MaxTurns *int `json:"maxTurns,omitempty"`
Skills []string `json:"skills,omitempty"`
MCPServers []string `json:"mcpServers,omitempty"`
Fields map[string]any `json:"fields,omitempty"`
}
// AgentPromptDefinition represents the parsed AGENT.md or AGENTS.md prompt file.
type AgentPromptDefinition struct {
Path string `json:"path"`
Raw string `json:"raw"`
Body string `json:"body"`
RawFrontmatter string `json:"raw_frontmatter,omitempty"`
Frontmatter AgentFrontmatter `json:"frontmatter"`
}
// SoulDefinition represents the resolved SOUL.md file linked to the agent.
type SoulDefinition struct {
Path string `json:"path"`
Content string `json:"content"`
}
// UserDefinition represents the resolved USER.md file linked to the workspace.
type UserDefinition struct {
Path string `json:"path"`
Content string `json:"content"`
}
// AgentContextDefinition captures the workspace agent definition in a runtime-friendly shape.
type AgentContextDefinition struct {
Source AgentDefinitionSource `json:"source,omitempty"`
Agent *AgentPromptDefinition `json:"agent,omitempty"`
Soul *SoulDefinition `json:"soul,omitempty"`
User *UserDefinition `json:"user,omitempty"`
}
// LoadAgentDefinition parses the workspace agent bootstrap files.
//
// It prefers the new AGENT.md format and its paired SOUL.md file. When the
// structured files are absent, it falls back to the legacy AGENTS.md layout so
// the current runtime can transition incrementally.
func (cb *ContextBuilder) LoadAgentDefinition() AgentContextDefinition {
return loadAgentDefinition(cb.workspace)
}
func loadAgentDefinition(workspace string) AgentContextDefinition {
definition := AgentContextDefinition{}
definition.User = loadUserDefinition(workspace)
agentPath := filepath.Join(workspace, string(AgentDefinitionSourceAgent))
if content, err := os.ReadFile(agentPath); err == nil {
prompt := parseAgentPromptDefinition(agentPath, string(content))
definition.Source = AgentDefinitionSourceAgent
definition.Agent = &prompt
soulPath := filepath.Join(workspace, "SOUL.md")
if content, err := os.ReadFile(soulPath); err == nil {
definition.Soul = &SoulDefinition{
Path: soulPath,
Content: string(content),
}
}
return definition
}
legacyPath := filepath.Join(workspace, string(AgentDefinitionSourceAgents))
if content, err := os.ReadFile(legacyPath); err == nil {
definition.Source = AgentDefinitionSourceAgents
definition.Agent = &AgentPromptDefinition{
Path: legacyPath,
Raw: string(content),
Body: string(content),
}
}
defaultSoulPath := filepath.Join(workspace, "SOUL.md")
if definition.Source != "" || fileExists(defaultSoulPath) {
if content, err := os.ReadFile(defaultSoulPath); err == nil {
definition.Soul = &SoulDefinition{
Path: defaultSoulPath,
Content: string(content),
}
}
}
return definition
}
func (definition AgentContextDefinition) trackedPaths(workspace string) []string {
paths := []string{
filepath.Join(workspace, string(AgentDefinitionSourceAgent)),
filepath.Join(workspace, "SOUL.md"),
filepath.Join(workspace, "USER.md"),
}
if definition.Source != AgentDefinitionSourceAgent {
paths = append(paths,
filepath.Join(workspace, string(AgentDefinitionSourceAgents)),
filepath.Join(workspace, "IDENTITY.md"),
)
}
return uniquePaths(paths)
}
func loadUserDefinition(workspace string) *UserDefinition {
userPath := filepath.Join(workspace, "USER.md")
if content, err := os.ReadFile(userPath); err == nil {
return &UserDefinition{
Path: userPath,
Content: string(content),
}
}
return nil
}
func parseAgentPromptDefinition(path, content string) AgentPromptDefinition {
frontmatter, body := splitAgentFrontmatter(content)
return AgentPromptDefinition{
Path: path,
Raw: content,
Body: body,
RawFrontmatter: frontmatter,
Frontmatter: parseAgentFrontmatter(path, frontmatter),
}
}
func parseAgentFrontmatter(path, frontmatter string) AgentFrontmatter {
frontmatter = strings.TrimSpace(frontmatter)
if frontmatter == "" {
return AgentFrontmatter{}
}
rawFields := make(map[string]any)
if err := yaml.Unmarshal([]byte(frontmatter), &rawFields); err != nil {
logger.WarnCF("agent", "Failed to parse AGENT.md frontmatter", map[string]any{
"path": path,
"error": err.Error(),
})
return AgentFrontmatter{}
}
var typed struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Tools []string `yaml:"tools"`
Model string `yaml:"model"`
MaxTurns *int `yaml:"maxTurns"`
Skills []string `yaml:"skills"`
MCPServers []string `yaml:"mcpServers"`
}
if err := yaml.Unmarshal([]byte(frontmatter), &typed); err != nil {
logger.WarnCF("agent", "Failed to decode AGENT.md frontmatter fields", map[string]any{
"path": path,
"error": err.Error(),
})
return AgentFrontmatter{}
}
return AgentFrontmatter{
Name: strings.TrimSpace(typed.Name),
Description: strings.TrimSpace(typed.Description),
Tools: append([]string(nil), typed.Tools...),
Model: strings.TrimSpace(typed.Model),
MaxTurns: typed.MaxTurns,
Skills: append([]string(nil), typed.Skills...),
MCPServers: append([]string(nil), typed.MCPServers...),
Fields: rawFields,
}
}
func splitAgentFrontmatter(content string) (frontmatter, body string) {
normalized := string(parser.NormalizeNewlines([]byte(content)))
lines := strings.Split(normalized, "\n")
if len(lines) == 0 || lines[0] != "---" {
return "", content
}
end := -1
for i := 1; i < len(lines); i++ {
if lines[i] == "---" {
end = i
break
}
}
if end == -1 {
return "", content
}
frontmatter = strings.Join(lines[1:end], "\n")
body = strings.Join(lines[end+1:], "\n")
body = strings.TrimLeft(body, "\n")
return frontmatter, body
}
func relativeWorkspacePath(workspace, path string) string {
if strings.TrimSpace(path) == "" {
return ""
}
relativePath, err := filepath.Rel(workspace, path)
if err == nil && relativePath != "." && !strings.HasPrefix(relativePath, "..") {
return filepath.ToSlash(relativePath)
}
return filepath.Clean(path)
}
func uniquePaths(paths []string) []string {
result := make([]string, 0, len(paths))
for _, path := range paths {
if strings.TrimSpace(path) == "" {
continue
}
cleaned := filepath.Clean(path)
if slices.Contains(result, cleaned) {
continue
}
result = append(result, cleaned)
}
return result
}
func fileExists(path string) bool {
_, err := os.Stat(path)
return err == nil
}
pkg/agent/definition_test.go
+302
View File
@@ -0,0 +1,302 @@
package agent
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestLoadAgentDefinitionParsesFrontmatterAndSoul(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"AGENT.md": `---
name: pico
description: Structured agent
model: claude-3-7-sonnet
tools:
- shell
- search
maxTurns: 8
skills:
- review
- search-docs
mcpServers:
- github
metadata:
mode: strict
---
# Agent
Act directly and use tools first.
`,
"SOUL.md": "# Soul\nStay precise.",
})
defer cleanupWorkspace(t, tmpDir)
cb := NewContextBuilder(tmpDir)
definition := cb.LoadAgentDefinition()
if definition.Source != AgentDefinitionSourceAgent {
t.Fatalf("expected source %q, got %q", AgentDefinitionSourceAgent, definition.Source)
}
if definition.Agent == nil {
t.Fatal("expected AGENT.md definition to be loaded")
}
if definition.Agent.Body == "" || !strings.Contains(definition.Agent.Body, "Act directly") {
t.Fatalf("expected AGENT.md body to be preserved, got %q", definition.Agent.Body)
}
if definition.Agent.Frontmatter.Name != "pico" {
t.Fatalf("expected name to be parsed, got %q", definition.Agent.Frontmatter.Name)
}
if definition.Agent.Frontmatter.Model != "claude-3-7-sonnet" {
t.Fatalf("expected model to be parsed, got %q", definition.Agent.Frontmatter.Model)
}
if len(definition.Agent.Frontmatter.Tools) != 2 {
t.Fatalf("expected tools to be parsed, got %v", definition.Agent.Frontmatter.Tools)
}
if definition.Agent.Frontmatter.MaxTurns == nil || *definition.Agent.Frontmatter.MaxTurns != 8 {
t.Fatalf("expected maxTurns to be parsed, got %v", definition.Agent.Frontmatter.MaxTurns)
}
if len(definition.Agent.Frontmatter.Skills) != 2 {
t.Fatalf("expected skills to be parsed, got %v", definition.Agent.Frontmatter.Skills)
}
if len(definition.Agent.Frontmatter.MCPServers) != 1 || definition.Agent.Frontmatter.MCPServers[0] != "github" {
t.Fatalf("expected mcpServers to be parsed, got %v", definition.Agent.Frontmatter.MCPServers)
}
if definition.Agent.Frontmatter.Fields["metadata"] == nil {
t.Fatal("expected arbitrary frontmatter fields to remain available")
}
if definition.Soul == nil {
t.Fatal("expected SOUL.md to be loaded")
}
if !strings.Contains(definition.Soul.Content, "Stay precise") {
t.Fatalf("expected soul content to be loaded, got %q", definition.Soul.Content)
}
if definition.Soul.Path != filepath.Join(tmpDir, "SOUL.md") {
t.Fatalf("expected default SOUL.md path, got %q", definition.Soul.Path)
}
}
func TestLoadAgentDefinitionFallsBackToLegacyAgentsMarkdown(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"AGENTS.md": "# Legacy Agent\nKeep compatibility.",
"SOUL.md": "# Soul\nLegacy soul.",
})
defer cleanupWorkspace(t, tmpDir)
cb := NewContextBuilder(tmpDir)
definition := cb.LoadAgentDefinition()
if definition.Source != AgentDefinitionSourceAgents {
t.Fatalf("expected source %q, got %q", AgentDefinitionSourceAgents, definition.Source)
}
if definition.Agent == nil {
t.Fatal("expected AGENTS.md to be loaded")
}
if definition.Agent.RawFrontmatter != "" {
t.Fatalf("legacy AGENTS.md should not have frontmatter, got %q", definition.Agent.RawFrontmatter)
}
if !strings.Contains(definition.Agent.Body, "Keep compatibility") {
t.Fatalf("expected legacy body to be preserved, got %q", definition.Agent.Body)
}
if definition.Soul == nil || !strings.Contains(definition.Soul.Content, "Legacy soul") {
t.Fatal("expected default SOUL.md to be loaded for legacy format")
}
}
func TestLoadAgentDefinitionLoadsWorkspaceUserMarkdown(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"AGENT.md": "# Agent\nStructured agent.",
"USER.md": "# User\nWorkspace preferences.",
})
defer cleanupWorkspace(t, tmpDir)
cb := NewContextBuilder(tmpDir)
definition := cb.LoadAgentDefinition()
if definition.User == nil {
t.Fatal("expected USER.md to be loaded")
}
if definition.User.Path != filepath.Join(tmpDir, "USER.md") {
t.Fatalf("expected workspace USER.md path, got %q", definition.User.Path)
}
if !strings.Contains(definition.User.Content, "Workspace preferences") {
t.Fatalf("expected workspace USER.md content, got %q", definition.User.Content)
}
}
func TestLoadAgentDefinitionInvalidFrontmatterFallsBackToEmptyStructuredFields(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"AGENT.md": `---
name: pico
tools:
- shell
broken
---
# Agent
Keep going.
`,
})
defer cleanupWorkspace(t, tmpDir)
cb := NewContextBuilder(tmpDir)
definition := cb.LoadAgentDefinition()
if definition.Agent == nil {
t.Fatal("expected AGENT.md definition to be loaded")
}
if !strings.Contains(definition.Agent.Body, "Keep going.") {
t.Fatalf("expected AGENT.md body to be preserved, got %q", definition.Agent.Body)
}
if definition.Agent.Frontmatter.Name != "" ||
definition.Agent.Frontmatter.Description != "" ||
definition.Agent.Frontmatter.Model != "" ||
definition.Agent.Frontmatter.MaxTurns != nil ||
len(definition.Agent.Frontmatter.Tools) != 0 ||
len(definition.Agent.Frontmatter.Skills) != 0 ||
len(definition.Agent.Frontmatter.MCPServers) != 0 ||
len(definition.Agent.Frontmatter.Fields) != 0 {
t.Fatalf("expected invalid frontmatter to decode as empty struct, got %+v", definition.Agent.Frontmatter)
}
}
func TestLoadBootstrapFilesUsesAgentBodyNotFrontmatter(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"AGENT.md": `---
name: pico
model: codex-mini
---
# Agent
Follow the body prompt.
`,
"SOUL.md": "# Soul\nSpeak plainly.",
"IDENTITY.md": "# Identity\nWorkspace identity.",
})
defer cleanupWorkspace(t, tmpDir)
cb := NewContextBuilder(tmpDir)
bootstrap := cb.LoadBootstrapFiles()
if !strings.Contains(bootstrap, "Follow the body prompt") {
t.Fatalf("expected AGENT.md body in bootstrap, got %q", bootstrap)
}
if !strings.Contains(bootstrap, "Speak plainly") {
t.Fatalf("expected resolved soul content in bootstrap, got %q", bootstrap)
}
if strings.Contains(bootstrap, "name: pico") {
t.Fatalf("bootstrap should not expose raw frontmatter, got %q", bootstrap)
}
if strings.Contains(bootstrap, "model: codex-mini") {
t.Fatalf("bootstrap should not expose raw frontmatter, got %q", bootstrap)
}
if !strings.Contains(bootstrap, "SOUL.md") {
t.Fatalf("expected bootstrap to label SOUL.md, got %q", bootstrap)
}
if strings.Contains(bootstrap, "Workspace identity") {
t.Fatalf("structured bootstrap should ignore IDENTITY.md, got %q", bootstrap)
}
}
func TestLoadBootstrapFilesIncludesWorkspaceUserMarkdown(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"AGENT.md": "# Agent\nFollow the new structure.",
"SOUL.md": "# Soul\nSpeak plainly.",
"USER.md": "# User\nShared profile.",
})
defer cleanupWorkspace(t, tmpDir)
cb := NewContextBuilder(tmpDir)
bootstrap := cb.LoadBootstrapFiles()
if !strings.Contains(bootstrap, "Shared profile") {
t.Fatalf("expected workspace USER.md in bootstrap, got %q", bootstrap)
}
if !strings.Contains(bootstrap, "## USER.md") {
t.Fatalf("expected USER.md heading in bootstrap, got %q", bootstrap)
}
}
func TestStructuredAgentIgnoresIdentityChanges(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"AGENT.md": "# Agent\nFollow the new structure.",
"SOUL.md": "# Soul\nVersion one.",
"IDENTITY.md": "# Identity\nLegacy identity.",
})
defer cleanupWorkspace(t, tmpDir)
cb := NewContextBuilder(tmpDir)
promptV1 := cb.BuildSystemPromptWithCache()
if strings.Contains(promptV1, "Legacy identity") {
t.Fatalf("structured prompt should not include IDENTITY.md, got %q", promptV1)
}
identityPath := filepath.Join(tmpDir, "IDENTITY.md")
if err := os.WriteFile(identityPath, []byte("# Identity\nVersion two."), 0o644); err != nil {
t.Fatal(err)
}
future := time.Now().Add(2 * time.Second)
if err := os.Chtimes(identityPath, future, future); err != nil {
t.Fatal(err)
}
cb.systemPromptMutex.RLock()
changed := cb.sourceFilesChangedLocked()
cb.systemPromptMutex.RUnlock()
if changed {
t.Fatal("IDENTITY.md should not invalidate cache for structured agent definitions")
}
promptV2 := cb.BuildSystemPromptWithCache()
if promptV1 != promptV2 {
t.Fatal("structured prompt should remain stable after IDENTITY.md changes")
}
}
func TestStructuredAgentUserChangesInvalidateCache(t *testing.T) {
tmpDir := setupWorkspace(t, map[string]string{
"AGENT.md": "# Agent\nFollow the new structure.",
"SOUL.md": "# Soul\nVersion one.",
"USER.md": "# User\nInitial workspace preferences.",
})
defer cleanupWorkspace(t, tmpDir)
cb := NewContextBuilder(tmpDir)
promptV1 := cb.BuildSystemPromptWithCache()
if !strings.Contains(promptV1, "Initial workspace preferences") {
t.Fatalf("expected workspace USER.md in prompt, got %q", promptV1)
}
userPath := filepath.Join(tmpDir, "USER.md")
if err := os.WriteFile(userPath, []byte("# User\nUpdated workspace preferences."), 0o644); err != nil {
t.Fatal(err)
}
future := time.Now().Add(2 * time.Second)
if err := os.Chtimes(userPath, future, future); err != nil {
t.Fatal(err)
}
cb.systemPromptMutex.RLock()
changed := cb.sourceFilesChangedLocked()
cb.systemPromptMutex.RUnlock()
if !changed {
t.Fatal("workspace USER.md changes should invalidate cache")
}
promptV2 := cb.BuildSystemPromptWithCache()
if !strings.Contains(promptV2, "Updated workspace preferences") {
t.Fatalf("expected updated workspace USER.md in prompt, got %q", promptV2)
}
}
func cleanupWorkspace(t *testing.T, path string) {
t.Helper()
if err := os.RemoveAll(path); err != nil {
t.Fatalf("failed to clean up workspace %s: %v", path, err)
}
}
pkg/agent/eventbus.go
+121
View File
@@ -0,0 +1,121 @@
package agent
import (
"sync"
"sync/atomic"
"time"
)
const defaultEventSubscriberBuffer = 16
// EventSubscription identifies a subscriber channel returned by EventBus.Subscribe.
type EventSubscription struct {
ID uint64
C <-chan Event
}
type eventSubscriber struct {
ch chan Event
}
// EventBus is a lightweight multi-subscriber broadcaster for agent-loop events.
type EventBus struct {
mu sync.RWMutex
subs map[uint64]eventSubscriber
nextID uint64
closed bool
dropped [eventKindCount]atomic.Int64
}
// NewEventBus creates a new in-process event broadcaster.
func NewEventBus() *EventBus {
return &EventBus{
subs: make(map[uint64]eventSubscriber),
}
}
// Subscribe registers a new subscriber with the requested channel buffer size.
// A non-positive buffer uses the default size.
func (b *EventBus) Subscribe(buffer int) EventSubscription {
if buffer <= 0 {
buffer = defaultEventSubscriberBuffer
}
b.mu.Lock()
defer b.mu.Unlock()
if b.closed {
ch := make(chan Event)
close(ch)
return EventSubscription{C: ch}
}
b.nextID++
id := b.nextID
ch := make(chan Event, buffer)
b.subs[id] = eventSubscriber{ch: ch}
return EventSubscription{ID: id, C: ch}
}
// Unsubscribe removes a subscriber and closes its channel.
func (b *EventBus) Unsubscribe(id uint64) {
b.mu.Lock()
defer b.mu.Unlock()
sub, ok := b.subs[id]
if !ok {
return
}
delete(b.subs, id)
close(sub.ch)
}
// Emit broadcasts an event to all current subscribers without blocking.
// When a subscriber channel is full, the event is dropped for that subscriber.
func (b *EventBus) Emit(evt Event) {
if evt.Time.IsZero() {
evt.Time = time.Now()
}
b.mu.RLock()
defer b.mu.RUnlock()
if b.closed {
return
}
for _, sub := range b.subs {
select {
case sub.ch <- evt:
default:
if evt.Kind < eventKindCount {
b.dropped[evt.Kind].Add(1)
}
}
}
}
// Dropped returns the number of dropped events for a given kind.
func (b *EventBus) Dropped(kind EventKind) int64 {
if kind >= eventKindCount {
return 0
}
return b.dropped[kind].Load()
}
// Close closes all subscriber channels and stops future broadcasts.
func (b *EventBus) Close() {
b.mu.Lock()
defer b.mu.Unlock()
if b.closed {
return
}
b.closed = true
for id, sub := range b.subs {
close(sub.ch)
delete(b.subs, id)
}
}
pkg/agent/eventbus_test.go
+684
View File
@@ -0,0 +1,684 @@
package agent
import (
"context"
"os"
"slices"
"testing"
"time"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/config"
"github.com/sipeed/picoclaw/pkg/providers"
"github.com/sipeed/picoclaw/pkg/tools"
)
func TestEventBus_SubscribeEmitUnsubscribeClose(t *testing.T) {
eventBus := NewEventBus()
sub := eventBus.Subscribe(1)
eventBus.Emit(Event{
Kind: EventKindTurnStart,
Meta: EventMeta{TurnID: "turn-1"},
})
select {
case evt := <-sub.C:
if evt.Kind != EventKindTurnStart {
t.Fatalf("expected %v, got %v", EventKindTurnStart, evt.Kind)
}
if evt.Meta.TurnID != "turn-1" {
t.Fatalf("expected turn id turn-1, got %q", evt.Meta.TurnID)
}
case <-time.After(time.Second):
t.Fatal("timed out waiting for event")
}
eventBus.Unsubscribe(sub.ID)
if _, ok := <-sub.C; ok {
t.Fatal("expected subscriber channel to be closed after unsubscribe")
}
eventBus.Close()
closedSub := eventBus.Subscribe(1)
if _, ok := <-closedSub.C; ok {
t.Fatal("expected closed bus to return a closed subscriber channel")
}
}
func TestEventBus_DropsWhenSubscriberIsFull(t *testing.T) {
eventBus := NewEventBus()
sub := eventBus.Subscribe(1)
defer eventBus.Unsubscribe(sub.ID)
start := time.Now()
for i := 0; i < 1000; i++ {
eventBus.Emit(Event{Kind: EventKindLLMRequest})
}
if elapsed := time.Since(start); elapsed > 100*time.Millisecond {
t.Fatalf("Emit took too long with a blocked subscriber: %s", elapsed)
}
if got := eventBus.Dropped(EventKindLLMRequest); got != 999 {
t.Fatalf("expected 999 dropped events, got %d", got)
}
}
type scriptedToolProvider struct {
calls int
}
func (m *scriptedToolProvider) Chat(
ctx context.Context,
messages []providers.Message,
toolDefs []providers.ToolDefinition,
model string,
opts map[string]any,
) (*providers.LLMResponse, error) {
m.calls++
if m.calls == 1 {
return &providers.LLMResponse{
ToolCalls: []providers.ToolCall{
{
ID: "call-1",
Name: "mock_custom",
Arguments: map[string]any{"task": "ping"},
},
},
}, nil
}
return &providers.LLMResponse{
Content: "done",
}, nil
}
func (m *scriptedToolProvider) GetDefaultModel() string {
return "scripted-tool-model"
}
func TestAgentLoop_EmitsMinimalTurnEvents(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "agent-eventbus-*")
if err != nil {
t.Fatalf("failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
cfg := &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
Workspace: tmpDir,
Model: "test-model",
MaxTokens: 4096,
MaxToolIterations: 10,
},
},
}
msgBus := bus.NewMessageBus()
provider := &scriptedToolProvider{}
al := NewAgentLoop(cfg, msgBus, provider)
al.RegisterTool(&mockCustomTool{})
defaultAgent := al.registry.GetDefaultAgent()
if defaultAgent == nil {
t.Fatal("expected default agent")
}
sub := al.SubscribeEvents(16)
defer al.UnsubscribeEvents(sub.ID)
response, err := al.runAgentLoop(context.Background(), defaultAgent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "run tool",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
if response != "done" {
t.Fatalf("expected final response 'done', got %q", response)
}
events := collectEventStream(sub.C)
if len(events) != 8 {
t.Fatalf("expected 8 events, got %d", len(events))
}
kinds := make([]EventKind, 0, len(events))
for _, evt := range events {
kinds = append(kinds, evt.Kind)
}
expectedKinds := []EventKind{
EventKindTurnStart,
EventKindLLMRequest,
EventKindLLMResponse,
EventKindToolExecStart,
EventKindToolExecEnd,
EventKindLLMRequest,
EventKindLLMResponse,
EventKindTurnEnd,
}
if !slices.Equal(kinds, expectedKinds) {
t.Fatalf("unexpected event sequence: got %v want %v", kinds, expectedKinds)
}
turnID := events[0].Meta.TurnID
for i, evt := range events {
if evt.Meta.TurnID != turnID {
t.Fatalf("event %d has mismatched turn id %q, want %q", i, evt.Meta.TurnID, turnID)
}
if evt.Meta.SessionKey != "session-1" {
t.Fatalf("event %d has session key %q, want session-1", i, evt.Meta.SessionKey)
}
}
startPayload, ok := events[0].Payload.(TurnStartPayload)
if !ok {
t.Fatalf("expected TurnStartPayload, got %T", events[0].Payload)
}
if startPayload.UserMessage != "run tool" {
t.Fatalf("expected user message 'run tool', got %q", startPayload.UserMessage)
}
toolStartPayload, ok := events[3].Payload.(ToolExecStartPayload)
if !ok {
t.Fatalf("expected ToolExecStartPayload, got %T", events[3].Payload)
}
if toolStartPayload.Tool != "mock_custom" {
t.Fatalf("expected tool name mock_custom, got %q", toolStartPayload.Tool)
}
toolEndPayload, ok := events[4].Payload.(ToolExecEndPayload)
if !ok {
t.Fatalf("expected ToolExecEndPayload, got %T", events[4].Payload)
}
if toolEndPayload.Tool != "mock_custom" {
t.Fatalf("expected tool end payload for mock_custom, got %q", toolEndPayload.Tool)
}
if toolEndPayload.IsError {
t.Fatal("expected mock_custom tool to succeed")
}
turnEndPayload, ok := events[len(events)-1].Payload.(TurnEndPayload)
if !ok {
t.Fatalf("expected TurnEndPayload, got %T", events[len(events)-1].Payload)
}
if turnEndPayload.Status != TurnEndStatusCompleted {
t.Fatalf("expected completed turn, got %q", turnEndPayload.Status)
}
if turnEndPayload.Iterations != 2 {
t.Fatalf("expected 2 iterations, got %d", turnEndPayload.Iterations)
}
}
func TestAgentLoop_EmitsSteeringAndSkippedToolEvents(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "agent-eventbus-steering-*")
if err != nil {
t.Fatalf("failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
cfg := &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
Workspace: tmpDir,
Model: "test-model",
MaxTokens: 4096,
MaxToolIterations: 10,
},
},
}
tool1ExecCh := make(chan struct{})
tool1 := &slowTool{name: "tool_one", duration: 50 * time.Millisecond, execCh: tool1ExecCh}
tool2 := &slowTool{name: "tool_two", duration: 50 * time.Millisecond}
provider := &toolCallProvider{
toolCalls: []providers.ToolCall{
{
ID: "call_1",
Type: "function",
Name: "tool_one",
Function: &providers.FunctionCall{
Name: "tool_one",
Arguments: "{}",
},
Arguments: map[string]any{},
},
{
ID: "call_2",
Type: "function",
Name: "tool_two",
Function: &providers.FunctionCall{
Name: "tool_two",
Arguments: "{}",
},
Arguments: map[string]any{},
},
},
finalResp: "steered response",
}
msgBus := bus.NewMessageBus()
al := NewAgentLoop(cfg, msgBus, provider)
al.RegisterTool(tool1)
al.RegisterTool(tool2)
sub := al.SubscribeEvents(32)
defer al.UnsubscribeEvents(sub.ID)
resultCh := make(chan string, 1)
go func() {
resp, _ := al.ProcessDirectWithChannel(context.Background(), "do something", "test-session", "test", "chat1")
resultCh <- resp
}()
select {
case <-tool1ExecCh:
case <-time.After(2 * time.Second):
t.Fatal("timeout waiting for tool_one to start")
}
if err := al.Steer(providers.Message{Role: "user", Content: "change course"}); err != nil {
t.Fatalf("Steer failed: %v", err)
}
select {
case resp := <-resultCh:
if resp != "steered response" {
t.Fatalf("expected steered response, got %q", resp)
}
case <-time.After(5 * time.Second):
t.Fatal("timeout waiting for steered response")
}
events := collectEventStream(sub.C)
steeringEvt, ok := findEvent(events, EventKindSteeringInjected)
if !ok {
t.Fatal("expected steering injected event")
}
steeringPayload, ok := steeringEvt.Payload.(SteeringInjectedPayload)
if !ok {
t.Fatalf("expected SteeringInjectedPayload, got %T", steeringEvt.Payload)
}
if steeringPayload.Count != 1 {
t.Fatalf("expected 1 steering message, got %d", steeringPayload.Count)
}
skippedEvt, ok := findEvent(events, EventKindToolExecSkipped)
if !ok {
t.Fatal("expected skipped tool event")
}
skippedPayload, ok := skippedEvt.Payload.(ToolExecSkippedPayload)
if !ok {
t.Fatalf("expected ToolExecSkippedPayload, got %T", skippedEvt.Payload)
}
if skippedPayload.Tool != "tool_two" {
t.Fatalf("expected skipped tool_two, got %q", skippedPayload.Tool)
}
interruptEvt, ok := findEvent(events, EventKindInterruptReceived)
if !ok {
t.Fatal("expected interrupt received event")
}
interruptPayload, ok := interruptEvt.Payload.(InterruptReceivedPayload)
if !ok {
t.Fatalf("expected InterruptReceivedPayload, got %T", interruptEvt.Payload)
}
if interruptPayload.Role != "user" {
t.Fatalf("expected interrupt role user, got %q", interruptPayload.Role)
}
if interruptPayload.Kind != InterruptKindSteering {
t.Fatalf("expected steering interrupt kind, got %q", interruptPayload.Kind)
}
if interruptPayload.ContentLen != len("change course") {
t.Fatalf("expected interrupt content len %d, got %d", len("change course"), interruptPayload.ContentLen)
}
}
func TestAgentLoop_EmitsContextCompressEventOnRetry(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "agent-eventbus-compress-*")
if err != nil {
t.Fatalf("failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
cfg := &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
Workspace: tmpDir,
Model: "test-model",
MaxTokens: 4096,
MaxToolIterations: 10,
},
},
}
contextErr := stringError("InvalidParameter: Total tokens of image and text exceed max message tokens")
provider := &failFirstMockProvider{
failures: 1,
failError: contextErr,
successResp: "Recovered from context error",
}
msgBus := bus.NewMessageBus()
al := NewAgentLoop(cfg, msgBus, provider)
defaultAgent := al.registry.GetDefaultAgent()
if defaultAgent == nil {
t.Fatal("expected default agent")
}
defaultAgent.Sessions.SetHistory("session-1", []providers.Message{
{Role: "user", Content: "Old message 1"},
{Role: "assistant", Content: "Old response 1"},
{Role: "user", Content: "Old message 2"},
{Role: "assistant", Content: "Old response 2"},
{Role: "user", Content: "Trigger message"},
})
sub := al.SubscribeEvents(16)
defer al.UnsubscribeEvents(sub.ID)
resp, err := al.runAgentLoop(context.Background(), defaultAgent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "Trigger message",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
if resp != "Recovered from context error" {
t.Fatalf("expected retry success, got %q", resp)
}
events := collectEventStream(sub.C)
retryEvt, ok := findEvent(events, EventKindLLMRetry)
if !ok {
t.Fatal("expected llm retry event")
}
retryPayload, ok := retryEvt.Payload.(LLMRetryPayload)
if !ok {
t.Fatalf("expected LLMRetryPayload, got %T", retryEvt.Payload)
}
if retryPayload.Reason != "context_limit" {
t.Fatalf("expected context_limit retry reason, got %q", retryPayload.Reason)
}
if retryPayload.Attempt != 1 {
t.Fatalf("expected retry attempt 1, got %d", retryPayload.Attempt)
}
compressEvt, ok := findEvent(events, EventKindContextCompress)
if !ok {
t.Fatal("expected context compress event")
}
payload, ok := compressEvt.Payload.(ContextCompressPayload)
if !ok {
t.Fatalf("expected ContextCompressPayload, got %T", compressEvt.Payload)
}
if payload.Reason != ContextCompressReasonRetry {
t.Fatalf("expected retry compress reason, got %q", payload.Reason)
}
if payload.DroppedMessages == 0 {
t.Fatal("expected dropped messages to be recorded")
}
}
func TestAgentLoop_EmitsSessionSummarizeEvent(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "agent-eventbus-summary-*")
if err != nil {
t.Fatalf("failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
cfg := &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
Workspace: tmpDir,
Model: "test-model",
MaxTokens: 4096,
MaxToolIterations: 10,
ContextWindow: 8000,
SummarizeMessageThreshold: 2,
SummarizeTokenPercent: 75,
},
},
}
msgBus := bus.NewMessageBus()
al := NewAgentLoop(cfg, msgBus, &simpleMockProvider{response: "summary text"})
defaultAgent := al.registry.GetDefaultAgent()
if defaultAgent == nil {
t.Fatal("expected default agent")
}
defaultAgent.Sessions.SetHistory("session-1", []providers.Message{
{Role: "user", Content: "Question one"},
{Role: "assistant", Content: "Answer one"},
{Role: "user", Content: "Question two"},
{Role: "assistant", Content: "Answer two"},
{Role: "user", Content: "Question three"},
{Role: "assistant", Content: "Answer three"},
})
sub := al.SubscribeEvents(16)
defer al.UnsubscribeEvents(sub.ID)
turnScope := al.newTurnEventScope(defaultAgent.ID, "session-1")
al.summarizeSession(defaultAgent, "session-1", turnScope)
events := collectEventStream(sub.C)
summaryEvt, ok := findEvent(events, EventKindSessionSummarize)
if !ok {
t.Fatal("expected session summarize event")
}
payload, ok := summaryEvt.Payload.(SessionSummarizePayload)
if !ok {
t.Fatalf("expected SessionSummarizePayload, got %T", summaryEvt.Payload)
}
if payload.SummaryLen == 0 {
t.Fatal("expected non-empty summary length")
}
}
func TestAgentLoop_EmitsFollowUpQueuedEvent(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "agent-eventbus-followup-*")
if err != nil {
t.Fatalf("failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
cfg := &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
Workspace: tmpDir,
Model: "test-model",
MaxTokens: 4096,
MaxToolIterations: 10,
},
},
}
provider := &toolCallProvider{
toolCalls: []providers.ToolCall{
{
ID: "call_async_1",
Type: "function",
Name: "async_followup",
Function: &providers.FunctionCall{
Name: "async_followup",
Arguments: "{}",
},
Arguments: map[string]any{},
},
},
finalResp: "async launched",
}
msgBus := bus.NewMessageBus()
al := NewAgentLoop(cfg, msgBus, provider)
doneCh := make(chan struct{})
al.RegisterTool(&asyncFollowUpTool{
name: "async_followup",
followUpText: "background result",
completionSig: doneCh,
})
defaultAgent := al.registry.GetDefaultAgent()
if defaultAgent == nil {
t.Fatal("expected default agent")
}
sub := al.SubscribeEvents(32)
defer al.UnsubscribeEvents(sub.ID)
resp, err := al.runAgentLoop(context.Background(), defaultAgent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "run async tool",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
if resp != "async launched" {
t.Fatalf("expected final response 'async launched', got %q", resp)
}
select {
case <-doneCh:
case <-time.After(2 * time.Second):
t.Fatal("timeout waiting for async tool completion")
}
followUpEvt := waitForEvent(t, sub.C, 2*time.Second, func(evt Event) bool {
return evt.Kind == EventKindFollowUpQueued
})
payload, ok := followUpEvt.Payload.(FollowUpQueuedPayload)
if !ok {
t.Fatalf("expected FollowUpQueuedPayload, got %T", followUpEvt.Payload)
}
if payload.SourceTool != "async_followup" {
t.Fatalf("expected source tool async_followup, got %q", payload.SourceTool)
}
if payload.Channel != "cli" {
t.Fatalf("expected channel cli, got %q", payload.Channel)
}
if payload.ChatID != "direct" {
t.Fatalf("expected chat id direct, got %q", payload.ChatID)
}
if payload.ContentLen != len("background result") {
t.Fatalf("expected content len %d, got %d", len("background result"), payload.ContentLen)
}
if followUpEvt.Meta.SessionKey != "session-1" {
t.Fatalf("expected session key session-1, got %q", followUpEvt.Meta.SessionKey)
}
if followUpEvt.Meta.TurnID == "" {
t.Fatal("expected follow-up event to include turn id")
}
}
func collectEventStream(ch <-chan Event) []Event {
var events []Event
for {
select {
case evt, ok := <-ch:
if !ok {
return events
}
events = append(events, evt)
default:
return events
}
}
}
func waitForEvent(t *testing.T, ch <-chan Event, timeout time.Duration, match func(Event) bool) Event {
t.Helper()
timer := time.NewTimer(timeout)
defer timer.Stop()
for {
select {
case evt, ok := <-ch:
if !ok {
t.Fatal("event stream closed before expected event arrived")
}
if match(evt) {
return evt
}
case <-timer.C:
t.Fatal("timed out waiting for expected event")
}
}
}
func findEvent(events []Event, kind EventKind) (Event, bool) {
for _, evt := range events {
if evt.Kind == kind {
return evt, true
}
}
return Event{}, false
}
type stringError string
func (e stringError) Error() string {
return string(e)
}
type asyncFollowUpTool struct {
name string
followUpText string
completionSig chan struct{}
}
func (t *asyncFollowUpTool) Name() string {
return t.name
}
func (t *asyncFollowUpTool) Description() string {
return "async follow-up tool for testing"
}
func (t *asyncFollowUpTool) Parameters() map[string]any {
return map[string]any{
"type": "object",
"properties": map[string]any{},
}
}
func (t *asyncFollowUpTool) Execute(ctx context.Context, args map[string]any) *tools.ToolResult {
return tools.AsyncResult("async follow-up scheduled")
}
func (t *asyncFollowUpTool) ExecuteAsync(
ctx context.Context,
args map[string]any,
cb tools.AsyncCallback,
) *tools.ToolResult {
go func() {
cb(ctx, &tools.ToolResult{ForLLM: t.followUpText})
if t.completionSig != nil {
close(t.completionSig)
}
}()
return tools.AsyncResult("async follow-up scheduled")
}
var (
_ tools.Tool = (*mockCustomTool)(nil)
_ tools.AsyncExecutor = (*asyncFollowUpTool)(nil)
)
pkg/agent/events.go
+271
View File
@@ -0,0 +1,271 @@
package agent
import (
"fmt"
"time"
)
// EventKind identifies a structured agent-loop event.
type EventKind uint8
const (
// EventKindTurnStart is emitted when a turn begins processing.
EventKindTurnStart EventKind = iota
// EventKindTurnEnd is emitted when a turn finishes, successfully or with an error.
EventKindTurnEnd
// EventKindLLMRequest is emitted before a provider chat request is made.
EventKindLLMRequest
// EventKindLLMDelta is emitted when a streaming provider yields a partial delta.
EventKindLLMDelta
// EventKindLLMResponse is emitted after a provider chat response is received.
EventKindLLMResponse
// EventKindLLMRetry is emitted when an LLM request is retried.
EventKindLLMRetry
// EventKindContextCompress is emitted when session history is forcibly compressed.
EventKindContextCompress
// EventKindSessionSummarize is emitted when asynchronous summarization completes.
EventKindSessionSummarize
// EventKindToolExecStart is emitted immediately before a tool executes.
EventKindToolExecStart
// EventKindToolExecEnd is emitted immediately after a tool finishes executing.
EventKindToolExecEnd
// EventKindToolExecSkipped is emitted when a queued tool call is skipped.
EventKindToolExecSkipped
// EventKindSteeringInjected is emitted when queued steering is injected into context.
EventKindSteeringInjected
// EventKindFollowUpQueued is emitted when an async tool queues a follow-up system message.
EventKindFollowUpQueued
// EventKindInterruptReceived is emitted when a soft interrupt message is accepted.
EventKindInterruptReceived
// EventKindSubTurnSpawn is emitted when a sub-turn is spawned.
EventKindSubTurnSpawn
// EventKindSubTurnEnd is emitted when a sub-turn finishes.
EventKindSubTurnEnd
// EventKindSubTurnResultDelivered is emitted when a sub-turn result is delivered.
EventKindSubTurnResultDelivered
// EventKindSubTurnOrphan is emitted when a sub-turn result cannot be delivered.
EventKindSubTurnOrphan
// EventKindError is emitted when a turn encounters an execution error.
EventKindError
eventKindCount
)
var eventKindNames = [...]string{
"turn_start",
"turn_end",
"llm_request",
"llm_delta",
"llm_response",
"llm_retry",
"context_compress",
"session_summarize",
"tool_exec_start",
"tool_exec_end",
"tool_exec_skipped",
"steering_injected",
"follow_up_queued",
"interrupt_received",
"subturn_spawn",
"subturn_end",
"subturn_result_delivered",
"subturn_orphan",
"error",
}
// String returns the stable string form of an EventKind.
func (k EventKind) String() string {
if k >= eventKindCount {
return fmt.Sprintf("event_kind(%d)", k)
}
return eventKindNames[k]
}
// Event is the structured envelope broadcast by the agent EventBus.
type Event struct {
Kind EventKind
Time time.Time
Meta EventMeta
Payload any
}
// EventMeta contains correlation fields shared by all agent-loop events.
type EventMeta struct {
AgentID string
TurnID string
ParentTurnID string
SessionKey string
Iteration int
TracePath string
Source string
}
// TurnEndStatus describes the terminal state of a turn.
type TurnEndStatus string
const (
// TurnEndStatusCompleted indicates the turn finished normally.
TurnEndStatusCompleted TurnEndStatus = "completed"
// TurnEndStatusError indicates the turn ended because of an error.
TurnEndStatusError TurnEndStatus = "error"
// TurnEndStatusAborted indicates the turn was hard-aborted and rolled back.
TurnEndStatusAborted TurnEndStatus = "aborted"
)
// TurnStartPayload describes the start of a turn.
type TurnStartPayload struct {
Channel string
ChatID string
UserMessage string
MediaCount int
}
// TurnEndPayload describes the completion of a turn.
type TurnEndPayload struct {
Status TurnEndStatus
Iterations int
Duration time.Duration
FinalContentLen int
}
// LLMRequestPayload describes an outbound LLM request.
type LLMRequestPayload struct {
Model string
MessagesCount int
ToolsCount int
MaxTokens int
Temperature float64
}
// LLMResponsePayload describes an inbound LLM response.
type LLMResponsePayload struct {
ContentLen int
ToolCalls int
HasReasoning bool
}
// LLMDeltaPayload describes a streamed LLM delta.
type LLMDeltaPayload struct {
ContentDeltaLen int
ReasoningDeltaLen int
}
// LLMRetryPayload describes a retry of an LLM request.
type LLMRetryPayload struct {
Attempt int
MaxRetries int
Reason string
Error string
Backoff time.Duration
}
// ContextCompressReason identifies why emergency compression ran.
type ContextCompressReason string
const (
// ContextCompressReasonProactive indicates compression before the first LLM call.
ContextCompressReasonProactive ContextCompressReason = "proactive_budget"
// ContextCompressReasonRetry indicates compression during context-error retry handling.
ContextCompressReasonRetry ContextCompressReason = "llm_retry"
)
// ContextCompressPayload describes a forced history compression.
type ContextCompressPayload struct {
Reason ContextCompressReason
DroppedMessages int
RemainingMessages int
}
// SessionSummarizePayload describes a completed async session summarization.
type SessionSummarizePayload struct {
SummarizedMessages int
KeptMessages int
SummaryLen int
OmittedOversized bool
}
// ToolExecStartPayload describes a tool execution request.
type ToolExecStartPayload struct {
Tool string
Arguments map[string]any
}
// ToolExecEndPayload describes the outcome of a tool execution.
type ToolExecEndPayload struct {
Tool string
Duration time.Duration
ForLLMLen int
ForUserLen int
IsError bool
Async bool
}
// ToolExecSkippedPayload describes a skipped tool call.
type ToolExecSkippedPayload struct {
Tool string
Reason string
}
// SteeringInjectedPayload describes steering messages appended before the next LLM call.
type SteeringInjectedPayload struct {
Count int
TotalContentLen int
}
// FollowUpQueuedPayload describes an async follow-up queued back into the inbound bus.
type FollowUpQueuedPayload struct {
SourceTool string
Channel string
ChatID string
ContentLen int
}
type InterruptKind string
const (
InterruptKindSteering InterruptKind = "steering"
InterruptKindGraceful InterruptKind = "graceful"
InterruptKindHard InterruptKind = "hard_abort"
)
// InterruptReceivedPayload describes accepted turn-control input.
type InterruptReceivedPayload struct {
Kind InterruptKind
Role string
ContentLen int
QueueDepth int
HintLen int
}
// SubTurnSpawnPayload describes the creation of a child turn.
type SubTurnSpawnPayload struct {
AgentID string
Label string
ParentTurnID string
}
// SubTurnEndPayload describes the completion of a child turn.
type SubTurnEndPayload struct {
AgentID string
Status string
}
// SubTurnResultDeliveredPayload describes delivery of a sub-turn result.
type SubTurnResultDeliveredPayload struct {
TargetChannel string
TargetChatID string
ContentLen int
}
// SubTurnOrphanPayload describes a sub-turn result that could not be delivered.
type SubTurnOrphanPayload struct {
ParentTurnID string
ChildTurnID string
Reason string
}
// ErrorPayload describes an execution error inside the agent loop.
type ErrorPayload struct {
Stage string
Message string
}
pkg/agent/hook_mount.go
+317
View File
@@ -0,0 +1,317 @@
package agent
import (
"context"
"fmt"
"sort"
"sync"
"time"
"github.com/sipeed/picoclaw/pkg/config"
)
type hookRuntime struct {
initOnce sync.Once
mu sync.Mutex
initErr error
mounted []string
}
func (r *hookRuntime) setInitErr(err error) {
r.mu.Lock()
r.initErr = err
r.mu.Unlock()
}
func (r *hookRuntime) getInitErr() error {
r.mu.Lock()
defer r.mu.Unlock()
return r.initErr
}
func (r *hookRuntime) setMounted(names []string) {
r.mu.Lock()
r.mounted = append([]string(nil), names...)
r.mu.Unlock()
}
func (r *hookRuntime) reset(al *AgentLoop) {
r.mu.Lock()
names := append([]string(nil), r.mounted...)
r.mounted = nil
r.initErr = nil
r.initOnce = sync.Once{}
r.mu.Unlock()
for _, name := range names {
al.UnmountHook(name)
}
}
// BuiltinHookFactory constructs an in-process hook from config.
type BuiltinHookFactory func(ctx context.Context, spec config.BuiltinHookConfig) (any, error)
var (
builtinHookRegistryMu sync.RWMutex
builtinHookRegistry = map[string]BuiltinHookFactory{}
)
// RegisterBuiltinHook registers a named in-process hook factory for config-driven mounting.
func RegisterBuiltinHook(name string, factory BuiltinHookFactory) error {
if name == "" {
return fmt.Errorf("builtin hook name is required")
}
if factory == nil {
return fmt.Errorf("builtin hook %q factory is nil", name)
}
builtinHookRegistryMu.Lock()
defer builtinHookRegistryMu.Unlock()
if _, exists := builtinHookRegistry[name]; exists {
return fmt.Errorf("builtin hook %q is already registered", name)
}
builtinHookRegistry[name] = factory
return nil
}
func unregisterBuiltinHook(name string) {
if name == "" {
return
}
builtinHookRegistryMu.Lock()
delete(builtinHookRegistry, name)
builtinHookRegistryMu.Unlock()
}
func lookupBuiltinHook(name string) (BuiltinHookFactory, bool) {
builtinHookRegistryMu.RLock()
defer builtinHookRegistryMu.RUnlock()
factory, ok := builtinHookRegistry[name]
return factory, ok
}
func configureHookManagerFromConfig(hm *HookManager, cfg *config.Config) {
if hm == nil || cfg == nil {
return
}
hm.ConfigureTimeouts(
hookTimeoutFromMS(cfg.Hooks.Defaults.ObserverTimeoutMS),
hookTimeoutFromMS(cfg.Hooks.Defaults.InterceptorTimeoutMS),
hookTimeoutFromMS(cfg.Hooks.Defaults.ApprovalTimeoutMS),
)
}
func hookTimeoutFromMS(ms int) time.Duration {
if ms <= 0 {
return 0
}
return time.Duration(ms) * time.Millisecond
}
func (al *AgentLoop) ensureHooksInitialized(ctx context.Context) error {
if al == nil || al.cfg == nil || al.hooks == nil {
return nil
}
al.hookRuntime.initOnce.Do(func() {
al.hookRuntime.setInitErr(al.loadConfiguredHooks(ctx))
})
return al.hookRuntime.getInitErr()
}
func (al *AgentLoop) loadConfiguredHooks(ctx context.Context) (err error) {
if al == nil || al.cfg == nil || !al.cfg.Hooks.Enabled {
return nil
}
mounted := make([]string, 0)
defer func() {
if err != nil {
for _, name := range mounted {
al.UnmountHook(name)
}
return
}
al.hookRuntime.setMounted(mounted)
}()
builtinNames := enabledBuiltinHookNames(al.cfg.Hooks.Builtins)
for _, name := range builtinNames {
spec := al.cfg.Hooks.Builtins[name]
factory, ok := lookupBuiltinHook(name)
if !ok {
return fmt.Errorf("builtin hook %q is not registered", name)
}
hook, factoryErr := factory(ctx, spec)
if factoryErr != nil {
return fmt.Errorf("build builtin hook %q: %w", name, factoryErr)
}
if err := al.MountHook(HookRegistration{
Name: name,
Priority: spec.Priority,
Source: HookSourceInProcess,
Hook: hook,
}); err != nil {
return fmt.Errorf("mount builtin hook %q: %w", name, err)
}
mounted = append(mounted, name)
}
processNames := enabledProcessHookNames(al.cfg.Hooks.Processes)
for _, name := range processNames {
spec := al.cfg.Hooks.Processes[name]
opts, buildErr := processHookOptionsFromConfig(spec)
if buildErr != nil {
return fmt.Errorf("configure process hook %q: %w", name, buildErr)
}
processHook, buildErr := NewProcessHook(ctx, name, opts)
if buildErr != nil {
return fmt.Errorf("start process hook %q: %w", name, buildErr)
}
if err := al.MountHook(HookRegistration{
Name: name,
Priority: spec.Priority,
Source: HookSourceProcess,
Hook: processHook,
}); err != nil {
_ = processHook.Close()
return fmt.Errorf("mount process hook %q: %w", name, err)
}
mounted = append(mounted, name)
}
return nil
}
func enabledBuiltinHookNames(specs map[string]config.BuiltinHookConfig) []string {
if len(specs) == 0 {
return nil
}
names := make([]string, 0, len(specs))
for name, spec := range specs {
if spec.Enabled {
names = append(names, name)
}
}
sort.Strings(names)
return names
}
func enabledProcessHookNames(specs map[string]config.ProcessHookConfig) []string {
if len(specs) == 0 {
return nil
}
names := make([]string, 0, len(specs))
for name, spec := range specs {
if spec.Enabled {
names = append(names, name)
}
}
sort.Strings(names)
return names
}
func processHookOptionsFromConfig(spec config.ProcessHookConfig) (ProcessHookOptions, error) {
transport := spec.Transport
if transport == "" {
transport = "stdio"
}
if transport != "stdio" {
return ProcessHookOptions{}, fmt.Errorf("unsupported transport %q", transport)
}
if len(spec.Command) == 0 {
return ProcessHookOptions{}, fmt.Errorf("command is required")
}
opts := ProcessHookOptions{
Command: append([]string(nil), spec.Command...),
Dir: spec.Dir,
Env: processHookEnvFromMap(spec.Env),
}
observeKinds, observeEnabled, err := processHookObserveKindsFromConfig(spec.Observe)
if err != nil {
return ProcessHookOptions{}, err
}
opts.Observe = observeEnabled
opts.ObserveKinds = observeKinds
for _, intercept := range spec.Intercept {
switch intercept {
case "before_llm", "after_llm":
opts.InterceptLLM = true
case "before_tool", "after_tool":
opts.InterceptTool = true
case "approve_tool":
opts.ApproveTool = true
case "":
continue
default:
return ProcessHookOptions{}, fmt.Errorf("unsupported intercept %q", intercept)
}
}
if !opts.Observe && !opts.InterceptLLM && !opts.InterceptTool && !opts.ApproveTool {
return ProcessHookOptions{}, fmt.Errorf("no hook modes enabled")
}
return opts, nil
}
func processHookEnvFromMap(envMap map[string]string) []string {
if len(envMap) == 0 {
return nil
}
keys := make([]string, 0, len(envMap))
for key := range envMap {
keys = append(keys, key)
}
sort.Strings(keys)
env := make([]string, 0, len(keys))
for _, key := range keys {
env = append(env, key+"="+envMap[key])
}
return env
}
func processHookObserveKindsFromConfig(observe []string) ([]string, bool, error) {
if len(observe) == 0 {
return nil, false, nil
}
validKinds := validHookEventKinds()
normalized := make([]string, 0, len(observe))
for _, kind := range observe {
switch kind {
case "", "*", "all":
return nil, true, nil
default:
if _, ok := validKinds[kind]; !ok {
return nil, false, fmt.Errorf("unsupported observe event %q", kind)
}
normalized = append(normalized, kind)
}
}
if len(normalized) == 0 {
return nil, false, nil
}
return normalized, true, nil
}
func validHookEventKinds() map[string]struct{} {
kinds := make(map[string]struct{}, int(eventKindCount))
for kind := EventKind(0); kind < eventKindCount; kind++ {
kinds[kind.String()] = struct{}{}
}
return kinds
}
pkg/agent/hook_mount_test.go
+179
View File
@@ -0,0 +1,179 @@
package agent
import (
"context"
"encoding/json"
"path/filepath"
"testing"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/config"
)
type builtinAutoHookConfig struct {
Model string `json:"model"`
Suffix string `json:"suffix"`
}
type builtinAutoHook struct {
model string
suffix string
}
func (h *builtinAutoHook) BeforeLLM(
ctx context.Context,
req *LLMHookRequest,
) (*LLMHookRequest, HookDecision, error) {
next := req.Clone()
next.Model = h.model
return next, HookDecision{Action: HookActionModify}, nil
}
func (h *builtinAutoHook) AfterLLM(
ctx context.Context,
resp *LLMHookResponse,
) (*LLMHookResponse, HookDecision, error) {
next := resp.Clone()
if next.Response != nil {
next.Response.Content += h.suffix
}
return next, HookDecision{Action: HookActionModify}, nil
}
func newConfiguredHookLoop(t *testing.T, provider *llmHookTestProvider, hooks config.HooksConfig) *AgentLoop {
t.Helper()
cfg := &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
Workspace: t.TempDir(),
Model: "test-model",
MaxTokens: 4096,
MaxToolIterations: 10,
},
},
Hooks: hooks,
}
return NewAgentLoop(cfg, bus.NewMessageBus(), provider)
}
func TestAgentLoop_ProcessDirectWithChannel_AutoMountsBuiltinHook(t *testing.T) {
const hookName = "test-auto-builtin-hook"
if err := RegisterBuiltinHook(hookName, func(
ctx context.Context,
spec config.BuiltinHookConfig,
) (any, error) {
var hookCfg builtinAutoHookConfig
if len(spec.Config) > 0 {
if err := json.Unmarshal(spec.Config, &hookCfg); err != nil {
return nil, err
}
}
return &builtinAutoHook{
model: hookCfg.Model,
suffix: hookCfg.Suffix,
}, nil
}); err != nil {
t.Fatalf("RegisterBuiltinHook failed: %v", err)
}
t.Cleanup(func() {
unregisterBuiltinHook(hookName)
})
rawCfg, err := json.Marshal(builtinAutoHookConfig{
Model: "builtin-model",
Suffix: "|builtin",
})
if err != nil {
t.Fatalf("json.Marshal failed: %v", err)
}
provider := &llmHookTestProvider{}
al := newConfiguredHookLoop(t, provider, config.HooksConfig{
Enabled: true,
Builtins: map[string]config.BuiltinHookConfig{
hookName: {
Enabled: true,
Config: rawCfg,
},
},
})
defer al.Close()
resp, err := al.ProcessDirectWithChannel(context.Background(), "hello", "session-1", "cli", "direct")
if err != nil {
t.Fatalf("ProcessDirectWithChannel failed: %v", err)
}
if resp != "provider content|builtin" {
t.Fatalf("expected builtin-hooked content, got %q", resp)
}
provider.mu.Lock()
lastModel := provider.lastModel
provider.mu.Unlock()
if lastModel != "builtin-model" {
t.Fatalf("expected builtin model, got %q", lastModel)
}
}
func TestAgentLoop_ProcessDirectWithChannel_AutoMountsProcessHook(t *testing.T) {
provider := &llmHookTestProvider{}
eventLog := filepath.Join(t.TempDir(), "events.log")
al := newConfiguredHookLoop(t, provider, config.HooksConfig{
Enabled: true,
Processes: map[string]config.ProcessHookConfig{
"ipc-auto": {
Enabled: true,
Command: processHookHelperCommand(),
Env: map[string]string{
"PICOCLAW_HOOK_HELPER": "1",
"PICOCLAW_HOOK_MODE": "rewrite",
"PICOCLAW_HOOK_EVENT_LOG": eventLog,
},
Observe: []string{"turn_end"},
Intercept: []string{"before_llm", "after_llm"},
},
},
})
defer al.Close()
resp, err := al.ProcessDirectWithChannel(context.Background(), "hello", "session-1", "cli", "direct")
if err != nil {
t.Fatalf("ProcessDirectWithChannel failed: %v", err)
}
if resp != "provider content|ipc" {
t.Fatalf("expected process-hooked content, got %q", resp)
}
provider.mu.Lock()
lastModel := provider.lastModel
provider.mu.Unlock()
if lastModel != "process-model" {
t.Fatalf("expected process model, got %q", lastModel)
}
waitForFileContains(t, eventLog, "turn_end")
}
func TestAgentLoop_ProcessDirectWithChannel_InvalidConfiguredHookFails(t *testing.T) {
provider := &llmHookTestProvider{}
al := newConfiguredHookLoop(t, provider, config.HooksConfig{
Enabled: true,
Processes: map[string]config.ProcessHookConfig{
"bad-hook": {
Enabled: true,
Command: processHookHelperCommand(),
Intercept: []string{"not_supported"},
},
},
})
defer al.Close()
_, err := al.ProcessDirectWithChannel(context.Background(), "hello", "session-1", "cli", "direct")
if err == nil {
t.Fatal("expected invalid configured hook error")
}
}
pkg/agent/hook_process.go
+511
View File
@@ -0,0 +1,511 @@
package agent
import (
"bufio"
"context"
"encoding/json"
"fmt"
"io"
"os"
"os/exec"
"sync"
"sync/atomic"
"time"
"github.com/sipeed/picoclaw/pkg/logger"
)
const (
processHookJSONRPCVersion = "2.0"
processHookReadBufferSize = 1024 * 1024
processHookCloseTimeout = 2 * time.Second
)
type ProcessHookOptions struct {
Command []string
Dir string
Env []string
Observe bool
ObserveKinds []string
InterceptLLM bool
InterceptTool bool
ApproveTool bool
}
type ProcessHook struct {
name string
opts ProcessHookOptions
cmd *exec.Cmd
stdin io.WriteCloser
observeKinds map[string]struct{}
writeMu sync.Mutex
pendingMu sync.Mutex
pending map[uint64]chan processHookRPCMessage
nextID atomic.Uint64
closed atomic.Bool
done chan struct{}
closeErr error
closeMu sync.Mutex
closeOnce sync.Once
}
type processHookRPCMessage struct {
JSONRPC string `json:"jsonrpc,omitempty"`
ID uint64 `json:"id,omitempty"`
Method string `json:"method,omitempty"`
Params json.RawMessage `json:"params,omitempty"`
Result json.RawMessage `json:"result,omitempty"`
Error *processHookRPCError `json:"error,omitempty"`
}
type processHookRPCError struct {
Code int `json:"code"`
Message string `json:"message"`
}
type processHookHelloParams struct {
Name string `json:"name"`
Version int `json:"version"`
Modes []string `json:"modes,omitempty"`
}
type processHookDecisionResponse struct {
Action HookAction `json:"action"`
Reason string `json:"reason,omitempty"`
}
type processHookBeforeLLMResponse struct {
processHookDecisionResponse
Request *LLMHookRequest `json:"request,omitempty"`
}
type processHookAfterLLMResponse struct {
processHookDecisionResponse
Response *LLMHookResponse `json:"response,omitempty"`
}
type processHookBeforeToolResponse struct {
processHookDecisionResponse
Call *ToolCallHookRequest `json:"call,omitempty"`
}
type processHookAfterToolResponse struct {
processHookDecisionResponse
Result *ToolResultHookResponse `json:"result,omitempty"`
}
func NewProcessHook(ctx context.Context, name string, opts ProcessHookOptions) (*ProcessHook, error) {
if len(opts.Command) == 0 {
return nil, fmt.Errorf("process hook command is required")
}
cmd := exec.Command(opts.Command[0], opts.Command[1:]...)
cmd.Dir = opts.Dir
if len(opts.Env) > 0 {
cmd.Env = append(os.Environ(), opts.Env...)
}
stdin, err := cmd.StdinPipe()
if err != nil {
return nil, fmt.Errorf("create process hook stdin: %w", err)
}
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("create process hook stdout: %w", err)
}
stderr, err := cmd.StderrPipe()
if err != nil {
return nil, fmt.Errorf("create process hook stderr: %w", err)
}
if err := cmd.Start(); err != nil {
return nil, fmt.Errorf("start process hook: %w", err)
}
ph := &ProcessHook{
name: name,
opts: opts,
cmd: cmd,
stdin: stdin,
observeKinds: newProcessHookObserveKinds(opts.ObserveKinds),
pending: make(map[uint64]chan processHookRPCMessage),
done: make(chan struct{}),
}
go ph.readLoop(stdout)
go ph.readStderr(stderr)
go ph.waitLoop()
helloCtx := ctx
if helloCtx == nil {
var cancel context.CancelFunc
helloCtx, cancel = context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
}
if err := ph.hello(helloCtx); err != nil {
_ = ph.Close()
return nil, err
}
return ph, nil
}
func (ph *ProcessHook) Close() error {
if ph == nil {
return nil
}
ph.closeOnce.Do(func() {
ph.closed.Store(true)
if ph.stdin != nil {
_ = ph.stdin.Close()
}
select {
case <-ph.done:
case <-time.After(processHookCloseTimeout):
if ph.cmd != nil && ph.cmd.Process != nil {
_ = ph.cmd.Process.Kill()
}
<-ph.done
}
})
ph.closeMu.Lock()
defer ph.closeMu.Unlock()
return ph.closeErr
}
func (ph *ProcessHook) OnEvent(ctx context.Context, evt Event) error {
if ph == nil || !ph.opts.Observe {
return nil
}
if len(ph.observeKinds) > 0 {
if _, ok := ph.observeKinds[evt.Kind.String()]; !ok {
return nil
}
}
return ph.notify(ctx, "hook.event", evt)
}
func (ph *ProcessHook) BeforeLLM(
ctx context.Context,
req *LLMHookRequest,
) (*LLMHookRequest, HookDecision, error) {
if ph == nil || !ph.opts.InterceptLLM {
return req, HookDecision{Action: HookActionContinue}, nil
}
var resp processHookBeforeLLMResponse
if err := ph.call(ctx, "hook.before_llm", req, &resp); err != nil {
return nil, HookDecision{}, err
}
if resp.Request == nil {
resp.Request = req
}
return resp.Request, HookDecision{Action: resp.Action, Reason: resp.Reason}, nil
}
func (ph *ProcessHook) AfterLLM(
ctx context.Context,
resp *LLMHookResponse,
) (*LLMHookResponse, HookDecision, error) {
if ph == nil || !ph.opts.InterceptLLM {
return resp, HookDecision{Action: HookActionContinue}, nil
}
var result processHookAfterLLMResponse
if err := ph.call(ctx, "hook.after_llm", resp, &result); err != nil {
return nil, HookDecision{}, err
}
if result.Response == nil {
result.Response = resp
}
return result.Response, HookDecision{Action: result.Action, Reason: result.Reason}, nil
}
func (ph *ProcessHook) BeforeTool(
ctx context.Context,
call *ToolCallHookRequest,
) (*ToolCallHookRequest, HookDecision, error) {
if ph == nil || !ph.opts.InterceptTool {
return call, HookDecision{Action: HookActionContinue}, nil
}
var resp processHookBeforeToolResponse
if err := ph.call(ctx, "hook.before_tool", call, &resp); err != nil {
return nil, HookDecision{}, err
}
if resp.Call == nil {
resp.Call = call
}
return resp.Call, HookDecision{Action: resp.Action, Reason: resp.Reason}, nil
}
func (ph *ProcessHook) AfterTool(
ctx context.Context,
result *ToolResultHookResponse,
) (*ToolResultHookResponse, HookDecision, error) {
if ph == nil || !ph.opts.InterceptTool {
return result, HookDecision{Action: HookActionContinue}, nil
}
var resp processHookAfterToolResponse
if err := ph.call(ctx, "hook.after_tool", result, &resp); err != nil {
return nil, HookDecision{}, err
}
if resp.Result == nil {
resp.Result = result
}
return resp.Result, HookDecision{Action: resp.Action, Reason: resp.Reason}, nil
}
func (ph *ProcessHook) ApproveTool(ctx context.Context, req *ToolApprovalRequest) (ApprovalDecision, error) {
if ph == nil || !ph.opts.ApproveTool {
return ApprovalDecision{Approved: true}, nil
}
var resp ApprovalDecision
if err := ph.call(ctx, "hook.approve_tool", req, &resp); err != nil {
return ApprovalDecision{}, err
}
return resp, nil
}
func (ph *ProcessHook) hello(ctx context.Context) error {
modes := make([]string, 0, 4)
if ph.opts.Observe {
modes = append(modes, "observe")
}
if ph.opts.InterceptLLM {
modes = append(modes, "llm")
}
if ph.opts.InterceptTool {
modes = append(modes, "tool")
}
if ph.opts.ApproveTool {
modes = append(modes, "approve")
}
var result map[string]any
return ph.call(ctx, "hook.hello", processHookHelloParams{
Name: ph.name,
Version: 1,
Modes: modes,
}, &result)
}
func (ph *ProcessHook) notify(ctx context.Context, method string, params any) error {
msg := processHookRPCMessage{
JSONRPC: processHookJSONRPCVersion,
Method: method,
}
if params != nil {
body, err := json.Marshal(params)
if err != nil {
return err
}
msg.Params = body
}
return ph.send(ctx, msg)
}
func (ph *ProcessHook) call(ctx context.Context, method string, params any, out any) error {
if ph.closed.Load() {
return fmt.Errorf("process hook %q is closed", ph.name)
}
id := ph.nextID.Add(1)
respCh := make(chan processHookRPCMessage, 1)
ph.pendingMu.Lock()
ph.pending[id] = respCh
ph.pendingMu.Unlock()
msg := processHookRPCMessage{
JSONRPC: processHookJSONRPCVersion,
ID: id,
Method: method,
}
if params != nil {
body, err := json.Marshal(params)
if err != nil {
ph.removePending(id)
return err
}
msg.Params = body
}
if err := ph.send(ctx, msg); err != nil {
ph.removePending(id)
return err
}
select {
case resp, ok := <-respCh:
if !ok {
return fmt.Errorf("process hook %q closed while waiting for %s", ph.name, method)
}
if resp.Error != nil {
return fmt.Errorf("process hook %q %s failed: %s", ph.name, method, resp.Error.Message)
}
if out != nil && len(resp.Result) > 0 {
if err := json.Unmarshal(resp.Result, out); err != nil {
return fmt.Errorf("decode process hook %q %s result: %w", ph.name, method, err)
}
}
return nil
case <-ctx.Done():
ph.removePending(id)
return ctx.Err()
}
}
func (ph *ProcessHook) send(ctx context.Context, msg processHookRPCMessage) error {
body, err := json.Marshal(msg)
if err != nil {
return err
}
body = append(body, '\n')
ph.writeMu.Lock()
defer ph.writeMu.Unlock()
if ph.closed.Load() {
return fmt.Errorf("process hook %q is closed", ph.name)
}
done := make(chan error, 1)
go func() {
_, writeErr := ph.stdin.Write(body)
done <- writeErr
}()
select {
case err := <-done:
if err != nil {
return fmt.Errorf("write process hook %q message: %w", ph.name, err)
}
return nil
case <-ctx.Done():
return ctx.Err()
}
}
func (ph *ProcessHook) readLoop(stdout io.Reader) {
scanner := bufio.NewScanner(stdout)
scanner.Buffer(make([]byte, 0, 64*1024), processHookReadBufferSize)
for scanner.Scan() {
var msg processHookRPCMessage
if err := json.Unmarshal(scanner.Bytes(), &msg); err != nil {
logger.WarnCF("hooks", "Failed to decode process hook message", map[string]any{
"hook": ph.name,
"error": err.Error(),
})
continue
}
if msg.ID == 0 {
continue
}
ph.pendingMu.Lock()
respCh, ok := ph.pending[msg.ID]
if ok {
delete(ph.pending, msg.ID)
}
ph.pendingMu.Unlock()
if ok {
respCh <- msg
close(respCh)
}
}
}
func (ph *ProcessHook) readStderr(stderr io.Reader) {
scanner := bufio.NewScanner(stderr)
scanner.Buffer(make([]byte, 0, 16*1024), processHookReadBufferSize)
for scanner.Scan() {
logger.WarnCF("hooks", "Process hook stderr", map[string]any{
"hook": ph.name,
"stderr": scanner.Text(),
})
}
}
func (ph *ProcessHook) waitLoop() {
err := ph.cmd.Wait()
ph.closeMu.Lock()
ph.closeErr = err
ph.closeMu.Unlock()
ph.failPending(err)
close(ph.done)
}
func (ph *ProcessHook) failPending(err error) {
ph.pendingMu.Lock()
defer ph.pendingMu.Unlock()
msg := processHookRPCMessage{
Error: &processHookRPCError{
Code: -32000,
Message: "process exited",
},
}
if err != nil {
msg.Error.Message = err.Error()
}
for id, ch := range ph.pending {
delete(ph.pending, id)
ch <- msg
close(ch)
}
}
func (ph *ProcessHook) removePending(id uint64) {
ph.pendingMu.Lock()
defer ph.pendingMu.Unlock()
if ch, ok := ph.pending[id]; ok {
delete(ph.pending, id)
close(ch)
}
}
func (al *AgentLoop) MountProcessHook(ctx context.Context, name string, opts ProcessHookOptions) error {
if al == nil {
return fmt.Errorf("agent loop is nil")
}
processHook, err := NewProcessHook(ctx, name, opts)
if err != nil {
return err
}
if err := al.MountHook(HookRegistration{
Name: name,
Source: HookSourceProcess,
Hook: processHook,
}); err != nil {
_ = processHook.Close()
return err
}
return nil
}
func newProcessHookObserveKinds(kinds []string) map[string]struct{} {
if len(kinds) == 0 {
return nil
}
normalized := make(map[string]struct{}, len(kinds))
for _, kind := range kinds {
if kind == "" {
continue
}
normalized[kind] = struct{}{}
}
if len(normalized) == 0 {
return nil
}
return normalized
}
pkg/agent/hook_process_test.go
+339
View File
@@ -0,0 +1,339 @@
package agent
import (
"bufio"
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/sipeed/picoclaw/pkg/providers"
)
func TestProcessHook_HelperProcess(t *testing.T) {
if os.Getenv("PICOCLAW_HOOK_HELPER") != "1" {
return
}
if err := runProcessHookHelper(); err != nil {
fmt.Fprintln(os.Stderr, err.Error())
os.Exit(1)
}
os.Exit(0)
}
func TestAgentLoop_MountProcessHook_LLMAndObserver(t *testing.T) {
provider := &llmHookTestProvider{}
al, agent, cleanup := newHookTestLoop(t, provider)
defer cleanup()
eventLog := filepath.Join(t.TempDir(), "events.log")
if err := al.MountProcessHook(context.Background(), "ipc-llm", ProcessHookOptions{
Command: processHookHelperCommand(),
Env: processHookHelperEnv("rewrite", eventLog),
Observe: true,
InterceptLLM: true,
}); err != nil {
t.Fatalf("MountProcessHook failed: %v", err)
}
resp, err := al.runAgentLoop(context.Background(), agent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "hello",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
if resp != "provider content|ipc" {
t.Fatalf("expected process-hooked llm content, got %q", resp)
}
provider.mu.Lock()
lastModel := provider.lastModel
provider.mu.Unlock()
if lastModel != "process-model" {
t.Fatalf("expected process model, got %q", lastModel)
}
waitForFileContains(t, eventLog, "turn_end")
}
func TestAgentLoop_MountProcessHook_ToolRewrite(t *testing.T) {
provider := &toolHookProvider{}
al, agent, cleanup := newHookTestLoop(t, provider)
defer cleanup()
al.RegisterTool(&echoTextTool{})
if err := al.MountProcessHook(context.Background(), "ipc-tool", ProcessHookOptions{
Command: processHookHelperCommand(),
Env: processHookHelperEnv("rewrite", ""),
InterceptTool: true,
}); err != nil {
t.Fatalf("MountProcessHook failed: %v", err)
}
resp, err := al.runAgentLoop(context.Background(), agent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "run tool",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
if resp != "ipc:ipc" {
t.Fatalf("expected rewritten process-hook tool result, got %q", resp)
}
}
type blockedToolProvider struct {
calls int
}
func (p *blockedToolProvider) Chat(
ctx context.Context,
messages []providers.Message,
tools []providers.ToolDefinition,
model string,
opts map[string]any,
) (*providers.LLMResponse, error) {
p.calls++
if p.calls == 1 {
return &providers.LLMResponse{
ToolCalls: []providers.ToolCall{
{
ID: "call-1",
Name: "blocked_tool",
Arguments: map[string]any{},
},
},
}, nil
}
return &providers.LLMResponse{
Content: messages[len(messages)-1].Content,
}, nil
}
func (p *blockedToolProvider) GetDefaultModel() string {
return "blocked-tool-provider"
}
func TestAgentLoop_MountProcessHook_ApprovalDeny(t *testing.T) {
provider := &blockedToolProvider{}
al, agent, cleanup := newHookTestLoop(t, provider)
defer cleanup()
if err := al.MountProcessHook(context.Background(), "ipc-approval", ProcessHookOptions{
Command: processHookHelperCommand(),
Env: processHookHelperEnv("deny", ""),
ApproveTool: true,
}); err != nil {
t.Fatalf("MountProcessHook failed: %v", err)
}
sub := al.SubscribeEvents(16)
defer al.UnsubscribeEvents(sub.ID)
resp, err := al.runAgentLoop(context.Background(), agent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "run blocked tool",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
expected := "Tool execution denied by approval hook: blocked by ipc hook"
if resp != expected {
t.Fatalf("expected %q, got %q", expected, resp)
}
events := collectEventStream(sub.C)
skippedEvt, ok := findEvent(events, EventKindToolExecSkipped)
if !ok {
t.Fatal("expected tool skipped event")
}
payload, ok := skippedEvt.Payload.(ToolExecSkippedPayload)
if !ok {
t.Fatalf("expected ToolExecSkippedPayload, got %T", skippedEvt.Payload)
}
if payload.Reason != expected {
t.Fatalf("expected reason %q, got %q", expected, payload.Reason)
}
}
func processHookHelperCommand() []string {
return []string{os.Args[0], "-test.run=TestProcessHook_HelperProcess", "--"}
}
func processHookHelperEnv(mode, eventLog string) []string {
env := []string{
"PICOCLAW_HOOK_HELPER=1",
"PICOCLAW_HOOK_MODE=" + mode,
}
if eventLog != "" {
env = append(env, "PICOCLAW_HOOK_EVENT_LOG="+eventLog)
}
return env
}
func waitForFileContains(t *testing.T, path, substring string) {
t.Helper()
deadline := time.Now().Add(3 * time.Second)
for time.Now().Before(deadline) {
data, err := os.ReadFile(path)
if err == nil && strings.Contains(string(data), substring) {
return
}
time.Sleep(20 * time.Millisecond)
}
data, _ := os.ReadFile(path)
t.Fatalf("timed out waiting for %q in %s; current content: %q", substring, path, string(data))
}
func runProcessHookHelper() error {
mode := os.Getenv("PICOCLAW_HOOK_MODE")
eventLog := os.Getenv("PICOCLAW_HOOK_EVENT_LOG")
scanner := bufio.NewScanner(os.Stdin)
scanner.Buffer(make([]byte, 0, 64*1024), processHookReadBufferSize)
encoder := json.NewEncoder(os.Stdout)
for scanner.Scan() {
var msg processHookRPCMessage
if err := json.Unmarshal(scanner.Bytes(), &msg); err != nil {
return err
}
if msg.ID == 0 {
if msg.Method == "hook.event" && eventLog != "" {
var evt map[string]any
if err := json.Unmarshal(msg.Params, &evt); err == nil {
if rawKind, ok := evt["Kind"].(float64); ok {
kind := EventKind(rawKind)
_ = os.WriteFile(eventLog, []byte(kind.String()+"\n"), 0o644)
}
}
}
continue
}
result, rpcErr := handleProcessHookRequest(mode, msg)
resp := processHookRPCMessage{
JSONRPC: processHookJSONRPCVersion,
ID: msg.ID,
}
if rpcErr != nil {
resp.Error = rpcErr
} else if result != nil {
body, err := json.Marshal(result)
if err != nil {
return err
}
resp.Result = body
} else {
resp.Result = []byte("{}")
}
if err := encoder.Encode(resp); err != nil {
return err
}
}
return scanner.Err()
}
func handleProcessHookRequest(mode string, msg processHookRPCMessage) (any, *processHookRPCError) {
switch msg.Method {
case "hook.hello":
return map[string]any{"ok": true}, nil
case "hook.before_llm":
if mode != "rewrite" {
return map[string]any{"action": HookActionContinue}, nil
}
var req map[string]any
_ = json.Unmarshal(msg.Params, &req)
req["model"] = "process-model"
return map[string]any{
"action": HookActionModify,
"request": req,
}, nil
case "hook.after_llm":
if mode != "rewrite" {
return map[string]any{"action": HookActionContinue}, nil
}
var resp map[string]any
_ = json.Unmarshal(msg.Params, &resp)
if rawResponse, ok := resp["response"].(map[string]any); ok {
if content, ok := rawResponse["content"].(string); ok {
rawResponse["content"] = content + "|ipc"
}
}
return map[string]any{
"action": HookActionModify,
"response": resp,
}, nil
case "hook.before_tool":
if mode != "rewrite" {
return map[string]any{"action": HookActionContinue}, nil
}
var call map[string]any
_ = json.Unmarshal(msg.Params, &call)
rawArgs, ok := call["arguments"].(map[string]any)
if !ok || rawArgs == nil {
rawArgs = map[string]any{}
}
rawArgs["text"] = "ipc"
call["arguments"] = rawArgs
return map[string]any{
"action": HookActionModify,
"call": call,
}, nil
case "hook.after_tool":
if mode != "rewrite" {
return map[string]any{"action": HookActionContinue}, nil
}
var result map[string]any
_ = json.Unmarshal(msg.Params, &result)
if rawResult, ok := result["result"].(map[string]any); ok {
if forLLM, ok := rawResult["for_llm"].(string); ok {
rawResult["for_llm"] = "ipc:" + forLLM
}
}
return map[string]any{
"action": HookActionModify,
"result": result,
}, nil
case "hook.approve_tool":
if mode == "deny" {
return ApprovalDecision{
Approved: false,
Reason: "blocked by ipc hook",
}, nil
}
return ApprovalDecision{Approved: true}, nil
default:
return nil, &processHookRPCError{
Code: -32601,
Message: "method not found",
}
}
}
pkg/agent/hooks.go
+809
View File
@@ -0,0 +1,809 @@
package agent
import (
"context"
"fmt"
"io"
"sort"
"sync"
"time"
"github.com/sipeed/picoclaw/pkg/logger"
"github.com/sipeed/picoclaw/pkg/providers"
"github.com/sipeed/picoclaw/pkg/tools"
)
const (
defaultHookObserverTimeout = 500 * time.Millisecond
defaultHookInterceptorTimeout = 5 * time.Second
defaultHookApprovalTimeout = 60 * time.Second
hookObserverBufferSize = 64
)
type HookAction string
const (
HookActionContinue HookAction = "continue"
HookActionModify HookAction = "modify"
HookActionDenyTool HookAction = "deny_tool"
HookActionAbortTurn HookAction = "abort_turn"
HookActionHardAbort HookAction = "hard_abort"
)
type HookDecision struct {
Action HookAction `json:"action"`
Reason string `json:"reason,omitempty"`
}
func (d HookDecision) normalizedAction() HookAction {
if d.Action == "" {
return HookActionContinue
}
return d.Action
}
type ApprovalDecision struct {
Approved bool `json:"approved"`
Reason string `json:"reason,omitempty"`
}
type HookSource uint8
const (
HookSourceInProcess HookSource = iota
HookSourceProcess
)
type HookRegistration struct {
Name string
Priority int
Source HookSource
Hook any
}
func NamedHook(name string, hook any) HookRegistration {
return HookRegistration{
Name: name,
Source: HookSourceInProcess,
Hook: hook,
}
}
type EventObserver interface {
OnEvent(ctx context.Context, evt Event) error
}
type LLMInterceptor interface {
BeforeLLM(ctx context.Context, req *LLMHookRequest) (*LLMHookRequest, HookDecision, error)
AfterLLM(ctx context.Context, resp *LLMHookResponse) (*LLMHookResponse, HookDecision, error)
}
type ToolInterceptor interface {
BeforeTool(ctx context.Context, call *ToolCallHookRequest) (*ToolCallHookRequest, HookDecision, error)
AfterTool(ctx context.Context, result *ToolResultHookResponse) (*ToolResultHookResponse, HookDecision, error)
}
type ToolApprover interface {
ApproveTool(ctx context.Context, req *ToolApprovalRequest) (ApprovalDecision, error)
}
type LLMHookRequest struct {
Meta EventMeta `json:"meta"`
Model string `json:"model"`
Messages []providers.Message `json:"messages,omitempty"`
Tools []providers.ToolDefinition `json:"tools,omitempty"`
Options map[string]any `json:"options,omitempty"`
Channel string `json:"channel,omitempty"`
ChatID string `json:"chat_id,omitempty"`
GracefulTerminal bool `json:"graceful_terminal,omitempty"`
}
func (r *LLMHookRequest) Clone() *LLMHookRequest {
if r == nil {
return nil
}
cloned := *r
cloned.Messages = cloneProviderMessages(r.Messages)
cloned.Tools = cloneToolDefinitions(r.Tools)
cloned.Options = cloneStringAnyMap(r.Options)
return &cloned
}
type LLMHookResponse struct {
Meta EventMeta `json:"meta"`
Model string `json:"model"`
Response *providers.LLMResponse `json:"response,omitempty"`
Channel string `json:"channel,omitempty"`
ChatID string `json:"chat_id,omitempty"`
}
func (r *LLMHookResponse) Clone() *LLMHookResponse {
if r == nil {
return nil
}
cloned := *r
cloned.Response = cloneLLMResponse(r.Response)
return &cloned
}
type ToolCallHookRequest struct {
Meta EventMeta `json:"meta"`
Tool string `json:"tool"`
Arguments map[string]any `json:"arguments,omitempty"`
Channel string `json:"channel,omitempty"`
ChatID string `json:"chat_id,omitempty"`
}
func (r *ToolCallHookRequest) Clone() *ToolCallHookRequest {
if r == nil {
return nil
}
cloned := *r
cloned.Arguments = cloneStringAnyMap(r.Arguments)
return &cloned
}
type ToolApprovalRequest struct {
Meta EventMeta `json:"meta"`
Tool string `json:"tool"`
Arguments map[string]any `json:"arguments,omitempty"`
Channel string `json:"channel,omitempty"`
ChatID string `json:"chat_id,omitempty"`
}
func (r *ToolApprovalRequest) Clone() *ToolApprovalRequest {
if r == nil {
return nil
}
cloned := *r
cloned.Arguments = cloneStringAnyMap(r.Arguments)
return &cloned
}
type ToolResultHookResponse struct {
Meta EventMeta `json:"meta"`
Tool string `json:"tool"`
Arguments map[string]any `json:"arguments,omitempty"`
Result *tools.ToolResult `json:"result,omitempty"`
Duration time.Duration `json:"duration"`
Channel string `json:"channel,omitempty"`
ChatID string `json:"chat_id,omitempty"`
}
func (r *ToolResultHookResponse) Clone() *ToolResultHookResponse {
if r == nil {
return nil
}
cloned := *r
cloned.Arguments = cloneStringAnyMap(r.Arguments)
cloned.Result = cloneToolResult(r.Result)
return &cloned
}
type HookManager struct {
eventBus *EventBus
observerTimeout time.Duration
interceptorTimeout time.Duration
approvalTimeout time.Duration
mu sync.RWMutex
hooks map[string]HookRegistration
ordered []HookRegistration
sub EventSubscription
done chan struct{}
closeOnce sync.Once
}
func NewHookManager(eventBus *EventBus) *HookManager {
hm := &HookManager{
eventBus: eventBus,
observerTimeout: defaultHookObserverTimeout,
interceptorTimeout: defaultHookInterceptorTimeout,
approvalTimeout: defaultHookApprovalTimeout,
hooks: make(map[string]HookRegistration),
done: make(chan struct{}),
}
if eventBus == nil {
close(hm.done)
return hm
}
hm.sub = eventBus.Subscribe(hookObserverBufferSize)
go hm.dispatchEvents()
return hm
}
func (hm *HookManager) Close() {
if hm == nil {
return
}
hm.closeOnce.Do(func() {
if hm.eventBus != nil {
hm.eventBus.Unsubscribe(hm.sub.ID)
}
<-hm.done
hm.closeAllHooks()
})
}
func (hm *HookManager) ConfigureTimeouts(observer, interceptor, approval time.Duration) {
if hm == nil {
return
}
if observer > 0 {
hm.observerTimeout = observer
}
if interceptor > 0 {
hm.interceptorTimeout = interceptor
}
if approval > 0 {
hm.approvalTimeout = approval
}
}
func (hm *HookManager) Mount(reg HookRegistration) error {
if hm == nil {
return fmt.Errorf("hook manager is nil")
}
if reg.Name == "" {
return fmt.Errorf("hook name is required")
}
if reg.Hook == nil {
return fmt.Errorf("hook %q is nil", reg.Name)
}
hm.mu.Lock()
defer hm.mu.Unlock()
if existing, ok := hm.hooks[reg.Name]; ok {
closeHookIfPossible(existing.Hook)
}
hm.hooks[reg.Name] = reg
hm.rebuildOrdered()
return nil
}
func (hm *HookManager) Unmount(name string) {
if hm == nil || name == "" {
return
}
hm.mu.Lock()
defer hm.mu.Unlock()
if existing, ok := hm.hooks[name]; ok {
closeHookIfPossible(existing.Hook)
}
delete(hm.hooks, name)
hm.rebuildOrdered()
}
func (hm *HookManager) dispatchEvents() {
defer close(hm.done)
for evt := range hm.sub.C {
for _, reg := range hm.snapshotHooks() {
observer, ok := reg.Hook.(EventObserver)
if !ok {
continue
}
hm.runObserver(reg.Name, observer, evt)
}
}
}
func (hm *HookManager) BeforeLLM(ctx context.Context, req *LLMHookRequest) (*LLMHookRequest, HookDecision) {
if hm == nil || req == nil {
return req, HookDecision{Action: HookActionContinue}
}
current := req.Clone()
for _, reg := range hm.snapshotHooks() {
interceptor, ok := reg.Hook.(LLMInterceptor)
if !ok {
continue
}
next, decision, ok := hm.callBeforeLLM(ctx, reg.Name, interceptor, current.Clone())
if !ok {
continue
}
switch decision.normalizedAction() {
case HookActionContinue, HookActionModify:
if next != nil {
current = next
}
case HookActionAbortTurn, HookActionHardAbort:
return current, decision
default:
hm.logUnsupportedAction(reg.Name, "before_llm", decision.Action)
}
}
return current, HookDecision{Action: HookActionContinue}
}
func (hm *HookManager) AfterLLM(ctx context.Context, resp *LLMHookResponse) (*LLMHookResponse, HookDecision) {
if hm == nil || resp == nil {
return resp, HookDecision{Action: HookActionContinue}
}
current := resp.Clone()
for _, reg := range hm.snapshotHooks() {
interceptor, ok := reg.Hook.(LLMInterceptor)
if !ok {
continue
}
next, decision, ok := hm.callAfterLLM(ctx, reg.Name, interceptor, current.Clone())
if !ok {
continue
}
switch decision.normalizedAction() {
case HookActionContinue, HookActionModify:
if next != nil {
current = next
}
case HookActionAbortTurn, HookActionHardAbort:
return current, decision
default:
hm.logUnsupportedAction(reg.Name, "after_llm", decision.Action)
}
}
return current, HookDecision{Action: HookActionContinue}
}
func (hm *HookManager) BeforeTool(
ctx context.Context,
call *ToolCallHookRequest,
) (*ToolCallHookRequest, HookDecision) {
if hm == nil || call == nil {
return call, HookDecision{Action: HookActionContinue}
}
current := call.Clone()
for _, reg := range hm.snapshotHooks() {
interceptor, ok := reg.Hook.(ToolInterceptor)
if !ok {
continue
}
next, decision, ok := hm.callBeforeTool(ctx, reg.Name, interceptor, current.Clone())
if !ok {
continue
}
switch decision.normalizedAction() {
case HookActionContinue, HookActionModify:
if next != nil {
current = next
}
case HookActionDenyTool, HookActionAbortTurn, HookActionHardAbort:
return current, decision
default:
hm.logUnsupportedAction(reg.Name, "before_tool", decision.Action)
}
}
return current, HookDecision{Action: HookActionContinue}
}
func (hm *HookManager) AfterTool(
ctx context.Context,
result *ToolResultHookResponse,
) (*ToolResultHookResponse, HookDecision) {
if hm == nil || result == nil {
return result, HookDecision{Action: HookActionContinue}
}
current := result.Clone()
for _, reg := range hm.snapshotHooks() {
interceptor, ok := reg.Hook.(ToolInterceptor)
if !ok {
continue
}
next, decision, ok := hm.callAfterTool(ctx, reg.Name, interceptor, current.Clone())
if !ok {
continue
}
switch decision.normalizedAction() {
case HookActionContinue, HookActionModify:
if next != nil {
current = next
}
case HookActionAbortTurn, HookActionHardAbort:
return current, decision
default:
hm.logUnsupportedAction(reg.Name, "after_tool", decision.Action)
}
}
return current, HookDecision{Action: HookActionContinue}
}
func (hm *HookManager) ApproveTool(ctx context.Context, req *ToolApprovalRequest) ApprovalDecision {
if hm == nil || req == nil {
return ApprovalDecision{Approved: true}
}
for _, reg := range hm.snapshotHooks() {
approver, ok := reg.Hook.(ToolApprover)
if !ok {
continue
}
decision, ok := hm.callApproveTool(ctx, reg.Name, approver, req.Clone())
if !ok {
return ApprovalDecision{
Approved: false,
Reason: fmt.Sprintf("tool approval hook %q failed", reg.Name),
}
}
if !decision.Approved {
return decision
}
}
return ApprovalDecision{Approved: true}
}
func (hm *HookManager) rebuildOrdered() {
hm.ordered = hm.ordered[:0]
for _, reg := range hm.hooks {
hm.ordered = append(hm.ordered, reg)
}
sort.SliceStable(hm.ordered, func(i, j int) bool {
if hm.ordered[i].Source != hm.ordered[j].Source {
return hm.ordered[i].Source < hm.ordered[j].Source
}
if hm.ordered[i].Priority == hm.ordered[j].Priority {
return hm.ordered[i].Name < hm.ordered[j].Name
}
return hm.ordered[i].Priority < hm.ordered[j].Priority
})
}
func (hm *HookManager) snapshotHooks() []HookRegistration {
hm.mu.RLock()
defer hm.mu.RUnlock()
snapshot := make([]HookRegistration, len(hm.ordered))
copy(snapshot, hm.ordered)
return snapshot
}
func (hm *HookManager) closeAllHooks() {
hm.mu.Lock()
defer hm.mu.Unlock()
for name, reg := range hm.hooks {
closeHookIfPossible(reg.Hook)
delete(hm.hooks, name)
}
hm.ordered = nil
}
func (hm *HookManager) runObserver(name string, observer EventObserver, evt Event) {
ctx, cancel := context.WithTimeout(context.Background(), hm.observerTimeout)
defer cancel()
done := make(chan error, 1)
go func() {
done <- observer.OnEvent(ctx, evt)
}()
select {
case err := <-done:
if err != nil {
logger.WarnCF("hooks", "Event observer failed", map[string]any{
"hook": name,
"event": evt.Kind.String(),
"error": err.Error(),
})
}
case <-ctx.Done():
logger.WarnCF("hooks", "Event observer timed out", map[string]any{
"hook": name,
"event": evt.Kind.String(),
"timeout_ms": hm.observerTimeout.Milliseconds(),
})
}
}
func (hm *HookManager) callBeforeLLM(
parent context.Context,
name string,
interceptor LLMInterceptor,
req *LLMHookRequest,
) (*LLMHookRequest, HookDecision, bool) {
return runInterceptorHook(
parent,
hm.interceptorTimeout,
name,
"before_llm",
func(ctx context.Context) (*LLMHookRequest, HookDecision, error) {
return interceptor.BeforeLLM(ctx, req)
},
)
}
func (hm *HookManager) callAfterLLM(
parent context.Context,
name string,
interceptor LLMInterceptor,
resp *LLMHookResponse,
) (*LLMHookResponse, HookDecision, bool) {
return runInterceptorHook(
parent,
hm.interceptorTimeout,
name,
"after_llm",
func(ctx context.Context) (*LLMHookResponse, HookDecision, error) {
return interceptor.AfterLLM(ctx, resp)
},
)
}
func (hm *HookManager) callBeforeTool(
parent context.Context,
name string,
interceptor ToolInterceptor,
call *ToolCallHookRequest,
) (*ToolCallHookRequest, HookDecision, bool) {
return runInterceptorHook(
parent,
hm.interceptorTimeout,
name,
"before_tool",
func(ctx context.Context) (*ToolCallHookRequest, HookDecision, error) {
return interceptor.BeforeTool(ctx, call)
},
)
}
func (hm *HookManager) callAfterTool(
parent context.Context,
name string,
interceptor ToolInterceptor,
resultView *ToolResultHookResponse,
) (*ToolResultHookResponse, HookDecision, bool) {
return runInterceptorHook(
parent,
hm.interceptorTimeout,
name,
"after_tool",
func(ctx context.Context) (*ToolResultHookResponse, HookDecision, error) {
return interceptor.AfterTool(ctx, resultView)
},
)
}
func (hm *HookManager) callApproveTool(
parent context.Context,
name string,
approver ToolApprover,
req *ToolApprovalRequest,
) (ApprovalDecision, bool) {
return runApprovalHook(
parent,
hm.approvalTimeout,
name,
"approve_tool",
func(ctx context.Context) (ApprovalDecision, error) {
return approver.ApproveTool(ctx, req)
},
)
}
func runInterceptorHook[T any](
parent context.Context,
timeout time.Duration,
name string,
stage string,
fn func(ctx context.Context) (T, HookDecision, error),
) (T, HookDecision, bool) {
var zero T
ctx, cancel := context.WithTimeout(parent, timeout)
defer cancel()
type result struct {
value T
decision HookDecision
err error
}
done := make(chan result, 1)
go func() {
value, decision, err := fn(ctx)
done <- result{value: value, decision: decision, err: err}
}()
select {
case res := <-done:
if res.err != nil {
logger.WarnCF("hooks", "Interceptor hook failed", map[string]any{
"hook": name,
"stage": stage,
"error": res.err.Error(),
})
return zero, HookDecision{}, false
}
return res.value, res.decision, true
case <-ctx.Done():
logger.WarnCF("hooks", "Interceptor hook timed out", map[string]any{
"hook": name,
"stage": stage,
"timeout_ms": timeout.Milliseconds(),
})
return zero, HookDecision{}, false
}
}
func runApprovalHook(
parent context.Context,
timeout time.Duration,
name string,
stage string,
fn func(ctx context.Context) (ApprovalDecision, error),
) (ApprovalDecision, bool) {
ctx, cancel := context.WithTimeout(parent, timeout)
defer cancel()
type result struct {
decision ApprovalDecision
err error
}
done := make(chan result, 1)
go func() {
decision, err := fn(ctx)
done <- result{decision: decision, err: err}
}()
select {
case res := <-done:
if res.err != nil {
logger.WarnCF("hooks", "Approval hook failed", map[string]any{
"hook": name,
"stage": stage,
"error": res.err.Error(),
})
return ApprovalDecision{}, false
}
return res.decision, true
case <-ctx.Done():
logger.WarnCF("hooks", "Approval hook timed out", map[string]any{
"hook": name,
"stage": stage,
"timeout_ms": timeout.Milliseconds(),
})
return ApprovalDecision{
Approved: false,
Reason: fmt.Sprintf("tool approval hook %q timed out", name),
}, true
}
}
func (hm *HookManager) logUnsupportedAction(name, stage string, action HookAction) {
logger.WarnCF("hooks", "Hook returned unsupported action for stage", map[string]any{
"hook": name,
"stage": stage,
"action": action,
})
}
func cloneProviderMessages(messages []providers.Message) []providers.Message {
if len(messages) == 0 {
return nil
}
cloned := make([]providers.Message, len(messages))
for i, msg := range messages {
cloned[i] = msg
if len(msg.Media) > 0 {
cloned[i].Media = append([]string(nil), msg.Media...)
}
if len(msg.SystemParts) > 0 {
cloned[i].SystemParts = append([]providers.ContentBlock(nil), msg.SystemParts...)
}
if len(msg.ToolCalls) > 0 {
cloned[i].ToolCalls = cloneProviderToolCalls(msg.ToolCalls)
}
}
return cloned
}
func cloneProviderToolCalls(calls []providers.ToolCall) []providers.ToolCall {
if len(calls) == 0 {
return nil
}
cloned := make([]providers.ToolCall, len(calls))
for i, call := range calls {
cloned[i] = call
if call.Function != nil {
fn := *call.Function
cloned[i].Function = &fn
}
if call.Arguments != nil {
cloned[i].Arguments = cloneStringAnyMap(call.Arguments)
}
if call.ExtraContent != nil {
extra := *call.ExtraContent
if call.ExtraContent.Google != nil {
google := *call.ExtraContent.Google
extra.Google = &google
}
cloned[i].ExtraContent = &extra
}
}
return cloned
}
func cloneToolDefinitions(defs []providers.ToolDefinition) []providers.ToolDefinition {
if len(defs) == 0 {
return nil
}
cloned := make([]providers.ToolDefinition, len(defs))
for i, def := range defs {
cloned[i] = def
cloned[i].Function.Parameters = cloneStringAnyMap(def.Function.Parameters)
}
return cloned
}
func cloneLLMResponse(resp *providers.LLMResponse) *providers.LLMResponse {
if resp == nil {
return nil
}
cloned := *resp
cloned.ToolCalls = cloneProviderToolCalls(resp.ToolCalls)
if len(resp.ReasoningDetails) > 0 {
cloned.ReasoningDetails = append(cloned.ReasoningDetails[:0:0], resp.ReasoningDetails...)
}
if resp.Usage != nil {
usage := *resp.Usage
cloned.Usage = &usage
}
return &cloned
}
func cloneStringAnyMap(src map[string]any) map[string]any {
if len(src) == 0 {
return nil
}
cloned := make(map[string]any, len(src))
for k, v := range src {
cloned[k] = v
}
return cloned
}
func cloneToolResult(result *tools.ToolResult) *tools.ToolResult {
if result == nil {
return nil
}
cloned := *result
if len(result.Media) > 0 {
cloned.Media = append([]string(nil), result.Media...)
}
return &cloned
}
func closeHookIfPossible(hook any) {
closer, ok := hook.(io.Closer)
if !ok {
return
}
if err := closer.Close(); err != nil {
logger.WarnCF("hooks", "Failed to close hook", map[string]any{
"error": err.Error(),
})
}
}
pkg/agent/hooks_test.go
+345
View File
@@ -0,0 +1,345 @@
package agent
import (
"context"
"os"
"sync"
"testing"
"time"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/config"
"github.com/sipeed/picoclaw/pkg/providers"
"github.com/sipeed/picoclaw/pkg/tools"
)
func newHookTestLoop(
t *testing.T,
provider providers.LLMProvider,
) (*AgentLoop, *AgentInstance, func()) {
t.Helper()
tmpDir, err := os.MkdirTemp("", "agent-hooks-*")
if err != nil {
t.Fatalf("failed to create temp dir: %v", err)
}
cfg := &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
Workspace: tmpDir,
Model: "test-model",
MaxTokens: 4096,
MaxToolIterations: 10,
},
},
}
al := NewAgentLoop(cfg, bus.NewMessageBus(), provider)
agent := al.registry.GetDefaultAgent()
if agent == nil {
t.Fatal("expected default agent")
}
return al, agent, func() {
al.Close()
_ = os.RemoveAll(tmpDir)
}
}
func TestHookManager_SortsInProcessBeforeProcess(t *testing.T) {
hm := NewHookManager(nil)
defer hm.Close()
if err := hm.Mount(HookRegistration{
Name: "process",
Priority: -10,
Source: HookSourceProcess,
Hook: struct{}{},
}); err != nil {
t.Fatalf("mount process hook: %v", err)
}
if err := hm.Mount(HookRegistration{
Name: "in-process",
Priority: 100,
Source: HookSourceInProcess,
Hook: struct{}{},
}); err != nil {
t.Fatalf("mount in-process hook: %v", err)
}
ordered := hm.snapshotHooks()
if len(ordered) != 2 {
t.Fatalf("expected 2 hooks, got %d", len(ordered))
}
if ordered[0].Name != "in-process" {
t.Fatalf("expected in-process hook first, got %q", ordered[0].Name)
}
if ordered[1].Name != "process" {
t.Fatalf("expected process hook second, got %q", ordered[1].Name)
}
}
type llmHookTestProvider struct {
mu sync.Mutex
lastModel string
}
func (p *llmHookTestProvider) Chat(
ctx context.Context,
messages []providers.Message,
tools []providers.ToolDefinition,
model string,
opts map[string]any,
) (*providers.LLMResponse, error) {
p.mu.Lock()
p.lastModel = model
p.mu.Unlock()
return &providers.LLMResponse{
Content: "provider content",
}, nil
}
func (p *llmHookTestProvider) GetDefaultModel() string {
return "llm-hook-provider"
}
type llmObserverHook struct {
eventCh chan Event
}
func (h *llmObserverHook) OnEvent(ctx context.Context, evt Event) error {
if evt.Kind == EventKindTurnEnd {
select {
case h.eventCh <- evt:
default:
}
}
return nil
}
func (h *llmObserverHook) BeforeLLM(
ctx context.Context,
req *LLMHookRequest,
) (*LLMHookRequest, HookDecision, error) {
next := req.Clone()
next.Model = "hook-model"
return next, HookDecision{Action: HookActionModify}, nil
}
func (h *llmObserverHook) AfterLLM(
ctx context.Context,
resp *LLMHookResponse,
) (*LLMHookResponse, HookDecision, error) {
next := resp.Clone()
next.Response.Content = "hooked content"
return next, HookDecision{Action: HookActionModify}, nil
}
func TestAgentLoop_Hooks_ObserverAndLLMInterceptor(t *testing.T) {
provider := &llmHookTestProvider{}
al, agent, cleanup := newHookTestLoop(t, provider)
defer cleanup()
hook := &llmObserverHook{eventCh: make(chan Event, 1)}
if err := al.MountHook(NamedHook("llm-observer", hook)); err != nil {
t.Fatalf("MountHook failed: %v", err)
}
resp, err := al.runAgentLoop(context.Background(), agent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "hello",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
if resp != "hooked content" {
t.Fatalf("expected hooked content, got %q", resp)
}
provider.mu.Lock()
lastModel := provider.lastModel
provider.mu.Unlock()
if lastModel != "hook-model" {
t.Fatalf("expected model hook-model, got %q", lastModel)
}
select {
case evt := <-hook.eventCh:
if evt.Kind != EventKindTurnEnd {
t.Fatalf("expected turn end event, got %v", evt.Kind)
}
case <-time.After(2 * time.Second):
t.Fatal("timed out waiting for hook observer event")
}
}
type toolHookProvider struct {
mu sync.Mutex
calls int
}
func (p *toolHookProvider) Chat(
ctx context.Context,
messages []providers.Message,
tools []providers.ToolDefinition,
model string,
opts map[string]any,
) (*providers.LLMResponse, error) {
p.mu.Lock()
defer p.mu.Unlock()
p.calls++
if p.calls == 1 {
return &providers.LLMResponse{
ToolCalls: []providers.ToolCall{
{
ID: "call-1",
Name: "echo_text",
Arguments: map[string]any{"text": "original"},
},
},
}, nil
}
last := messages[len(messages)-1]
return &providers.LLMResponse{
Content: last.Content,
}, nil
}
func (p *toolHookProvider) GetDefaultModel() string {
return "tool-hook-provider"
}
type echoTextTool struct{}
func (t *echoTextTool) Name() string {
return "echo_text"
}
func (t *echoTextTool) Description() string {
return "echo a text argument"
}
func (t *echoTextTool) Parameters() map[string]any {
return map[string]any{
"type": "object",
"properties": map[string]any{
"text": map[string]any{
"type": "string",
},
},
}
}
func (t *echoTextTool) Execute(ctx context.Context, args map[string]any) *tools.ToolResult {
text, _ := args["text"].(string)
return tools.SilentResult(text)
}
type toolRewriteHook struct{}
func (h *toolRewriteHook) BeforeTool(
ctx context.Context,
call *ToolCallHookRequest,
) (*ToolCallHookRequest, HookDecision, error) {
next := call.Clone()
next.Arguments["text"] = "modified"
return next, HookDecision{Action: HookActionModify}, nil
}
func (h *toolRewriteHook) AfterTool(
ctx context.Context,
result *ToolResultHookResponse,
) (*ToolResultHookResponse, HookDecision, error) {
next := result.Clone()
next.Result.ForLLM = "after:" + next.Result.ForLLM
return next, HookDecision{Action: HookActionModify}, nil
}
func TestAgentLoop_Hooks_ToolInterceptorCanRewrite(t *testing.T) {
provider := &toolHookProvider{}
al, agent, cleanup := newHookTestLoop(t, provider)
defer cleanup()
al.RegisterTool(&echoTextTool{})
if err := al.MountHook(NamedHook("tool-rewrite", &toolRewriteHook{})); err != nil {
t.Fatalf("MountHook failed: %v", err)
}
resp, err := al.runAgentLoop(context.Background(), agent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "run tool",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
if resp != "after:modified" {
t.Fatalf("expected rewritten tool result, got %q", resp)
}
}
type denyApprovalHook struct{}
func (h *denyApprovalHook) ApproveTool(ctx context.Context, req *ToolApprovalRequest) (ApprovalDecision, error) {
return ApprovalDecision{
Approved: false,
Reason: "blocked",
}, nil
}
func TestAgentLoop_Hooks_ToolApproverCanDeny(t *testing.T) {
provider := &toolHookProvider{}
al, agent, cleanup := newHookTestLoop(t, provider)
defer cleanup()
al.RegisterTool(&echoTextTool{})
if err := al.MountHook(NamedHook("deny-approval", &denyApprovalHook{})); err != nil {
t.Fatalf("MountHook failed: %v", err)
}
sub := al.SubscribeEvents(16)
defer al.UnsubscribeEvents(sub.ID)
resp, err := al.runAgentLoop(context.Background(), agent, processOptions{
SessionKey: "session-1",
Channel: "cli",
ChatID: "direct",
UserMessage: "run tool",
DefaultResponse: defaultResponse,
EnableSummary: false,
SendResponse: false,
})
if err != nil {
t.Fatalf("runAgentLoop failed: %v", err)
}
expected := "Tool execution denied by approval hook: blocked"
if resp != expected {
t.Fatalf("expected %q, got %q", expected, resp)
}
events := collectEventStream(sub.C)
skippedEvt, ok := findEvent(events, EventKindToolExecSkipped)
if !ok {
t.Fatal("expected tool skipped event")
}
payload, ok := skippedEvt.Payload.(ToolExecSkippedPayload)
if !ok {
t.Fatalf("expected ToolExecSkippedPayload, got %T", skippedEvt.Payload)
}
if payload.Reason != expected {
t.Fatalf("expected skipped reason %q, got %q", expected, payload.Reason)
}
}
pkg/agent/instance.go
+12 -1
View File
@@ -130,6 +130,17 @@ func NewAgentInstance(
maxTokens = 8192
}
contextWindow := defaults.ContextWindow
if contextWindow == 0 {
// Default heuristic: 4x the output token limit.
// Most models have context windows well above their output limits
// (e.g., GPT-4o 128k ctx / 16k out, Claude 200k ctx / 8k out).
// 4x is a conservative lower bound that avoids premature
// summarization while remaining safe — the reactive
// forceCompression handles any overshoot.
contextWindow = maxTokens * 4
}
temperature := 0.7
if defaults.Temperature != nil {
temperature = *defaults.Temperature
@@ -182,7 +193,7 @@ func NewAgentInstance(
MaxTokens: maxTokens,
Temperature: temperature,
ThinkingLevel: thinkingLevel,
ContextWindow: maxTokens,
ContextWindow: contextWindow,
SummarizeMessageThreshold: summarizeMessageThreshold,
SummarizeTokenPercent: summarizeTokenPercent,
Provider: provider,
pkg/agent/loop.go
+1559 -419
View File
File diff suppressed because it is too large Load Diff
pkg/agent/loop_test.go
+8 -9
View File
@@ -1235,11 +1235,11 @@ func TestAgentLoop_ContextExhaustionRetry(t *testing.T) {
al := NewAgentLoop(cfg, msgBus, provider)
// Inject some history to simulate a full context
// Inject some history to simulate a full context.
// Session history only stores user/assistant/tool messages — the system
// prompt is built dynamically by BuildMessages and is NOT stored here.
sessionKey := "test-session-context"
// Create dummy history
history := []providers.Message{
{Role: "system", Content: "System prompt"},
{Role: "user", Content: "Old message 1"},
{Role: "assistant", Content: "Old response 1"},
{Role: "user", Content: "Old message 2"},
@@ -1277,12 +1277,11 @@ func TestAgentLoop_ContextExhaustionRetry(t *testing.T) {
// Check final history length
finalHistory := defaultAgent.Sessions.GetHistory(sessionKey)
// We verify that the history has been modified (compressed)
// Original length: 6
// Expected behavior: compression drops ~50% of history (mid slice)
// We can assert that the length is NOT what it would be without compression.
// Without compression: 6 + 1 (new user msg) + 1 (assistant msg) = 8
if len(finalHistory) >= 8 {
t.Errorf("Expected history to be compressed (len < 8), got %d", len(finalHistory))
// Original length: 5
// Expected behavior: compression drops ~50% of Turns
// Without compression: 5 + 1 (new user msg) + 1 (assistant msg) = 7
if len(finalHistory) >= 7 {
t.Errorf("Expected history to be compressed (len < 7), got %d", len(finalHistory))
}
}
pkg/agent/steering.go
+503
View File
@@ -0,0 +1,503 @@
package agent
import (
"context"
"fmt"
"strings"
"sync"
"github.com/sipeed/picoclaw/pkg/logger"
"github.com/sipeed/picoclaw/pkg/providers"
"github.com/sipeed/picoclaw/pkg/routing"
"github.com/sipeed/picoclaw/pkg/tools"
)
// SteeringMode controls how queued steering messages are dequeued.
type SteeringMode string
const (
// SteeringOneAtATime dequeues only the first queued message per poll.
SteeringOneAtATime SteeringMode = "one-at-a-time"
// SteeringAll drains the entire queue in a single poll.
SteeringAll SteeringMode = "all"
// MaxQueueSize number of possible messages in the Steering Queue
MaxQueueSize = 10
// manualSteeringScope is the legacy fallback queue used when no active
// turn/session scope is available.
manualSteeringScope = "__manual__"
)
// parseSteeringMode normalizes a config string into a SteeringMode.
func parseSteeringMode(s string) SteeringMode {
switch s {
case "all":
return SteeringAll
default:
return SteeringOneAtATime
}
}
// steeringQueue is a thread-safe queue of user messages that can be injected
// into a running agent loop to interrupt it between tool calls.
type steeringQueue struct {
mu sync.Mutex
queues map[string][]providers.Message
mode SteeringMode
}
func newSteeringQueue(mode SteeringMode) *steeringQueue {
return &steeringQueue{
queues: make(map[string][]providers.Message),
mode: mode,
}
}
func normalizeSteeringScope(scope string) string {
scope = strings.TrimSpace(scope)
if scope == "" {
return manualSteeringScope
}
return scope
}
// push enqueues a steering message in the legacy fallback scope.
func (sq *steeringQueue) push(msg providers.Message) error {
return sq.pushScope(manualSteeringScope, msg)
}
// pushScope enqueues a steering message for the provided scope.
func (sq *steeringQueue) pushScope(scope string, msg providers.Message) error {
sq.mu.Lock()
defer sq.mu.Unlock()
scope = normalizeSteeringScope(scope)
queue := sq.queues[scope]
if len(queue) >= MaxQueueSize {
return fmt.Errorf("steering queue is full")
}
sq.queues[scope] = append(queue, msg)
return nil
}
// dequeue removes and returns pending steering messages from the legacy
// fallback scope according to the configured mode.
func (sq *steeringQueue) dequeue() []providers.Message {
return sq.dequeueScope(manualSteeringScope)
}
// dequeueScope removes and returns pending steering messages for the provided
// scope according to the configured mode.
func (sq *steeringQueue) dequeueScope(scope string) []providers.Message {
sq.mu.Lock()
defer sq.mu.Unlock()
return sq.dequeueLocked(normalizeSteeringScope(scope))
}
// dequeueScopeWithFallback drains the scoped queue first and falls back to the
// legacy manual scope for backwards compatibility.
func (sq *steeringQueue) dequeueScopeWithFallback(scope string) []providers.Message {
sq.mu.Lock()
defer sq.mu.Unlock()
scope = strings.TrimSpace(scope)
if scope != "" {
if msgs := sq.dequeueLocked(scope); len(msgs) > 0 {
return msgs
}
}
return sq.dequeueLocked(manualSteeringScope)
}
func (sq *steeringQueue) dequeueLocked(scope string) []providers.Message {
queue := sq.queues[scope]
if len(queue) == 0 {
return nil
}
switch sq.mode {
case SteeringAll:
msgs := append([]providers.Message(nil), queue...)
delete(sq.queues, scope)
return msgs
default:
msg := queue[0]
queue[0] = providers.Message{} // Clear reference for GC
queue = queue[1:]
if len(queue) == 0 {
delete(sq.queues, scope)
} else {
sq.queues[scope] = queue
}
return []providers.Message{msg}
}
}
// len returns the number of queued messages across all scopes.
func (sq *steeringQueue) len() int {
sq.mu.Lock()
defer sq.mu.Unlock()
total := 0
for _, queue := range sq.queues {
total += len(queue)
}
return total
}
// lenScope returns the number of queued messages for a specific scope.
func (sq *steeringQueue) lenScope(scope string) int {
sq.mu.Lock()
defer sq.mu.Unlock()
return len(sq.queues[normalizeSteeringScope(scope)])
}
// setMode updates the steering mode.
func (sq *steeringQueue) setMode(mode SteeringMode) {
sq.mu.Lock()
defer sq.mu.Unlock()
sq.mode = mode
}
// getMode returns the current steering mode.
func (sq *steeringQueue) getMode() SteeringMode {
sq.mu.Lock()
defer sq.mu.Unlock()
return sq.mode
}
// Steer enqueues a user message to be injected into the currently running
// agent loop. The message will be picked up after the current tool finishes
// executing, causing any remaining tool calls in the batch to be skipped.
func (al *AgentLoop) Steer(msg providers.Message) error {
scope := ""
agentID := ""
if ts := al.getAnyActiveTurnState(); ts != nil {
scope = ts.sessionKey
agentID = ts.agentID
}
return al.enqueueSteeringMessage(scope, agentID, msg)
}
func (al *AgentLoop) enqueueSteeringMessage(scope, agentID string, msg providers.Message) error {
if al.steering == nil {
return fmt.Errorf("steering queue is not initialized")
}
if err := al.steering.pushScope(scope, msg); err != nil {
logger.WarnCF("agent", "Failed to enqueue steering message", map[string]any{
"error": err.Error(),
"role": msg.Role,
"scope": normalizeSteeringScope(scope),
})
return err
}
queueDepth := al.steering.lenScope(scope)
logger.DebugCF("agent", "Steering message enqueued", map[string]any{
"role": msg.Role,
"content_len": len(msg.Content),
"media_count": len(msg.Media),
"queue_len": queueDepth,
"scope": normalizeSteeringScope(scope),
})
meta := EventMeta{
Source: "Steer",
TracePath: "turn.interrupt.received",
}
if ts := al.getAnyActiveTurnState(); ts != nil {
meta = ts.eventMeta("Steer", "turn.interrupt.received")
} else {
if strings.TrimSpace(agentID) != "" {
meta.AgentID = agentID
}
normalizedScope := normalizeSteeringScope(scope)
if normalizedScope != manualSteeringScope {
meta.SessionKey = normalizedScope
}
if meta.AgentID == "" {
if registry := al.GetRegistry(); registry != nil {
if agent := registry.GetDefaultAgent(); agent != nil {
meta.AgentID = agent.ID
}
}
}
}
al.emitEvent(
EventKindInterruptReceived,
meta,
InterruptReceivedPayload{
Kind: InterruptKindSteering,
Role: msg.Role,
ContentLen: len(msg.Content),
QueueDepth: queueDepth,
},
)
return nil
}
// SteeringMode returns the current steering mode.
func (al *AgentLoop) SteeringMode() SteeringMode {
if al.steering == nil {
return SteeringOneAtATime
}
return al.steering.getMode()
}
// SetSteeringMode updates the steering mode.
func (al *AgentLoop) SetSteeringMode(mode SteeringMode) {
if al.steering == nil {
return
}
al.steering.setMode(mode)
}
// dequeueSteeringMessages is the internal method called by the agent loop
// to poll for steering messages in the legacy fallback scope.
func (al *AgentLoop) dequeueSteeringMessages() []providers.Message {
if al.steering == nil {
return nil
}
return al.steering.dequeue()
}
func (al *AgentLoop) dequeueSteeringMessagesForScope(scope string) []providers.Message {
if al.steering == nil {
return nil
}
return al.steering.dequeueScope(scope)
}
func (al *AgentLoop) dequeueSteeringMessagesForScopeWithFallback(scope string) []providers.Message {
if al.steering == nil {
return nil
}
return al.steering.dequeueScopeWithFallback(scope)
}
func (al *AgentLoop) pendingSteeringCountForScope(scope string) int {
if al.steering == nil {
return 0
}
return al.steering.lenScope(scope)
}
func (al *AgentLoop) continueWithSteeringMessages(
ctx context.Context,
agent *AgentInstance,
sessionKey, channel, chatID string,
steeringMsgs []providers.Message,
) (string, error) {
return al.runAgentLoop(ctx, agent, processOptions{
SessionKey: sessionKey,
Channel: channel,
ChatID: chatID,
DefaultResponse: defaultResponse,
EnableSummary: true,
SendResponse: false,
InitialSteeringMessages: steeringMsgs,
SkipInitialSteeringPoll: true,
})
}
func (al *AgentLoop) agentForSession(sessionKey string) *AgentInstance {
registry := al.GetRegistry()
if registry == nil {
return nil
}
if parsed := routing.ParseAgentSessionKey(sessionKey); parsed != nil {
if agent, ok := registry.GetAgent(parsed.AgentID); ok {
return agent
}
}
return registry.GetDefaultAgent()
}
// Continue resumes an idle agent by dequeuing any pending steering messages
// and running them through the agent loop. This is used when the agent's last
// message was from the assistant (i.e., it has stopped processing) and the
// user has since enqueued steering messages.
//
// If no steering messages are pending, it returns an empty string.
func (al *AgentLoop) Continue(ctx context.Context, sessionKey, channel, chatID string) (string, error) {
if active := al.GetActiveTurn(); active != nil {
return "", fmt.Errorf("turn %s is still active", active.TurnID)
}
if err := al.ensureHooksInitialized(ctx); err != nil {
return "", err
}
if err := al.ensureMCPInitialized(ctx); err != nil {
return "", err
}
steeringMsgs := al.dequeueSteeringMessagesForScopeWithFallback(sessionKey)
if len(steeringMsgs) == 0 {
return "", nil
}
agent := al.agentForSession(sessionKey)
if agent == nil {
return "", fmt.Errorf("no agent available for session %q", sessionKey)
}
if tool, ok := agent.Tools.Get("message"); ok {
if resetter, ok := tool.(interface{ ResetSentInRound() }); ok {
resetter.ResetSentInRound()
}
}
return al.continueWithSteeringMessages(ctx, agent, sessionKey, channel, chatID, steeringMsgs)
}
func (al *AgentLoop) InterruptGraceful(hint string) error {
ts := al.getAnyActiveTurnState()
if ts == nil {
return fmt.Errorf("no active turn")
}
if !ts.requestGracefulInterrupt(hint) {
return fmt.Errorf("turn %s cannot accept graceful interrupt", ts.turnID)
}
al.emitEvent(
EventKindInterruptReceived,
ts.eventMeta("InterruptGraceful", "turn.interrupt.received"),
InterruptReceivedPayload{
Kind: InterruptKindGraceful,
HintLen: len(hint),
},
)
return nil
}
func (al *AgentLoop) InterruptHard() error {
ts := al.getAnyActiveTurnState()
if ts == nil {
return fmt.Errorf("no active turn")
}
if !ts.requestHardAbort() {
return fmt.Errorf("turn %s is already aborting", ts.turnID)
}
al.emitEvent(
EventKindInterruptReceived,
ts.eventMeta("InterruptHard", "turn.interrupt.received"),
InterruptReceivedPayload{
Kind: InterruptKindHard,
},
)
return nil
}
// ====================== SubTurn Result Polling ======================
// dequeuePendingSubTurnResults polls the SubTurn result channel for the given
// session and returns all available results without blocking.
// Returns nil if no active turn state exists for this session.
func (al *AgentLoop) dequeuePendingSubTurnResults(sessionKey string) []*tools.ToolResult {
tsInterface, ok := al.activeTurnStates.Load(sessionKey)
if !ok {
return nil
}
ts, ok := tsInterface.(*turnState)
if !ok {
return nil
}
var results []*tools.ToolResult
for {
select {
case result, ok := <-ts.pendingResults:
if !ok {
return results
}
if result != nil {
results = append(results, result)
}
default:
return results
}
}
}
// ====================== Hard Abort ======================
// HardAbort immediately cancels the running agent loop for the given session,
// cascading the cancellation to all child SubTurns. This is a destructive operation
// that terminates execution without waiting for graceful cleanup.
//
// Use this when the user explicitly requests immediate termination (e.g., "stop now", "abort").
// For graceful interruption that allows the agent to finish the current tool and summarize,
// use Steer() instead.
func (al *AgentLoop) HardAbort(sessionKey string) error {
tsInterface, ok := al.activeTurnStates.Load(sessionKey)
if !ok {
return fmt.Errorf("no active turn state found for session %s", sessionKey)
}
ts, ok := tsInterface.(*turnState)
if !ok {
return fmt.Errorf("invalid turn state type for session %s", sessionKey)
}
logger.InfoCF("agent", "Hard abort triggered", map[string]any{
"session_key": sessionKey,
"turn_id": ts.turnID,
"depth": ts.depth,
"initial_history_length": ts.initialHistoryLength,
})
// IMPORTANT: Trigger cascading cancellation FIRST to stop all child SubTurns
// from adding more messages to the session. This prevents race conditions
// where rollback happens while children are still writing.
// Use isHardAbort=true for hard abort to immediately cancel all children.
ts.Finish(true)
// Roll back session history to the state before the turn started.
if ts.session != nil {
history := ts.session.GetHistory(sessionKey)
if ts.initialHistoryLength < len(history) {
ts.session.SetHistory(sessionKey, history[:ts.initialHistoryLength])
}
}
return nil
}
// ====================== Follow-Up Injection ======================
// InjectFollowUp enqueues a message to be automatically processed after the current
// turn completes. Unlike Steer(), which interrupts the current execution, InjectFollowUp
// waits for the current turn to finish naturally before processing the message.
//
// This is useful for:
// - Automated workflows that need to chain multiple turns
// - Background tasks that should run after the main task completes
// - Scheduled follow-up actions
//
// The message will be processed via Continue() when the agent becomes idle.
func (al *AgentLoop) InjectFollowUp(msg providers.Message) error {
// InjectFollowUp uses the same steering queue mechanism as Steer(),
// but the semantic difference is in when it's called:
// - Steer() is called during active execution to interrupt
// - InjectFollowUp() is called when planning future work
//
// Both end up in the same queue and are processed by Continue()
// when the agent is idle.
return al.Steer(msg)
}
// ====================== API Aliases for Design Document Compatibility ======================
// InjectSteering is an alias for Steer() to match the design document naming.
// It injects a steering message into the currently running agent loop.
func (al *AgentLoop) InjectSteering(msg providers.Message) error {
return al.Steer(msg)
}
pkg/agent/steering_test.go
+1591
View File
File diff suppressed because it is too large Load Diff
pkg/agent/subturn.go
+671
View File
@@ -0,0 +1,671 @@
package agent
import (
"context"
"errors"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/sipeed/picoclaw/pkg/logger"
"github.com/sipeed/picoclaw/pkg/providers"
"github.com/sipeed/picoclaw/pkg/tools"
)
// ====================== Config & Constants ======================
const (
// Default values for SubTurn configuration (used when config is not set or is zero)
defaultMaxSubTurnDepth = 3
defaultMaxConcurrentSubTurns = 5
defaultConcurrencyTimeout = 30 * time.Second
defaultSubTurnTimeout = 5 * time.Minute
// maxEphemeralHistorySize limits the number of messages stored in ephemeral sessions.
// This prevents memory accumulation in long-running sub-turns.
maxEphemeralHistorySize = 50
)
var (
ErrDepthLimitExceeded = errors.New("sub-turn depth limit exceeded")
ErrInvalidSubTurnConfig = errors.New("invalid sub-turn config")
ErrConcurrencyTimeout = errors.New("timeout waiting for concurrency slot")
)
// getSubTurnConfig returns the effective SubTurn configuration with defaults applied.
func (al *AgentLoop) getSubTurnConfig() subTurnRuntimeConfig {
cfg := al.cfg.Agents.Defaults.SubTurn
maxDepth := cfg.MaxDepth
if maxDepth <= 0 {
maxDepth = defaultMaxSubTurnDepth
}
maxConcurrent := cfg.MaxConcurrent
if maxConcurrent <= 0 {
maxConcurrent = defaultMaxConcurrentSubTurns
}
concurrencyTimeout := time.Duration(cfg.ConcurrencyTimeoutSec) * time.Second
if concurrencyTimeout <= 0 {
concurrencyTimeout = defaultConcurrencyTimeout
}
defaultTimeout := time.Duration(cfg.DefaultTimeoutMinutes) * time.Minute
if defaultTimeout <= 0 {
defaultTimeout = defaultSubTurnTimeout
}
return subTurnRuntimeConfig{
maxDepth: maxDepth,
maxConcurrent: maxConcurrent,
concurrencyTimeout: concurrencyTimeout,
defaultTimeout: defaultTimeout,
defaultTokenBudget: cfg.DefaultTokenBudget,
}
}
// subTurnRuntimeConfig holds the effective runtime configuration for SubTurn execution.
type subTurnRuntimeConfig struct {
maxDepth int
maxConcurrent int
concurrencyTimeout time.Duration
defaultTimeout time.Duration
defaultTokenBudget int
}
// ====================== SubTurn Config ======================
// SubTurnConfig configures the execution of a child sub-turn.
//
// Usage Examples:
//
// Synchronous sub-turn (Async=false):
//
// cfg := SubTurnConfig{
// Model: "gpt-4o-mini",
// SystemPrompt: "Analyze this code",
// Async: false, // Result returned immediately
// }
// result, err := SpawnSubTurn(ctx, cfg)
// // Use result directly here
// processResult(result)
//
// Asynchronous sub-turn (Async=true):
//
// cfg := SubTurnConfig{
// Model: "gpt-4o-mini",
// SystemPrompt: "Background analysis",
// Async: true, // Result delivered to channel
// }
// result, err := SpawnSubTurn(ctx, cfg)
// // Result also available in parent's pendingResults channel
// // Parent turn will poll and process it in a later iteration
type SubTurnConfig struct {
Model string
Tools []tools.Tool
SystemPrompt string
MaxTokens int
// Async controls the result delivery mechanism:
//
// When Async = false (synchronous sub-turn):
// - The caller blocks until the sub-turn completes
// - The result is ONLY returned via the function return value
// - The result is NOT delivered to the parent's pendingResults channel
// - This prevents double delivery: caller gets result immediately, no need for channel
// - Use case: When the caller needs the result immediately to continue execution
// - Example: A tool that needs to process the sub-turn result before returning
//
// When Async = true (asynchronous sub-turn):
// - The sub-turn runs in the background (still blocks the caller, but semantically async)
// - The result is delivered to the parent's pendingResults channel
// - The result is ALSO returned via the function return value (for consistency)
// - The parent turn can poll pendingResults in later iterations to process results
// - Use case: Fire-and-forget operations, or when results are processed in batches
// - Example: Spawning multiple sub-turns in parallel and collecting results later
//
// IMPORTANT: The Async flag does NOT make the call non-blocking. It only controls
// whether the result is delivered via the channel. For true non-blocking execution,
// the caller must spawn the sub-turn in a separate goroutine.
Async bool
// Critical indicates this SubTurn's result is important and should continue
// running even after the parent turn finishes gracefully.
//
// When parent finishes gracefully (Finish(false)):
// - Critical=true: SubTurn continues running, delivers result as orphan
// - Critical=false: SubTurn exits gracefully without error
//
// When parent finishes with hard abort (Finish(true)):
// - All SubTurns are canceled regardless of Critical flag
Critical bool
// Timeout is the maximum duration for this SubTurn.
// If the SubTurn runs longer than this, it will be canceled.
// Default is 5 minutes (defaultSubTurnTimeout) if not specified.
Timeout time.Duration
// MaxContextRunes limits the context size (in runes) passed to the SubTurn.
// This prevents context window overflow by truncating message history before LLM calls.
//
// Values:
// 0 = Auto-calculate based on model's ContextWindow * 0.75 (default, recommended)
// -1 = No limit (disable soft truncation, rely only on hard context errors)
// >0 = Use specified rune limit
//
// The soft limit acts as a first line of defense before hitting the provider's
// hard context window limit. When exceeded, older messages are intelligently
// truncated while preserving system messages and recent context.
MaxContextRunes int
// ActualSystemPrompt is injected as the true 'system' role message for the childAgent.
// The legacy SystemPrompt field is actually used as the first 'user' message (task description).
ActualSystemPrompt string
// InitialMessages preloads the ephemeral session history before the agent loop starts.
// Used by evaluator-optimizer patterns to pass the full worker context across multiple iterations.
InitialMessages []providers.Message
// InitialTokenBudget is a shared atomic counter for tracking remaining tokens.
// If set, the SubTurn will inherit this budget and deduct tokens after each LLM call.
// If nil, the SubTurn will inherit the parent's tokenBudget (if any).
// Used by team tool to enforce token limits across all team members.
InitialTokenBudget *atomic.Int64
// Can be extended with temperature, topP, etc.
}
// ====================== Context Keys ======================
type agentLoopKeyType struct{}
var agentLoopKey = agentLoopKeyType{}
// WithAgentLoop injects AgentLoop into context for tool access
func WithAgentLoop(ctx context.Context, al *AgentLoop) context.Context {
return context.WithValue(ctx, agentLoopKey, al)
}
// AgentLoopFromContext retrieves AgentLoop from context
func AgentLoopFromContext(ctx context.Context) *AgentLoop {
al, _ := ctx.Value(agentLoopKey).(*AgentLoop)
return al
}
// ====================== Helper Functions ======================
func (al *AgentLoop) generateSubTurnID() string {
return fmt.Sprintf("subturn-%d", al.subTurnCounter.Add(1))
}
// ====================== Core Function: spawnSubTurn ======================
// AgentLoopSpawner implements tools.SubTurnSpawner interface.
// This allows tools to spawn sub-turns without circular dependency.
type AgentLoopSpawner struct {
al *AgentLoop
}
// SpawnSubTurn implements tools.SubTurnSpawner interface.
func (s *AgentLoopSpawner) SpawnSubTurn(
ctx context.Context,
cfg tools.SubTurnConfig,
) (*tools.ToolResult, error) {
parentTS := turnStateFromContext(ctx)
if parentTS == nil {
return nil, errors.New(
"parent turnState not found in context - cannot spawn sub-turn outside of a turn",
)
}
// Convert tools.SubTurnConfig to agent.SubTurnConfig
agentCfg := SubTurnConfig{
Model: cfg.Model,
Tools: cfg.Tools,
SystemPrompt: cfg.SystemPrompt,
ActualSystemPrompt: cfg.ActualSystemPrompt,
InitialMessages: cfg.InitialMessages,
InitialTokenBudget: cfg.InitialTokenBudget,
MaxTokens: cfg.MaxTokens,
Async: cfg.Async,
Critical: cfg.Critical,
Timeout: cfg.Timeout,
MaxContextRunes: cfg.MaxContextRunes,
}
return spawnSubTurn(ctx, s.al, parentTS, agentCfg)
}
// NewSubTurnSpawner creates a SubTurnSpawner for the given AgentLoop.
func NewSubTurnSpawner(al *AgentLoop) *AgentLoopSpawner {
return &AgentLoopSpawner{al: al}
}
// SpawnSubTurn is the exported entry point for tools to spawn sub-turns.
// It retrieves AgentLoop and parent turnState from context and delegates to spawnSubTurn.
func SpawnSubTurn(ctx context.Context, cfg SubTurnConfig) (*tools.ToolResult, error) {
al := AgentLoopFromContext(ctx)
if al == nil {
return nil, errors.New(
"AgentLoop not found in context - ensure context is properly initialized",
)
}
parentTS := turnStateFromContext(ctx)
if parentTS == nil {
return nil, errors.New(
"parent turnState not found in context - cannot spawn sub-turn outside of a turn",
)
}
return spawnSubTurn(ctx, al, parentTS, cfg)
}
func spawnSubTurn(
ctx context.Context,
al *AgentLoop,
parentTS *turnState,
cfg SubTurnConfig,
) (result *tools.ToolResult, err error) {
// Get effective SubTurn configuration
rtCfg := al.getSubTurnConfig()
// 0. Acquire concurrency semaphore FIRST to ensure it's released even if early validation fails.
// Blocks if parent already has maxConcurrentSubTurns running, with a timeout to prevent indefinite blocking.
// Also respects context cancellation so we don't block forever if parent is aborted.
// NOTE: The semaphore is released immediately after runTurn completes (not in a defer) to
// ensure it is freed before the cleanup phase (async result delivery), which may block on
// a full pendingResults channel. Holding the semaphore through cleanup would allow the
// parent's goroutine to be blocked waiting for a semaphore slot while child turns are
// blocked delivering results — a deadlock.
var semAcquired bool
if parentTS.concurrencySem != nil {
// Create a timeout context for semaphore acquisition
timeoutCtx, cancel := context.WithTimeout(ctx, rtCfg.concurrencyTimeout)
defer cancel()
select {
case parentTS.concurrencySem <- struct{}{}:
semAcquired = true
defer func() {
if semAcquired {
<-parentTS.concurrencySem
}
}()
case <-timeoutCtx.Done():
// Check parent context first - if it was canceled, propagate that error
if ctx.Err() != nil {
return nil, ctx.Err()
}
// Otherwise it's our timeout
return nil, fmt.Errorf("%w: all %d slots occupied for %v",
ErrConcurrencyTimeout, rtCfg.maxConcurrent, rtCfg.concurrencyTimeout)
}
}
// 1. Depth limit check
if parentTS.depth >= rtCfg.maxDepth {
logger.WarnCF("subturn", "Depth limit exceeded", map[string]any{
"parent_id": parentTS.turnID,
"depth": parentTS.depth,
"max_depth": rtCfg.maxDepth,
})
return nil, ErrDepthLimitExceeded
}
// 2. Config validation
if cfg.Model == "" {
return nil, ErrInvalidSubTurnConfig
}
// 3. Determine timeout for child SubTurn
timeout := cfg.Timeout
if timeout <= 0 {
timeout = rtCfg.defaultTimeout
}
// 4. Create INDEPENDENT child context (not derived from parent ctx).
// This allows the child to continue running after parent finishes gracefully.
// The child has its own timeout for self-protection.
childCtx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
childID := al.generateSubTurnID()
// Get the agent instance from parent, falling back to the default agent.
// Wrap it in a shallow copy that uses an ephemeral (in-memory only) session store
// so that child turns never pollute or persist to the parent's session history.
baseAgent := parentTS.agent
if baseAgent == nil {
baseAgent = al.registry.GetDefaultAgent()
}
if baseAgent == nil {
return nil, errors.New("parent turnState has no agent instance")
}
ephemeralStore := newEphemeralSession(nil)
agent := *baseAgent // shallow copy
agent.Sessions = ephemeralStore
// Clone the tool registry so child turn's tool registrations
// don't pollute the parent's registry.
if baseAgent.Tools != nil {
agent.Tools = baseAgent.Tools.Clone()
}
// Create processOptions for the child turn
opts := processOptions{
SessionKey: childID,
Channel: parentTS.channel,
ChatID: parentTS.chatID,
SenderID: parentTS.opts.SenderID,
SenderDisplayName: parentTS.opts.SenderDisplayName,
UserMessage: cfg.SystemPrompt, // Task description becomes the first user message
SystemPromptOverride: cfg.ActualSystemPrompt,
Media: nil,
InitialSteeringMessages: cfg.InitialMessages,
DefaultResponse: "",
EnableSummary: false,
SendResponse: false,
NoHistory: true, // SubTurns don't use session history
SkipInitialSteeringPoll: true,
}
// Create event scope for the child turn
scope := al.newTurnEventScope(agent.ID, childID)
// Create child turnState using the new API
childTS := newTurnState(&agent, opts, scope)
// Set SubTurn-specific fields
childTS.cancelFunc = cancel
childTS.critical = cfg.Critical
childTS.depth = parentTS.depth + 1
childTS.parentTurnID = parentTS.turnID
childTS.parentTurnState = parentTS
childTS.pendingResults = make(chan *tools.ToolResult, 16)
childTS.concurrencySem = make(chan struct{}, rtCfg.maxConcurrent)
childTS.al = al // back-ref for hard abort cascade
childTS.session = ephemeralStore // same store as agent.Sessions
// Token budget initialization/inheritance
// If InitialTokenBudget is explicitly provided (e.g., by team tool), use it.
// Otherwise, inherit from parent's tokenBudget (for nested SubTurns).
if cfg.InitialTokenBudget != nil {
childTS.tokenBudget = cfg.InitialTokenBudget
} else if parentTS.tokenBudget != nil {
childTS.tokenBudget = parentTS.tokenBudget
} else if rtCfg.defaultTokenBudget > 0 {
// Apply default token budget from config if no budget is set
budget := &atomic.Int64{}
budget.Store(int64(rtCfg.defaultTokenBudget))
childTS.tokenBudget = budget
}
// IMPORTANT: Put childTS into childCtx so that code inside runTurn can retrieve it
childCtx = withTurnState(childCtx, childTS)
childCtx = WithAgentLoop(childCtx, al) // Propagate AgentLoop to child turn
childTS.ctx = childCtx
// Register child turn state so GetAllActiveTurns/Subagents can find it
al.activeTurnStates.Store(childID, childTS)
defer al.activeTurnStates.Delete(childID)
// 5. Establish parent-child relationship (thread-safe)
parentTS.mu.Lock()
parentTS.childTurnIDs = append(parentTS.childTurnIDs, childID)
parentTS.mu.Unlock()
// 6. Emit Spawn event
al.emitEvent(EventKindSubTurnSpawn,
childTS.eventMeta("spawnSubTurn", "subturn.spawn"),
SubTurnSpawnPayload{
AgentID: childTS.agentID,
Label: childID,
ParentTurnID: parentTS.turnID,
},
)
// 7. Defer cleanup: deliver result (for async), emit End event, and recover from panics
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("subturn panicked: %v", r)
result = nil
logger.ErrorCF("subturn", "SubTurn panicked", map[string]any{
"child_id": childID,
"parent_id": parentTS.turnID,
"panic": r,
})
}
// Result Delivery Strategy (Async vs Sync)
if cfg.Async {
deliverSubTurnResult(al, parentTS, childID, result)
}
status := "completed"
if err != nil {
status = "error"
}
al.emitEvent(EventKindSubTurnEnd,
childTS.eventMeta("spawnSubTurn", "subturn.end"),
SubTurnEndPayload{
AgentID: childTS.agentID,
Status: status,
},
)
}()
// 8. Execute sub-turn via the real agent loop.
turnRes, turnErr := al.runTurn(childCtx, childTS)
// Release the concurrency semaphore immediately after runTurn completes,
// before the cleanup defer runs. This prevents a deadlock where:
// - All semaphore slots are held by sub-turns in their cleanup phase
// - Cleanup blocks on a full pendingResults channel
// - The parent goroutine is blocked waiting for a semaphore slot
// - The parent cannot consume pendingResults because it is blocked on the semaphore
if semAcquired {
<-parentTS.concurrencySem
semAcquired = false // prevent the defer from double-releasing
}
// Convert turnResult to tools.ToolResult
if turnErr != nil {
err = turnErr
result = &tools.ToolResult{
Err: turnErr,
ForLLM: fmt.Sprintf("SubTurn failed: %v", turnErr),
}
} else {
result = &tools.ToolResult{
ForLLM: turnRes.finalContent,
ForUser: turnRes.finalContent,
}
}
return result, err
}
// ====================== Result Delivery ======================
// deliverSubTurnResult delivers a sub-turn result to the parent turn's pendingResults channel.
//
// IMPORTANT: This function is ONLY called for asynchronous sub-turns (Async=true).
// For synchronous sub-turns (Async=false), results are returned directly via the function
// return value to avoid double delivery.
//
// Delivery behavior:
// - If parent turn is still running: attempts to deliver to pendingResults channel
// - If channel is full: emits SubTurnOrphanResultEvent (result is lost from channel but tracked)
// - If parent turn has finished: emits SubTurnOrphanResultEvent (late arrival)
//
// Thread safety:
// - Reads parent state under lock, then releases lock before channel send
// - Small race window exists but is acceptable (worst case: result becomes orphan)
//
// Event emissions:
// - SubTurnResultDeliveredEvent: successful delivery to channel
// - SubTurnOrphanResultEvent: delivery failed (parent finished or channel full)
func deliverSubTurnResult(al *AgentLoop, parentTS *turnState, childID string, result *tools.ToolResult) {
// Let GC clean up the pendingResults channel; parent Finish will no longer close it.
// We use defer/recover to catch any unlikely channel panics if it were ever closed.
defer func() {
if r := recover(); r != nil {
logger.WarnCF("subturn", "recovered panic sending to pendingResults", map[string]any{
"parent_id": parentTS.turnID,
"child_id": childID,
"recover": r,
})
if result != nil && al != nil {
al.emitEvent(EventKindSubTurnOrphan,
parentTS.eventMeta("deliverSubTurnResult", "subturn.orphan"),
SubTurnOrphanPayload{ParentTurnID: parentTS.turnID, ChildTurnID: childID, Reason: "panic"},
)
}
}
}()
parentTS.mu.Lock()
isFinished := parentTS.isFinished.Load()
resultChan := parentTS.pendingResults
parentTS.mu.Unlock()
// If parent turn has already finished, treat this as an orphan result
if isFinished || resultChan == nil {
if result != nil && al != nil {
al.emitEvent(EventKindSubTurnOrphan,
parentTS.eventMeta("deliverSubTurnResult", "subturn.orphan"),
SubTurnOrphanPayload{ParentTurnID: parentTS.turnID, ChildTurnID: childID, Reason: "parent_finished"},
)
}
return
}
// Parent Turn is still running → attempt to deliver result
// We use a select statement with parentTS.Finished() to ensure that if the
// parent turn finishes while we are waiting to send the result (e.g. channel
// is full), we don't leak this goroutine by blocking forever.
select {
case resultChan <- result:
// Successfully delivered
if al != nil {
al.emitEvent(EventKindSubTurnResultDelivered,
parentTS.eventMeta("deliverSubTurnResult", "subturn.result_delivered"),
SubTurnResultDeliveredPayload{ContentLen: len(result.ForLLM)},
)
}
case <-parentTS.Finished():
// Parent finished while we were waiting to deliver.
// The result cannot be delivered to the LLM, so it becomes an orphan.
logger.WarnCF("subturn", "parent finished before result could be delivered", map[string]any{
"parent_id": parentTS.turnID,
"child_id": childID,
})
if result != nil && al != nil {
al.emitEvent(
EventKindSubTurnOrphan,
parentTS.eventMeta("deliverSubTurnResult", "subturn.orphan"),
SubTurnOrphanPayload{
ParentTurnID: parentTS.turnID,
ChildTurnID: childID,
Reason: "parent_finished_waiting",
},
)
}
}
}
// ====================== Other Types ======================
// ephemeralSessionStore is an in-memory session.SessionStore used by SubTurns.
// It does not persist to disk and auto-truncates history to maxEphemeralHistorySize.
type ephemeralSessionStore struct {
mu sync.Mutex
history []providers.Message
summary string
}
func newEphemeralSession(initial []providers.Message) ephemeralSessionStoreIface {
s := &ephemeralSessionStore{}
if len(initial) > 0 {
s.history = append(s.history, initial...)
}
return s
}
// ephemeralSessionStoreIface is satisfied by *ephemeralSessionStore.
// Declared so newEphemeralSession can return a typed interface.
type ephemeralSessionStoreIface interface {
AddMessage(sessionKey, role, content string)
AddFullMessage(sessionKey string, msg providers.Message)
GetHistory(key string) []providers.Message
GetSummary(key string) string
SetSummary(key, summary string)
SetHistory(key string, history []providers.Message)
TruncateHistory(key string, keepLast int)
Save(key string) error
Close() error
}
func (e *ephemeralSessionStore) AddMessage(_, role, content string) {
e.mu.Lock()
defer e.mu.Unlock()
e.history = append(e.history, providers.Message{Role: role, Content: content})
e.truncateLocked()
}
func (e *ephemeralSessionStore) AddFullMessage(_ string, msg providers.Message) {
e.mu.Lock()
defer e.mu.Unlock()
e.history = append(e.history, msg)
e.truncateLocked()
}
func (e *ephemeralSessionStore) GetHistory(_ string) []providers.Message {
e.mu.Lock()
defer e.mu.Unlock()
out := make([]providers.Message, len(e.history))
copy(out, e.history)
return out
}
func (e *ephemeralSessionStore) GetSummary(_ string) string {
e.mu.Lock()
defer e.mu.Unlock()
return e.summary
}
func (e *ephemeralSessionStore) SetSummary(_, summary string) {
e.mu.Lock()
defer e.mu.Unlock()
e.summary = summary
}
func (e *ephemeralSessionStore) SetHistory(_ string, history []providers.Message) {
e.mu.Lock()
defer e.mu.Unlock()
e.history = make([]providers.Message, len(history))
copy(e.history, history)
e.truncateLocked()
}
func (e *ephemeralSessionStore) TruncateHistory(_ string, keepLast int) {
e.mu.Lock()
defer e.mu.Unlock()
if keepLast <= 0 {
e.history = nil
return
}
if keepLast >= len(e.history) {
return
}
e.history = e.history[len(e.history)-keepLast:]
}
func (e *ephemeralSessionStore) Save(_ string) error { return nil }
func (e *ephemeralSessionStore) Close() error { return nil }
func (e *ephemeralSessionStore) truncateLocked() {
if len(e.history) > maxEphemeralHistorySize {
e.history = e.history[len(e.history)-maxEphemeralHistorySize:]
}
}
pkg/agent/subturn_test.go
+2067
View File
File diff suppressed because it is too large Load Diff
pkg/agent/turn.go
+481
View File
@@ -0,0 +1,481 @@
package agent
import (
"context"
"reflect"
"sync"
"sync/atomic"
"time"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/providers"
"github.com/sipeed/picoclaw/pkg/session"
"github.com/sipeed/picoclaw/pkg/tools"
)
type TurnPhase string
const (
TurnPhaseSetup TurnPhase = "setup"
TurnPhaseRunning TurnPhase = "running"
TurnPhaseTools TurnPhase = "tools"
TurnPhaseFinalizing TurnPhase = "finalizing"
TurnPhaseCompleted TurnPhase = "completed"
TurnPhaseAborted TurnPhase = "aborted"
)
type ActiveTurnInfo struct {
TurnID string
AgentID string
SessionKey string
Channel string
ChatID string
UserMessage string
Phase TurnPhase
Iteration int
StartedAt time.Time
Depth int
ParentTurnID string
ChildTurnIDs []string
}
type turnResult struct {
finalContent string
status TurnEndStatus
followUps []bus.InboundMessage
}
type turnState struct {
mu sync.RWMutex
agent *AgentInstance
opts processOptions
scope turnEventScope
turnID string
agentID string
sessionKey string
channel string
chatID string
userMessage string
media []string
phase TurnPhase
iteration int
startedAt time.Time
finalContent string
followUps []bus.InboundMessage
gracefulInterrupt bool
gracefulInterruptHint string
gracefulTerminalUsed bool
hardAbort bool
providerCancel context.CancelFunc
turnCancel context.CancelFunc
restorePointHistory []providers.Message
restorePointSummary string
persistedMessages []providers.Message
// SubTurn support (from HEAD)
depth int // SubTurn depth (0 for root turn)
parentTurnID string // Parent turn ID (empty for root turn)
childTurnIDs []string // Child turn IDs
pendingResults chan *tools.ToolResult // Channel for SubTurn results
concurrencySem chan struct{} // Semaphore for limiting concurrent SubTurns
isFinished atomic.Bool // Whether this turn has finished
session session.SessionStore // Session store reference
initialHistoryLength int // Snapshot of history length at turn start
// Additional SubTurn fields
ctx context.Context // Context for this turn
cancelFunc context.CancelFunc // Cancel function for this turn's context
critical bool // Whether this SubTurn should continue after parent ends
parentTurnState *turnState // Reference to parent turnState
parentEnded atomic.Bool // Whether parent has ended
closeOnce sync.Once // Ensures pendingResults channel is closed once
finishedChan chan struct{} // Closed when turn finishes
// Token budget tracking
tokenBudget *atomic.Int64 // Shared token budget counter
lastFinishReason string // Last LLM finish_reason
lastUsage *providers.UsageInfo // Last LLM usage info
// Back-reference to the owning AgentLoop (set for SubTurns only, used for hard abort cascade)
al *AgentLoop
}
func newTurnState(agent *AgentInstance, opts processOptions, scope turnEventScope) *turnState {
ts := &turnState{
agent: agent,
opts: opts,
scope: scope,
turnID: scope.turnID,
agentID: agent.ID,
sessionKey: opts.SessionKey,
channel: opts.Channel,
chatID: opts.ChatID,
userMessage: opts.UserMessage,
media: append([]string(nil), opts.Media...),
phase: TurnPhaseSetup,
startedAt: time.Now(),
}
// Bind session store and capture initial history length for rollback logic
if agent != nil && agent.Sessions != nil {
ts.session = agent.Sessions
ts.initialHistoryLength = len(agent.Sessions.GetHistory(opts.SessionKey))
}
return ts
}
func (al *AgentLoop) registerActiveTurn(ts *turnState) {
al.activeTurnStates.Store(ts.sessionKey, ts)
}
func (al *AgentLoop) clearActiveTurn(ts *turnState) {
al.activeTurnStates.Delete(ts.sessionKey)
}
func (al *AgentLoop) getActiveTurnState(sessionKey string) *turnState {
if val, ok := al.activeTurnStates.Load(sessionKey); ok {
return val.(*turnState)
}
return nil
}
// getAnyActiveTurnState returns any active turn state (for backward compatibility)
func (al *AgentLoop) getAnyActiveTurnState() *turnState {
var firstTS *turnState
al.activeTurnStates.Range(func(key, value any) bool {
firstTS = value.(*turnState)
return false // stop after first
})
return firstTS
}
func (al *AgentLoop) GetActiveTurn() *ActiveTurnInfo {
// For backward compatibility, return the first active turn found
// In the new architecture, there can be multiple concurrent turns
var firstTS *turnState
al.activeTurnStates.Range(func(key, value any) bool {
firstTS = value.(*turnState)
return false // stop after first
})
if firstTS == nil {
return nil
}
info := firstTS.snapshot()
return &info
}
func (al *AgentLoop) GetActiveTurnBySession(sessionKey string) *ActiveTurnInfo {
ts := al.getActiveTurnState(sessionKey)
if ts == nil {
return nil
}
info := ts.snapshot()
return &info
}
func (ts *turnState) snapshot() ActiveTurnInfo {
ts.mu.RLock()
defer ts.mu.RUnlock()
return ActiveTurnInfo{
TurnID: ts.turnID,
AgentID: ts.agentID,
SessionKey: ts.sessionKey,
Channel: ts.channel,
ChatID: ts.chatID,
UserMessage: ts.userMessage,
Phase: ts.phase,
Iteration: ts.iteration,
StartedAt: ts.startedAt,
Depth: ts.depth,
ParentTurnID: ts.parentTurnID,
ChildTurnIDs: append([]string(nil), ts.childTurnIDs...),
}
}
func (ts *turnState) setPhase(phase TurnPhase) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.phase = phase
}
func (ts *turnState) setIteration(iteration int) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.iteration = iteration
}
func (ts *turnState) currentIteration() int {
ts.mu.RLock()
defer ts.mu.RUnlock()
return ts.iteration
}
func (ts *turnState) setFinalContent(content string) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.finalContent = content
}
func (ts *turnState) finalContentLen() int {
ts.mu.RLock()
defer ts.mu.RUnlock()
return len(ts.finalContent)
}
func (ts *turnState) setTurnCancel(cancel context.CancelFunc) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.turnCancel = cancel
}
func (ts *turnState) setProviderCancel(cancel context.CancelFunc) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.providerCancel = cancel
}
func (ts *turnState) clearProviderCancel(_ context.CancelFunc) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.providerCancel = nil
}
func (ts *turnState) requestGracefulInterrupt(hint string) bool {
ts.mu.Lock()
defer ts.mu.Unlock()
if ts.hardAbort {
return false
}
ts.gracefulInterrupt = true
ts.gracefulInterruptHint = hint
return true
}
func (ts *turnState) gracefulInterruptRequested() (bool, string) {
ts.mu.RLock()
defer ts.mu.RUnlock()
return ts.gracefulInterrupt && !ts.gracefulTerminalUsed, ts.gracefulInterruptHint
}
func (ts *turnState) markGracefulTerminalUsed() {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.gracefulTerminalUsed = true
}
func (ts *turnState) requestHardAbort() bool {
ts.mu.Lock()
if ts.hardAbort {
ts.mu.Unlock()
return false
}
ts.hardAbort = true
turnCancel := ts.turnCancel
providerCancel := ts.providerCancel
ts.mu.Unlock()
if providerCancel != nil {
providerCancel()
}
if turnCancel != nil {
turnCancel()
}
return true
}
func (ts *turnState) hardAbortRequested() bool {
ts.mu.RLock()
defer ts.mu.RUnlock()
return ts.hardAbort
}
func (ts *turnState) eventMeta(source, tracePath string) EventMeta {
snap := ts.snapshot()
return EventMeta{
AgentID: snap.AgentID,
TurnID: snap.TurnID,
SessionKey: snap.SessionKey,
Iteration: snap.Iteration,
Source: source,
TracePath: tracePath,
}
}
func (ts *turnState) captureRestorePoint(history []providers.Message, summary string) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.restorePointHistory = append([]providers.Message(nil), history...)
ts.restorePointSummary = summary
}
func (ts *turnState) recordPersistedMessage(msg providers.Message) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.persistedMessages = append(ts.persistedMessages, msg)
}
func (ts *turnState) refreshRestorePointFromSession(agent *AgentInstance) {
history := agent.Sessions.GetHistory(ts.sessionKey)
summary := agent.Sessions.GetSummary(ts.sessionKey)
ts.mu.RLock()
persisted := append([]providers.Message(nil), ts.persistedMessages...)
ts.mu.RUnlock()
if matched := matchingTurnMessageTail(history, persisted); matched > 0 {
history = append([]providers.Message(nil), history[:len(history)-matched]...)
}
ts.captureRestorePoint(history, summary)
}
func (ts *turnState) restoreSession(agent *AgentInstance) error {
ts.mu.RLock()
history := append([]providers.Message(nil), ts.restorePointHistory...)
summary := ts.restorePointSummary
ts.mu.RUnlock()
agent.Sessions.SetHistory(ts.sessionKey, history)
agent.Sessions.SetSummary(ts.sessionKey, summary)
return agent.Sessions.Save(ts.sessionKey)
}
func matchingTurnMessageTail(history, persisted []providers.Message) int {
maxMatch := min(len(history), len(persisted))
for size := maxMatch; size > 0; size-- {
if reflect.DeepEqual(history[len(history)-size:], persisted[len(persisted)-size:]) {
return size
}
}
return 0
}
func (ts *turnState) interruptHintMessage() providers.Message {
_, hint := ts.gracefulInterruptRequested()
content := "Interrupt requested. Stop scheduling tools and provide a short final summary."
if hint != "" {
content += "\n\nInterrupt hint: " + hint
}
return providers.Message{
Role: "user",
Content: content,
}
}
// SubTurn-related methods
// Finish marks the turn as finished and closes the pendingResults channel
func (ts *turnState) Finish(isHardAbort bool) {
ts.isFinished.Store(true)
// Close pendingResults channel exactly once
ts.closeOnce.Do(func() {
if ts.pendingResults != nil {
close(ts.pendingResults)
}
ts.mu.Lock()
if ts.finishedChan == nil {
ts.finishedChan = make(chan struct{})
}
close(ts.finishedChan)
ts.mu.Unlock()
})
// If this is a graceful finish (not hard abort), signal to children
if !isHardAbort && ts.parentTurnState == nil {
// This is a root turn finishing gracefully
ts.parentEnded.Store(true)
}
// Cancel the turn context
if ts.cancelFunc != nil {
ts.cancelFunc()
}
// Hard abort cascades to all child turns
if isHardAbort && ts.al != nil {
ts.mu.RLock()
children := append([]string(nil), ts.childTurnIDs...)
ts.mu.RUnlock()
for _, childID := range children {
if val, ok := ts.al.activeTurnStates.Load(childID); ok {
val.(*turnState).Finish(true)
}
}
}
}
// Finished returns whether the turn has finished
func (ts *turnState) Finished() chan struct{} {
ts.mu.Lock()
defer ts.mu.Unlock()
if ts.finishedChan == nil {
ts.finishedChan = make(chan struct{})
}
return ts.finishedChan
}
// IsParentEnded checks if the parent turn has ended
func (ts *turnState) IsParentEnded() bool {
if ts.parentTurnState == nil {
return false
}
return ts.parentTurnState.parentEnded.Load()
}
// GetLastFinishReason returns the last LLM finish_reason
func (ts *turnState) GetLastFinishReason() string {
ts.mu.RLock()
defer ts.mu.RUnlock()
return ts.lastFinishReason
}
// SetLastFinishReason sets the last LLM finish_reason
func (ts *turnState) SetLastFinishReason(reason string) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.lastFinishReason = reason
}
// GetLastUsage returns the last LLM usage info
func (ts *turnState) GetLastUsage() *providers.UsageInfo {
ts.mu.RLock()
defer ts.mu.RUnlock()
return ts.lastUsage
}
// SetLastUsage sets the last LLM usage info
func (ts *turnState) SetLastUsage(usage *providers.UsageInfo) {
ts.mu.Lock()
defer ts.mu.Unlock()
ts.lastUsage = usage
}
// Context helper functions for SubTurn
type turnStateKeyType struct{}
var turnStateKey = turnStateKeyType{}
func withTurnState(ctx context.Context, ts *turnState) context.Context {
return context.WithValue(ctx, turnStateKey, ts)
}
func turnStateFromContext(ctx context.Context) *turnState {
ts, _ := ctx.Value(turnStateKey).(*turnState)
return ts
}
// TurnStateFromContext retrieves turnState from context (exported for tools)
func TurnStateFromContext(ctx context.Context) *turnState {
return turnStateFromContext(ctx)
}
pkg/commands/builtin.go
+1
View File
@@ -14,6 +14,7 @@ func BuiltinDefinitions() []Definition {
switchCommand(),
checkCommand(),
clearCommand(),
subagentsCommand(),
reloadCommand(),
}
}
pkg/commands/cmd_subagents.go
+42
View File
@@ -0,0 +1,42 @@
package commands
import (
"context"
"fmt"
)
// TurnInfo is a mirrored struct from agent.TurnInfo to avoid circular dependencies.
type TurnInfo struct {
TurnID string
ParentTurnID string
Depth int
ChildTurnIDs []string
IsFinished bool
}
func subagentsCommand() Definition {
return Definition{
Name: "subagents",
Description: "Show running subagents and task tree",
Handler: func(ctx context.Context, req Request, rt *Runtime) error {
getTurnFn := rt.GetActiveTurn
if getTurnFn == nil {
return req.Reply("Runtime does not support querying active turns.")
}
turnRaw := getTurnFn()
if turnRaw == nil {
return req.Reply("No active tasks running in this session.")
}
if treeStr, ok := turnRaw.(string); ok {
if treeStr == "" {
return req.Reply("No active tasks running in this session.")
}
return req.Reply(fmt.Sprintf("🤖 **Active Subagents Tree**\n```text\n%s\n```", treeStr))
}
return req.Reply(fmt.Sprintf("🤖 **Active Subagents List**\n```text\n%+v\n```", turnRaw))
},
}
}
pkg/commands/runtime.go
+1
View File
@@ -12,6 +12,7 @@ type Runtime struct {
ListDefinitions func() []Definition
ListSkillNames func() []string
GetEnabledChannels func() []string
GetActiveTurn func() any // Returning any to avoid circular dependency with agent package
SwitchModel func(value string) (oldModel string, err error)
SwitchChannel func(value string) error
ClearHistory func() error
pkg/config/config.go
+58 -8
View File
@@ -84,6 +84,7 @@ type Config struct {
Providers ProvidersConfig `json:"providers,omitempty"`
ModelList []ModelConfig `json:"model_list"` // New model-centric provider configuration
Gateway GatewayConfig `json:"gateway"`
Hooks HooksConfig `json:"hooks,omitempty"`
Tools ToolsConfig `json:"tools"`
Heartbeat HeartbeatConfig `json:"heartbeat"`
Devices DevicesConfig `json:"devices"`
@@ -92,6 +93,36 @@ type Config struct {
BuildInfo BuildInfo `json:"build_info,omitempty"`
}
type HooksConfig struct {
Enabled bool `json:"enabled"`
Defaults HookDefaultsConfig `json:"defaults,omitempty"`
Builtins map[string]BuiltinHookConfig `json:"builtins,omitempty"`
Processes map[string]ProcessHookConfig `json:"processes,omitempty"`
}
type HookDefaultsConfig struct {
ObserverTimeoutMS int `json:"observer_timeout_ms,omitempty"`
InterceptorTimeoutMS int `json:"interceptor_timeout_ms,omitempty"`
ApprovalTimeoutMS int `json:"approval_timeout_ms,omitempty"`
}
type BuiltinHookConfig struct {
Enabled bool `json:"enabled"`
Priority int `json:"priority,omitempty"`
Config json.RawMessage `json:"config,omitempty"`
}
type ProcessHookConfig struct {
Enabled bool `json:"enabled"`
Priority int `json:"priority,omitempty"`
Transport string `json:"transport,omitempty"`
Command []string `json:"command,omitempty"`
Dir string `json:"dir,omitempty"`
Env map[string]string `json:"env,omitempty"`
Observe []string `json:"observe,omitempty"`
Intercept []string `json:"intercept,omitempty"`
}
// BuildInfo contains build-time version information
type BuildInfo struct {
Version string `json:"version"`
@@ -219,9 +250,15 @@ type RoutingConfig struct {
Threshold float64 `json:"threshold"` // complexity score in [0,1]; score >= threshold → primary model
}
// ToolFeedbackConfig controls whether tool execution details are sent to the
// chat channel as real-time feedback messages. When enabled, every tool call
// produces a short notification with the tool name and its parameters.
// SubTurnConfig configures the SubTurn execution system.
type SubTurnConfig struct {
MaxDepth int `json:"max_depth" env:"PICOCLAW_AGENTS_DEFAULTS_SUBTURN_MAX_DEPTH"`
MaxConcurrent int `json:"max_concurrent" env:"PICOCLAW_AGENTS_DEFAULTS_SUBTURN_MAX_CONCURRENT"`
DefaultTimeoutMinutes int `json:"default_timeout_minutes" env:"PICOCLAW_AGENTS_DEFAULTS_SUBTURN_DEFAULT_TIMEOUT_MINUTES"`
DefaultTokenBudget int `json:"default_token_budget" env:"PICOCLAW_AGENTS_DEFAULTS_SUBTURN_DEFAULT_TOKEN_BUDGET"`
ConcurrencyTimeoutSec int `json:"concurrency_timeout_sec" env:"PICOCLAW_AGENTS_DEFAULTS_SUBTURN_CONCURRENCY_TIMEOUT_SEC"`
}
type ToolFeedbackConfig struct {
Enabled bool `json:"enabled" env:"PICOCLAW_AGENTS_DEFAULTS_TOOL_FEEDBACK_ENABLED"`
MaxArgsLength int `json:"max_args_length" env:"PICOCLAW_AGENTS_DEFAULTS_TOOL_FEEDBACK_MAX_ARGS_LENGTH"`
@@ -238,12 +275,15 @@ type AgentDefaults struct {
ImageModel string `json:"image_model,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_IMAGE_MODEL"`
ImageModelFallbacks []string `json:"image_model_fallbacks,omitempty"`
MaxTokens int `json:"max_tokens" env:"PICOCLAW_AGENTS_DEFAULTS_MAX_TOKENS"`
ContextWindow int `json:"context_window,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_CONTEXT_WINDOW"`
Temperature *float64 `json:"temperature,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_TEMPERATURE"`
MaxToolIterations int `json:"max_tool_iterations" env:"PICOCLAW_AGENTS_DEFAULTS_MAX_TOOL_ITERATIONS"`
SummarizeMessageThreshold int `json:"summarize_message_threshold" env:"PICOCLAW_AGENTS_DEFAULTS_SUMMARIZE_MESSAGE_THRESHOLD"`
SummarizeTokenPercent int `json:"summarize_token_percent" env:"PICOCLAW_AGENTS_DEFAULTS_SUMMARIZE_TOKEN_PERCENT"`
MaxMediaSize int `json:"max_media_size,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_MAX_MEDIA_SIZE"`
Routing *RoutingConfig `json:"routing,omitempty"`
SteeringMode string `json:"steering_mode,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_STEERING_MODE"` // "one-at-a-time" (default) or "all"
SubTurn SubTurnConfig `json:"subturn" envPrefix:"PICOCLAW_AGENTS_DEFAULTS_SUBTURN_"`
ToolFeedback ToolFeedbackConfig `json:"tool_feedback,omitempty"`
LogLevel string `json:"log_level,omitempty" env:"PICOCLAW_LOG_LEVEL"`
}
@@ -923,10 +963,13 @@ func LoadConfig(path string) (*Config, error) {
if passphrase := credential.PassphraseProvider(); passphrase != "" {
for _, m := range cfg.ModelList {
if m.APIKey != "" && !strings.HasPrefix(m.APIKey, "enc://") && !strings.HasPrefix(m.APIKey, "file://") {
fmt.Fprintf(os.Stderr,
if m.APIKey != "" && !strings.HasPrefix(m.APIKey, "enc://") &&
!strings.HasPrefix(m.APIKey, "file://") {
fmt.Fprintf(
os.Stderr,
"picoclaw: warning: model %q has a plaintext api_key; call SaveConfig to encrypt it\n",
m.ModelName)
m.ModelName,
)
}
}
}
@@ -979,7 +1022,8 @@ func encryptPlaintextAPIKeys(models []ModelConfig, passphrase string) ([]ModelCo
changed := false
for i := range sealed {
m := &sealed[i]
if m.APIKey == "" || strings.HasPrefix(m.APIKey, "enc://") || strings.HasPrefix(m.APIKey, "file://") {
if m.APIKey == "" || strings.HasPrefix(m.APIKey, "enc://") ||
strings.HasPrefix(m.APIKey, "file://") {
continue
}
encrypted, err := credential.Encrypt(passphrase, "", m.APIKey)
@@ -1012,7 +1056,13 @@ func resolveAPIKeys(models []ModelConfig, configDir string) error {
for j, key := range models[i].APIKeys {
resolved, err := cr.Resolve(key)
if err != nil {
return fmt.Errorf("model_list[%d] (%s): api_keys[%d]: %w", i, models[i].ModelName, j, err)
return fmt.Errorf(
"model_list[%d] (%s): api_keys[%d]: %w",
i,
models[i].ModelName,
j,
err,
)
}
models[i].APIKeys[j] = resolved
}
pkg/config/config_test.go
+98
View File
@@ -470,6 +470,22 @@ func TestDefaultConfig_CronAllowCommandEnabled(t *testing.T) {
}
}
func TestDefaultConfig_HooksDefaults(t *testing.T) {
cfg := DefaultConfig()
if !cfg.Hooks.Enabled {
t.Fatal("DefaultConfig().Hooks.Enabled should be true")
}
if cfg.Hooks.Defaults.ObserverTimeoutMS != 500 {
t.Fatalf("ObserverTimeoutMS = %d, want 500", cfg.Hooks.Defaults.ObserverTimeoutMS)
}
if cfg.Hooks.Defaults.InterceptorTimeoutMS != 5000 {
t.Fatalf("InterceptorTimeoutMS = %d, want 5000", cfg.Hooks.Defaults.InterceptorTimeoutMS)
}
if cfg.Hooks.Defaults.ApprovalTimeoutMS != 60000 {
t.Fatalf("ApprovalTimeoutMS = %d, want 60000", cfg.Hooks.Defaults.ApprovalTimeoutMS)
}
}
func TestDefaultConfig_LogLevel(t *testing.T) {
cfg := DefaultConfig()
if cfg.Agents.Defaults.LogLevel != "fatal" {
@@ -562,6 +578,88 @@ func TestLoadConfig_WebToolsProxy(t *testing.T) {
}
}
func TestLoadConfig_HooksProcessConfig(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.json")
configJSON := `{
"hooks": {
"processes": {
"review-gate": {
"enabled": true,
"transport": "stdio",
"command": ["uvx", "picoclaw-hook-reviewer"],
"dir": "/tmp/hooks",
"env": {
"HOOK_MODE": "rewrite"
},
"observe": ["turn_start", "turn_end"],
"intercept": ["before_tool", "approve_tool"]
}
},
"builtins": {
"audit": {
"enabled": true,
"priority": 5,
"config": {
"label": "audit"
}
}
}
}
}`
if err := os.WriteFile(configPath, []byte(configJSON), 0o600); err != nil {
t.Fatalf("os.WriteFile() error: %v", err)
}
cfg, err := LoadConfig(configPath)
if err != nil {
t.Fatalf("LoadConfig() error: %v", err)
}
processCfg, ok := cfg.Hooks.Processes["review-gate"]
if !ok {
t.Fatal("expected review-gate process hook")
}
if !processCfg.Enabled {
t.Fatal("expected review-gate process hook to be enabled")
}
if processCfg.Transport != "stdio" {
t.Fatalf("Transport = %q, want stdio", processCfg.Transport)
}
if len(processCfg.Command) != 2 || processCfg.Command[0] != "uvx" {
t.Fatalf("Command = %v", processCfg.Command)
}
if processCfg.Dir != "/tmp/hooks" {
t.Fatalf("Dir = %q, want /tmp/hooks", processCfg.Dir)
}
if processCfg.Env["HOOK_MODE"] != "rewrite" {
t.Fatalf("HOOK_MODE = %q, want rewrite", processCfg.Env["HOOK_MODE"])
}
if len(processCfg.Observe) != 2 || processCfg.Observe[1] != "turn_end" {
t.Fatalf("Observe = %v", processCfg.Observe)
}
if len(processCfg.Intercept) != 2 || processCfg.Intercept[1] != "approve_tool" {
t.Fatalf("Intercept = %v", processCfg.Intercept)
}
builtinCfg, ok := cfg.Hooks.Builtins["audit"]
if !ok {
t.Fatal("expected audit builtin hook")
}
if !builtinCfg.Enabled {
t.Fatal("expected audit builtin hook to be enabled")
}
if builtinCfg.Priority != 5 {
t.Fatalf("Priority = %d, want 5", builtinCfg.Priority)
}
if !strings.Contains(string(builtinCfg.Config), `"audit"`) {
t.Fatalf("Config = %s", string(builtinCfg.Config))
}
if cfg.Hooks.Defaults.ApprovalTimeoutMS != 60000 {
t.Fatalf("ApprovalTimeoutMS = %d, want 60000", cfg.Hooks.Defaults.ApprovalTimeoutMS)
}
}
// TestDefaultConfig_DMScope verifies the default dm_scope value
// TestDefaultConfig_SummarizationThresholds verifies summarization defaults
func TestDefaultConfig_SummarizationThresholds(t *testing.T) {
pkg/config/defaults.go
+9
View File
@@ -36,6 +36,7 @@ func DefaultConfig() *Config {
MaxToolIterations: 50,
SummarizeMessageThreshold: 20,
SummarizeTokenPercent: 75,
SteeringMode: "one-at-a-time",
ToolFeedback: ToolFeedbackConfig{
Enabled: true,
MaxArgsLength: 300,
@@ -193,6 +194,14 @@ func DefaultConfig() *Config {
AllowFrom: FlexibleStringSlice{},
},
},
Hooks: HooksConfig{
Enabled: true,
Defaults: HookDefaultsConfig{
ObserverTimeoutMS: 500,
InterceptorTimeoutMS: 5000,
ApprovalTimeoutMS: 60000,
},
},
Providers: ProvidersConfig{
OpenAI: OpenAIProviderConfig{WebSearch: true},
},
pkg/providers/common/common.go
+10 -1
View File
@@ -214,11 +214,20 @@ func ParseResponse(body io.Reader) (*LLMResponse, error) {
Reasoning: choice.Message.Reasoning,
ReasoningDetails: choice.Message.ReasoningDetails,
ToolCalls: toolCalls,
FinishReason: choice.FinishReason,
FinishReason: normalizeFinishReason(choice.FinishReason),
Usage: apiResponse.Usage,
}, nil
}
// normalizeFinishReason normalizes finish_reason values across providers.
// Converts "length" to "truncated" for consistent handling.
func normalizeFinishReason(reason string) string {
if reason == "length" {
return "truncated"
}
return reason
}
// DecodeToolCallArguments decodes a tool call's arguments from raw JSON.
func DecodeToolCallArguments(raw json.RawMessage, name string) map[string]any {
arguments := make(map[string]any)
pkg/tools/registry.go
+19
View File
@@ -384,3 +384,22 @@ func (r *ToolRegistry) GetSummaries() []string {
}
return summaries
}
// GetAll returns all registered tools (both core and non-core with TTL > 0).
// Used by SubTurn to inherit parent's tool set.
func (r *ToolRegistry) GetAll() []Tool {
r.mu.RLock()
defer r.mu.RUnlock()
sorted := r.sortedToolNames()
tools := make([]Tool, 0, len(sorted))
for _, name := range sorted {
entry := r.tools[name]
// Include core tools and non-core tools with active TTL
if entry.IsCore || entry.TTL > 0 {
tools = append(tools, entry.Tool)
}
}
return tools
}
pkg/tools/result.go
+10 -1
View File
@@ -1,6 +1,10 @@
package tools
import "encoding/json"
import (
"encoding/json"
"github.com/sipeed/picoclaw/pkg/providers"
)
// ToolResult represents the structured return value from tool execution.
// It provides clear semantics for different types of results and supports
@@ -34,6 +38,11 @@ type ToolResult struct {
// Media contains media store refs produced by this tool.
// When non-empty, the agent will publish these as OutboundMediaMessage.
Media []string `json:"media,omitempty"`
// Messages holds the ephemeral session history after execution.
// Only populated by SubTurn executions; used by evaluator_optimizer
// to carry stateful worker context across evaluation iterations.
Messages []providers.Message `json:"-"`
}
// NewToolResult creates a basic ToolResult with content for the LLM.
pkg/tools/spawn.go
+71 -25
View File
@@ -7,7 +7,10 @@ import (
)
type SpawnTool struct {
manager *SubagentManager
spawner SubTurnSpawner
defaultModel string
maxTokens int
temperature float64
allowlistCheck func(targetAgentID string) bool
}
@@ -15,9 +18,19 @@ type SpawnTool struct {
var _ AsyncExecutor = (*SpawnTool)(nil)
func NewSpawnTool(manager *SubagentManager) *SpawnTool {
return &SpawnTool{
manager: manager,
if manager == nil {
return &SpawnTool{}
}
return &SpawnTool{
defaultModel: manager.defaultModel,
maxTokens: manager.maxTokens,
temperature: manager.temperature,
}
}
// SetSpawner sets the SubTurnSpawner for direct sub-turn execution.
func (t *SpawnTool) SetSpawner(spawner SubTurnSpawner) {
t.spawner = spawner
}
func (t *SpawnTool) Name() string {
@@ -59,11 +72,19 @@ func (t *SpawnTool) Execute(ctx context.Context, args map[string]any) *ToolResul
// ExecuteAsync implements AsyncExecutor. The callback is passed through to the
// subagent manager as a call parameter — never stored on the SpawnTool instance.
func (t *SpawnTool) ExecuteAsync(ctx context.Context, args map[string]any, cb AsyncCallback) *ToolResult {
func (t *SpawnTool) ExecuteAsync(
ctx context.Context,
args map[string]any,
cb AsyncCallback,
) *ToolResult {
return t.execute(ctx, args, cb)
}
func (t *SpawnTool) execute(ctx context.Context, args map[string]any, cb AsyncCallback) *ToolResult {
func (t *SpawnTool) execute(
ctx context.Context,
args map[string]any,
cb AsyncCallback,
) *ToolResult {
task, ok := args["task"].(string)
if !ok || strings.TrimSpace(task) == "" {
return ErrorResult("task is required and must be a non-empty string")
@@ -79,28 +100,53 @@ func (t *SpawnTool) execute(ctx context.Context, args map[string]any, cb AsyncCa
}
}
if t.manager == nil {
return ErrorResult("Subagent manager not configured")
// Build system prompt for spawned subagent
systemPrompt := fmt.Sprintf(
`You are a spawned subagent running in the background. Complete the given task independently and report back when done.
Task: %s`,
task,
)
if label != "" {
systemPrompt = fmt.Sprintf(
`You are a spawned subagent labeled "%s" running in the background. Complete the given task independently and report back when done.
Task: %s`,
label,
task,
)
}
// Read channel/chatID from context (injected by registry).
// Fall back to "cli"/"direct" for non-conversation callers (e.g., CLI, tests)
// to preserve the same defaults as the original NewSpawnTool constructor.
channel := ToolChannel(ctx)
if channel == "" {
channel = "cli"
}
chatID := ToolChatID(ctx)
if chatID == "" {
chatID = "direct"
// Use spawner if available (direct SpawnSubTurn call)
if t.spawner != nil {
// Launch async sub-turn in goroutine
go func() {
result, err := t.spawner.SpawnSubTurn(ctx, SubTurnConfig{
Model: t.defaultModel,
Tools: nil, // Will inherit from parent via context
SystemPrompt: systemPrompt,
MaxTokens: t.maxTokens,
Temperature: t.temperature,
Async: true, // Async execution
})
if err != nil {
result = ErrorResult(fmt.Sprintf("Spawn failed: %v", err)).WithError(err)
}
// Call callback if provided
if cb != nil {
cb(ctx, result)
}
}()
// Return immediate acknowledgment
if label != "" {
return AsyncResult(fmt.Sprintf("Spawned subagent '%s' for task: %s", label, task))
}
return AsyncResult(fmt.Sprintf("Spawned subagent for task: %s", task))
}
// Pass callback to manager for async completion notification
result, err := t.manager.Spawn(ctx, task, label, agentID, channel, chatID, cb)
if err != nil {
return ErrorResult(fmt.Sprintf("failed to spawn subagent: %v", err))
}
// Return AsyncResult since the task runs in background
return AsyncResult(result)
// Fallback: spawner not configured
return ErrorResult("Subagent manager not configured")
}
pkg/tools/spawn_test.go
+19
View File
@@ -6,6 +6,24 @@ import (
"testing"
)
// mockSpawner implements SubTurnSpawner for testing
type mockSpawner struct{}
func (m *mockSpawner) SpawnSubTurn(ctx context.Context, cfg SubTurnConfig) (*ToolResult, error) {
// Extract task from system prompt for response
task := cfg.SystemPrompt
if strings.Contains(task, "Task: ") {
parts := strings.Split(task, "Task: ")
if len(parts) > 1 {
task = parts[1]
}
}
return &ToolResult{
ForLLM: "Task completed: " + task,
ForUser: "Task completed",
}, nil
}
func TestSpawnTool_Execute_EmptyTask(t *testing.T) {
provider := &MockLLMProvider{}
manager := NewSubagentManager(provider, "test-model", "/tmp/test")
@@ -44,6 +62,7 @@ func TestSpawnTool_Execute_ValidTask(t *testing.T) {
provider := &MockLLMProvider{}
manager := NewSubagentManager(provider, "test-model", "/tmp/test")
tool := NewSpawnTool(manager)
tool.SetSpawner(&mockSpawner{})
ctx := context.Background()
args := map[string]any{
pkg/tools/subagent.go
+180 -127
View File
@@ -4,11 +4,34 @@ import (
"context"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/sipeed/picoclaw/pkg/providers"
)
// SubTurnSpawner is an interface for spawning sub-turns.
// This avoids circular dependency between tools and agent packages.
type SubTurnSpawner interface {
SpawnSubTurn(ctx context.Context, cfg SubTurnConfig) (*ToolResult, error)
}
// SubTurnConfig holds configuration for spawning a sub-turn.
type SubTurnConfig struct {
Model string
Tools []Tool
SystemPrompt string
MaxTokens int
Temperature float64
Async bool // true for async (spawn), false for sync (subagent)
Critical bool // continue running after parent finishes gracefully
Timeout time.Duration // 0 = use default (5 minutes)
MaxContextRunes int // 0 = auto, -1 = no limit, >0 = explicit limit
ActualSystemPrompt string
InitialMessages []providers.Message
InitialTokenBudget *atomic.Int64 // Shared token budget for team members; nil if no budget
}
type SubagentTask struct {
ID string
Task string
@@ -21,6 +44,15 @@ type SubagentTask struct {
Created int64
}
type SpawnSubTurnFunc func(
ctx context.Context,
task, label, agentID string,
tools *ToolRegistry,
maxTokens int,
temperature float64,
hasMaxTokens, hasTemperature bool,
) (*ToolResult, error)
type SubagentManager struct {
tasks map[string]*SubagentTask
mu sync.RWMutex
@@ -34,6 +66,7 @@ type SubagentManager struct {
hasMaxTokens bool
hasTemperature bool
nextID int
spawner SpawnSubTurnFunc
}
func NewSubagentManager(
@@ -51,6 +84,12 @@ func NewSubagentManager(
}
}
func (sm *SubagentManager) SetSpawner(spawner SpawnSubTurnFunc) {
sm.mu.Lock()
defer sm.mu.Unlock()
sm.spawner = spawner
}
// SetLLMOptions sets max tokens and temperature for subagent LLM calls.
func (sm *SubagentManager) SetLLMOptions(maxTokens int, temperature float64) {
sm.mu.Lock()
@@ -108,22 +147,16 @@ func (sm *SubagentManager) Spawn(
return fmt.Sprintf("Spawned subagent for task: %s", task), nil
}
func (sm *SubagentManager) runTask(ctx context.Context, task *SubagentTask, callback AsyncCallback) {
// Build system prompt for subagent
systemPrompt := `You are a subagent. Complete the given task independently and report the result.
You have access to tools - use them as needed to complete your task.
After completing the task, provide a clear summary of what was done.`
messages := []providers.Message{
{
Role: "system",
Content: systemPrompt,
},
{
Role: "user",
Content: task.Task,
},
}
func (sm *SubagentManager) runTask(
ctx context.Context,
task *SubagentTask,
callback AsyncCallback,
) {
task.Status = "running"
task.Created = time.Now().UnixMilli()
// TODO(eventbus): once subagents are modeled as child turns inside
// pkg/agent, emit SubTurnEnd and SubTurnResultDelivered from the parent
// AgentLoop instead of this legacy manager.
// Check if context is already canceled before starting
select {
@@ -136,8 +169,8 @@ After completing the task, provide a clear summary of what was done.`
default:
}
// Run tool loop with access to tools
sm.mu.RLock()
spawner := sm.spawner
tools := sm.tools
maxIter := sm.maxIterations
maxTokens := sm.maxTokens
@@ -146,27 +179,69 @@ After completing the task, provide a clear summary of what was done.`
hasTemperature := sm.hasTemperature
sm.mu.RUnlock()
var llmOptions map[string]any
if hasMaxTokens || hasTemperature {
llmOptions = map[string]any{}
if hasMaxTokens {
llmOptions["max_tokens"] = maxTokens
var result *ToolResult
var err error
if spawner != nil {
result, err = spawner(
ctx,
task.Task,
task.Label,
task.AgentID,
tools,
maxTokens,
temperature,
hasMaxTokens,
hasTemperature,
)
} else {
// Fallback to legacy RunToolLoop
systemPrompt := `You are a subagent. Complete the given task independently and report the result.
You have access to tools - use them as needed to complete your task.
After completing the task, provide a clear summary of what was done.`
messages := []providers.Message{
{Role: "system", Content: systemPrompt},
{Role: "user", Content: task.Task},
}
if hasTemperature {
llmOptions["temperature"] = temperature
var llmOptions map[string]any
if hasMaxTokens || hasTemperature {
llmOptions = map[string]any{}
if hasMaxTokens {
llmOptions["max_tokens"] = maxTokens
}
if hasTemperature {
llmOptions["temperature"] = temperature
}
}
var loopResult *ToolLoopResult
loopResult, err = RunToolLoop(ctx, ToolLoopConfig{
Provider: sm.provider,
Model: sm.defaultModel,
Tools: tools,
MaxIterations: maxIter,
LLMOptions: llmOptions,
}, messages, task.OriginChannel, task.OriginChatID)
if err == nil {
result = &ToolResult{
ForLLM: fmt.Sprintf(
"Subagent '%s' completed (iterations: %d): %s",
task.Label,
loopResult.Iterations,
loopResult.Content,
),
ForUser: loopResult.Content,
Silent: false,
IsError: false,
Async: false,
}
}
}
loopResult, err := RunToolLoop(ctx, ToolLoopConfig{
Provider: sm.provider,
Model: sm.defaultModel,
Tools: tools,
MaxIterations: maxIter,
LLMOptions: llmOptions,
}, messages, task.OriginChannel, task.OriginChatID)
sm.mu.Lock()
var result *ToolResult
defer func() {
sm.mu.Unlock()
// Call callback if provided and result is set
@@ -193,19 +268,7 @@ After completing the task, provide a clear summary of what was done.`
}
} else {
task.Status = "completed"
task.Result = loopResult.Content
result = &ToolResult{
ForLLM: fmt.Sprintf(
"Subagent '%s' completed (iterations: %d): %s",
task.Label,
loopResult.Iterations,
loopResult.Content,
),
ForUser: loopResult.Content,
Silent: false,
IsError: false,
Async: false,
}
task.Result = result.ForLLM
}
}
@@ -253,16 +316,28 @@ func (sm *SubagentManager) ListTaskCopies() []SubagentTask {
}
// SubagentTool executes a subagent task synchronously and returns the result.
// Unlike SpawnTool which runs tasks asynchronously, SubagentTool waits for completion
// and returns the result directly in the ToolResult.
// It directly calls SubTurnSpawner with Async=false for synchronous execution.
type SubagentTool struct {
manager *SubagentManager
spawner SubTurnSpawner
defaultModel string
maxTokens int
temperature float64
}
func NewSubagentTool(manager *SubagentManager) *SubagentTool {
return &SubagentTool{
manager: manager,
if manager == nil {
return &SubagentTool{}
}
return &SubagentTool{
defaultModel: manager.defaultModel,
maxTokens: manager.maxTokens,
temperature: manager.temperature,
}
}
// SetSpawner sets the SubTurnSpawner for direct sub-turn execution.
func (t *SubagentTool) SetSpawner(spawner SubTurnSpawner) {
t.spawner = spawner
}
func (t *SubagentTool) Name() string {
@@ -298,86 +373,64 @@ func (t *SubagentTool) Execute(ctx context.Context, args map[string]any) *ToolRe
label, _ := args["label"].(string)
if t.manager == nil {
return ErrorResult("Subagent manager not configured").WithError(fmt.Errorf("manager is nil"))
// Build system prompt for subagent
systemPrompt := fmt.Sprintf(
`You are a subagent. Complete the given task independently and provide a clear, concise result.
Task: %s`,
task,
)
if label != "" {
systemPrompt = fmt.Sprintf(
`You are a subagent labeled "%s". Complete the given task independently and provide a clear, concise result.
Task: %s`,
label,
task,
)
}
// Build messages for subagent
messages := []providers.Message{
{
Role: "system",
Content: "You are a subagent. Complete the given task independently and provide a clear, concise result.",
},
{
Role: "user",
Content: task,
},
}
// Use RunToolLoop to execute with tools (same as async SpawnTool)
sm := t.manager
sm.mu.RLock()
tools := sm.tools
maxIter := sm.maxIterations
maxTokens := sm.maxTokens
temperature := sm.temperature
hasMaxTokens := sm.hasMaxTokens
hasTemperature := sm.hasTemperature
sm.mu.RUnlock()
var llmOptions map[string]any
if hasMaxTokens || hasTemperature {
llmOptions = map[string]any{}
if hasMaxTokens {
llmOptions["max_tokens"] = maxTokens
// Use spawner if available (direct SpawnSubTurn call)
if t.spawner != nil {
result, err := t.spawner.SpawnSubTurn(ctx, SubTurnConfig{
Model: t.defaultModel,
Tools: nil, // Will inherit from parent via context
SystemPrompt: systemPrompt,
MaxTokens: t.maxTokens,
Temperature: t.temperature,
Async: false, // Synchronous execution
})
if err != nil {
return ErrorResult(fmt.Sprintf("Subagent execution failed: %v", err)).WithError(err)
}
if hasTemperature {
llmOptions["temperature"] = temperature
// Format result for display
userContent := result.ForLLM
if result.ForUser != "" {
userContent = result.ForUser
}
maxUserLen := 500
if len(userContent) > maxUserLen {
userContent = userContent[:maxUserLen] + "..."
}
labelStr := label
if labelStr == "" {
labelStr = "(unnamed)"
}
llmContent := fmt.Sprintf("Subagent task completed:\nLabel: %s\nResult: %s",
labelStr, result.ForLLM)
return &ToolResult{
ForLLM: llmContent,
ForUser: userContent,
Silent: false,
IsError: result.IsError,
Async: false,
}
}
// Fall back to "cli"/"direct" for non-conversation callers (e.g., CLI, tests)
// to preserve the same defaults as the original NewSubagentTool constructor.
channel := ToolChannel(ctx)
if channel == "" {
channel = "cli"
}
chatID := ToolChatID(ctx)
if chatID == "" {
chatID = "direct"
}
loopResult, err := RunToolLoop(ctx, ToolLoopConfig{
Provider: sm.provider,
Model: sm.defaultModel,
Tools: tools,
MaxIterations: maxIter,
LLMOptions: llmOptions,
}, messages, channel, chatID)
if err != nil {
return ErrorResult(fmt.Sprintf("Subagent execution failed: %v", err)).WithError(err)
}
// ForUser: Brief summary for user (truncated if too long)
userContent := loopResult.Content
maxUserLen := 500
if len(userContent) > maxUserLen {
userContent = userContent[:maxUserLen] + "..."
}
// ForLLM: Full execution details
labelStr := label
if labelStr == "" {
labelStr = "(unnamed)"
}
llmContent := fmt.Sprintf("Subagent task completed:\nLabel: %s\nIterations: %d\nResult: %s",
labelStr, loopResult.Iterations, loopResult.Content)
return &ToolResult{
ForLLM: llmContent,
ForUser: userContent,
Silent: false,
IsError: false,
Async: false,
}
// Fallback: spawner not configured
return ErrorResult("Subagent manager not configured").WithError(fmt.Errorf("spawner not set"))
}
pkg/tools/subagent_tool_test.go
+13 -14
View File
@@ -48,24 +48,19 @@ func TestSubagentManager_SetLLMOptions_AppliesToRunToolLoop(t *testing.T) {
provider := &MockLLMProvider{}
manager := NewSubagentManager(provider, "test-model", "/tmp/test")
manager.SetLLMOptions(2048, 0.6)
tool := NewSubagentTool(manager)
ctx := WithToolContext(context.Background(), "cli", "direct")
args := map[string]any{"task": "Do something"}
result := tool.Execute(ctx, args)
if result == nil || result.IsError {
t.Fatalf("Expected successful result, got: %+v", result)
// Verify options are set on manager
if manager.maxTokens != 2048 {
t.Errorf("manager.maxTokens = %d, want 2048", manager.maxTokens)
}
if provider.lastOptions == nil {
t.Fatal("Expected LLM options to be passed, got nil")
if manager.temperature != 0.6 {
t.Errorf("manager.temperature = %f, want 0.6", manager.temperature)
}
if provider.lastOptions["max_tokens"] != 2048 {
t.Fatalf("max_tokens = %v, want %d", provider.lastOptions["max_tokens"], 2048)
if !manager.hasMaxTokens {
t.Error("manager.hasMaxTokens should be true")
}
if provider.lastOptions["temperature"] != 0.6 {
t.Fatalf("temperature = %v, want %v", provider.lastOptions["temperature"], 0.6)
if !manager.hasTemperature {
t.Error("manager.hasTemperature should be true")
}
}
@@ -150,6 +145,7 @@ func TestSubagentTool_Execute_Success(t *testing.T) {
provider := &MockLLMProvider{}
manager := NewSubagentManager(provider, "test-model", "/tmp/test")
tool := NewSubagentTool(manager)
tool.SetSpawner(&mockSpawner{})
ctx := WithToolContext(context.Background(), "telegram", "chat-123")
args := map[string]any{
@@ -204,6 +200,7 @@ func TestSubagentTool_Execute_NoLabel(t *testing.T) {
provider := &MockLLMProvider{}
manager := NewSubagentManager(provider, "test-model", "/tmp/test")
tool := NewSubagentTool(manager)
tool.SetSpawner(&mockSpawner{})
ctx := context.Background()
args := map[string]any{
@@ -277,6 +274,7 @@ func TestSubagentTool_Execute_ContextPassing(t *testing.T) {
provider := &MockLLMProvider{}
manager := NewSubagentManager(provider, "test-model", "/tmp/test")
tool := NewSubagentTool(manager)
tool.SetSpawner(&mockSpawner{})
channel := "test-channel"
chatID := "test-chat"
@@ -302,6 +300,7 @@ func TestSubagentTool_ForUserTruncation(t *testing.T) {
provider := &MockLLMProvider{}
manager := NewSubagentManager(provider, "test-model", "/tmp/test")
tool := NewSubagentTool(manager)
tool.SetSpawner(&mockSpawner{})
ctx := context.Background()
pkg/utils/context.go
+173
View File
@@ -0,0 +1,173 @@
// PicoClaw - Ultra-lightweight personal AI agent
// Inspired by and based on nanobot: https://github.com/HKUDS/nanobot
// License: MIT
//
// Copyright (c) 2026 PicoClaw contributors
package utils
import (
"encoding/json"
"fmt"
"unicode/utf8"
"github.com/sipeed/picoclaw/pkg/providers"
)
// CalculateDefaultMaxContextRunes computes a default context limit based on the model's context window.
// Strategy: Use 75% of the context window and convert to rune estimate.
//
// Token-to-rune conversion ratios (conservative estimates):
// - English: ~4 chars per token
// - Chinese: ~1.5-2 chars per token
// - Mixed: ~3 chars per token (used here for safety)
func CalculateDefaultMaxContextRunes(contextWindow int) int {
if contextWindow <= 0 {
// Conservative fallback when context window is unknown
return 8000 // ~2000 tokens
}
// Use 75% of context window to leave headroom
targetTokens := int(float64(contextWindow) * 0.75)
// Convert tokens to runes using conservative ratio
const avgCharsPerToken = 3
return targetTokens * avgCharsPerToken
}
// ResolveMaxContextRunes determines the final MaxContextRunes value to use.
// Priority: explicit config > auto-calculate > conservative default
func ResolveMaxContextRunes(configValue, contextWindow int) int {
switch {
case configValue > 0:
// Explicitly configured, use as-is
return configValue
case configValue == -1:
// Explicitly disabled
return -1
default:
// 0 or unset: auto-calculate
return CalculateDefaultMaxContextRunes(contextWindow)
}
}
// MeasureContextRunes calculates the total rune count of a message list.
// Includes content, reasoning content, and estimates for tool calls.
func MeasureContextRunes(messages []providers.Message) int {
totalRunes := 0
for _, msg := range messages {
totalRunes += utf8.RuneCountInString(msg.Content)
totalRunes += utf8.RuneCountInString(msg.ReasoningContent)
// Tool calls: serialize to JSON and count
if len(msg.ToolCalls) > 0 {
for _, tc := range msg.ToolCalls {
totalRunes += utf8.RuneCountInString(tc.Name)
// Arguments: serialize and count
if argsJSON, err := json.Marshal(tc.Arguments); err == nil {
totalRunes += utf8.RuneCount(argsJSON)
} else {
// Fallback estimate if serialization fails
totalRunes += 100
}
}
}
// ToolCallID
totalRunes += utf8.RuneCountInString(msg.ToolCallID)
}
return totalRunes
}
// TruncateContextSmart intelligently truncates message history to fit within maxRunes.
//
// Strategy:
// 1. Always preserve system messages (they define the agent's behavior)
// 2. Keep the most recent messages (they contain current context)
// 3. Drop older middle messages when necessary
// 4. Insert a truncation notice to inform the LLM
//
// Returns the truncated message list.
func TruncateContextSmart(messages []providers.Message, maxRunes int) []providers.Message {
if len(messages) == 0 {
return messages
}
// Separate system messages from others
var systemMsgs []providers.Message
var otherMsgs []providers.Message
for _, msg := range messages {
if msg.Role == "system" {
systemMsgs = append(systemMsgs, msg)
} else {
otherMsgs = append(otherMsgs, msg)
}
}
// Calculate system message size
systemRunes := 0
for _, msg := range systemMsgs {
systemRunes += utf8.RuneCountInString(msg.Content)
systemRunes += utf8.RuneCountInString(msg.ReasoningContent)
}
// Reserve space for truncation notice (estimate ~80 runes)
const truncationNoticeEstimate = 80
// Allocate remaining space for other messages
remainingRunes := maxRunes - systemRunes - truncationNoticeEstimate
if remainingRunes <= 0 {
// System messages already exceed limit - return only system messages
return systemMsgs
}
// Collect recent messages in reverse order until we hit the limit
var keptMsgs []providers.Message
currentRunes := 0
for i := len(otherMsgs) - 1; i >= 0; i-- {
msg := otherMsgs[i]
msgRunes := utf8.RuneCountInString(msg.Content) +
utf8.RuneCountInString(msg.ReasoningContent)
// Estimate tool call size
if len(msg.ToolCalls) > 0 {
for _, tc := range msg.ToolCalls {
msgRunes += utf8.RuneCountInString(tc.Name)
if argsJSON, err := json.Marshal(tc.Arguments); err == nil {
msgRunes += utf8.RuneCount(argsJSON)
} else {
msgRunes += 100
}
}
}
msgRunes += utf8.RuneCountInString(msg.ToolCallID)
if currentRunes+msgRunes > remainingRunes {
// Would exceed limit, stop collecting
break
}
// Prepend to maintain chronological order
keptMsgs = append([]providers.Message{msg}, keptMsgs...)
currentRunes += msgRunes
}
// If we dropped messages, add a truncation notice
result := systemMsgs
if len(keptMsgs) < len(otherMsgs) {
droppedCount := len(otherMsgs) - len(keptMsgs)
truncationNotice := providers.Message{
Role: "system",
Content: fmt.Sprintf(
"[Context truncated: %d earlier messages omitted to stay within context limits]",
droppedCount,
),
}
result = append(result, truncationNotice)
}
result = append(result, keptMsgs...)
return result
}
pkg/utils/context_test.go
+450
View File
@@ -0,0 +1,450 @@
// PicoClaw - Ultra-lightweight personal AI agent
// License: MIT
//
// Copyright (c) 2026 PicoClaw contributors
package utils
import (
"testing"
"github.com/sipeed/picoclaw/pkg/providers"
)
func TestCalculateDefaultMaxContextRunes(t *testing.T) {
tests := []struct {
name string
contextWindow int
want int
}{
{
name: "zero context window uses fallback",
contextWindow: 0,
want: 8000,
},
{
name: "negative context window uses fallback",
contextWindow: -1,
want: 8000,
},
{
name: "small context window (4k tokens)",
contextWindow: 4000,
want: 9000, // 4000 * 0.75 * 3 = 9000
},
{
name: "medium context window (128k tokens)",
contextWindow: 128000,
want: 288000, // 128000 * 0.75 * 3 = 288000
},
{
name: "large context window (1M tokens)",
contextWindow: 1000000,
want: 2250000, // 1000000 * 0.75 * 3 = 2250000
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := CalculateDefaultMaxContextRunes(tt.contextWindow)
if got != tt.want {
t.Errorf("CalculateDefaultMaxContextRunes(%d) = %d, want %d",
tt.contextWindow, got, tt.want)
}
})
}
}
func TestResolveMaxContextRunes(t *testing.T) {
tests := []struct {
name string
configValue int
contextWindow int
want int
}{
{
name: "explicit positive value",
configValue: 12000,
contextWindow: 4000,
want: 12000,
},
{
name: "explicit disable (-1)",
configValue: -1,
contextWindow: 4000,
want: -1,
},
{
name: "zero uses auto-calculate",
configValue: 0,
contextWindow: 4000,
want: 9000, // 4000 * 0.75 * 3
},
{
name: "unset (0) with unknown context window",
configValue: 0,
contextWindow: 0,
want: 8000, // fallback
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := ResolveMaxContextRunes(tt.configValue, tt.contextWindow)
if got != tt.want {
t.Errorf("ResolveMaxContextRunes(%d, %d) = %d, want %d",
tt.configValue, tt.contextWindow, got, tt.want)
}
})
}
}
func TestMeasureContextRunes(t *testing.T) {
tests := []struct {
name string
messages []providers.Message
want int
}{
{
name: "empty messages",
messages: []providers.Message{},
want: 0,
},
{
name: "single simple message",
messages: []providers.Message{
{Role: "user", Content: "Hello"},
},
want: 5, // "Hello" = 5 runes
},
{
name: "message with reasoning",
messages: []providers.Message{
{
Role: "assistant",
Content: "Answer",
ReasoningContent: "Thinking",
},
},
want: 14, // "Answer" (6) + "Thinking" (8) = 14
},
{
name: "message with tool call",
messages: []providers.Message{
{
Role: "assistant",
Content: "Using tool",
ToolCalls: []providers.ToolCall{
{
Name: "test_tool",
Arguments: map[string]any{"key": "value"},
},
},
},
},
want: 10 + 9 + 15, // "Using tool" + "test_tool" + {"key":"value"}
},
{
name: "multiple messages",
messages: []providers.Message{
{Role: "system", Content: "You are helpful"},
{Role: "user", Content: "Hi"},
{Role: "assistant", Content: "Hello!"},
},
want: 15 + 2 + 6, // 15 + 2 + 6 = 23
},
{
name: "unicode characters",
messages: []providers.Message{
{Role: "user", Content: "\u4f60\u597d\u4e16\u754c"}, // 4 Chinese characters
},
want: 4,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := MeasureContextRunes(tt.messages)
if got != tt.want {
t.Errorf("MeasureContextRunes() = %d, want %d", got, tt.want)
}
})
}
}
func TestTruncateContextSmart(t *testing.T) {
tests := []struct {
name string
messages []providers.Message
maxRunes int
wantLen int
wantHas []string // Content strings that should be present
wantNot []string // Content strings that should be absent
}{
{
name: "empty messages",
messages: []providers.Message{},
maxRunes: 100,
wantLen: 0,
},
{
name: "no truncation needed",
messages: []providers.Message{
{Role: "system", Content: "System"},
{Role: "user", Content: "Hello"},
},
maxRunes: 100,
wantLen: 2,
wantHas: []string{"System", "Hello"},
},
{
name: "truncate when limit is tight",
messages: []providers.Message{
{Role: "system", Content: "System"},
{Role: "user", Content: "Message 1 with some content here"},
{Role: "assistant", Content: "Response 1 with some content here"},
{Role: "user", Content: "Message 2 with some content here"},
{Role: "assistant", Content: "Response 2 with some content here"},
{Role: "user", Content: "Latest"},
},
maxRunes: 120, // Tight limit to force truncation
wantLen: -1, // Don't check exact length, just verify truncation occurred
wantHas: []string{"System", "Latest"},
wantNot: []string{"Message 1", "Response 1"},
},
{
name: "system messages exceed limit",
messages: []providers.Message{
{Role: "system", Content: "Very long system message"},
{Role: "user", Content: "User message"},
},
maxRunes: 10, // Less than system message
wantLen: 1, // Only system message
wantHas: []string{"Very long system message"},
wantNot: []string{"User message"},
},
{
name: "preserve multiple system messages",
messages: []providers.Message{
{Role: "system", Content: "Sys1"},
{Role: "system", Content: "Sys2"},
{Role: "user", Content: "Old"},
{Role: "user", Content: "New"},
},
maxRunes: 200, // Generous limit
wantLen: 4, // Both system + truncation notice + new
wantHas: []string{"Sys1", "Sys2", "New"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := TruncateContextSmart(tt.messages, tt.maxRunes)
if tt.wantLen >= 0 && len(got) != tt.wantLen {
t.Errorf("TruncateContextSmart() returned %d messages, want %d",
len(got), tt.wantLen)
}
// Check for expected content
allContent := ""
for _, msg := range got {
allContent += msg.Content + " "
}
for _, want := range tt.wantHas {
found := false
for _, msg := range got {
if msg.Content == want || containsSubstring(msg.Content, want) {
found = true
break
}
}
if !found {
t.Errorf("Expected content %q not found in truncated messages", want)
}
}
for _, notWant := range tt.wantNot {
for _, msg := range got {
if containsSubstring(msg.Content, notWant) {
t.Errorf("Unexpected content %q found in truncated messages", notWant)
}
}
}
})
}
}
func containsSubstring(s, substr string) bool {
return len(s) >= len(substr) && findSubstring(s, substr)
}
func findSubstring(s, substr string) bool {
for i := 0; i <= len(s)-len(substr); i++ {
if s[i:i+len(substr)] == substr {
return true
}
}
return false
}
// TestSubTurnConfigMaxContextRunes verifies that MaxContextRunes configuration
// is properly integrated into the SubTurn execution flow.
func TestSubTurnConfigMaxContextRunes(t *testing.T) {
tests := []struct {
name string
maxContextRunes int
contextWindow int
wantResolved int
}{
{
name: "default (0) auto-calculates from context window",
maxContextRunes: 0,
contextWindow: 4000,
wantResolved: 9000, // 4000 * 0.75 * 3
},
{
name: "explicit value is used",
maxContextRunes: 12000,
contextWindow: 4000,
wantResolved: 12000,
},
{
name: "disabled (-1) returns -1",
maxContextRunes: -1,
contextWindow: 4000,
wantResolved: -1,
},
{
name: "fallback when context window unknown",
maxContextRunes: 0,
contextWindow: 0,
wantResolved: 8000, // conservative fallback
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := ResolveMaxContextRunes(tt.maxContextRunes, tt.contextWindow)
if got != tt.wantResolved {
t.Errorf("utils.ResolveMaxContextRunes(%d, %d) = %d, want %d",
tt.maxContextRunes, tt.contextWindow, got, tt.wantResolved)
}
})
}
}
// TestContextTruncationFlow verifies the complete context truncation flow:
// 1. Messages accumulate beyond soft limit
// 2. Truncation is triggered
// 3. System messages are preserved
// 4. Recent messages are kept
func TestContextTruncationFlow(t *testing.T) {
// Build a message history that exceeds the limit
messages := []providers.Message{
{Role: "system", Content: "You are a helpful assistant"}, // ~27 runes
{Role: "user", Content: "First question"}, // ~14 runes
{Role: "assistant", Content: "First answer"}, // ~12 runes
{Role: "user", Content: "Second question"}, // ~15 runes
{Role: "assistant", Content: "Second answer"}, // ~13 runes
{Role: "user", Content: "Third question"}, // ~14 runes
{Role: "assistant", Content: "Third answer"}, // ~12 runes
{Role: "user", Content: "Latest question"}, // ~15 runes
}
// Total: ~122 runes
totalRunes := MeasureContextRunes(messages)
if totalRunes < 100 {
t.Errorf("Expected total runes > 100, got %d", totalRunes)
}
// Set limit to 150 runes - should force truncation of old messages
// but preserve system + truncation notice + recent messages
maxRunes := 150
truncated := TruncateContextSmart(messages, maxRunes)
// Verify truncation occurred
if len(truncated) >= len(messages) {
t.Errorf("Expected truncation, but got %d messages (original: %d)",
len(truncated), len(messages))
}
// Verify system message is preserved
foundSystem := false
for _, msg := range truncated {
if msg.Role == "system" && msg.Content == "You are a helpful assistant" {
foundSystem = true
break
}
}
if !foundSystem {
t.Error("System message was not preserved after truncation")
}
// Verify latest message is preserved
foundLatest := false
for _, msg := range truncated {
if msg.Content == "Latest question" {
foundLatest = true
break
}
}
if !foundLatest {
t.Error("Latest message was not preserved after truncation")
}
// Verify truncation notice is present
foundNotice := false
for _, msg := range truncated {
if msg.Role == "system" && containsSubstring(msg.Content, "truncated") {
foundNotice = true
break
}
}
if !foundNotice {
t.Error("Truncation notice was not added")
}
// Verify result is within limit (with some tolerance for estimation)
resultRunes := MeasureContextRunes(truncated)
if resultRunes > maxRunes+20 { // Allow 20 rune tolerance
t.Errorf("Truncated context (%d runes) significantly exceeds limit (%d runes)",
resultRunes, maxRunes)
}
}
// TestContextTruncationPreservesToolCalls verifies that tool calls are
// properly handled during context truncation.
func TestContextTruncationPreservesToolCalls(t *testing.T) {
messages := []providers.Message{
{Role: "system", Content: "System"},
{Role: "user", Content: "Old message that should be dropped"},
{
Role: "assistant",
Content: "Recent tool use",
ToolCalls: []providers.ToolCall{
{
Name: "important_tool",
Arguments: map[string]any{"key": "value"},
},
},
},
}
// Set a generous limit that should keep the tool call message
maxRunes := 200
truncated := TruncateContextSmart(messages, maxRunes)
// Verify tool call message is preserved
foundToolCall := false
for _, msg := range truncated {
if len(msg.ToolCalls) > 0 && msg.ToolCalls[0].Name == "important_tool" {
foundToolCall = true
break
}
}
if !foundToolCall {
t.Error("Tool call message was not preserved during truncation")
}
}
web/backend/api/gateway_test.go
+10
View File
@@ -596,6 +596,11 @@ func TestGatewayStatusReturnsErrorAfterStartupWindowExpires(t *testing.T) {
func TestGatewayStatusReturnsRestartingDuringRestartGap(t *testing.T) {
resetGatewayTestState(t)
// Mock health check to return error, so it won't override our "restarting" status
gatewayHealthGet = func(url string, timeout time.Duration) (*http.Response, error) {
return nil, errors.New("mock health check error")
}
configPath := filepath.Join(t.TempDir(), "config.json")
h := NewHandler(configPath)
mux := http.NewServeMux()
@@ -738,6 +743,11 @@ func TestGatewayRestartKeepsOldProcessWhenItDoesNotExitInTime(t *testing.T) {
func TestGatewayRestartReturnsErrorStatusWhenReplacementFailsToStart(t *testing.T) {
resetGatewayTestState(t)
// Mock health check to return error, so it won't override our "error" status
gatewayHealthGet = func(url string, timeout time.Duration) (*http.Response, error) {
return nil, errors.New("mock health check error")
}
configPath := filepath.Join(t.TempDir(), "config.json")
cfg := config.DefaultConfig()
cfg.Agents.Defaults.ModelName = cfg.ModelList[0].ModelName
web/frontend/src/components/config/config-page.tsx
+4
View File
@@ -147,6 +147,9 @@ export function ConfigPage() {
const maxTokens = parseIntField(form.maxTokens, "Max tokens", {
min: 1,
})
const contextWindow = form.contextWindow.trim()
? parseIntField(form.contextWindow, "Context window", { min: 1 })
: undefined
const maxToolIterations = parseIntField(
form.maxToolIterations,
"Max tool iterations",
@@ -201,6 +204,7 @@ export function ConfigPage() {
workspace,
restrict_to_workspace: form.restrictToWorkspace,
max_tokens: maxTokens,
context_window: contextWindow,
max_tool_iterations: maxToolIterations,
summarize_message_threshold: summarizeMessageThreshold,
summarize_token_percent: summarizeTokenPercent,
web/frontend/src/components/config/config-sections.tsx
+14
View File
@@ -106,6 +106,20 @@ export function AgentDefaultsSection({
/>
</Field>
<Field
label={t("pages.config.context_window")}
hint={t("pages.config.context_window_hint")}
layout="setting-row"
>
<Input
type="number"
min={1}
value={form.contextWindow}
onChange={(e) => onFieldChange("contextWindow", e.target.value)}
placeholder="131072"
/>
</Field>
<Field
label={t("pages.config.max_tool_iterations")}
hint={t("pages.config.max_tool_iterations_hint")}
web/frontend/src/components/config/form-model.ts
+3
View File
@@ -12,6 +12,7 @@ export interface CoreConfigForm {
allowCommand: boolean
cronExecTimeoutMinutes: string
maxTokens: string
contextWindow: string
maxToolIterations: string
summarizeMessageThreshold: string
summarizeTokenPercent: string
@@ -71,6 +72,7 @@ export const EMPTY_FORM: CoreConfigForm = {
allowCommand: true,
cronExecTimeoutMinutes: "5",
maxTokens: "32768",
contextWindow: "",
maxToolIterations: "50",
summarizeMessageThreshold: "20",
summarizeTokenPercent: "75",
@@ -164,6 +166,7 @@ export function buildFormFromConfig(config: unknown): CoreConfigForm {
EMPTY_FORM.cronExecTimeoutMinutes,
),
maxTokens: asNumberString(defaults.max_tokens, EMPTY_FORM.maxTokens),
contextWindow: asNumberString(defaults.context_window, EMPTY_FORM.contextWindow),
maxToolIterations: asNumberString(
defaults.max_tool_iterations,
EMPTY_FORM.maxToolIterations,
web/frontend/src/i18n/locales/en.json
+2
View File
@@ -415,6 +415,8 @@
"cron_exec_timeout_hint": "Maximum runtime for scheduled commands. Set to 0 to disable the timeout.",
"max_tokens": "Max Tokens",
"max_tokens_hint": "Upper token limit per model response.",
"context_window": "Context Window",
"context_window_hint": "Model input context capacity in tokens. Leave empty to use the default (4x max tokens).",
"max_tool_iterations": "Max Tool Iterations",
"max_tool_iterations_hint": "Maximum tool-call loops in a single task.",
"summarize_threshold": "Summarize Message Threshold",
web/frontend/src/i18n/locales/zh.json
+2
View File
@@ -415,6 +415,8 @@
"cron_exec_timeout_hint": "定时任务中命令的最长运行时间。设置为 0 表示不限制超时。",
"max_tokens": "最大 Token 数",
"max_tokens_hint": "单次模型响应允许的最大 Token 数。",
"context_window": "上下文窗口",
"context_window_hint": "模型输入上下文容量（Token 数）。留空使用默认值（最大 Token 数的 4 倍）。",
"max_tool_iterations": "最大工具迭代次数",
"max_tool_iterations_hint": "单个任务中允许的工具调用循环上限。",
"summarize_threshold": "触发摘要的消息阈值",
workspace/AGENT.md
+45
View File
@@ -0,0 +1,45 @@
---
name: pico
description: >
The default general-purpose assistant for everyday conversation, problem
solving, and workspace help.
---
You are Pico, the default assistant for this workspace.
Your name is PicoClaw 🦞.
## Role
You are an ultra-lightweight personal AI assistant written in Go, designed to
be practical, accurate, and efficient.
## Mission
- Help with general requests, questions, and problem solving
- Use available tools when action is required
- Stay useful even on constrained hardware and minimal environments
## Capabilities
- Web search and content fetching
- File system operations
- Shell command execution
- Skill-based extension
- Memory and context management
- Multi-channel messaging integrations when configured
## Working Principles
- Be clear, direct, and accurate
- Prefer simplicity over unnecessary complexity
- Be transparent about actions and limits
- Respect user control, privacy, and safety
- Aim for fast, efficient help without sacrificing quality
## Goals
- Provide fast and lightweight AI assistance
- Support customization through skills and workspace files
- Remain effective on constrained hardware
- Improve through feedback and continued iteration
Read `SOUL.md` as part of your identity and communication style.
workspace/AGENTS.md
-12
View File
@@ -1,12 +0,0 @@
# Agent Instructions
You are a helpful AI assistant. Be concise, accurate, and friendly.
## Guidelines
- Always explain what you're doing before taking actions
- Ask for clarification when request is ambiguous
- Use tools to help accomplish tasks
- Remember important information in your memory files
- Be proactive and helpful
- Learn from user feedback
workspace/IDENTITY.md
-53
View File
@@ -1,53 +0,0 @@
# Identity
## Name
PicoClaw 🦞
## Description
Ultra-lightweight personal AI assistant written in Go, inspired by nanobot.
## Purpose
- Provide intelligent AI assistance with minimal resource usage
- Support multiple LLM providers (OpenAI, Anthropic, Zhipu, etc.)
- Enable easy customization through skills system
- Run on minimal hardware ($10 boards, <10MB RAM)
## Capabilities
- Web search and content fetching
- File system operations (read, write, edit)
- Shell command execution
- Multi-channel messaging (Telegram, WhatsApp, Feishu)
- Skill-based extensibility
- Memory and context management
## Philosophy
- Simplicity over complexity
- Performance over features
- User control and privacy
- Transparent operation
- Community-driven development
## Goals
- Provide a fast, lightweight AI assistant
- Support offline-first operation where possible
- Enable easy customization and extension
- Maintain high quality responses
- Run efficiently on constrained hardware
## License
MIT License - Free and open source
## Repository
https://github.com/sipeed/picoclaw
## Contact
Issues: https://github.com/sipeed/picoclaw/issues
Discussions: https://github.com/sipeed/picoclaw/discussions
---
"Every bit helps, every bit matters."
- Picoclaw
workspace/SOUL.md
+4 -2
View File
@@ -1,6 +1,6 @@
# Soul
I am picoclaw, a lightweight AI assistant powered by AI.
I am PicoClaw: calm, helpful, and practical.
## Personality
@@ -8,10 +8,12 @@ I am picoclaw, a lightweight AI assistant powered by AI.
- Concise and to the point
- Curious and eager to learn
- Honest and transparent
- Calm under uncertainty
## Values
- Accuracy over speed
- User privacy and safety
- Transparency in actions
- Continuous improvement
- Continuous improvement
- Simplicity over unnecessary complexity
workspace/USER.md
+2 -2
View File
@@ -1,6 +1,6 @@
# User
Information about user goes here.
Information about the user goes here.
## Preferences
@@ -18,4 +18,4 @@ Information about user goes here.
- What the user wants to learn from AI
- Preferred interaction style
- Areas of interest
- Areas of interest