Compare commits

...

42 Commits

Author SHA1 Message Date
lxowalle 639b32703a feat: support streaming (#2892)
* Support streaming

* fix: stream pico reasoning updates

Route Pico reasoning through the active streamer and hide empty thought placeholders.

* fix: harden configured streaming delivery

* fix ci

* fix split issue
2026-05-19 16:38:47 +08:00
LC 941bac2332 feat(web): add chat detail visibility selector (#2886) 2026-05-18 14:50:57 +08:00
sky5454 cb5d33124c fix(powershell): windows security enhancement, sec deny powershell encoding bypass via iex inje… (#2836)
* fix(powershell):  sec deny  powershell encoding bypass   via iex injection.

* fix(exec): security guard bypass fixes for PowerShell/CMD encoding and path traversal

- Split deny patterns into defaultDenyPatterns (all platforms) and
  windowsDenyPatterns (Windows-only) to avoid false positives
- Add PowerShell encoding bypass detection:
  - [Text.Encoding] and [System.Text.Encoding] variants
  - -EncodedCommand short forms (-e, -ec, -enc)
  - .GetString([byte[]] with whitespace variations
  - FromBase64String decoding
  - PowerShell variable = [byte[](...) patterns
  - Literal \uXXXX Unicode escape sequences
- Expand PowerShell ($env:VAR) and CMD (%VAR%) environment variables
  before workspace path checking to prevent $env:USERPROFILE bypass
- Expand ~ to home directory on Windows
- Add .../.../ path traversal variant detection (blocks .../.../, ..../..../)
- Add symlink/junction resolution before workspace check
- Add Windows path normalization for ADS (file.txt:stream) and
  extended-length paths (\?\)
- Add comprehensive tests for all new patterns

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(exec): fix -EncodedCommand regex and rename Windows tests with expanded payloads

- Fix -EncodedCommand regex to match all short forms: -e, -ec, -enc, -en
- Rename Windows-specific tests with TestWindows_ prefix for clarity:
  - TestWindows_TildeBypassPrevented
  - TestWindows_SymlinkBypassPrevented
  - TestWindows_PowerShellEncodingBypass
- Expand test payloads:
  - [Text.Encoding]: add UTF8 and Unicode variants
  - -EncodedCommand: add -enc and -en forms
  - Unicode escape: add multiple \uXXXX forms

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: retest

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:28:03 +08:00
Mauro 68e572f969 fix(config): make load_image configurable (#2879) 2026-05-18 10:17:34 +08:00
LC 57876248e2 feat(provider): add SiliconFlow provider support (#2885) 2026-05-18 10:16:09 +08:00
LC 789f907f6d feat(chat): add independent code block copy and collapse controls (#2882)
* feat(chat): add independent copy and collapse controls for code blocks

* fix(chat): unify code block rendering styles

* fix(chat): refine code block labels

* feat(chat): highlight tool call code blocks as json
2026-05-18 10:01:39 +08:00
Guoguo feacd84b84 docs: update wechat qrcode (#2889) 2026-05-18 09:50:36 +08:00
肆月 604187e312 feat(web,api): test connection with real connectivity verification (#2833)
* feat(web,api): add fetch models and saved catalog support

Split from PR #2752 (part 2 of 3).

Backend:
- /api/models/catalog endpoint for browsing remote model catalogs
- /api/models/fetch endpoint for fetching available models from providers
- Credential reuse with provider/API base matching for security
- Default API base resolution for providers without explicit base

Frontend:
- FetchModelsDialog for importing models from remote providers
- CatalogDialog for browsing and importing from model catalogs
- Static import for FetchModelsDialog (replaces dynamic import from PR1)
- Dynamic import retained for TestModelDialog (PR3 territory)

* feat(web,api): add test connection with real connectivity verification

Split from PR #2752 (part 3 of 3).

Backend:
- /api/models/{index}/test endpoint for testing saved model configs
- /api/models/test-inline endpoint for testing unsaved form values
- Real network probe (GET /models) for connectivity verification
- Credential reuse with provider/API base matching for security
- Default API base resolution for providers without explicit base

Frontend:
- TestModelDialog for testing model connectivity
- Inline test support for add/edit model sheets
- Static import for TestModelDialog (replaces dynamic import from PR1)
2026-05-18 09:47:44 +08:00
美電球 0df050ff2e Merge pull request #2766 from SiYue-ZO/docs/v3-config-format-sync
docs: sync all documentation to V3 config format
2026-05-15 22:06:08 +08:00
Mauro 6817aa5311 Merge pull request #2811 from afjcjsbx/fix/mcp-streamable-http-support
fix(mcp): support streamable HTTP alias, request-response mode and integration tests
2026-05-15 12:55:58 +02:00
lxowalle 412705783d fix(pico): preserve image media across pico attachments and client (#2874)
* fix(pico): preserve image media across pico attachments and client

* * fix ci

* fix(pico): preserve text when client media parsing fails

- Skip non-inline Pico attachment URLs instead of treating them as invalid inline media
- Preserve pico_client text messages when malformed media payloads are received
- Add regression coverage for media.create, download attachments, and invalid media payloads

* fix lint
2026-05-15 15:49:07 +08:00
wenjie bfb2b35f74 chore: update slack-go to v0.23.1 (#2875)
Adapt Slack media uploads to the renamed UploadFile API.
2026-05-15 13:58:11 +08:00
wenjie b225629af8 build(deps): update @tailwindcss/vite to 4.3.0 (#2876) 2026-05-15 13:58:09 +08:00
美電球 c62a9bf55b Merge pull request #2862 from lc6464/fix/mimo-reasoning-history-replay
fix(openai_compat): align MiMo reasoning replay with DeepSeek
2026-05-15 12:09:52 +08:00
美電球 f7d25c6546 Merge pull request #2741 from lc6464/fix/deepseek-stream-reasoning-content
fix(openai_compat): parse reasoning_content in streaming responses
2026-05-15 12:09:26 +08:00
dependabot[bot] 215d98aa78 build(deps): bump tailwindcss from 4.2.4 to 4.3.0 in /web/frontend (#2872)
Bumps [tailwindcss](https://github.com/tailwindlabs/tailwindcss/tree/HEAD/packages/tailwindcss) from 4.2.4 to 4.3.0.
- [Release notes](https://github.com/tailwindlabs/tailwindcss/releases)
- [Changelog](https://github.com/tailwindlabs/tailwindcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tailwindlabs/tailwindcss/commits/v4.3.0/packages/tailwindcss)

---
updated-dependencies:
- dependency-name: tailwindcss
  dependency-version: 4.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 11:23:13 +08:00
dependabot[bot] dab8391344 build(deps-dev): bump typescript-eslint in /web/frontend (#2871)
Bumps [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) from 8.59.1 to 8.59.3.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.59.3/packages/typescript-eslint)

---
updated-dependencies:
- dependency-name: typescript-eslint
  dependency-version: 8.59.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 11:08:46 +08:00
dependabot[bot] a4abbf62e2 build(deps-dev): bump prettier-plugin-tailwindcss in /web/frontend (#2870)
Bumps [prettier-plugin-tailwindcss](https://github.com/tailwindlabs/prettier-plugin-tailwindcss) from 0.7.2 to 0.8.0.
- [Release notes](https://github.com/tailwindlabs/prettier-plugin-tailwindcss/releases)
- [Changelog](https://github.com/tailwindlabs/prettier-plugin-tailwindcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tailwindlabs/prettier-plugin-tailwindcss/compare/v0.7.2...v0.8.0)

---
updated-dependencies:
- dependency-name: prettier-plugin-tailwindcss
  dependency-version: 0.8.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 10:58:34 +08:00
dependabot[bot] 8ab455171c build(deps): bump jotai from 2.19.1 to 2.20.0 in /web/frontend (#2869)
Bumps [jotai](https://github.com/pmndrs/jotai) from 2.19.1 to 2.20.0.
- [Release notes](https://github.com/pmndrs/jotai/releases)
- [Commits](https://github.com/pmndrs/jotai/compare/v2.19.1...v2.20.0)

---
updated-dependencies:
- dependency-name: jotai
  dependency-version: 2.20.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 10:46:39 +08:00
dependabot[bot] eec4436e64 build(deps): bump github.com/adhocore/gronx from 1.19.6 to 1.19.7 (#2868)
Bumps [github.com/adhocore/gronx](https://github.com/adhocore/gronx) from 1.19.6 to 1.19.7.
- [Release notes](https://github.com/adhocore/gronx/releases)
- [Changelog](https://github.com/adhocore/gronx/blob/main/CHANGELOG.md)
- [Commits](https://github.com/adhocore/gronx/compare/v1.19.6...v1.19.7)

---
updated-dependencies:
- dependency-name: github.com/adhocore/gronx
  dependency-version: 1.19.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 10:31:30 +08:00
dependabot[bot] dc41c9c566 build(deps): bump golang.org/x/net from 0.53.0 to 0.54.0 (#2867)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.53.0 to 0.54.0.
- [Commits](https://github.com/golang/net/compare/v0.53.0...v0.54.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.54.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 10:21:14 +08:00
dependabot[bot] 2f8429f57c build(deps): bump github.com/mymmrac/telego from 1.8.0 to 1.9.0 (#2866)
Bumps [github.com/mymmrac/telego](https://github.com/mymmrac/telego) from 1.8.0 to 1.9.0.
- [Release notes](https://github.com/mymmrac/telego/releases)
- [Commits](https://github.com/mymmrac/telego/compare/v1.8.0...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/mymmrac/telego
  dependency-version: 1.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 10:13:57 +08:00
dependabot[bot] d8385ce0a7 build(deps-dev): bump vite from 8.0.10 to 8.0.13 in /web/frontend (#2865)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 8.0.10 to 8.0.13.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.13/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.0.13
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 10:08:02 +08:00
dependabot[bot] 89631b8671 build(deps): bump github.com/larksuite/oapi-sdk-go/v3 (#2864)
Bumps [github.com/larksuite/oapi-sdk-go/v3](https://github.com/larksuite/oapi-sdk-go) from 3.6.1 to 3.7.5.
- [Release notes](https://github.com/larksuite/oapi-sdk-go/releases)
- [Changelog](https://github.com/larksuite/oapi-sdk-go/blob/v3_main/changelog.md)
- [Commits](https://github.com/larksuite/oapi-sdk-go/compare/v3.6.1...v3.7.5)

---
updated-dependencies:
- dependency-name: github.com/larksuite/oapi-sdk-go/v3
  dependency-version: 3.7.5
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 10:06:33 +08:00
dependabot[bot] 4db1168962 build(deps): bump modernc.org/sqlite from 1.48.2 to 1.50.1 (#2863)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.48.2 to 1.50.1.
- [Changelog](https://gitlab.com/cznic/sqlite/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.48.2...v1.50.1)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.50.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-15 09:57:11 +08:00
肆月 f6190b54de feat(web,api): fetch models and saved catalog support (#2832)
* feat(web,api): add fetch models and saved catalog support

Split from PR #2752 (part 2 of 3).

Backend:
- /api/models/catalog endpoint for browsing remote model catalogs
- /api/models/fetch endpoint for fetching available models from providers
- Credential reuse with provider/API base matching for security
- Default API base resolution for providers without explicit base

Frontend:
- FetchModelsDialog for importing models from remote providers
- CatalogDialog for browsing and importing from model catalogs
- Static import for FetchModelsDialog (replaces dynamic import from PR1)
- Dynamic import retained for TestModelDialog (PR3 territory)

* fix(web,api): support bare-array responses in fetchOpenAICompatibleModels

* fix(web,api): tighten maskAPIKeyValue to match maskAPIKey policy

For 9-12 character keys, maskAPIKeyValue exposed first 4 + last 4
chars (only 1 char masked for a 9-char key). Now uses the same
policy as maskAPIKey: first 3 + last 2 for 9-12 chars, first 3 +
last 4 for longer keys. Adds tests covering all key length boundaries.
2026-05-15 09:49:03 +08:00
lc6464 6ae7dc38b9 fix(openai_compat): align MiMo reasoning replay with DeepSeek 2026-05-14 20:59:01 +08:00
lc6464 10f4466a7e fix(openai_compat): parse SSE events and reasoning variants in streams 2026-05-14 10:20:01 +08:00
Anton Bogdanovich 794eb04f32 feat(providers): add gemini web search provider (#2763)
* add gemini web search provider

* fix(web): prefer free providers before Gemini in auto mode

* fix(web): expose gemini api key and model settings

* fix(web): prefer configured providers before Gemini in auto mode

* fix(web): satisfy gemini lint checks

* fix(web): address gemini provider review feedback

* test(web): align auto-provider expectations

* fix(web): let gemini ignore search range
2026-05-14 09:50:47 +08:00
afjcjsbx ffb8243721 fix(integration): docker runner workspace mounts and go command path 2026-05-13 20:03:55 +02:00
afjcjsbx ec21ddc222 fix lint 2026-05-13 19:57:38 +02:00
afjcjsbx ffe091d8b2 docs(cli): mention streamable-http in mcp add transport help 2026-05-13 19:56:08 +02:00
afjcjsbx 4edbc73b64 fix(integration): execute suite commands directly in docker runner 2026-05-13 19:54:32 +02:00
afjcjsbx ffc8bdba36 fix(mcp): normalize streamable-http before config validation 2026-05-13 19:30:59 +02:00
afjcjsbx 0ac8703e0f docs(tests): document integration test strategy 2026-05-07 20:45:05 +02:00
afjcjsbx 131f33f084 fix(tests): add t.Parallel() to mcp streamable subtests 2026-05-07 20:39:53 +02:00
afjcjsbx 6dd30a0c77 fix(ci): ensure go binary resolution and correct command execution 2026-05-07 20:31:57 +02:00
afjcjsbx 7a8d7fb218 fix(scripts): allow command expansion in integration runner 2026-05-07 20:28:07 +02:00
afjcjsbx 703f630f33 fix(ci): define golang image and volumes for integration-runner 2026-05-07 19:44:06 +02:00
afjcjsbx 6e8590900b fix(mcp): support streamable HTTP alias and request-response mode 2026-05-07 19:24:02 +02:00
SiYue e304dce40e docs: sync all documentation to V3 config format
- Replace config/config.example.json with V3 format (version: 3, api_keys array, channel_list)
- Update config-versioning.md: version 2→3, ConfigV2→ConfigV3, CurrentVersion=3
- Update 7 project READMEs: api_key→api_keys, add version: 3 to quick-start examples
- Update 12 security docs (ANTIGRAVITY_AUTH + credential_encryption): api_key→api_keys
- Update provider-refactoring.md: api_key→api_keys in all config examples
- Update security_configuration.md: api_key→api_keys in Before example
- Update 3 channel docs: channels→channel_list in JSON examples
2026-05-06 18:02:46 +08:00
lc6464 b00ff5bc5d fix(openai_compat): parse reasoning_content in streaming responses 2026-05-01 15:20:35 +08:00
161 changed files with 12139 additions and 1074 deletions
+11
View File
@@ -5,7 +5,18 @@ on:
branches: [ "main" ]
jobs:
integration:
name: Integration Tests
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Run Docker-backed integration suites
run: bash ./scripts/run-integration-tests.sh
build:
needs: integration
runs-on: ubuntu-latest
steps:
- name: Checkout
+10
View File
@@ -64,3 +64,13 @@ jobs:
- name: Run go test
run: go test -tags goolm,stdjson ./...
integration:
name: Integration Tests
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Run Docker-backed integration suites
run: bash ./scripts/run-integration-tests.sh
+3
View File
@@ -73,10 +73,13 @@ make check # Full pre-commit check: deps + fmt + vet + test + docs consist
```bash
make test # Run all tests
make integration-test # Run Docker-backed integration suites
go test -run TestName -v ./pkg/session/ # Run a single test
go test -bench=. -benchmem -run='^$' ./... # Run benchmarks
```
Docker-backed integration suites are auto-discovered from [`integration/suites/`](integration/suites/). See [`integration/README.md`](integration/README.md) for the suite layout and the conventions used by CI.
### Code Style
```bash
+5 -1
View File
@@ -1,4 +1,4 @@
.PHONY: all build install uninstall clean help test build-all lint-docs
.PHONY: all build install uninstall clean help test integration-test build-all lint-docs
# Build variables
BINARY_NAME=picoclaw
@@ -379,6 +379,10 @@ test: generate
@$(GO) test $(GOFLAGS) $$($(GO) list $(GOFLAGS) ./... | grep -v github.com/sipeed/picoclaw/web/)
@cd web && make test
## integration-test: Run Docker-backed integration test suites
integration-test:
@bash ./scripts/run-integration-tests.sh
## fmt: Format Go code
fmt:
@$(GOLANGCI_LINT) fmt
+1
View File
@@ -493,6 +493,7 @@ PicoClaw can search the web to provide up-to-date information. Configure in `too
| Search Engine | API Key | Free Tier | Link |
|--------------|---------|-----------|------|
| DuckDuckGo | Not needed | Unlimited | Built-in fallback |
| [Gemini Google Search](https://aistudio.google.com/apikey) | Required | Varies | Gemini with Google Search grounding |
| [Baidu Search](https://cloud.baidu.com/doc/qianfan-api/s/Wmbq4z7e5) | Required | 1500/month (daily allocation) | AI-powered, China-optimized |
| [Tavily](https://tavily.com) | Required | 1000 queries/month | Optimized for AI Agents |
| [Brave Search](https://brave.com/search/api) | Required | 2000 queries/month | Fast and private |
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 432 KiB

After

Width:  |  Height:  |  Size: 327 KiB

+2 -2
View File
@@ -74,7 +74,7 @@ func newAddCommand() *cobra.Command {
flags.StringArrayP("env", "e", nil, "Environment variable in KEY=value format (repeatable, saved to config)")
flags.String("env-file", "", "Path to an env file for stdio servers (recommended for secrets)")
flags.StringArrayP("header", "H", nil, "HTTP header in 'Name: Value' or 'Name=Value' format (repeatable)")
flags.StringP("transport", "t", "stdio", "Transport type: stdio, http, or sse")
flags.StringP("transport", "t", "stdio", "Transport type: stdio, http / streamable-http, or sse")
flags.BoolP("force", "f", false, "Overwrite an existing server without prompting")
flags.Bool("deferred", false, "Mark server as deferred (tools hidden until explicitly activated)")
flags.Bool("no-deferred", false, "Mark server as non-deferred (tools always active)")
@@ -173,7 +173,7 @@ func parseAddArgs(args []string) (addOptions, string, string, []string, bool, er
}
func buildServerConfig(target string, args []string, opts addOptions) (config.MCPServerConfig, error) {
transport := strings.ToLower(strings.TrimSpace(opts.Transport))
transport := config.NormalizeMCPTransportType(opts.Transport)
if transport == "" {
transport = "stdio"
}
+41
View File
@@ -296,6 +296,47 @@ func TestMCPAddHTTPServer(t *testing.T) {
assert.Empty(t, server.Command)
}
func TestMCPAddSupportsStreamableHTTPAlias(t *testing.T) {
configPath := setupMCPConfigEnv(t)
cmd := NewMCPCommand()
_, err := executeCommand(cmd, []string{
"add",
"context7",
"--transport",
"streamable-http",
"https://mcp.context7.com/mcp",
}, "")
require.NoError(t, err)
cfg := readMCPConfig(t, configPath)
server := cfg.Tools.MCP.Servers["context7"]
assert.Equal(t, "http", server.Type)
assert.Equal(t, "https://mcp.context7.com/mcp", server.URL)
}
func TestSaveValidatedConfigNormalizesStreamableHTTPAlias(t *testing.T) {
configPath := setupMCPConfigEnv(t)
cfg := config.DefaultConfig()
cfg.Tools.MCP.Enabled = true
cfg.Tools.MCP.Servers = map[string]config.MCPServerConfig{
"context7": {
Enabled: true,
Type: "streamable-http",
URL: "https://mcp.context7.com/mcp",
},
}
require.NoError(t, saveValidatedConfig(cfg))
saved := readMCPConfig(t, configPath)
server := saved.Tools.MCP.Servers["context7"]
assert.Equal(t, "http", server.Type)
assert.Equal(t, "https://mcp.context7.com/mcp", server.URL)
assert.Equal(t, "streamable-http", cfg.Tools.MCP.Servers["context7"].Type)
}
func TestMCPRemoveRemovesLastServerAndDisablesMCP(t *testing.T) {
configPath := setupMCPConfigEnv(t)
writeMCPConfig(t, configPath, &config.Config{
+27 -12
View File
@@ -108,7 +108,9 @@ func saveValidatedConfig(cfg *config.Config) error {
return fmt.Errorf("config is nil")
}
data, err := json.Marshal(cfg)
normalizedCfg := normalizedConfigForSave(cfg)
data, err := json.Marshal(normalizedCfg)
if err != nil {
return fmt.Errorf("failed to serialize config: %w", err)
}
@@ -117,13 +119,32 @@ func saveValidatedConfig(cfg *config.Config) error {
return err
}
if err := config.SaveConfig(internal.GetConfigPath(), cfg); err != nil {
if err := config.SaveConfig(internal.GetConfigPath(), normalizedCfg); err != nil {
return fmt.Errorf("failed to save config: %w", err)
}
return nil
}
func normalizedConfigForSave(cfg *config.Config) *config.Config {
clone := *cfg
if cfg.Tools.MCP.Servers == nil {
return &clone
}
clone.Tools = cfg.Tools
clone.Tools.MCP = cfg.Tools.MCP
clone.Tools.MCP.Servers = make(map[string]config.MCPServerConfig, len(cfg.Tools.MCP.Servers))
for name, server := range cfg.Tools.MCP.Servers {
if server.Type != "" {
server.Type = config.NormalizeMCPTransportType(server.Type)
}
clone.Tools.MCP.Servers[name] = server
}
return &clone
}
func validateConfigDocument(data []byte) error {
var instance map[string]any
if err := json.Unmarshal(data, &instance); err != nil {
@@ -156,17 +177,11 @@ func loadMCPConfigSchema() (*jsonschema.Resolved, error) {
}
func inferTransportType(server config.MCPServerConfig) string {
switch server.Type {
case "stdio", "http", "sse":
return server.Type
transport := config.EffectiveMCPTransportType(server)
if transport == "" {
return "unknown"
}
if server.URL != "" {
return "sse"
}
if server.Command != "" {
return "stdio"
}
return "unknown"
return transport
}
func renderServerTarget(server config.MCPServerConfig) string {
+143 -90
View File
@@ -1,4 +1,5 @@
{
"version": 3,
"agents": {
"defaults": {
"workspace": "~/.picoclaw/workspace",
@@ -33,13 +34,13 @@
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-openai-key",
"api_keys": ["sk-your-openai-key"],
"api_base": "https://api.openai.com/v1"
},
{
"model_name": "claude-sonnet-4.6",
"model": "anthropic/claude-sonnet-4.6",
"api_key": "sk-ant-your-key",
"api_keys": ["sk-ant-your-key"],
"api_base": "https://api.anthropic.com/v1",
"thinking_level": "high"
},
@@ -47,7 +48,7 @@
"_comment": "Anthropic Messages API - use native format for direct Anthropic API access",
"model_name": "claude-opus-4-6",
"model": "anthropic-messages/claude-opus-4-6",
"api_key": "sk-ant-your-key",
"api_keys": ["sk-ant-your-key"],
"api_base": "https://api.anthropic.com"
},
{
@@ -59,12 +60,12 @@
{
"model_name": "deepseek",
"model": "deepseek/deepseek-chat",
"api_key": "sk-your-deepseek-key"
"api_keys": ["sk-your-deepseek-key"]
},
{
"model_name": "venice-uncensored",
"model": "venice/venice-uncensored",
"api_key": "your-venice-api-key"
"api_keys": ["your-venice-api-key"]
},
{
"model_name": "lmstudio-local",
@@ -73,114 +74,134 @@
{
"model_name": "longcat",
"model": "longcat/LongCat-Flash-Thinking",
"api_key": "your-longcat-api-key"
"api_keys": ["your-longcat-api-key"]
},
{
"model_name": "modelscope-qwen",
"model": "modelscope/Qwen/Qwen3-235B-A22B-Instruct-2507",
"api_key": "your-modelscope-access-token",
"api_keys": ["your-modelscope-access-token"],
"api_base": "https://api-inference.modelscope.cn/v1"
},
{
"model_name": "azure-gpt5",
"model": "azure/my-gpt5-deployment",
"api_key": "your-azure-api-key",
"api_keys": ["your-azure-api-key"],
"api_base": "https://your-resource.openai.azure.com"
},
{
"model_name": "loadbalanced-gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-key1",
"api_keys": ["sk-key1"],
"api_base": "https://api1.example.com/v1"
},
{
"model_name": "loadbalanced-gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-key2",
"api_keys": ["sk-key2"],
"api_base": "https://api2.example.com/v1"
}
],
"channels": {
"channel_list": {
"telegram": {
"enabled": false,
"token": "YOUR_TELEGRAM_BOT_TOKEN",
"base_url": "",
"proxy": "",
"type": "telegram",
"allow_from": ["YOUR_USER_ID"],
"use_markdown_v2": false,
"reasoning_channel_id": "",
"streaming": {
"enabled": true
"settings": {
"token": "YOUR_TELEGRAM_BOT_TOKEN",
"base_url": "",
"proxy": "",
"use_markdown_v2": false,
"streaming": {
"enabled": true
}
}
},
"discord": {
"enabled": false,
"token": "YOUR_DISCORD_BOT_TOKEN",
"proxy": "",
"type": "discord",
"allow_from": [],
"group_trigger": {
"mention_only": false
},
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"token": "YOUR_DISCORD_BOT_TOKEN",
"proxy": ""
}
},
"qq": {
"enabled": false,
"app_id": "YOUR_QQ_APP_ID",
"app_secret": "YOUR_QQ_APP_SECRET",
"type": "qq",
"allow_from": [],
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"app_id": "YOUR_QQ_APP_ID",
"app_secret": "YOUR_QQ_APP_SECRET"
}
},
"maixcam": {
"enabled": false,
"host": "0.0.0.0",
"port": 18790,
"type": "maixcam",
"allow_from": [],
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"host": "0.0.0.0",
"port": 18790
}
},
"whatsapp": {
"enabled": false,
"bridge_url": "ws://localhost:3001",
"use_native": false,
"session_store_path": "",
"type": "whatsapp",
"allow_from": [],
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"bridge_url": "ws://localhost:3001",
"use_native": false,
"session_store_path": ""
}
},
"feishu": {
"enabled": false,
"app_id": "",
"app_secret": "",
"encrypt_key": "",
"verification_token": "",
"type": "feishu",
"allow_from": [],
"reasoning_channel_id": "",
"placeholder": {
"enabled": true,
"text": ["Thinking...", "Processing...", "Typing..."]
},
"reasoning_channel_id": "",
"random_reaction_emoji": [],
"is_lark": false
"settings": {
"app_id": "",
"app_secret": "",
"encrypt_key": "",
"verification_token": "",
"random_reaction_emoji": [],
"is_lark": false
}
},
"dingtalk": {
"enabled": false,
"client_id": "YOUR_CLIENT_ID",
"client_secret": "YOUR_CLIENT_SECRET",
"type": "dingtalk",
"allow_from": [],
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"client_id": "YOUR_CLIENT_ID",
"client_secret": "YOUR_CLIENT_SECRET"
}
},
"slack": {
"enabled": false,
"bot_token": "xoxb-YOUR-BOT-TOKEN",
"app_token": "xapp-YOUR-APP-TOKEN",
"type": "slack",
"allow_from": [],
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"bot_token": "xoxb-YOUR-BOT-TOKEN",
"app_token": "xapp-YOUR-APP-TOKEN"
}
},
"matrix": {
"enabled": false,
"homeserver": "https://matrix.org",
"user_id": "@your-bot:matrix.org",
"access_token": "YOUR_MATRIX_ACCESS_TOKEN",
"device_id": "",
"join_on_invite": true,
"type": "matrix",
"allow_from": [],
"group_trigger": {
"mention_only": true
@@ -190,68 +211,82 @@
"text": ["Thinking...", "Processing...", "Typing..."]
},
"reasoning_channel_id": "",
"crypto_database_path": "",
"crypto_passphrase": "YOUR_MATRIX_CRYPTO_PICKLE_KEY"
"settings": {
"homeserver": "https://matrix.org",
"user_id": "@your-bot:matrix.org",
"access_token": "YOUR_MATRIX_ACCESS_TOKEN",
"device_id": "",
"join_on_invite": true,
"crypto_database_path": "",
"crypto_passphrase": "YOUR_MATRIX_CRYPTO_PICKLE_KEY"
}
},
"line": {
"enabled": false,
"channel_secret": "YOUR_LINE_CHANNEL_SECRET",
"channel_access_token": "YOUR_LINE_CHANNEL_ACCESS_TOKEN",
"webhook_path": "/webhook/line",
"type": "line",
"allow_from": [],
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"channel_secret": "YOUR_LINE_CHANNEL_SECRET",
"channel_access_token": "YOUR_LINE_CHANNEL_ACCESS_TOKEN",
"webhook_path": "/webhook/line"
}
},
"onebot": {
"enabled": false,
"ws_url": "ws://127.0.0.1:3001",
"access_token": "",
"reconnect_interval": 5,
"group_trigger_prefix": [],
"type": "onebot",
"allow_from": [],
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"group_trigger": {
"prefixes": []
},
"settings": {
"ws_url": "ws://127.0.0.1:3001",
"access_token": "",
"reconnect_interval": 5
}
},
"wecom": {
"_comment": "WeCom AI Bot over WebSocket.",
"enabled": false,
"bot_id": "YOUR_BOT_ID",
"secret": "YOUR_SECRET",
"websocket_url": "wss://openws.work.weixin.qq.com",
"send_thinking_message": true,
"type": "wecom",
"allow_from": [],
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"bot_id": "YOUR_BOT_ID",
"secret": "YOUR_SECRET",
"websocket_url": "wss://openws.work.weixin.qq.com",
"send_thinking_message": true
}
},
"pico": {
"enabled": false,
"token": "YOUR_PICO_TOKEN",
"allow_token_query": false,
"allow_origins": [],
"ping_interval": 30,
"read_timeout": 60,
"max_connections": 100,
"allow_from": []
"type": "pico",
"allow_from": [],
"settings": {
"token": "YOUR_PICO_TOKEN",
"allow_token_query": false,
"allow_origins": [],
"ping_interval": 30,
"read_timeout": 60,
"max_connections": 100
}
},
"pico_client": {
"enabled": false,
"url": "wss://remote-pico-server/pico/ws",
"token": "YOUR_PICO_TOKEN",
"session_id": "",
"ping_interval": 30,
"read_timeout": 60,
"allow_from": []
"type": "pico_client",
"allow_from": [],
"settings": {
"url": "wss://remote-pico-server/pico/ws",
"token": "YOUR_PICO_TOKEN",
"session_id": "",
"ping_interval": 30,
"read_timeout": 60
}
},
"irc": {
"enabled": false,
"server": "irc.libera.chat:6697",
"tls": true,
"nick": "mybot",
"user": "",
"real_name": "",
"password": "",
"nickserv_password": "",
"sasl_user": "",
"sasl_password": "",
"channels": ["#mychannel"],
"request_caps": ["server-time", "message-tags"],
"type": "irc",
"allow_from": [],
"group_trigger": {
"mention_only": true
@@ -259,7 +294,20 @@
"typing": {
"enabled": false
},
"reasoning_channel_id": ""
"reasoning_channel_id": "",
"settings": {
"server": "irc.libera.chat:6697",
"tls": true,
"nick": "mybot",
"user": "",
"real_name": "",
"password": "",
"nickserv_password": "",
"sasl_user": "",
"sasl_password": "",
"channels": ["#mychannel"],
"request_caps": ["server-time", "message-tags"]
}
}
},
"tools": {
@@ -268,7 +316,6 @@
"web": {
"enabled": true,
"prefer_native": true,
"fetch_limit_bytes": 10485760,
"format": "plaintext",
"brave": {
"enabled": false,
@@ -291,6 +338,12 @@
"enabled": false,
"max_results": 5
},
"gemini": {
"enabled": false,
"api_key": "",
"model": "gemini-2.5-flash",
"max_results": 5
},
"perplexity": {
"enabled": false,
"api_key": "pplx-xxx",
+5 -5
View File
@@ -78,17 +78,17 @@ Inspired by [LiteLLM](https://docs.litellm.ai/docs/proxy/configs) design:
"model_name": "deepseek-chat",
"model": "openai/deepseek-chat",
"api_base": "https://api.deepseek.com/v1",
"api_key": "sk-xxx"
"api_keys": ["sk-xxx"]
},
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-xxx"
"api_keys": ["sk-xxx"]
},
{
"model_name": "claude-sonnet-4.6",
"model": "anthropic/claude-sonnet-4.6",
"api_key": "sk-xxx"
"api_keys": ["sk-xxx"]
},
{
"model_name": "gemini-3-flash",
@@ -99,7 +99,7 @@ Inspired by [LiteLLM](https://docs.litellm.ai/docs/proxy/configs) design:
"model_name": "my-company-llm",
"model": "openai/company-model-v1",
"api_base": "https://llm.company.com/v1",
"api_key": "xxx"
"api_keys": ["xxx"]
}
],
@@ -252,7 +252,7 @@ func (c *Config) GetModelConfig(modelName string) (*ModelConfig, error) {
{
"providers": {
"deepseek": {
"api_key": "sk-xxx",
"api_keys": ["sk-xxx"],
"api_base": "https://api.deepseek.com/v1"
}
},
+49
View File
@@ -364,6 +364,55 @@ Configurez plusieurs endpoints pour le même nom de modèle — PicoClaw effectu
L'ancienne configuration `providers` est **dépréciée** et a été supprimée dans V2. Les configs V0/V1 existantes sont auto-migrées. Voir [docs/migration/model-list-migration.md](../migration/model-list-migration.md).
#### Configuration du Streaming
Le streaming provider utilise un double opt-in et est désactivé par défaut. L'agent ne tente le streaming que lorsque le channel courant a `settings.streaming.enabled: true`, que l'entrée de modèle active a `streaming.enabled: true`, et que le provider comme le channel prennent en charge le streaming. Si une condition manque, PicoClaw utilise le chemin de requête non-streaming normal.
Pico WebUI est le premier channel entièrement câblé. Pico crée le premier message assistant avec le message wire existant `message.create`, puis met à jour ce même message avec `message.update`; aucun nouveau type de message Pico n'est introduit.
Laissez `streaming` absent si vous ne voulez pas de streaming. Un bloc `streaming` omis signifie désactivé; il n'est pas nécessaire d'écrire `"streaming": {"enabled": false}`.
Exemple d'activation :
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"provider": "openai",
"model": "gpt-5.4",
"api_keys": ["sk-your-openai-key"],
"streaming": {
"enabled": true
}
}
],
"channel_list": {
"pico": {
"enabled": true,
"type": "pico",
"settings": {
"token": "YOUR_PICO_TOKEN",
"streaming": {
"enabled": true
}
}
}
}
}
```
| Champ | Type | Défaut | Description |
| ----- | ---- | ------ | ----------- |
| `channel_list.<name>.settings.streaming.enabled` | bool | `false` | Autorise ce channel à afficher la sortie streaming du provider |
| `channel_list.<name>.settings.streaming.throttle_seconds` | int | Défaut Pico après activation : `0` | Intervalle minimal entre les mises à jour intermédiaires; le contenu final est toujours envoyé |
| `channel_list.<name>.settings.streaming.min_growth_chars` | int | Défaut Pico après activation : `1` | Croissance minimale du texte avant une mise à jour intermédiaire; le contenu final est toujours envoyé |
| `model_list[].streaming.enabled` | bool | `false` | Autorise cette entrée de modèle à tenter des requêtes provider en streaming |
Les anciennes variables d'environnement Telegram restent compatibles : `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED`, `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS` et `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS`. Elles s'appliquent uniquement aux settings Telegram et n'activent ni ne modifient `settings.streaming` de Pico.
Le comportement d'échec est volontairement conservateur : si le streaming échoue avant l'envoi d'un chunk visible, PicoClaw réessaie une fois via le chemin `Chat()` normal. Si un chunk a déjà été affiché à l'utilisateur, PicoClaw n'envoie pas une deuxième réponse non-streaming, afin d'éviter une sortie dupliquée.
### Architecture des Providers
PicoClaw route les providers par famille de protocole :
+49
View File
@@ -31,6 +31,55 @@ PICOCLAW_HOME=/opt/picoclaw picoclaw agent
PICOCLAW_HOME=/srv/picoclaw PICOCLAW_CONFIG=/srv/picoclaw/main.json picoclaw gateway
```
### Configurazione Streaming
Lo streaming del provider usa un double opt-in ed è disattivato per impostazione predefinita. L'agent prova lo streaming solo quando il canale corrente ha `settings.streaming.enabled: true`, l'entry del modello attivo ha `streaming.enabled: true`, e sia il provider sia il canale supportano lo streaming. Se manca una qualsiasi condizione, PicoClaw usa il normale percorso di richiesta non streaming.
Pico WebUI è il primo canale completamente collegato. Pico crea il primo messaggio assistant con il wire message esistente `message.create`, poi aggiorna lo stesso messaggio con `message.update`; non viene introdotto alcun nuovo tipo di wire message Pico.
Lascia `streaming` assente quando non vuoi usare lo streaming. Un blocco `streaming` omesso significa disattivato; non è necessario scrivere `"streaming": {"enabled": false}`.
Esempio di attivazione:
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"provider": "openai",
"model": "gpt-5.4",
"api_keys": ["sk-your-openai-key"],
"streaming": {
"enabled": true
}
}
],
"channel_list": {
"pico": {
"enabled": true,
"type": "pico",
"settings": {
"token": "YOUR_PICO_TOKEN",
"streaming": {
"enabled": true
}
}
}
}
}
```
| Campo | Tipo | Predefinito | Descrizione |
| ----- | ---- | ----------- | ----------- |
| `channel_list.<name>.settings.streaming.enabled` | bool | `false` | Permette a questo canale di mostrare l'output streaming del provider |
| `channel_list.<name>.settings.streaming.throttle_seconds` | int | Predefinito Pico dopo l'attivazione: `0` | Intervallo minimo tra aggiornamenti intermedi; il contenuto finale viene sempre inviato |
| `channel_list.<name>.settings.streaming.min_growth_chars` | int | Predefinito Pico dopo l'attivazione: `1` | Crescita minima del testo prima di inviare un aggiornamento intermedio; il contenuto finale viene sempre inviato |
| `model_list[].streaming.enabled` | bool | `false` | Permette a questa entry di modello di provare richieste provider streaming |
Le variabili d'ambiente legacy di Telegram restano compatibili: `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED`, `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS` e `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS`. Si applicano solo alle settings Telegram e non attivano né modificano `settings.streaming` di Pico.
Il comportamento in caso di errore è intenzionalmente conservativo: se lo streaming fallisce prima che venga inviato un chunk visibile, PicoClaw riprova una volta tramite il normale percorso `Chat()`. Se un chunk è già stato mostrato all'utente, PicoClaw non invia una seconda risposta non streaming, evitando output duplicato.
### Struttura del Workspace
PicoClaw salva i dati nel workspace configurato (predefinito: `~/.picoclaw/workspace`):
+49
View File
@@ -365,6 +365,55 @@ HEARTBEAT_OK を返信 ユーザーが直接結果を受信
`providers` 設定は**非推奨**となり、V2 で削除されました。既存の V0/V1 設定は自動的に移行されます。[docs/migration/model-list-migration.md](../migration/model-list-migration.md) を参照してください。
#### ストリーミング設定
Provider ストリーミングは二重の opt-in 方式で、デフォルトでは無効です。現在の channel に `settings.streaming.enabled: true` があり、アクティブなモデルエントリに `streaming.enabled: true` があり、さらに provider と channel の両方がストリーミングをサポートしている場合にのみ、agent はストリーミングリクエストを試行します。いずれかの条件が欠ける場合、PicoClaw は通常の非ストリーミングリクエスト経路を使います。
Pico WebUI が最初に完全対応した channel です。Pico は既存の `message.create` wire message で最初の assistant メッセージを作成し、その後 `message.update` で同じメッセージを更新します。新しい Pico wire message type は追加されません。
ストリーミングを使わない場合は `streaming` を省略してください。`streaming` ブロックの省略は無効を意味するため、`"streaming": {"enabled": false}` を書く必要はありません。
有効化例:
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"provider": "openai",
"model": "gpt-5.4",
"api_keys": ["sk-your-openai-key"],
"streaming": {
"enabled": true
}
}
],
"channel_list": {
"pico": {
"enabled": true,
"type": "pico",
"settings": {
"token": "YOUR_PICO_TOKEN",
"streaming": {
"enabled": true
}
}
}
}
}
```
| フィールド | 型 | デフォルト | 説明 |
| ---------- | -- | ---------- | ---- |
| `channel_list.<name>.settings.streaming.enabled` | bool | `false` | この channel で provider のストリーミング出力を表示できるようにします |
| `channel_list.<name>.settings.streaming.throttle_seconds` | int | Pico で有効化後のデフォルト:`0` | 中間更新の最小間隔。最終内容は常に flush されます |
| `channel_list.<name>.settings.streaming.min_growth_chars` | int | Pico で有効化後のデフォルト:`1` | 次の中間更新を送るために必要な最小文字増加数。最終内容は常に flush されます |
| `model_list[].streaming.enabled` | bool | `false` | このモデルエントリで provider ストリーミングリクエストを試行できるようにします |
既存の Telegram 環境変数 `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED``PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS``PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS` は互換性のため引き続き使えます。これらは Telegram settings にのみ適用され、Pico の `settings.streaming` を有効化または変更しません。
失敗時の動作は保守的です。可視 chunk が送信される前にストリーミングが失敗した場合、PicoClaw は通常の `Chat()` 経路で一度だけ再試行します。すでに chunk がユーザーに表示されている場合は、表示済み出力の重複を避けるため、二つ目の非ストリーミング回答は送信しません。
### Provider アーキテクチャ
PicoClaw はプロトコルファミリーで Provider をルーティングします:
+49
View File
@@ -744,6 +744,55 @@ Resolution rules:
- If `provider` is omitted, PicoClaw treats the first `/` segment in `model` as the provider and everything after that first `/` as the runtime model ID.
- This means `"model": "openrouter/openai/gpt-5.4"` still works as a compatibility form and sends `openai/gpt-5.4` to OpenRouter.
#### Streaming Configuration
Provider streaming uses a double opt-in and is disabled by default. The agent only tries streaming when the current channel has `settings.streaming.enabled: true`, the active model entry has `streaming.enabled: true`, and both the provider and channel support streaming. If any condition is missing, PicoClaw uses the normal non-streaming request path.
Pico WebUI is the first fully wired channel. Pico creates the first assistant message with the existing `message.create` wire message, then updates that same message with `message.update`; no new Pico wire message type is introduced.
Leave `streaming` unset when you do not want streaming. An omitted `streaming` block means disabled; you do not need to write `"streaming": {"enabled": false}`.
Opt-in example:
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"provider": "openai",
"model": "gpt-5.4",
"api_keys": ["sk-your-openai-key"],
"streaming": {
"enabled": true
}
}
],
"channel_list": {
"pico": {
"enabled": true,
"type": "pico",
"settings": {
"token": "YOUR_PICO_TOKEN",
"streaming": {
"enabled": true
}
}
}
}
}
```
| Field | Type | Default | Description |
| ----- | ---- | ------- | ----------- |
| `channel_list.<name>.settings.streaming.enabled` | bool | `false` | Allows this channel to display provider streaming output |
| `channel_list.<name>.settings.streaming.throttle_seconds` | int | Pico default after enabling: `0` | Minimum interval for intermediate updates; final content is always flushed |
| `channel_list.<name>.settings.streaming.min_growth_chars` | int | Pico default after enabling: `1` | Minimum character growth before sending an intermediate update; final content is always flushed |
| `model_list[].streaming.enabled` | bool | `false` | Allows this model entry to try provider streaming requests |
Legacy Telegram environment variables remain compatible: `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED`, `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS`, and `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS`. They only apply to Telegram settings and do not enable or modify Pico `settings.streaming`.
Failure behavior is intentionally conservative: if streaming fails before any visible chunk is sent, PicoClaw retries once through the normal `Chat()` path. If a chunk has already been shown to the user, PicoClaw does not send a second non-streaming answer, because that would duplicate visible output.
#### Vendor-Specific Examples
> **Tip**: You can omit `api_key` fields and store them in `.security.yml` for better security. See [Security Configuration](#-security-configuration-recommended).
+49
View File
@@ -31,6 +31,55 @@ PICOCLAW_HOME=/opt/picoclaw picoclaw agent
PICOCLAW_HOME=/srv/picoclaw PICOCLAW_CONFIG=/srv/picoclaw/main.json picoclaw gateway
```
### Konfigurasi Streaming
Provider streaming menggunakan double opt-in dan dimatikan secara lalai. Agent hanya mencuba streaming apabila saluran semasa mempunyai `settings.streaming.enabled: true`, entry model aktif mempunyai `streaming.enabled: true`, dan kedua-dua provider serta saluran menyokong streaming. Jika mana-mana syarat tiada, PicoClaw menggunakan laluan permintaan bukan streaming biasa.
Pico WebUI ialah saluran pertama yang disambungkan sepenuhnya. Pico mencipta mesej assistant pertama dengan wire message sedia ada `message.create`, kemudian mengemas kini mesej yang sama dengan `message.update`; tiada jenis wire message Pico baharu ditambah.
Biarkan `streaming` tidak ditetapkan jika anda tidak mahu streaming. Blok `streaming` yang tiada bermaksud dimatikan; anda tidak perlu menulis `"streaming": {"enabled": false}`.
Contoh mengaktifkan streaming:
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"provider": "openai",
"model": "gpt-5.4",
"api_keys": ["sk-your-openai-key"],
"streaming": {
"enabled": true
}
}
],
"channel_list": {
"pico": {
"enabled": true,
"type": "pico",
"settings": {
"token": "YOUR_PICO_TOKEN",
"streaming": {
"enabled": true
}
}
}
}
}
```
| Kunci | Jenis | Lalai | Penerangan |
| ----- | ----- | ----- | ---------- |
| `channel_list.<name>.settings.streaming.enabled` | bool | `false` | Membenarkan saluran ini memaparkan output streaming provider |
| `channel_list.<name>.settings.streaming.throttle_seconds` | int | Lalai Pico selepas diaktifkan: `0` | Jarak masa minimum antara kemas kini pertengahan; kandungan akhir sentiasa dihantar |
| `channel_list.<name>.settings.streaming.min_growth_chars` | int | Lalai Pico selepas diaktifkan: `1` | Pertambahan aksara minimum sebelum menghantar kemas kini pertengahan; kandungan akhir sentiasa dihantar |
| `model_list[].streaming.enabled` | bool | `false` | Membenarkan entry model ini mencuba permintaan provider streaming |
Pemboleh ubah persekitaran Telegram lama masih serasi: `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED`, `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS`, dan `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS`. Ia hanya digunakan untuk settings Telegram dan tidak mengaktifkan atau mengubah `settings.streaming` Pico.
Tingkah laku kegagalan adalah konservatif: jika streaming gagal sebelum mana-mana chunk kelihatan dihantar, PicoClaw mencuba semula sekali melalui laluan `Chat()` biasa. Jika chunk sudah dipaparkan kepada pengguna, PicoClaw tidak menghantar jawapan bukan streaming kedua untuk mengelakkan output berganda.
### Susun Atur Workspace
PicoClaw menyimpan data dalam workspace yang dikonfigurasikan (lalai: `~/.picoclaw/workspace`):
+49
View File
@@ -365,6 +365,55 @@ Configure múltiplos endpoints para o mesmo nome de modelo — PicoClaw fará ro
A configuração antiga `providers` está **depreciada** e foi removida no V2. Configs V0/V1 existentes são auto-migradas. Veja [docs/migration/model-list-migration.md](../migration/model-list-migration.md).
#### Configuração de Streaming
O streaming do provider usa double opt-in e fica desativado por padrão. O agent só tenta streaming quando o canal atual tem `settings.streaming.enabled: true`, a entrada de modelo ativa tem `streaming.enabled: true`, e tanto o provider quanto o canal suportam streaming. Se qualquer condição estiver ausente, o PicoClaw usa o caminho normal de requisição sem streaming.
O Pico WebUI é o primeiro canal totalmente integrado. O Pico cria a primeira mensagem assistant com o wire message existente `message.create` e depois atualiza a mesma mensagem com `message.update`; nenhum novo tipo de wire message do Pico é introduzido.
Deixe `streaming` ausente quando não quiser streaming. Um bloco `streaming` omitido significa desativado; você não precisa escrever `"streaming": {"enabled": false}`.
Exemplo de ativação:
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"provider": "openai",
"model": "gpt-5.4",
"api_keys": ["sk-your-openai-key"],
"streaming": {
"enabled": true
}
}
],
"channel_list": {
"pico": {
"enabled": true,
"type": "pico",
"settings": {
"token": "YOUR_PICO_TOKEN",
"streaming": {
"enabled": true
}
}
}
}
}
```
| Campo | Tipo | Padrão | Descrição |
| ----- | ---- | ------ | --------- |
| `channel_list.<name>.settings.streaming.enabled` | bool | `false` | Permite que este canal exiba output streaming do provider |
| `channel_list.<name>.settings.streaming.throttle_seconds` | int | Padrão do Pico após ativar: `0` | Intervalo mínimo entre atualizações intermediárias; o conteúdo final sempre é enviado |
| `channel_list.<name>.settings.streaming.min_growth_chars` | int | Padrão do Pico após ativar: `1` | Crescimento mínimo de texto antes de enviar outra atualização intermediária; o conteúdo final sempre é enviado |
| `model_list[].streaming.enabled` | bool | `false` | Permite que esta entrada de modelo tente requisições de provider streaming |
As variáveis de ambiente legadas do Telegram continuam compatíveis: `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED`, `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS` e `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS`. Elas se aplicam apenas às settings do Telegram e não ativam nem modificam `settings.streaming` do Pico.
O comportamento de falha é intencionalmente conservador: se o streaming falhar antes de qualquer chunk visível ser enviado, o PicoClaw tenta novamente uma vez pelo caminho normal `Chat()`. Se um chunk já foi mostrado ao usuário, o PicoClaw não envia uma segunda resposta sem streaming, evitando output duplicado.
### Arquitetura de Providers
PicoClaw roteia providers por família de protocolo:
+49
View File
@@ -365,6 +365,55 @@ Cấu hình nhiều endpoint cho cùng tên mô hình — PicoClaw sẽ tự đ
Cấu hình `providers` cũ đã **bị deprecated** và đã được loại bỏ trong V2. Các cấu hình V0/V1 hiện có sẽ được tự động migrate. Xem [docs/migration/model-list-migration.md](../migration/model-list-migration.md).
#### Cấu Hình Streaming
Provider streaming dùng cơ chế double opt-in và bị tắt theo mặc định. Agent chỉ thử streaming khi channel hiện tại có `settings.streaming.enabled: true`, entry model đang dùng có `streaming.enabled: true`, và cả provider lẫn channel đều hỗ trợ streaming. Nếu thiếu bất kỳ điều kiện nào, PicoClaw dùng đường dẫn yêu cầu không streaming thông thường.
Pico WebUI là channel đầu tiên được nối đầy đủ. Pico tạo message assistant đầu tiên bằng wire message hiện có `message.create`, sau đó cập nhật chính message đó bằng `message.update`; không thêm loại wire message Pico mới.
Hãy để trống `streaming` khi bạn không muốn dùng streaming. Bỏ qua block `streaming` nghĩa là đã tắt; bạn không cần viết `"streaming": {"enabled": false}`.
Ví dụ bật streaming:
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"provider": "openai",
"model": "gpt-5.4",
"api_keys": ["sk-your-openai-key"],
"streaming": {
"enabled": true
}
}
],
"channel_list": {
"pico": {
"enabled": true,
"type": "pico",
"settings": {
"token": "YOUR_PICO_TOKEN",
"streaming": {
"enabled": true
}
}
}
}
}
```
| Trường | Kiểu | Mặc định | Mô tả |
| ------ | ---- | -------- | ----- |
| `channel_list.<name>.settings.streaming.enabled` | bool | `false` | Cho phép channel này hiển thị output streaming từ provider |
| `channel_list.<name>.settings.streaming.throttle_seconds` | int | Mặc định Pico sau khi bật: `0` | Khoảng cách tối thiểu giữa các cập nhật trung gian; nội dung cuối luôn được flush |
| `channel_list.<name>.settings.streaming.min_growth_chars` | int | Mặc định Pico sau khi bật: `1` | Số ký tự tăng tối thiểu trước khi gửi cập nhật trung gian; nội dung cuối luôn được flush |
| `model_list[].streaming.enabled` | bool | `false` | Cho phép entry model này thử yêu cầu provider streaming |
Các biến môi trường Telegram cũ vẫn tương thích: `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED`, `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS`, và `PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS`. Chúng chỉ áp dụng cho Telegram settings và không bật hoặc thay đổi `settings.streaming` của Pico.
Hành vi lỗi được giữ thận trọng: nếu streaming lỗi trước khi gửi bất kỳ chunk hiển thị nào, PicoClaw thử lại một lần qua đường dẫn `Chat()` thông thường. Nếu đã có chunk hiển thị cho người dùng, PicoClaw không gửi thêm một câu trả lời non-streaming thứ hai để tránh lặp output.
### Kiến Trúc Provider
PicoClaw định tuyến provider theo họ giao thức:
+50 -1
View File
@@ -293,7 +293,7 @@ PicoClaw 默认在沙箱环境中运行。Agent 只能访问配置的工作区
| `tools.exec.custom_deny_patterns` | string[] | `[]` | 自定义阻止的正则表达式模式 |
| `tools.exec.custom_allow_patterns` | string[] | `[]` | 自定义允许的正则表达式模式 |
> **安全提示:** Symlink 保护默认启用——所有文件路径在白名单匹配前都会通过 `filepath.EvalSymlinks` 解析,防止符号链接逃逸攻击。
> **安全提示:** Symlink 保护默认启用——所有文件路径在允许列表匹配前都会通过 `filepath.EvalSymlinks` 解析,防止符号链接逃逸攻击。
#### 已知限制:构建工具的子进程
@@ -537,6 +537,55 @@ Agent 读取 HEARTBEAT.md
- 如果未设置 `provider`PicoClaw 会把 `model` 第一个 `/` 之前的字段当作 provider,并把第一个 `/` 之后的全部内容当作最终模型 ID。
- 这意味着 `"model": "openrouter/openai/gpt-5.4"` 这样的兼容写法仍然可用,并会把 `openai/gpt-5.4` 发送给 OpenRouter。
#### 流式输出配置
Provider 流式输出采用双开关,默认关闭。只有当前 channel 的 `settings.streaming.enabled` 和当前模型条目的 `streaming.enabled` 都为 `true`,并且 provider 与 channel 都支持流式能力时,Agent 才会尝试流式请求;任一条件不满足时仍使用普通非流式请求。
当前完整落地的是 Pico WebUI。Pico 使用已有的 `message.create` 创建第一条 assistant 消息,随后用 `message.update` 更新同一条消息,不新增协议消息类型。
不需要流式时请省略 `streaming` 配置块。省略表示关闭,不需要写 `"streaming": {"enabled": false}`
开启示例:
```json
{
"model_list": [
{
"model_name": "gpt-5.4",
"provider": "openai",
"model": "gpt-5.4",
"api_keys": ["sk-your-openai-key"],
"streaming": {
"enabled": true
}
}
],
"channel_list": {
"pico": {
"enabled": true,
"type": "pico",
"settings": {
"token": "YOUR_PICO_TOKEN",
"streaming": {
"enabled": true
}
}
}
}
}
```
| 字段 | 类型 | 默认值 | 说明 |
|------|------|--------|------|
| `channel_list.<name>.settings.streaming.enabled` | bool | `false` | 是否允许该 channel 尝试展示 provider 流式输出 |
| `channel_list.<name>.settings.streaming.throttle_seconds` | int | Pico 开启后默认 `0` | 中间更新的最小时间间隔,最终内容不受此限制 |
| `channel_list.<name>.settings.streaming.min_growth_chars` | int | Pico 开启后默认 `1` | 中间更新相比上次发送至少增长的字符数,最终内容不受此限制 |
| `model_list[].streaming.enabled` | bool | `false` | 是否允许该模型条目尝试 provider 流式请求 |
Telegram 旧环境变量仍兼容:`PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED``PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS``PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS`。这些环境变量只作用于 Telegram settings,不会开启或修改 Pico 的 `settings.streaming`
失败处理保持保守:如果还没有任何可见 chunk 就失败,PicoClaw 会回退到普通 `Chat()` 路径重试一次;如果已经有 chunk 展示给用户,则不会再发送一条非流式最终答案,避免界面重复输出。
#### 各厂商配置示例
<details>
+3
View File
@@ -113,10 +113,13 @@ Cette conception permet également le **support multi-agents** avec une sélecti
| `max_tokens_field` | string | Non | Remplace le nom du champ max tokens dans le corps de la requête (ex : `max_completion_tokens` pour les modèles o1) |
| `thinking_level` | string | Non | Niveau de pensée étendue : `off`, `low`, `medium`, `high`, `xhigh` ou `adaptive` |
| `extra_body` | object | Non | Champs supplémentaires à injecter dans chaque corps de requête |
| `streaming.enabled` | bool | Non | Opt-in pour le streaming provider sur cette entrée de modèle. Par défaut `false`, et le channel actif doit aussi avoir `settings.streaming.enabled` à `true` |
| `rpm` | int | Non | Limite de requêtes par minute |
| `fallbacks` | string[] | Non | Noms des modèles de secours pour le basculement automatique |
| `enabled` | bool | Non | Activer ou désactiver cette entrée de modèle (par défaut : `true`) |
Lorsque le streaming est désactivé, omettez le bloc `streaming`. Écrire `"streaming": {"enabled": false}` est optionnel et n'est pas nécessaire.
#### Exemples par Vendor
**OpenAI**
+3
View File
@@ -114,10 +114,13 @@
| `max_tokens_field` | string | いいえ | リクエストボディの max tokens フィールド名を上書き(例:o1 モデルでは `max_completion_tokens` |
| `thinking_level` | string | いいえ | 拡張思考レベル:`off``low``medium``high``xhigh``adaptive` |
| `extra_body` | object | いいえ | 各リクエストボディに注入する追加フィールド |
| `streaming.enabled` | bool | いいえ | このモデルエントリで provider ストリーミングを試行するための opt-in。デフォルトは `false` で、アクティブな channel の `settings.streaming.enabled``true` である必要があります |
| `rpm` | int | いいえ | 1 分あたりのリクエストレート制限 |
| `fallbacks` | string[] | いいえ | 自動フェイルオーバーのフォールバックモデル名 |
| `enabled` | bool | いいえ | このモデルエントリを有効にするかどうか(デフォルト:`true` |
ストリーミングを無効にする場合は `streaming` ブロックを省略してください。`"streaming": {"enabled": false}` を書くことは任意であり、必須ではありません。
#### ベンダー別設定例
**OpenAI**
+3
View File
@@ -131,10 +131,13 @@ This design also enables **multi-agent support** with flexible provider selectio
| `tool_schema_transform` | string | No | Optional compatibility transform for tool parameter schemas. Default: disabled. Supported values: `simple`. |
| `extra_body` | object | No | Additional fields to inject into every request body |
| `custom_headers` | object | No | Additional HTTP headers to inject into every request (e.g., `{"X-Source":"coding-plan"}`). If a key matches a built-in header, the custom value overrides the built-in one (e.g., `Authorization`, `User-Agent`, `Content-Type`, `Accept`). |
| `streaming.enabled` | bool | No | Opt-in for provider streaming on this model entry. Defaults to `false` and also requires the active channel's `settings.streaming.enabled` to be `true`. |
| `rpm` | int | No | Per-minute request rate limit |
| `fallbacks` | string[] | No | Fallback model names for automatic failover |
| `enabled` | bool | No | Whether this model entry is active (default: `true`) |
When streaming is disabled, omit the `streaming` block. Writing `"streaming": {"enabled": false}` is optional and not needed in generated or hand-written config.
#### Tool Schema Compatibility
By default, PicoClaw now forwards tool JSON Schemas unchanged.
+3
View File
@@ -113,10 +113,13 @@ Este design também permite **suporte multi-agente** com seleção flexível de
| `max_tokens_field` | string | Não | Substitui o nome do campo max tokens no corpo da requisição (ex: `max_completion_tokens` para modelos o1) |
| `thinking_level` | string | Não | Nível de pensamento estendido: `off`, `low`, `medium`, `high`, `xhigh` ou `adaptive` |
| `extra_body` | object | Não | Campos adicionais para injetar em cada corpo de requisição |
| `streaming.enabled` | bool | Não | Opt-in para provider streaming nesta entrada de modelo. O padrão é `false` e o canal ativo também precisa de `settings.streaming.enabled` como `true` |
| `rpm` | int | Não | Limite de requisições por minuto |
| `fallbacks` | string[] | Não | Nomes dos modelos de fallback para failover automático |
| `enabled` | bool | Não | Ativar ou desativar esta entrada de modelo (padrão: `true`) |
Quando streaming estiver desativado, omita o bloco `streaming`. Escrever `"streaming": {"enabled": false}` é opcional e não é necessário.
#### Exemplos por Vendor
**OpenAI**
+3
View File
@@ -113,10 +113,13 @@ Thiết kế này cũng cho phép **hỗ trợ đa agent** với lựa chọn pr
| `max_tokens_field` | string | Không | Ghi đè tên trường max tokens trong request body (ví dụ: `max_completion_tokens` cho model o1) |
| `thinking_level` | string | Không | Mức độ tư duy mở rộng: `off`, `low`, `medium`, `high`, `xhigh` hoặc `adaptive` |
| `extra_body` | object | Không | Các trường bổ sung để chèn vào mỗi request body |
| `streaming.enabled` | bool | Không | Opt-in cho provider streaming trên entry model này. Mặc định là `false` và channel đang hoạt động cũng cần `settings.streaming.enabled``true` |
| `rpm` | int | Không | Giới hạn tốc độ yêu cầu mỗi phút |
| `fallbacks` | string[] | Không | Tên model dự phòng cho failover tự động |
| `enabled` | bool | Không | Kích hoạt hay vô hiệu hóa entry model này (mặc định: `true`) |
Khi không dùng streaming, hãy bỏ qua block `streaming`. Viết `"streaming": {"enabled": false}` là tùy chọn và không cần thiết.
#### Ví Dụ Theo Vendor
**OpenAI**
+3
View File
@@ -123,6 +123,7 @@
| `proxy` | string | 否 | 此模型条目的 HTTP 代理 URL |
| `user_agent` | string | 否 | 自定义 `User-Agent` 请求头(支持 OpenAI 兼容、Gemini、Anthropic 和 Azure provider |
| `request_timeout` | int | 否 | 请求超时时间(秒),默认值因 provider 而异 |
| `streaming.enabled` | bool | 否 | 是否允许此模型条目尝试 provider 流式请求,默认 `false`。它只表达模型/端点能力 opt-in,实际还需要当前 channel 的 `settings.streaming.enabled` 同时开启 |
| `max_tokens_field` | string | 否 | 覆盖请求体中 max tokens 的字段名(如 o1 模型使用 `max_completion_tokens` |
| `thinking_level` | string | 否 | 扩展思考级别:`off``low``medium``high``xhigh``adaptive` |
| `extra_body` | object | 否 | 注入到每个请求体中的额外字段 |
@@ -131,6 +132,8 @@
| `fallbacks` | string[] | 否 | 自动故障转移的备用模型名称 |
| `enabled` | bool | 否 | 是否启用此模型条目(默认:`true` |
不需要流式时请省略 `streaming` 配置块。写 `"streaming": {"enabled": false}` 是可选的,手写或生成配置时都不需要。
#### `provider` / `model` 解析规则
PicoClaw 按下面的规则解析 `provider` 和最终发给上游的模型 ID
+2 -1
View File
@@ -354,6 +354,7 @@ Cela crée `~/.picoclaw/config.json` et le répertoire workspace.
```json
{
"version": 3,
"agents": {
"defaults": {
"model_name": "gpt-5.4"
@@ -363,7 +364,7 @@ Cela crée `~/.picoclaw/config.json` et le répertoire workspace.
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-api-key"
"api_keys": ["sk-your-api-key"]
}
]
}
+2 -1
View File
@@ -350,6 +350,7 @@ Ini membuat `~/.picoclaw/config.json` dan direktori workspace.
```json
{
"version": 3,
"agents": {
"defaults": {
"model_name": "gpt-5.4"
@@ -359,7 +360,7 @@ Ini membuat `~/.picoclaw/config.json` dan direktori workspace.
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-api-key"
"api_keys": ["sk-your-api-key"]
}
]
}
+2 -1
View File
@@ -350,6 +350,7 @@ Questo crea `~/.picoclaw/config.json` e la directory workspace.
```json
{
"version": 3,
"agents": {
"defaults": {
"model_name": "gpt-5.4"
@@ -359,7 +360,7 @@ Questo crea `~/.picoclaw/config.json` e la directory workspace.
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-api-key"
"api_keys": ["sk-your-api-key"]
}
]
}
+2 -1
View File
@@ -351,6 +351,7 @@ picoclaw onboard
```json
{
"version": 3,
"agents": {
"defaults": {
"model_name": "gpt-5.4"
@@ -360,7 +361,7 @@ picoclaw onboard
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-api-key"
"api_keys": ["sk-your-api-key"]
}
]
}
+2 -1
View File
@@ -351,6 +351,7 @@ Isso cria `~/.picoclaw/config.json` e o diretório workspace.
```json
{
"version": 3,
"agents": {
"defaults": {
"model_name": "gpt-5.4"
@@ -360,7 +361,7 @@ Isso cria `~/.picoclaw/config.json` e o diretório workspace.
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-api-key"
"api_keys": ["sk-your-api-key"]
}
]
}
+2 -1
View File
@@ -351,6 +351,7 @@ Lệnh này tạo `~/.picoclaw/config.json` và thư mục workspace.
```json
{
"version": 3,
"agents": {
"defaults": {
"model_name": "gpt-5.4"
@@ -360,7 +361,7 @@ Lệnh này tạo `~/.picoclaw/config.json` và thư mục workspace.
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-api-key"
"api_keys": ["sk-your-api-key"]
}
]
}
+2 -1
View File
@@ -351,6 +351,7 @@ picoclaw onboard
```json
{
"version": 3,
"agents": {
"defaults": {
"model_name": "gpt-5.4"
@@ -360,7 +361,7 @@ picoclaw onboard
{
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_key": "sk-your-api-key"
"api_keys": ["sk-your-api-key"]
}
]
}
+9 -8
View File
@@ -45,7 +45,8 @@ When you load a config file:
The `version` field in `config.json` indicates the schema version:
- `0` or missing: Legacy config (no version field)
- `1`: Previous version (will be auto-migrated to V2 on load)
- `2`: Current version
- `2`: Previous version (will be auto-migrated to V3 on load)
- `3`: Current version
```json
{
@@ -64,8 +65,8 @@ When making breaking changes to the config schema:
Create a new struct for the new version if the structure changes significantly:
```go
// ConfigV2 represents version 2 config structure
type ConfigV2 struct {
// ConfigV3 represents version 3 config structure
type ConfigV3 struct {
Version int `json:"version"`
Agents AgentsConfig `json:"agents"`
// ... other fields with new structure
@@ -75,7 +76,7 @@ type ConfigV2 struct {
### Step 2: Update Current Config Version
```go
const CurrentVersion = 2 // Increment this
const CurrentVersion = 3 // Increment this
```
### Step 3: Add a Loader Function
@@ -141,9 +142,9 @@ Create a test in `config_migration_test.go`:
```go
func TestMigrateV2ToV3(t *testing.T) {
// Create a version 2 config
v2Config := Config{
Version: 2,
// Create a version 3 config
v3Config := Config{
Version: 3,
// ... set up test data
}
@@ -224,7 +225,7 @@ Backups are created in the same directory as your config file:
### Scenario: Adding a new field with default value
Old config (version 2):
Old config (version 3):
```json
{
"version": 3,
+4 -2
View File
@@ -117,7 +117,7 @@ Supported flags:
| `--env`, `-e` | Add a stdio environment variable in `KEY=value` format. Repeatable. Values are saved to config. |
| `--env-file` | Attach an env file path to a stdio server. Recommended for secrets you do not want stored inline in `config.json`. |
| `--header`, `-H` | Add an HTTP header in `Name: Value` or `Name=Value` format. Repeatable. |
| `--transport`, `-t` | Transport type: `stdio` (default), `http`, or `sse`. |
| `--transport`, `-t` | Transport type: `stdio` (default), `http` / `streamable-http`, or `sse`. |
| `--force`, `-f` | Overwrite an existing server entry without confirmation. |
| `--deferred` | Mark the server as deferred: tools are hidden and discoverable on demand. |
| `--no-deferred` | Mark the server as non-deferred: tools are always loaded into context. |
@@ -198,13 +198,15 @@ For `stdio`:
- `--header` is rejected
- `-- <command> [args...]` is supported and recommended for unambiguous parsing
For `http` / `sse`:
For `http` / `streamable-http` / `sse`:
- `<command-or-url>` must be a valid URL
- extra command args are rejected
- `--env` is rejected
- `--env-file` is rejected
- `--header` is supported and stored in `headers`
- `http` and `streamable-http` use streamable HTTP request-response mode
- `sse` uses the same streamable HTTP transport, but also enables the optional standalone SSE listener for server-initiated notifications
Overwrite behavior:
+26
View File
@@ -66,6 +66,32 @@ General settings for fetching and processing webpage content.
| `enabled` | bool | true | Enable DuckDuckGo search |
| `max_results` | int | 5 | Maximum number of results |
### Gemini Google Search
Gemini search uses Gemini with Google Search grounding. It returns an AI-synthesized answer with citations from Google Search.
| Config | Type | Default | Description |
|---------------|--------|----------------------|-----------------------------------|
| `enabled` | bool | false | Enable Gemini Google Search |
| `api_key` | string | - | Google Gemini API key |
| `model` | string | `gemini-2.5-flash` | Gemini model used for search |
| `max_results` | int | 5 | Maximum number of citations |
```json
{
"tools": {
"web": {
"gemini": {
"enabled": true,
"api_key": "YOUR_GEMINI_API_KEY",
"model": "gemini-2.5-flash",
"max_results": 5
}
}
}
}
```
### Baidu Search
Baidu Search uses the [Qianfan AI Search API](https://cloud.baidu.com/doc/qianfan-api/s/Wmbq4z7e5), which is AI-powered and optimized for Chinese-language queries.
+2 -2
View File
@@ -691,7 +691,7 @@ case "your-provider":
{
"model_name": "your-model",
"model": "your-provider/model-name",
"api_key": "your-api-key",
"api_keys": ["your-api-key"],
"api_base": "https://api.your-provider.com/v1"
}
]
@@ -725,7 +725,7 @@ picoclaw agent -m "Hello" --model your-model
export PICOCLAW_AGENTS_DEFAULTS_MODEL=your-model
# Remplacer les paramètres du fournisseur
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_key":"..."}]'
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_keys":["..."]}]'
```
---
+2 -2
View File
@@ -691,7 +691,7 @@ case "your-provider":
{
"model_name": "your-model",
"model": "your-provider/model-name",
"api_key": "your-api-key",
"api_keys": ["your-api-key"],
"api_base": "https://api.your-provider.com/v1"
}
]
@@ -725,7 +725,7 @@ picoclaw agent -m "Hello" --model your-model
export PICOCLAW_AGENTS_DEFAULTS_MODEL=your-model
# プロバイダー設定の上書き
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_key":"..."}]'
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_keys":["..."]}]'
```
---
+2 -2
View File
@@ -689,7 +689,7 @@ case "your-provider":
{
"model_name": "your-model",
"model": "your-provider/model-name",
"api_key": "your-api-key",
"api_keys": ["your-api-key"],
"api_base": "https://api.your-provider.com/v1"
}
]
@@ -723,7 +723,7 @@ picoclaw agent -m "Hello" --model your-model
export PICOCLAW_AGENTS_DEFAULTS_MODEL=your-model
# Override provider settings
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_key":"..."}]'
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_keys":["..."]}]'
```
---
+2 -2
View File
@@ -691,7 +691,7 @@ case "your-provider":
{
"model_name": "your-model",
"model": "your-provider/model-name",
"api_key": "your-api-key",
"api_keys": ["your-api-key"],
"api_base": "https://api.your-provider.com/v1"
}
]
@@ -725,7 +725,7 @@ picoclaw agent -m "Hello" --model your-model
export PICOCLAW_AGENTS_DEFAULTS_MODEL=your-model
# Substituir configurações do provedor
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_key":"..."}]'
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_keys":["..."]}]'
```
---
+2 -2
View File
@@ -691,7 +691,7 @@ case "your-provider":
{
"model_name": "your-model",
"model": "your-provider/model-name",
"api_key": "your-api-key",
"api_keys": ["your-api-key"],
"api_base": "https://api.your-provider.com/v1"
}
]
@@ -725,7 +725,7 @@ picoclaw agent -m "Hello" --model your-model
export PICOCLAW_AGENTS_DEFAULTS_MODEL=your-model
# Ghi đè cài đặt nhà cung cấp
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_key":"..."}]'
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_keys":["..."]}]'
```
---
+2 -2
View File
@@ -691,7 +691,7 @@ case "your-provider":
{
"model_name": "your-model",
"model": "your-provider/model-name",
"api_key": "your-api-key",
"api_keys": ["your-api-key"],
"api_base": "https://api.your-provider.com/v1"
}
]
@@ -725,7 +725,7 @@ picoclaw agent -m "Hello" --model your-model
export PICOCLAW_AGENTS_DEFAULTS_MODEL=your-model
# 覆盖提供商设置
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_key":"..."}]'
export PICOCLAW_MODEL_LIST='[{"model_name":"your-model","model":"your-provider/model-name","api_keys":["..."]}]'
```
---
+1 -1
View File
@@ -33,7 +33,7 @@ enc://AAAA...base64...
{
"model_name": "gpt-4o",
"model": "openai/gpt-4o",
"api_key": "enc://AAAA...base64...",
"api_keys": ["enc://AAAA...base64..."],
"api_base": "https://api.openai.com/v1"
}
]
+1 -1
View File
@@ -32,7 +32,7 @@ enc://AAAA...base64...
{
"model_name": "gpt-4o",
"model": "openai/gpt-4o",
"api_key": "enc://AAAA...base64...",
"api_keys": ["enc://AAAA...base64..."],
"api_base": "https://api.openai.com/v1"
}
]
+1 -1
View File
@@ -31,7 +31,7 @@ enc://AAAA...base64...
{
"model_name": "gpt-4o",
"model": "openai/gpt-4o",
// "api_key": "enc://AAAA...base64..." move to .security.yml
// "api_keys": ["enc://AAAA...base64..."] move to .security.yml
"api_base": "https://api.openai.com/v1"
}
]
+1 -1
View File
@@ -33,7 +33,7 @@ enc://AAAA...base64...
{
"model_name": "gpt-4o",
"model": "openai/gpt-4o",
"api_key": "enc://AAAA...base64...",
"api_keys": ["enc://AAAA...base64..."],
"api_base": "https://api.openai.com/v1"
}
]
+1 -1
View File
@@ -33,7 +33,7 @@ enc://AAAA...base64...
{
"model_name": "gpt-4o",
"model": "openai/gpt-4o",
"api_key": "enc://AAAA...base64...",
"api_keys": ["enc://AAAA...base64..."],
"api_base": "https://api.openai.com/v1"
}
]
+1 -1
View File
@@ -32,7 +32,7 @@ enc://AAAA...base64...
{
"model_name": "gpt-4o",
"model": "openai/gpt-4o",
"api_key": "enc://AAAA...base64...",
"api_keys": ["enc://AAAA...base64..."],
"api_base": "https://api.openai.com/v1"
}
]
+1 -1
View File
@@ -152,7 +152,7 @@ You can now remove sensitive fields from `config.json` since they're loaded from
"model_name": "gpt-5.4",
"model": "openai/gpt-5.4",
"api_base": "https://api.openai.com/v1",
"api_key": "sk-your-actual-api-key-here"
"api_keys": ["sk-your-actual-api-key-here"]
}
],
"channel_list": {
+1 -1
View File
@@ -33,7 +33,7 @@ go run ./examples/pico-echo-server -addr :9090 -token secret
2. Configure `pico_client` in your `config.json`:
```json
{
"channels": {
"channel_list": {
"pico_client": {
"enabled": true,
"url": "ws://localhost:9090/ws",
+18 -18
View File
@@ -5,7 +5,7 @@ go 1.25.10
require (
fyne.io/systray v1.12.1
github.com/SevereCloud/vksdk/v3 v3.3.1
github.com/adhocore/gronx v1.19.6
github.com/adhocore/gronx v1.19.7
github.com/anthropics/anthropic-sdk-go v1.26.0
github.com/atc0005/go-teams-notify/v2 v2.14.0
github.com/aws/aws-sdk-go-v2 v1.41.7
@@ -15,25 +15,27 @@ require (
github.com/caarlos0/env/v11 v11.4.0
github.com/charmbracelet/lipgloss v1.1.0
github.com/creack/pty v1.1.24
github.com/eclipse/paho.mqtt.golang v1.5.1
github.com/ergochat/irc-go v0.6.0
github.com/ergochat/readline v0.1.3
github.com/gomarkdown/markdown v0.0.0-20260411013819-759bbc3e3207
github.com/google/uuid v1.6.0
github.com/gorilla/websocket v1.5.3
github.com/h2non/filetype v1.1.3
github.com/larksuite/oapi-sdk-go/v3 v3.6.1
github.com/larksuite/oapi-sdk-go/v3 v3.7.5
github.com/line/line-bot-sdk-go/v8 v8.19.0
github.com/mdp/qrterminal/v3 v3.2.1
github.com/minio/selfupdate v0.6.0
github.com/modelcontextprotocol/go-sdk v1.5.0
github.com/muesli/termenv v0.16.0
github.com/mymmrac/telego v1.8.0
github.com/mymmrac/telego v1.9.0
github.com/open-dingtalk/dingtalk-stream-sdk-go v0.9.1
github.com/openai/openai-go/v3 v3.22.0
github.com/pion/rtp v1.10.1
github.com/pion/webrtc/v3 v3.3.6
github.com/pmezard/go-difflib v1.0.0
github.com/rs/zerolog v1.35.1
github.com/slack-go/slack v0.17.3
github.com/slack-go/slack v0.23.1
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/stretchr/testify v1.11.1
@@ -41,12 +43,12 @@ require (
go.mau.fi/util v0.9.8
go.mau.fi/whatsmeow v0.0.0-20260219150138-7ae702b1eed4
golang.org/x/oauth2 v0.36.0
golang.org/x/term v0.42.0
golang.org/x/term v0.43.0
golang.org/x/time v0.15.0
google.golang.org/protobuf v1.36.11
gopkg.in/yaml.v3 v3.0.1
maunium.net/go/mautrix v0.27.0
modernc.org/sqlite v1.48.2
modernc.org/sqlite v1.50.1
rsc.io/qr v0.2.0
)
@@ -76,7 +78,6 @@ require (
github.com/coder/websocket v1.8.14 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/eclipse/paho.mqtt.golang v1.5.1 // indirect
github.com/elliotchance/orderedmap/v3 v3.1.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
@@ -90,7 +91,6 @@ require (
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/petermattis/goid v0.0.0-20260330135022-df67b199bc81 // indirect
github.com/pion/randutil v0.1.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/segmentio/asm v1.1.3 // indirect
@@ -105,24 +105,24 @@ require (
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
golang.org/x/text v0.36.0 // indirect
modernc.org/libc v1.70.0 // indirect
golang.org/x/text v0.37.0 // indirect
modernc.org/libc v1.72.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)
require (
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/andybalholm/brotli v1.2.1 // indirect
github.com/bytedance/gopkg v0.1.3 // indirect
github.com/bytedance/sonic v1.15.0 // indirect
github.com/bytedance/sonic/loader v0.5.0 // indirect
github.com/bytedance/sonic v1.15.1 // indirect
github.com/bytedance/sonic/loader v0.5.1 // indirect
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/github/copilot-sdk/go v0.2.0
github.com/go-resty/resty/v2 v2.17.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/google/jsonschema-go v0.4.3
github.com/grbit/go-json v0.11.0 // indirect
github.com/klauspost/compress v1.18.4 // indirect
github.com/klauspost/compress v1.18.6 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.2.0 // indirect
@@ -130,14 +130,14 @@ require (
github.com/tidwall/sjson v1.2.5 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasthttp v1.69.0 // indirect
github.com/valyala/fasthttp v1.71.0 // indirect
github.com/valyala/fastjson v1.6.10 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
golang.org/x/arch v0.24.0 // indirect
golang.org/x/crypto v0.50.0
golang.org/x/net v0.53.0
golang.org/x/crypto v0.51.0
golang.org/x/net v0.54.0
golang.org/x/sync v0.20.0
golang.org/x/sys v0.43.0
golang.org/x/sys v0.44.0
)
replace github.com/bwmarrin/discordgo => github.com/yeongaori/discordgo-fork v0.0.0-20260319072544-e8e546f5d532
+38 -38
View File
@@ -9,14 +9,14 @@ github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7Oputl
github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU=
github.com/SevereCloud/vksdk/v3 v3.3.1 h1:O86zsp5LQnHE+O5acvuXM/s6S1LyxzVTkF6+Lup0Jyg=
github.com/SevereCloud/vksdk/v3 v3.3.1/go.mod h1:c6WaA5aocUYsXfkcUbg2qy45V9M1VDcqHHmHIN14NAw=
github.com/adhocore/gronx v1.19.6 h1:5KNVcoR9ACgL9HhEqCm5QXsab/gI4QDIybTAWcXDKDc=
github.com/adhocore/gronx v1.19.6/go.mod h1:7oUY1WAU8rEJWmAxXR2DN0JaO4gi9khSgKjiRypqteg=
github.com/adhocore/gronx v1.19.7 h1:7hhFwChgDw9eHC3+TQ+OKKBqJnP44oWkDCnnW9nrsuA=
github.com/adhocore/gronx v1.19.7/go.mod h1:7oUY1WAU8rEJWmAxXR2DN0JaO4gi9khSgKjiRypqteg=
github.com/agnivade/levenshtein v1.2.1 h1:EHBY3UOn1gwdy/VbFwgo4cxecRznFk7fKWN1KOX7eoM=
github.com/agnivade/levenshtein v1.2.1/go.mod h1:QVVI16kDrtSuwcpd0p1+xMC6Z/VfhtCyDIjcwga4/DU=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883 h1:bvNMNQO63//z+xNgfBlViaCIJKLlCJ6/fmUseuG0wVQ=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/anthropics/anthropic-sdk-go v1.26.0 h1:oUTzFaUpAevfuELAP1sjL6CQJ9HHAfT7CoSYSac11PY=
github.com/anthropics/anthropic-sdk-go v1.26.0/go.mod h1:qUKmaW+uuPB64iy1l+4kOSvaLqPXnHTTBKH6RVZ7q5Q=
github.com/atc0005/go-teams-notify/v2 v2.14.0 h1:7N+xw+COnYANLREaAveQ65rsNQ12nIZJED9nMLyscCo=
@@ -59,10 +59,10 @@ github.com/beeper/argo-go v1.1.2 h1:UQI2G8F+NLfGTOmTUI0254pGKx/HUU/etbUGTJv91Fs=
github.com/beeper/argo-go v1.1.2/go.mod h1:M+LJAnyowKVQ6Rdj6XYGEn+qcVFkb3R/MUpqkGR0hM4=
github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M=
github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE=
github.com/bytedance/sonic v1.15.0/go.mod h1:tFkWrPz0/CUCLEF4ri4UkHekCIcdnkqXw9VduqpJh0k=
github.com/bytedance/sonic/loader v0.5.0 h1:gXH3KVnatgY7loH5/TkeVyXPfESoqSBSBEiDd5VjlgE=
github.com/bytedance/sonic/loader v0.5.0/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo=
github.com/bytedance/sonic v1.15.1 h1:nJD5PmM0vY7J8CT6MxoqbVAAMhkSmV2HgRAUrrpLoOw=
github.com/bytedance/sonic v1.15.1/go.mod h1:mT2NbXunuaEbnZ+mRIX/vYqKISmgEuHFDI4UzmKx2SA=
github.com/bytedance/sonic/loader v0.5.1 h1:Ygpfa9zwRCCKSlrp5bBP/b/Xzc3VxsAW+5NIYXrOOpI=
github.com/bytedance/sonic/loader v0.5.1/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo=
github.com/caarlos0/env/v11 v11.4.0 h1:Kcb6t5kIIr4XkoQC9AF2j+8E1Jsrl3Wz/hhm1LtoGAc=
github.com/caarlos0/env/v11 v11.4.0/go.mod h1:qupehSf/Y0TUTsxKywqRt/vJjN5nz6vauiYEUUr8P4U=
github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
@@ -166,8 +166,8 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c=
github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
@@ -179,8 +179,8 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/larksuite/oapi-sdk-go/v3 v3.6.1 h1:vAdu+sX9yXNkKnKnYQeIv6yBkjP37Q1JEJHmMa2eCjQ=
github.com/larksuite/oapi-sdk-go/v3 v3.6.1/go.mod h1:ZEplY+kwuIrj/nqw5uSCINNATcH3KdxSN7y+UxYY5fI=
github.com/larksuite/oapi-sdk-go/v3 v3.7.5 h1:dimv+ZAGia01f4xCDGvCiBHKWMf4K1AB7fGsM+lv5Jw=
github.com/larksuite/oapi-sdk-go/v3 v3.7.5/go.mod h1:ZEplY+kwuIrj/nqw5uSCINNATcH3KdxSN7y+UxYY5fI=
github.com/line/line-bot-sdk-go/v8 v8.19.0 h1:5FD/1SprRZ8Y0FiUI6syYiBewOs0ak2tuUBMYN0wzE4=
github.com/line/line-bot-sdk-go/v8 v8.19.0/go.mod h1:AeSRUuu7WGgveGDJb6DyKyFUOst2UB2aF6LO2cQeuXs=
github.com/lucasb-eyer/go-colorful v1.3.0 h1:2/yBRLdWBZKrf7gB40FoiKfAWYQ0lqNcbuQwVHXptag=
@@ -201,8 +201,8 @@ github.com/modelcontextprotocol/go-sdk v1.5.0 h1:CHU0FIX9kpueNkxuYtfYQn1Z0slhFzB
github.com/modelcontextprotocol/go-sdk v1.5.0/go.mod h1:gggDIhoemhWs3BGkGwd1umzEXCEMMvAnhTrnbXJKKKA=
github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc=
github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
github.com/mymmrac/telego v1.8.0 h1:EvIprWo9Cn0MHgumvvqNXPAXO1yJj3pu2cdCCeDxbow=
github.com/mymmrac/telego v1.8.0/go.mod h1:pdLV346EgVuq7Xrh3kMggeBiazeHhsdEoK0RTEOPXRM=
github.com/mymmrac/telego v1.9.0 h1:ZUJxZaPx/1IgRvVb5lXnUB8FgW5rNYfRe6Q2EJ4OJ+Y=
github.com/mymmrac/telego v1.9.0/go.mod h1:tVEB7OqiOPx8elRk9+ETkwiDQrUhWSB2XmAKIY9KmWY=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
@@ -246,8 +246,8 @@ github.com/segmentio/encoding v0.5.4 h1:OW1VRern8Nw6ITAtwSZ7Idrl3MXCFwXHPgqESYfv
github.com/segmentio/encoding v0.5.4/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0=
github.com/sergi/go-diff v1.3.1 h1:xkr+Oxo4BOQKmkn/B9eMK0g5Kg/983T9DqqPHwYqD+8=
github.com/sergi/go-diff v1.3.1/go.mod h1:aMJSSKb2lpPvRNec0+w3fl7LP9IOFzdc9Pa4NFbPK1I=
github.com/slack-go/slack v0.17.3 h1:zV5qO3Q+WJAQ/XwbGfNFrRMaJ5T/naqaonyPV/1TP4g=
github.com/slack-go/slack v0.17.3/go.mod h1:X+UqOufi3LYQHDnMG1vxf0J8asC6+WllXrVrhl8/Prk=
github.com/slack-go/slack v0.23.1 h1:ZS5B96wxxYQRwvJ3/vJFtqtUZi3tXhsZCyT44Nv7M80=
github.com/slack-go/slack v0.23.1/go.mod h1:H0yR/YBuRJ39RkE+JpV/d/oEsbanzTRowR82bCN0cEs=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
@@ -283,8 +283,8 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasthttp v1.69.0 h1:fNLLESD2SooWeh2cidsuFtOcrEi4uB4m1mPrkJMZyVI=
github.com/valyala/fasthttp v1.69.0/go.mod h1:4wA4PfAraPlAsJ5jMSqCE2ug5tqUPwKXxVj8oNECGcw=
github.com/valyala/fasthttp v1.71.0 h1:tepR7H+Guh9VUqxxcPggYi8R3lGUu2Rsdh+z7/FCY3k=
github.com/valyala/fasthttp v1.71.0/go.mod h1:z1sDUvOShhXq/C9mwH/fSm1Vb71tUJwmQdgkBrBNwnA=
github.com/valyala/fastjson v1.6.10 h1:/yjJg8jaVQdYR3arGxPE2X5z89xrlhS0eGXdv+ADTh4=
github.com/valyala/fastjson v1.6.10/go.mod h1:e6FubmQouUNP73jtMLmcbxS6ydWIpOfhz34TSfO3JaE=
github.com/vektah/gqlparser/v2 v2.5.27 h1:RHPD3JOplpk5mP5JGX8RKZkt2/Vwj/PZv0HxTdwFp0s=
@@ -330,8 +330,8 @@ golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWP
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.16.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
@@ -354,8 +354,8 @@ golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/net v0.54.0 h1:2zJIZAxAHV/OHCDTCOHAYehQzLfSXuf/5SoL/Dv6w/w=
golang.org/x/net v0.54.0/go.mod h1:Sj4oj8jK6XmHpBZU/zWHw3BV3abl4Kvi+Ut7cQcY+cQ=
golang.org/x/oauth2 v0.23.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
@@ -388,16 +388,16 @@ golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ=
golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0=
golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
@@ -405,8 +405,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -449,10 +449,10 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
maunium.net/go/mautrix v0.27.0 h1:yfEYwoIluVWkofUgbZl9gP4i5nQTF+QNsxtb+r5bKlM=
maunium.net/go/mautrix v0.27.0/go.mod h1:7QpEQiTy6p4LHkXXaZI+N46tGYy8HMhD0JjzZAFoFWs=
modernc.org/cc/v4 v4.27.1 h1:9W30zRlYrefrDV2JE2O8VDtJ1yPGownxciz5rrbQZis=
modernc.org/cc/v4 v4.27.1/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
modernc.org/ccgo/v4 v4.32.0 h1:hjG66bI/kqIPX1b2yT6fr/jt+QedtP2fqojG2VrFuVw=
modernc.org/ccgo/v4 v4.32.0/go.mod h1:6F08EBCx5uQc38kMGl+0Nm0oWczoo1c7cgpzEry7Uc0=
modernc.org/cc/v4 v4.28.2 h1:3tQ0lf2ADtoby2EtSP+J7IE2SHwEJdP8ioR59wx7XpY=
modernc.org/cc/v4 v4.28.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.34.0 h1:yRLPFZieg532OT4rp4JFNIVcquwalMX26G95WQDqwCQ=
modernc.org/ccgo/v4 v4.34.0/go.mod h1:AS5WYMyBakQ+fhsHhtP8mWB82KTGPkNNJDGfGQCe0/A=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
@@ -461,18 +461,18 @@ modernc.org/gc/v3 v3.1.2 h1:ZtDCnhonXSZexk/AYsegNRV1lJGgaNZJuKjJSWKyEqo=
modernc.org/gc/v3 v3.1.2/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.70.0 h1:U58NawXqXbgpZ/dcdS9kMshu08aiA6b7gusEusqzNkw=
modernc.org/libc v1.70.0/go.mod h1:OVmxFGP1CI/Z4L3E0Q3Mf1PDE0BucwMkcXjjLntvHJo=
modernc.org/libc v1.72.3 h1:ZnDF4tXn4NBXFutMMQC4vtbTFSXhhKzR73fv0beZEAU=
modernc.org/libc v1.72.3/go.mod h1:dn0dZNnnn1clLyvRxLxYExxiKRZIRENOfqQ8XEeg4Qs=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.48.2 h1:5CnW4uP8joZtA0LedVqLbZV5GD7F/0x91AXeSyjoh5c=
modernc.org/sqlite v1.48.2/go.mod h1:hWjRO6Tj/5Ik8ieqxQybiEOUXy0NJFNp2tpvVpKlvig=
modernc.org/sqlite v1.50.1 h1:l+cQvn0sd0zJJtfygGHuQJ5AjlrwXmWPw4KP3ZMwr9w=
modernc.org/sqlite v1.50.1/go.mod h1:tcNzv5p84E0skkmJn038y+hWJbLQXQqEnQfeh5r2JLM=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
+255
View File
@@ -0,0 +1,255 @@
# Integration Test Suites
This directory contains the integration test suites that CI runs before merging PRs and before building `main`.
These tests exist to catch regressions that are easy to miss with unit tests alone, especially when different PRs touch adjacent code paths and the breakage only appears once the pieces are wired together. Typical examples are:
- protocol compatibility across package boundaries
- request and response behavior over real transports
- subprocess, CLI, or container wiring
- configuration passed through environment variables
- startup, discovery, and teardown flows involving more than one component
## Two Layers of Integration Testing
PicoClaw currently uses two related mechanisms:
1. Go integration tests, usually in `*_integration_test.go` files and guarded by `//go:build integration`
2. Docker-backed suites in `integration/suites/` that start real dependencies and run one or more of those Go tests in CI
That distinction matters:
- a tagged Go integration test is the test implementation
- a Docker-backed suite is how we make that test reproducible and CI-safe
If a test should actively protect merges between PRs, it should be reachable from [`scripts/run-integration-tests.sh`](../scripts/run-integration-tests.sh), either by extending an existing suite or by adding a new one.
Some integration-tagged tests are intentionally opt-in and not part of the Docker suites. For example, real CLI smoke tests under `pkg/providers/cli/` depend on external binaries being installed locally. Those are useful for manual verification, but they do not gate PR merges.
## What CI Runs
The integration jobs in [`.github/workflows/pr.yml`](../.github/workflows/pr.yml) and [`.github/workflows/build.yml`](../.github/workflows/build.yml) both execute:
```bash
bash ./scripts/run-integration-tests.sh
```
The runner auto-discovers every suite under `integration/suites/`, so adding a suite does not require editing the GitHub Actions workflow.
## How the Runner Works
- The shared runner is defined in [`integration/docker-compose.runner.yml`](docker-compose.runner.yml).
- Each suite lives in `integration/suites/<suite-name>/`.
- The runner script loads `suite.env`, merges the shared compose file with the suite-specific compose files, starts dependency services, runs the suite command, and then tears everything down.
- The shared runner container sets `GOFLAGS=-tags=goolm,stdjson,integration`, so tests run with the same build tags used by CI.
In practice, each suite gives us:
- a deterministic dependency graph
- a stable execution environment
- automatic cleanup after the run
- a clean place to encode the exact regression we want to prevent
## Current Reference Suite
[`integration/suites/mcp-streamable/`](suites/mcp-streamable/) is the reference example today.
It does three things:
- builds and starts a fixture MCP server from [`integration/fixtures/mcp-streamable-server/`](fixtures/mcp-streamable-server/)
- injects connection details into the runner container through environment variables
- runs [`TestIntegration_RealConfiguredServer`](../pkg/mcp/manager_real_server_integration_test.go) to verify that PicoClaw can connect to a real server, discover tools, invoke one, and validate the response payload
That suite complements [`TestIntegration_StreamableHTTPCompatibility`](../pkg/mcp/manager_integration_test.go), which exercises the same area in-process. Together they cover both protocol behavior and real service wiring.
## Suite Layout
Each suite directory must contain:
- `suite.env`
- at least one `docker-compose.yml` or `docker-compose.*.yml`
Example:
```text
integration/suites/my-suite/
├── docker-compose.yml
└── suite.env
```
## Required Manifest Fields
`suite.env` is sourced by the runner script and must define:
- `TEST_COMMAND`: shell command executed inside the integration runner container
Optional fields:
- `RUNNER_SERVICE`: override the default runner service name (`integration-runner`)
Example:
```bash
TEST_COMMAND='go test ./pkg/mcp -run TestIntegration_RealConfiguredServer -v'
```
## Running Integration Tests Locally
### Prerequisites
- Docker with the `docker compose` plugin for Docker-backed suites
- Go 1.25+ only if you want to run tagged integration tests directly on your host instead of through Docker
### Run Everything That CI Runs
```bash
make integration-test
```
Equivalent direct command:
```bash
bash ./scripts/run-integration-tests.sh
```
### Run a Single Suite
```bash
bash ./scripts/run-integration-tests.sh mcp-streamable
```
This is the fastest way to reproduce exactly what the CI integration job does for one suite.
### Run a Tagged Integration Test Directly
For faster iteration while writing the test, you can run the Go test itself without Docker:
```bash
go test -tags=goolm,stdjson,integration ./pkg/mcp -run TestIntegration_StreamableHTTPCompatibility -v
```
You can also run the real-server smoke test directly if you provide the same environment variables that the Docker suite would inject:
```bash
PICOCLAW_MCP_REAL_SERVER_JSON='{"enabled":true,"type":"http","url":"http://127.0.0.1:8080/mcp"}' \
PICOCLAW_MCP_REAL_TOOL_NAME=echo \
PICOCLAW_MCP_REAL_TOOL_ARGS_JSON='{"message":"hello"}' \
PICOCLAW_MCP_REAL_EXPECT_SUBSTRING=hello \
go test -tags=goolm,stdjson,integration ./pkg/mcp -run TestIntegration_RealConfiguredServer -v
```
Notes:
- avoid `-short`: the current integration tests skip in short mode
- use direct `go test` for tight feedback loops, then validate the Docker suite before committing
## When to Add an Integration Test
Reach for an integration test when the risk lives in the interaction, not just in the function body. Good candidates include:
- code that crosses process or container boundaries
- transport-specific behavior such as HTTP, SSE, stdio, or streamable MCP flows
- CLI parsing and wiring that depends on real subprocess execution
- configuration propagation through files, env vars, headers, or service discovery
- regressions that usually appear only after merging separately reasonable PRs
Prefer a unit test when the behavior is pure, local, and fully controllable in-process.
## How to Add a New Integration Test
### 1. Start from the regression you want to prevent
Describe the real workflow that could break after a merge. The sharper the scenario, the better the test will age.
Examples:
- "PicoClaw can still connect to a streamable MCP server after transport normalization changes."
- "A provider wrapper still parses the real CLI output format after refactors in response handling."
### 2. Implement the Go test
Add or extend a `*_integration_test.go` file in the package that owns the behavior and gate it with:
```go
//go:build integration
```
Guidelines:
- keep the assertions focused on observable behavior
- use bounded timeouts
- prefer deterministic fixtures over internet access or shared external state
- skip only when the dependency is intentionally optional, such as a locally installed third-party CLI
### 3. Decide whether it must gate CI merges
Use this rule of thumb:
- if the test is only a manual smoke check, a tagged Go test may be enough
- if the test should prevent regressions from landing through PR merges, wire it into a Docker-backed suite
### 4. Reuse or add a suite
If an existing suite already exercises the same subsystem, extend it. Otherwise create:
```text
integration/suites/<name>/
├── docker-compose.yml
└── suite.env
```
Use `integration/fixtures/` for reusable helper services or fake servers.
### 5. Define the suite command
In `suite.env`, point `TEST_COMMAND` at the Go test you want CI to run.
Examples:
```bash
TEST_COMMAND='go test ./pkg/mcp -run TestIntegration_RealConfiguredServer -v'
```
```bash
TEST_COMMAND='go test ./pkg/somepkg -run TestIntegration_MyScenario -v'
```
You can also run multiple tests if they share the same environment, but keep suites cohesive and easy to diagnose when they fail.
### 6. Model the dependencies in Docker Compose
Suite compose files can:
- define dependency services needed by the tests
- extend or override the shared `integration-runner` service
- inject environment variables into the runner for the tests to consume
Practical advice:
- prefer Docker service names over hard-coded host ports
- add health checks when the runner depends on service readiness
- keep the suite self-contained and deterministic
### 7. Validate locally before committing
At minimum:
```bash
go test -tags=goolm,stdjson,integration ./path/to/package -run TestIntegration_Name -v
bash ./scripts/run-integration-tests.sh <suite-name>
```
The first command helps while authoring. The second proves that the CI path works end to end.
## Review Checklist for New Suites
Before opening the PR, check that the new suite:
- reproduces a realistic cross-component failure mode
- is deterministic and isolated
- does not require manual setup in CI
- has clear failure output
- finishes in a reasonable amount of time
- cleans up after itself through the normal runner teardown
Once committed, the suite will be auto-discovered by the CI integration job.
+19
View File
@@ -0,0 +1,19 @@
services:
integration-runner:
image: golang:1.25-bookworm
working_dir: /workspace
entrypoint: ["bash", "-c"]
volumes:
- ${INTEGRATION_REPO_ROOT}:/workspace
- picoclaw-integration-gocache:/go-build-cache
- picoclaw-integration-gomodcache:/go-mod-cache
environment:
GOCACHE: /go-build-cache
GOMODCACHE: /go-mod-cache
GOTOOLCHAIN: local
CGO_ENABLED: "0"
GOFLAGS: -tags=goolm,stdjson,integration
volumes:
picoclaw-integration-gocache:
picoclaw-integration-gomodcache:
@@ -0,0 +1,23 @@
FROM golang:1.25-bookworm AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /out/mcp-streamable-server ./integration/fixtures/mcp-streamable-server
FROM alpine:3.22
RUN adduser -D -u 10001 appuser
COPY --from=build /out/mcp-streamable-server /usr/local/bin/mcp-streamable-server
USER appuser
EXPOSE 8080
HEALTHCHECK --interval=5s --timeout=3s --retries=12 CMD wget -qO- http://127.0.0.1:8080/healthz || exit 1
ENTRYPOINT ["/usr/local/bin/mcp-streamable-server"]
@@ -0,0 +1,71 @@
package main
import (
"context"
"log"
"net/http"
"os"
"strings"
"time"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func main() {
server := mcp.NewServer(&mcp.Implementation{
Name: "picoclaw-integration-streamable-server",
Version: "1.0.0",
}, nil)
mcp.AddTool(server, &mcp.Tool{
Name: "echo",
Description: "Echo back the provided message",
}, func(ctx context.Context, req *mcp.CallToolRequest, args map[string]any) (*mcp.CallToolResult, any, error) {
message, _ := args["message"].(string)
return &mcp.CallToolResult{
Content: []mcp.Content{
&mcp.TextContent{Text: message},
},
}, nil, nil
})
streamable := mcp.NewStreamableHTTPHandler(func(*http.Request) *mcp.Server {
return server
}, &mcp.StreamableHTTPOptions{
JSONResponse: envBool("STREAMABLE_JSON_RESPONSE", true),
})
mux := http.NewServeMux()
mux.Handle("/mcp", streamable)
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("ok"))
})
srv := &http.Server{
Addr: ":8080",
Handler: mux,
ReadHeaderTimeout: 5 * time.Second,
}
log.Printf("streamable MCP integration server listening on %s", srv.Addr)
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatal(err)
}
}
func envBool(name string, fallback bool) bool {
value := strings.TrimSpace(strings.ToLower(os.Getenv(name)))
if value == "" {
return fallback
}
switch value {
case "1", "true", "yes", "on":
return true
case "0", "false", "no", "off":
return false
default:
return fallback
}
}
@@ -0,0 +1,156 @@
package integration
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
func TestRunIntegrationTestsScriptExecutesSuiteCommand(t *testing.T) {
bashPath, err := exec.LookPath("bash")
if err != nil {
t.Skip("bash not available")
}
repoRoot := repoRootFromTestFile(t)
suitesRoot := filepath.Join(repoRoot, "integration", "suites")
suiteDir, err := os.MkdirTemp(suitesRoot, "runner-script-")
if err != nil {
t.Fatalf("MkdirTemp() error = %v", err)
}
t.Cleanup(func() {
_ = os.RemoveAll(suiteDir)
})
suiteName := filepath.Base(suiteDir)
err = os.WriteFile(
filepath.Join(suiteDir, "suite.env"),
[]byte("TEST_COMMAND='printf runner-ok'\n"),
0o644,
)
if err != nil {
t.Fatalf("WriteFile(suite.env) error = %v", err)
}
err = os.WriteFile(
filepath.Join(suiteDir, "docker-compose.yml"),
[]byte("services:\n fake-dependency:\n image: busybox\n"),
0o644,
)
if err != nil {
t.Fatalf("WriteFile(docker-compose.yml) error = %v", err)
}
stubDir := t.TempDir()
logPath := filepath.Join(t.TempDir(), "docker.log")
err = os.WriteFile(filepath.Join(stubDir, "docker"), []byte(`#!/bin/sh
set -eu
log_file="${DOCKER_LOG:?}"
{
printf '%s\n' '---'
for arg in "$@"; do
printf '%s\n' "$arg"
done
} >>"$log_file"
subcommand=""
for arg in "$@"; do
case "$arg" in
config|up|run|down)
subcommand="$arg"
;;
esac
done
case "$subcommand" in
config)
printf '%s\n' integration-runner fake-dependency
;;
up)
;;
run)
printf '%s\n' runner-ok
;;
down)
;;
*)
printf 'unexpected docker invocation: %s\n' "$*" >&2
exit 1
;;
esac
`), 0o755)
if err != nil {
t.Fatalf("WriteFile(docker stub) error = %v", err)
}
cmd := exec.Command(bashPath, filepath.Join(repoRoot, "scripts", "run-integration-tests.sh"), suiteName)
cmd.Dir = repoRoot
cmd.Env = append(os.Environ(),
"PATH="+stubDir+string(os.PathListSeparator)+os.Getenv("PATH"),
"DOCKER_LOG="+logPath,
)
output, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("run-integration-tests.sh error = %v\noutput:\n%s", err, output)
}
if !strings.Contains(string(output), "runner-ok") {
t.Fatalf("script output did not include runner output:\n%s", output)
}
logData, err := os.ReadFile(logPath)
if err != nil {
t.Fatalf("ReadFile(logPath) error = %v", err)
}
runArgs := findLoggedDockerInvocation(t, string(logData), "run")
if strings.Contains(strings.Join(runArgs, "\n"), "\nsh\n-c\n") {
t.Fatalf("docker compose run unexpectedly wrapped TEST_COMMAND with sh -c:\n%v", runArgs)
}
if !containsArg(runArgs, "integration-runner") {
t.Fatalf("docker compose run args missing runner service:\n%v", runArgs)
}
if !containsArg(runArgs, "printf runner-ok") {
t.Fatalf("docker compose run args missing suite command as a single argument:\n%v", runArgs)
}
}
func repoRootFromTestFile(t *testing.T) string {
t.Helper()
wd, err := os.Getwd()
if err != nil {
t.Fatalf("Getwd() error = %v", err)
}
return filepath.Dir(wd)
}
func findLoggedDockerInvocation(t *testing.T, logData, subcommand string) []string {
t.Helper()
for _, block := range strings.Split(logData, "---\n") {
block = strings.TrimSpace(block)
if block == "" {
continue
}
args := strings.Split(block, "\n")
if containsArg(args, subcommand) {
return args
}
}
t.Fatalf("did not find docker %q invocation in log:\n%s", subcommand, logData)
return nil
}
func containsArg(args []string, want string) bool {
for _, arg := range args {
if arg == want {
return true
}
}
return false
}
@@ -0,0 +1,19 @@
services:
integration-runner:
depends_on:
mcp-streamable-server:
condition: service_healthy
environment:
PICOCLAW_MCP_REAL_SERVER_JSON: >-
{"enabled":true,"type":"http","url":"http://mcp-streamable-server:8080/mcp"}
PICOCLAW_MCP_REAL_TOOL_NAME: echo
PICOCLAW_MCP_REAL_TOOL_ARGS_JSON: >-
{"message":"hello from docker integration suite"}
PICOCLAW_MCP_REAL_EXPECT_SUBSTRING: hello from docker integration suite
mcp-streamable-server:
build:
context: ${INTEGRATION_REPO_ROOT}
dockerfile: integration/fixtures/mcp-streamable-server/Dockerfile
environment:
STREAMABLE_JSON_RESPONSE: "true"
@@ -0,0 +1 @@
TEST_COMMAND='go test ./pkg/mcp -run TestIntegration_RealConfiguredServer -v'
+4
View File
@@ -31,6 +31,10 @@ func (a *messageBusAdapter) PublishOutboundMedia(ctx context.Context, msg bus.Ou
return a.inner.PublishOutboundMedia(ctx, msg)
}
func (a *messageBusAdapter) GetStreamer(ctx context.Context, channel, chatID, sessionKey string) (bus.Streamer, bool) {
return a.inner.GetStreamer(ctx, channel, chatID, sessionKey)
}
func (a *messageBusAdapter) InboundChan() <-chan bus.InboundMessage {
return a.inner.InboundChan()
}
+6 -2
View File
@@ -119,9 +119,11 @@ const (
pendingTurnPrefix = "pending-"
metadataKeyMessageKind = "message_kind"
metadataKeyToolCalls = "tool_calls"
metadataKeyOutboundKind = "outbound_kind"
messageKindThought = "thought"
messageKindToolFeedback = "tool_feedback"
messageKindToolCalls = "tool_calls"
outboundKindFinal = "final"
metadataKeyAccountID = "account_id"
metadataKeyGuildID = "guild_id"
metadataKeyTeamID = "team_id"
@@ -585,7 +587,7 @@ func (al *AgentLoop) runAgentLoop(
opts.Dispatch.SessionKey,
opts.Dispatch.SessionScope,
)
al.bus.PublishOutbound(ctx, bus.OutboundMessage{
msg := bus.OutboundMessage{
Context: outboundContextFromInbound(
opts.Dispatch.InboundContext,
opts.Dispatch.Channel(),
@@ -597,7 +599,9 @@ func (al *AgentLoop) runAgentLoop(
Scope: scope,
Content: result.finalContent,
ContextUsage: computeContextUsage(agent, opts.Dispatch.SessionKey),
})
}
markFinalOutbound(&msg)
al.bus.PublishOutbound(ctx, msg)
}
if result.finalContent != "" {
+7 -4
View File
@@ -75,12 +75,14 @@ func (al *AgentLoop) PublishResponseIfNeeded(ctx context.Context, channel, chatI
}
msg := bus.OutboundMessage{
Context: bus.NewOutboundContext(channel, chatID, ""),
Content: response,
Context: bus.NewOutboundContext(channel, chatID, ""),
SessionKey: sessionKey,
Content: response,
}
if sessionKey != "" {
msg.ContextUsage = computeContextUsage(al.agentForSession(sessionKey), sessionKey)
}
markFinalOutbound(&msg)
al.bus.PublishOutbound(ctx, msg)
logger.InfoCF("agent", "Published outbound response",
map[string]any{
@@ -100,7 +102,7 @@ func (al *AgentLoop) targetReasoningChannelID(channelName string) (chatID string
return ""
}
func (al *AgentLoop) publishPicoReasoning(ctx context.Context, reasoningContent, chatID string) {
func (al *AgentLoop) publishPicoReasoning(ctx context.Context, reasoningContent, chatID, sessionKey string) {
if reasoningContent == "" || chatID == "" {
return
}
@@ -120,7 +122,8 @@ func (al *AgentLoop) publishPicoReasoning(ctx context.Context, reasoningContent,
metadataKeyMessageKind: messageKindThought,
},
},
Content: reasoningContent,
SessionKey: sessionKey,
Content: reasoningContent,
}); err != nil {
if errors.Is(err, context.DeadlineExceeded) || errors.Is(err, context.Canceled) ||
errors.Is(err, bus.ErrBusClosed) {
+49
View File
@@ -284,6 +284,55 @@ func TestPublishResponseIfNeeded_DismissesToolFeedbackWhenMessageToolAlreadySent
}
}
func TestPublishResponseIfNeeded_MarksFinalOutbound(t *testing.T) {
al, _, msgBus, provider, cleanup := newTestAgentLoop(t)
defer cleanup()
_ = provider
al.PublishResponseIfNeeded(context.Background(), "pico", "pico:session-1", "session-1", "final reply")
select {
case outbound := <-msgBus.OutboundChan():
if outbound.Content != "final reply" {
t.Fatalf("outbound content = %q, want final reply", outbound.Content)
}
if outbound.Context.Raw[metadataKeyOutboundKind] != outboundKindFinal {
t.Fatalf("outbound kind = %q, want %q", outbound.Context.Raw[metadataKeyOutboundKind], outboundKindFinal)
}
if outbound.SessionKey != "session-1" {
t.Fatalf("outbound session key = %q, want session-1", outbound.SessionKey)
}
case <-time.After(time.Second):
t.Fatal("expected final outbound")
}
}
func TestPublishPicoReasoningIncludesSessionKey(t *testing.T) {
al, _, msgBus, provider, cleanup := newTestAgentLoop(t)
defer cleanup()
_ = provider
al.publishPicoReasoning(context.Background(), "reasoning", "pico-chat", "session-1")
select {
case outbound := <-msgBus.OutboundChan():
if outbound.Channel != "pico" || outbound.ChatID != "pico-chat" {
t.Fatalf("unexpected outbound target: %+v", outbound)
}
if outbound.Content != "reasoning" {
t.Fatalf("outbound content = %q, want reasoning", outbound.Content)
}
if outbound.SessionKey != "session-1" {
t.Fatalf("outbound session key = %q, want session-1", outbound.SessionKey)
}
if outbound.Context.Raw[metadataKeyMessageKind] != messageKindThought {
t.Fatalf("message kind = %q, want %q", outbound.Context.Raw[metadataKeyMessageKind], messageKindThought)
}
case <-time.After(time.Second):
t.Fatal("expected pico reasoning outbound")
}
}
func TestProcessMessage_IncludesCurrentSenderInDynamicContext(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "agent-test-*")
if err != nil {
+10
View File
@@ -86,6 +86,16 @@ func outboundMessageForTurn(ts *turnState, content string) bus.OutboundMessage {
}
}
func markFinalOutbound(msg *bus.OutboundMessage) {
if msg == nil {
return
}
if msg.Context.Raw == nil {
msg.Context.Raw = make(map[string]string, 1)
}
msg.Context.Raw[metadataKeyOutboundKind] = outboundKindFinal
}
func outboundMessageForTurnWithKind(ts *turnState, content, kind string) bus.OutboundMessage {
msg := outboundMessageForTurn(ts, content)
if strings.TrimSpace(kind) == "" {
+3
View File
@@ -21,6 +21,9 @@ type MessageBus interface {
// PublishOutboundMedia sends an outbound media message.
PublishOutboundMedia(ctx context.Context, msg bus.OutboundMediaMessage) error
// GetStreamer returns a channel streamer when the active channel supports streaming.
GetStreamer(ctx context.Context, channel, chatID, sessionKey string) (bus.Streamer, bool)
// InboundChan returns the channel for receiving inbound messages.
InboundChan() <-chan bus.InboundMessage
}
+87 -5
View File
@@ -29,6 +29,36 @@ func modelConfigIdentityKey(mc *config.ModelConfig) string {
return ""
}
func effectiveDefaultProvider(defaultProvider string) string {
defaultProvider = strings.TrimSpace(defaultProvider)
if defaultProvider == "" {
return "openai"
}
return providers.NormalizeProvider(defaultProvider)
}
func modelProviderAndIDForResolution(defaultProvider string, mc *config.ModelConfig) (provider string, modelID string) {
if mc == nil {
return "", ""
}
return providers.ExtractProtocol(mc)
}
func cloneModelConfigForResolution(
defaultProvider string,
mc *config.ModelConfig,
workspace string,
) *config.ModelConfig {
if mc == nil {
return nil
}
clone := *mc
if clone.Workspace == "" {
clone.Workspace = workspace
}
return &clone
}
func candidateFromModelConfig(
defaultProvider string,
mc *config.ModelConfig,
@@ -37,7 +67,7 @@ func candidateFromModelConfig(
return providers.FallbackCandidate{}, false
}
protocol, modelID := providers.ExtractProtocol(mc)
protocol, modelID := modelProviderAndIDForResolution(defaultProvider, mc)
if strings.TrimSpace(modelID) == "" {
return providers.FallbackCandidate{}, false
}
@@ -50,7 +80,7 @@ func candidateFromModelConfig(
}, true
}
func lookupModelConfigByRef(cfg *config.Config, raw string) *config.ModelConfig {
func lookupModelConfigByRef(cfg *config.Config, raw string, defaultProvider ...string) *config.ModelConfig {
raw = strings.TrimSpace(raw)
if raw == "" || cfg == nil {
return nil
@@ -66,6 +96,10 @@ func lookupModelConfigByRef(cfg *config.Config, raw string) *config.ModelConfig
rawKey = providers.ModelKey(rawRef.Provider, rawRef.Model)
}
fallbackProvider := ""
if len(defaultProvider) > 0 {
fallbackProvider = effectiveDefaultProvider(defaultProvider[0])
}
for i := range cfg.ModelList {
mc := cfg.ModelList[i]
if mc == nil {
@@ -75,12 +109,14 @@ func lookupModelConfigByRef(cfg *config.Config, raw string) *config.ModelConfig
if fullModel == "" {
continue
}
protocol, modelID := modelProviderAndIDForResolution(fallbackProvider, mc)
if fullModel == raw {
return mc
}
protocol, modelID := providers.ExtractProtocol(mc)
if modelID == raw {
return mc
if fallbackProvider == "" || providers.NormalizeProvider(protocol) == fallbackProvider {
return mc
}
}
if rawKey != "" && providers.ModelKey(protocol, modelID) == rawKey {
return mc
@@ -99,8 +135,9 @@ func resolveModelCandidate(
if raw == "" {
return providers.FallbackCandidate{}, false
}
defaultProvider = effectiveDefaultProvider(defaultProvider)
if mc := lookupModelConfigByRef(cfg, raw); mc != nil {
if mc := lookupModelConfigByRef(cfg, raw, defaultProvider); mc != nil {
return candidateFromModelConfig(defaultProvider, mc)
}
@@ -177,3 +214,48 @@ func resolvedModelConfig(cfg *config.Config, modelName, workspace string) (*conf
return &clone, nil
}
func resolveActiveModelConfig(
cfg *config.Config,
workspace string,
candidates []providers.FallbackCandidate,
activeModel string,
defaultProvider string,
) *config.ModelConfig {
if cfg == nil {
return nil
}
defaultProvider = effectiveDefaultProvider(defaultProvider)
if len(candidates) > 0 {
candidate := candidates[0]
identityKey := strings.TrimSpace(candidate.IdentityKey)
if identityKey != "" {
for _, mc := range cfg.ModelList {
if mc == nil || modelConfigIdentityKey(mc) != identityKey {
continue
}
protocol, modelID := modelProviderAndIDForResolution(defaultProvider, mc)
if providers.ModelKey(protocol, modelID) == providers.ModelKey(candidate.Provider, candidate.Model) {
return cloneModelConfigForResolution(defaultProvider, mc, workspace)
}
}
}
for _, mc := range cfg.ModelList {
if mc == nil {
continue
}
protocol, modelID := modelProviderAndIDForResolution(defaultProvider, mc)
if providers.ModelKey(protocol, modelID) == providers.ModelKey(candidate.Provider, candidate.Model) {
return cloneModelConfigForResolution(defaultProvider, mc, workspace)
}
}
return nil
}
if mc := lookupModelConfigByRef(cfg, activeModel, defaultProvider); mc != nil {
return cloneModelConfigForResolution(defaultProvider, mc, workspace)
}
return nil
}
+116
View File
@@ -0,0 +1,116 @@
package agent
import (
"testing"
"github.com/sipeed/picoclaw/pkg/config"
"github.com/sipeed/picoclaw/pkg/providers"
)
func TestResolveActiveModelConfig_PrefersCandidateIdentityKey(t *testing.T) {
cfg := &config.Config{
ModelList: []*config.ModelConfig{
{
ModelName: "glm-4.7",
Provider: "zhipu",
Model: "glm-4.7",
Streaming: config.ModelStreamingConfig{Enabled: false},
},
{
ModelName: "suanneng-glm-4.7",
Provider: "zhipu",
Model: "glm-4.7",
Streaming: config.ModelStreamingConfig{Enabled: true},
},
},
}
got := resolveActiveModelConfig(
cfg,
"/workspace",
[]providers.FallbackCandidate{{
Provider: "zhipu",
Model: "glm-4.7",
IdentityKey: "model_name:suanneng-glm-4.7",
}},
"glm-4.7",
"openai",
)
if got == nil {
t.Fatal("resolveActiveModelConfig() = nil, want model config")
}
if got.ModelName != "suanneng-glm-4.7" {
t.Fatalf("model_name = %q, want %q", got.ModelName, "suanneng-glm-4.7")
}
if !got.Streaming.Enabled {
t.Fatal("streaming.enabled = false, want true from identity-matched model config")
}
}
func TestResolveActiveModelConfig_LoadBalancedAliasUsesSelectedCandidate(t *testing.T) {
cfg := &config.Config{
ModelList: []*config.ModelConfig{
{
ModelName: "lb-model",
Model: "openai/primary",
Streaming: config.ModelStreamingConfig{Enabled: false},
},
{
ModelName: "lb-model",
Model: "openai/secondary",
Streaming: config.ModelStreamingConfig{Enabled: true},
},
},
}
got := resolveActiveModelConfig(
cfg,
"/workspace",
[]providers.FallbackCandidate{{
Provider: "openai",
Model: "secondary",
IdentityKey: "model_name:lb-model",
}},
"lb-model",
"openai",
)
if got == nil {
t.Fatal("resolveActiveModelConfig() = nil, want model config")
}
if got.Model != "openai/secondary" {
t.Fatalf("model = %q, want openai/secondary", got.Model)
}
if !got.Streaming.Enabled {
t.Fatal("streaming.enabled = false, want true from selected load-balanced entry")
}
}
func TestResolveActiveModelConfig_DoesNotFallbackToOpenAIForDefaultProviderCandidate(t *testing.T) {
cfg := &config.Config{
ModelList: []*config.ModelConfig{
{
ModelName: "openai-gpt",
Provider: "openai",
Model: "gpt-4o",
Streaming: config.ModelStreamingConfig{Enabled: true},
},
},
}
got := resolveActiveModelConfig(
cfg,
"/workspace",
[]providers.FallbackCandidate{{
Provider: "nvidia",
Model: "gpt-4o",
}},
"gpt-4o",
"nvidia",
)
if got != nil {
t.Fatalf("resolveActiveModelConfig() = %#v, want nil for non-active provider config", got)
}
}
+36
View File
@@ -58,6 +58,7 @@ func (p *Pipeline) Finalize(
Message: err.Error(),
},
)
cancelConfiguredStreamingLLM(turnCtx, exec)
return turnResult{status: TurnEndStatusError}, err
}
}
@@ -73,6 +74,41 @@ func (p *Pipeline) Finalize(
)
}
contextUsage := computeContextUsage(ts.agent, ts.sessionKey)
streamErr := finalizeConfiguredStreamingLLM(turnCtx, ts, exec, finalContent, contextUsage)
// If streaming never became visible, keep the legacy Pico interim publish path
// so the final answer is still delivered outside normal SendResponse.
if ((streamErr != nil && !isConfiguredStreamingVisibleError(streamErr)) || exec.streamingFallback) &&
!ts.opts.SendResponse && ts.opts.AllowInterimPicoPublish && finalContent != "" {
agentID, sessionKey, scope := outboundTurnMetadata(
ts.agent.ID,
ts.opts.Dispatch.SessionKey,
ts.opts.Dispatch.SessionScope,
)
msg := bus.OutboundMessage{
Context: outboundContextFromInbound(
ts.opts.Dispatch.InboundContext,
ts.opts.Dispatch.Channel(),
ts.opts.Dispatch.ChatID(),
ts.opts.Dispatch.ReplyToMessageID(),
),
AgentID: agentID,
SessionKey: sessionKey,
Scope: scope,
Content: finalContent,
ContextUsage: contextUsage,
}
markFinalOutbound(&msg)
_ = al.bus.PublishOutbound(turnCtx, msg)
}
if streamErr != nil && isConfiguredStreamingVisibleError(streamErr) {
ts.setPhase(TurnPhaseCompleted)
return turnResult{
finalContent: finalContent,
status: TurnEndStatusError,
followUps: append([]bus.InboundMessage(nil), ts.followUps...),
}, streamErr
}
ts.setPhase(TurnPhaseCompleted)
return turnResult{
finalContent: finalContent,
+88 -4
View File
@@ -98,15 +98,21 @@ func (p *Pipeline) CallLLM(
switch decision.normalizedAction() {
case HookActionContinue, HookActionModify:
if llmReq != nil {
prevModel := exec.llmModel
exec.llmModel = llmReq.Model
exec.callMessages = llmReq.Messages
exec.providerToolDefs = llmReq.Tools
exec.llmOpts = llmReq.Options
if strings.TrimSpace(exec.llmModel) != "" && exec.llmModel != prevModel {
p.applyBeforeLLMModelRewrite(ts, exec)
}
}
case HookActionAbortTurn:
cancelConfiguredStreamingLLM(turnCtx, exec)
exec.abortedByHook = true
return ControlBreak, nil
case HookActionHardAbort:
cancelConfiguredStreamingLLM(turnCtx, exec)
_ = ts.requestHardAbort()
exec.abortedByHardAbort = true
return ControlBreak, nil
@@ -155,14 +161,30 @@ func (p *Pipeline) CallLLM(
al.activeRequests.Add(1)
defer al.activeRequests.Done()
if response, handled, streamErr := p.tryConfiguredStreamingLLM(
providerCtx,
ts,
exec,
messagesForCall,
toolDefsForCall,
); handled {
return response, streamErr
}
if len(exec.activeCandidates) > 1 && p.Fallback != nil {
fbResult, fbErr := p.Fallback.Execute(
providerCtx,
exec.activeCandidates,
func(ctx context.Context, provider, model string) (*providers.LLMResponse, error) {
candidateProvider := exec.activeProvider
if cp, ok := ts.agent.CandidateProviders[providers.ModelKey(provider, model)]; ok {
candidateProvider = cp
candidateProvider, err := providerForFallbackCandidate(
ts.agent,
exec.activeProvider,
exec.activeCandidates,
provider,
model,
)
if err != nil {
return nil, err
}
return candidateProvider.Chat(ctx, messagesForCall, toolDefsForCall, model, exec.llmOpts)
},
@@ -203,6 +225,9 @@ func (p *Pipeline) CallLLM(
exec.abortedByHardAbort = true
return ControlBreak, nil
}
if isConfiguredStreamingVisibleError(err) {
break
}
// Retry without media if vision is unsupported
if hasMediaRefs(exec.callMessages) && isVisionUnsupportedError(err) && retry < maxRetries {
@@ -415,9 +440,11 @@ func (p *Pipeline) CallLLM(
exec.response = llmResp.Response
}
case HookActionAbortTurn:
cancelConfiguredStreamingLLM(turnCtx, exec)
exec.abortedByHook = true
return ControlBreak, nil
case HookActionHardAbort:
cancelConfiguredStreamingLLM(turnCtx, exec)
_ = ts.requestHardAbort()
exec.abortedByHardAbort = true
return ControlBreak, nil
@@ -438,7 +465,20 @@ func (p *Pipeline) CallLLM(
// Pico tool-call turns publish their reasoning/content/tool summary as a
// structured sequence after the tool-call payload is normalized below.
} else if ts.channel == "pico" {
go al.publishPicoReasoning(turnCtx, reasoningContent, ts.chatID)
if exec.streamingPublisher != nil && exec.streamingPublisher.ReasoningPublished() {
if err := exec.streamingPublisher.FinalizeReasoning(turnCtx, reasoningContent); err != nil {
logger.WarnCF("agent", "Failed to finalize streamed pico reasoning", map[string]any{
"channel": ts.channel,
"chat_id": ts.chatID,
"error": err.Error(),
})
}
} else {
// Publish pico thoughts before the turn context is canceled at return time.
// The async variant can race with turn teardown and intermittently drop the
// thought message in CI even though the LLM produced reasoning content.
al.publishPicoReasoning(turnCtx, reasoningContent, ts.chatID, ts.sessionKey)
}
} else {
go al.handleReasoning(
turnCtx,
@@ -480,6 +520,7 @@ func (p *Pipeline) CallLLM(
responseContent = exec.response.ReasoningContent
}
if steerMsgs := al.dequeueSteeringMessagesForScope(ts.sessionKey); len(steerMsgs) > 0 {
cancelConfiguredStreamingLLM(turnCtx, exec)
logger.InfoCF("agent", "Steering arrived after direct LLM response; continuing turn",
map[string]any{
"agent_id": ts.agent.ID,
@@ -489,6 +530,7 @@ func (p *Pipeline) CallLLM(
exec.pendingMessages = append(exec.pendingMessages, steerMsgs...)
return ControlContinue, nil
}
exec.finalContent = responseContent
logger.InfoCF("agent", "LLM response without tool calls (direct answer)",
map[string]any{
@@ -498,6 +540,7 @@ func (p *Pipeline) CallLLM(
})
return ControlBreak, nil
}
cancelConfiguredStreamingLLM(turnCtx, exec)
// Tool-call path: normalize and prepare for tool execution
exec.normalizedToolCalls = make([]providers.ToolCall, 0, len(exec.response.ToolCalls))
@@ -572,3 +615,44 @@ func (p *Pipeline) CallLLM(
return ControlToolLoop, nil
}
func (p *Pipeline) applyBeforeLLMModelRewrite(ts *turnState, exec *turnExecution) {
if p == nil || ts == nil || ts.agent == nil || exec == nil {
return
}
rawModel := strings.TrimSpace(exec.llmModel)
if rawModel == "" {
return
}
defaultProvider := "openai"
if p.Cfg != nil {
if provider := strings.TrimSpace(p.Cfg.Agents.Defaults.Provider); provider != "" {
defaultProvider = provider
}
}
defaultProvider = effectiveDefaultProvider(defaultProvider)
candidates := resolveModelCandidates(p.Cfg, defaultProvider, rawModel, nil)
exec.activeCandidates = candidates
exec.activeModel = resolvedCandidateModel(candidates, rawModel)
exec.llmModel = exec.activeModel
exec.activeModelConfig = resolveActiveModelConfig(p.Cfg, ts.agent.Workspace, candidates, rawModel, defaultProvider)
}
func providerForFallbackCandidate(
agent *AgentInstance,
activeProvider providers.LLMProvider,
activeCandidates []providers.FallbackCandidate,
provider string,
model string,
) (providers.LLMProvider, error) {
if agent != nil {
if cp, ok := agent.CandidateProviders[providers.ModelKey(provider, model)]; ok && cp != nil {
return cp, nil
}
}
if activeProvider == nil {
return nil, fmt.Errorf("fallback model %q has no active provider", model)
}
return activeProvider, nil
}
+7
View File
@@ -99,6 +99,13 @@ func (p *Pipeline) SetupTurn(ctx context.Context, ts *turnState) (*turnExecution
)
exec.activeCandidates = activeCandidates
exec.activeModel = activeModel
exec.activeModelConfig = resolveActiveModelConfig(
p.Cfg,
ts.agent.Workspace,
activeCandidates,
activeModel,
p.Cfg.Agents.Defaults.Provider,
)
exec.activeProvider = activeProvider
exec.usedLight = usedLight
+478
View File
@@ -0,0 +1,478 @@
package agent
import (
"context"
"errors"
"fmt"
"reflect"
"strings"
"time"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/config"
"github.com/sipeed/picoclaw/pkg/logger"
"github.com/sipeed/picoclaw/pkg/providers"
)
func (p *Pipeline) tryConfiguredStreamingLLM(
ctx context.Context,
ts *turnState,
exec *turnExecution,
messagesForCall []providers.Message,
toolDefsForCall []providers.ToolDefinition,
) (*providers.LLMResponse, bool, error) {
exec.streamingPublisher = nil
exec.streamingFallback = false
if !p.configuredStreamingEligible(ts, exec) {
return nil, false, nil
}
streamProvider, ok := exec.activeProvider.(providers.StreamingProvider)
if !ok {
logger.DebugCF("agent", "configured streaming not used", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"model": exec.activeModel,
"reason": "provider_not_streaming",
})
return nil, false, nil
}
streamer, ok := p.Bus.GetStreamer(ctx, ts.channel, ts.chatID, ts.sessionKey)
if !ok || streamer == nil {
logger.DebugCF("agent", "configured streaming not used", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"chat_id": ts.chatID,
"model": exec.activeModel,
"reason": "streamer_unavailable",
})
return nil, false, nil
}
publisher := &streamingChunkPublisher{
streamer: streamer,
channel: ts.channel,
chatID: ts.chatID,
}
logger.DebugCF("agent", "configured streaming enabled", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"chat_id": ts.chatID,
"model": exec.llmModel,
})
chunkCount := 0
firstChunkAt := time.Time{}
lastChunkAt := time.Time{}
recordChunk := func() {
now := time.Now()
chunkCount++
if firstChunkAt.IsZero() {
firstChunkAt = now
}
lastChunkAt = now
}
var response *providers.LLMResponse
var streamErr error
if eventProvider, ok := exec.activeProvider.(providers.StreamingEventProvider); ok {
response, streamErr = eventProvider.ChatStreamEvents(
ctx,
messagesForCall,
toolDefsForCall,
exec.llmModel,
exec.llmOpts,
func(chunk providers.StreamChunk) {
recordChunk()
if strings.TrimSpace(chunk.ReasoningContent) != "" {
publisher.UpdateReasoning(ctx, chunk.ReasoningContent)
}
if strings.TrimSpace(chunk.Content) != "" {
publisher.Update(ctx, chunk.Content)
}
},
)
} else {
response, streamErr = streamProvider.ChatStream(
ctx,
messagesForCall,
toolDefsForCall,
exec.llmModel,
exec.llmOpts,
func(accumulated string) {
recordChunk()
publisher.Update(ctx, accumulated)
},
)
}
logConfiguredStreamingSummary(ts, exec, chunkCount, firstChunkAt, lastChunkAt, streamErr)
if streamErr == nil {
if updateErr := publisher.Err(); updateErr != nil {
logFields := map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"model": exec.llmModel,
"error": updateErr.Error(),
}
if publisher.Published() {
logger.WarnCF("agent", "ChatStream update failed after visible output", logFields)
return nil, true, configuredStreamingVisibleError{err: updateErr}
}
logger.WarnCF("agent", "ChatStream update failed before visible output; retrying with Chat", logFields)
publisher.Cancel(ctx)
fallbackResponse, err := exec.activeProvider.Chat(
ctx,
messagesForCall,
toolDefsForCall,
exec.llmModel,
exec.llmOpts,
)
if err == nil && fallbackResponse != nil {
exec.streamingFallback = true
}
return fallbackResponse, true, err
}
}
if streamErr != nil {
if !publisher.Published() {
logger.WarnCF("agent", "ChatStream failed before visible output; retrying with Chat", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"model": exec.llmModel,
"error": streamErr.Error(),
})
publisher.Cancel(ctx)
fallbackResponse, err := exec.activeProvider.Chat(
ctx,
messagesForCall,
toolDefsForCall,
exec.llmModel,
exec.llmOpts,
)
if err == nil && fallbackResponse != nil {
exec.streamingFallback = true
}
return fallbackResponse, true, err
}
return nil, true, configuredStreamingVisibleError{err: streamErr}
}
if response != nil {
exec.streamingPublisher = publisher
}
return response, true, nil
}
func logConfiguredStreamingSummary(
ts *turnState,
exec *turnExecution,
chunkCount int,
firstChunkAt time.Time,
lastChunkAt time.Time,
streamErr error,
) {
fields := map[string]any{
"chunks": chunkCount,
}
if ts != nil {
fields["agent_id"] = ts.agent.ID
fields["channel"] = ts.channel
}
if exec != nil {
fields["model"] = exec.llmModel
}
if !firstChunkAt.IsZero() && !lastChunkAt.IsZero() {
fields["chunk_span_ms"] = lastChunkAt.Sub(firstChunkAt).Milliseconds()
}
if streamErr != nil {
fields["error"] = streamErr.Error()
}
logger.DebugCF("agent", "configured streaming completed", fields)
}
type configuredStreamingVisibleError struct {
err error
}
func (e configuredStreamingVisibleError) Error() string {
if e.err == nil {
return "configured streaming failed after visible output"
}
return e.err.Error()
}
func (e configuredStreamingVisibleError) Unwrap() error {
return e.err
}
func isConfiguredStreamingVisibleError(err error) bool {
var visibleErr configuredStreamingVisibleError
return errors.As(err, &visibleErr)
}
func finalizeConfiguredStreamingLLM(
ctx context.Context,
ts *turnState,
exec *turnExecution,
content string,
contextUsage *bus.ContextUsage,
) error {
if exec == nil || exec.streamingPublisher == nil {
return nil
}
publisher := exec.streamingPublisher
exec.streamingPublisher = nil
visibleBeforeFinalize := publisher.Published()
if err := publisher.Finalize(ctx, content, contextUsage); err != nil {
if visibleBeforeFinalize {
logger.WarnCF("agent", "stream final flush failed after visible output", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"model": exec.llmModel,
"error": err.Error(),
})
return configuredStreamingVisibleError{err: err}
}
publisher.Cancel(ctx)
logger.WarnCF("agent", "stream final flush failed", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"model": exec.llmModel,
"error": err.Error(),
})
return err
}
return nil
}
func cancelConfiguredStreamingLLM(ctx context.Context, exec *turnExecution) {
if exec == nil || exec.streamingPublisher == nil {
return
}
publisher := exec.streamingPublisher
exec.streamingPublisher = nil
publisher.Cancel(ctx)
}
func (p *Pipeline) configuredStreamingEligible(ts *turnState, exec *turnExecution) bool {
if p == nil || ts == nil || exec == nil || p.Bus == nil {
logger.DebugCF("agent", "configured streaming not used", map[string]any{
"reason": "missing_pipeline_state",
})
return false
}
if strings.TrimSpace(ts.channel) == "" || strings.TrimSpace(ts.chatID) == "" {
logger.DebugCF("agent", "configured streaming not used", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"chat_id": ts.chatID,
"model": exec.activeModel,
"reason": "missing_channel_context",
})
return false
}
if !ts.opts.SendResponse && !ts.opts.AllowInterimPicoPublish {
logger.DebugCF("agent", "configured streaming not used", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"chat_id": ts.chatID,
"model": exec.activeModel,
"reason": "turn_output_disabled",
})
return false
}
if len(exec.activeCandidates) != 1 {
logger.DebugCF("agent", "configured streaming not used", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"model": exec.activeModel,
"candidates": len(exec.activeCandidates),
"reason": "fallback_candidates_enabled",
})
return false
}
if exec.activeModelConfig == nil || !exec.activeModelConfig.Streaming.Enabled {
modelName := ""
modelStreaming := false
if exec.activeModelConfig != nil {
modelName = exec.activeModelConfig.ModelName
modelStreaming = exec.activeModelConfig.Streaming.Enabled
}
logger.DebugCF("agent", "configured streaming not used", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"model": exec.activeModel,
"model_name": modelName,
"model_streaming": modelStreaming,
"has_model_config": exec.activeModelConfig != nil,
"reason": "model_streaming_disabled",
})
return false
}
channelStreaming, ok := p.channelStreamingConfig(ts.channel)
if !ok || !channelStreaming.Enabled {
logger.DebugCF("agent", "configured streaming not used", map[string]any{
"agent_id": ts.agent.ID,
"channel": ts.channel,
"model": exec.activeModel,
"channel_streaming": channelStreaming.Enabled,
"has_channel_config": ok,
"reason": "channel_streaming_disabled",
})
return false
}
return true
}
func (p *Pipeline) channelStreamingConfig(channelName string) (config.StreamingConfig, bool) {
if p == nil || p.Cfg == nil || p.Cfg.Channels == nil {
return config.StreamingConfig{}, false
}
ch := p.Cfg.Channels[channelName]
if ch == nil {
return config.StreamingConfig{}, false
}
decoded, err := ch.GetDecoded()
if err != nil {
logger.WarnCF("agent", "channel streaming config decode failed", map[string]any{
"channel": channelName,
"error": err.Error(),
})
return config.StreamingConfig{}, false
}
return streamingConfigFromDecodedSettings(decoded)
}
func streamingConfigFromDecodedSettings(decoded any) (config.StreamingConfig, bool) {
value := reflect.ValueOf(decoded)
if !value.IsValid() {
return config.StreamingConfig{}, false
}
if value.Kind() == reflect.Ptr {
if value.IsNil() {
return config.StreamingConfig{}, false
}
value = value.Elem()
}
if value.Kind() != reflect.Struct {
return config.StreamingConfig{}, false
}
field := value.FieldByName("Streaming")
if !field.IsValid() || !field.CanInterface() {
return config.StreamingConfig{}, false
}
streaming, ok := field.Interface().(config.StreamingConfig)
return streaming, ok
}
type streamingChunkPublisher struct {
streamer bus.Streamer
channel string
chatID string
published bool
reasoningPublished bool
err error
}
func (p *streamingChunkPublisher) Update(ctx context.Context, accumulated string) {
if p == nil || p.streamer == nil || strings.TrimSpace(accumulated) == "" {
return
}
if err := p.streamer.Update(ctx, accumulated); err != nil {
p.err = err
logger.WarnCF("agent", "stream update failed", map[string]any{
"channel": p.channel,
"chat_id": p.chatID,
"error": err.Error(),
})
return
}
p.published = true
}
func (p *streamingChunkPublisher) UpdateReasoning(ctx context.Context, accumulated string) {
if p == nil || p.streamer == nil || strings.TrimSpace(accumulated) == "" {
return
}
reasoningStreamer, ok := p.streamer.(bus.ReasoningStreamer)
if !ok {
return
}
if err := reasoningStreamer.UpdateReasoning(ctx, accumulated); err != nil {
p.err = err
logger.WarnCF("agent", "stream reasoning update failed", map[string]any{
"channel": p.channel,
"chat_id": p.chatID,
"error": err.Error(),
})
return
}
p.reasoningPublished = true
}
func (p *streamingChunkPublisher) Published() bool {
return p != nil && p.published
}
func (p *streamingChunkPublisher) ReasoningPublished() bool {
return p != nil && p.reasoningPublished
}
func (p *streamingChunkPublisher) Err() error {
if p == nil {
return nil
}
return p.err
}
func (p *streamingChunkPublisher) Finalize(ctx context.Context, content string, contextUsage *bus.ContextUsage) error {
if p == nil || p.streamer == nil {
return nil
}
if strings.TrimSpace(content) == "" && !p.published {
return nil
}
var err error
if streamer, ok := p.streamer.(bus.ContextUsageStreamer); ok {
err = streamer.FinalizeWithContext(ctx, content, contextUsage)
} else {
err = p.streamer.Finalize(ctx, content)
}
if err != nil {
return fmt.Errorf("stream finalize: %w", err)
}
p.published = true
return nil
}
func (p *streamingChunkPublisher) FinalizeReasoning(ctx context.Context, content string) error {
if p == nil || p.streamer == nil || !p.reasoningPublished || strings.TrimSpace(content) == "" {
return nil
}
reasoningStreamer, ok := p.streamer.(bus.ReasoningStreamer)
if !ok {
return nil
}
if err := reasoningStreamer.FinalizeReasoning(ctx, content); err != nil {
return fmt.Errorf("stream reasoning finalize: %w", err)
}
return nil
}
func (p *streamingChunkPublisher) ClearFinalizedStreamMarker() {
if p == nil || p.streamer == nil {
return
}
if cleaner, ok := p.streamer.(interface{ ClearFinalizedStreamMarker() }); ok {
cleaner.ClearFinalizedStreamMarker()
}
}
func (p *streamingChunkPublisher) Cancel(ctx context.Context) {
if p == nil || p.streamer == nil {
return
}
p.streamer.Cancel(ctx)
}
File diff suppressed because it is too large Load Diff
+8 -4
View File
@@ -11,6 +11,7 @@ import (
"time"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/config"
"github.com/sipeed/picoclaw/pkg/logger"
"github.com/sipeed/picoclaw/pkg/providers"
"github.com/sipeed/picoclaw/pkg/session"
@@ -124,15 +125,18 @@ type turnExecution struct {
iteration int
// Per-iteration state set by Pipeline.PreLLM
activeCandidates []providers.FallbackCandidate
activeModel string
activeProvider providers.LLMProvider
usedLight bool
activeCandidates []providers.FallbackCandidate
activeModel string
activeModelConfig *config.ModelConfig
activeProvider providers.LLMProvider
usedLight bool
// LLM call per-iteration state
response *providers.LLMResponse
normalizedToolCalls []providers.ToolCall
allResponsesHandled bool
streamingPublisher *streamingChunkPublisher
streamingFallback bool
callMessages []providers.Message
providerToolDefs []providers.ToolDefinition
llmModel string
+18 -4
View File
@@ -26,7 +26,7 @@ const defaultBusBufferSize = 64
type StreamDelegate interface {
// GetStreamer returns a Streamer for the given channel+chatID if the channel
// supports streaming. Returns nil, false if streaming is unavailable.
GetStreamer(ctx context.Context, channel, chatID string) (Streamer, bool)
GetStreamer(ctx context.Context, channel, chatID, sessionKey string) (Streamer, bool)
}
// Streamer pushes incremental content to a streaming-capable channel.
@@ -37,6 +37,20 @@ type Streamer interface {
Cancel(ctx context.Context)
}
// ContextUsageStreamer can attach final context-window usage metadata when a
// streaming channel's final message replaces the normal outbound response.
type ContextUsageStreamer interface {
Streamer
FinalizeWithContext(ctx context.Context, content string, usage *ContextUsage) error
}
// ReasoningStreamer can show incremental model reasoning/thought content
// separately from the final user-visible answer stream.
type ReasoningStreamer interface {
UpdateReasoning(ctx context.Context, content string) error
FinalizeReasoning(ctx context.Context, content string) error
}
type MessageBus struct {
inbound chan InboundMessage
outbound chan OutboundMessage
@@ -182,10 +196,10 @@ func (mb *MessageBus) SetEventPublisher(p EventPublisher) {
mb.eventPublisher.Store(p)
}
// GetStreamer returns a Streamer for the given channel+chatID via the delegate.
func (mb *MessageBus) GetStreamer(ctx context.Context, channel, chatID string) (Streamer, bool) {
// GetStreamer returns a Streamer for the given channel+chatID+session via the delegate.
func (mb *MessageBus) GetStreamer(ctx context.Context, channel, chatID, sessionKey string) (Streamer, bool) {
if d, ok := mb.streamDelegate.Load().(StreamDelegate); ok && d != nil {
return d.GetStreamer(ctx, channel, chatID)
return d.GetStreamer(ctx, channel, chatID, sessionKey)
}
return nil, false
}
+1 -1
View File
@@ -791,7 +791,7 @@ the top level, with channel-specific settings in the `settings` sub-key:
```json
{
"channels": {
"channel_list": {
"matrix": {
"enabled": true,
"type": "matrix",
+1 -1
View File
@@ -790,7 +790,7 @@ channel 特定的设置放在 `settings` 子键中:
```json
{
"channels": {
"channel_list": {
"matrix": {
"enabled": true,
"type": "matrix",
+387 -73
View File
@@ -41,6 +41,8 @@ const (
janitorInterval = 10 * time.Second
typingStopTTL = 5 * time.Minute
placeholderTTL = 10 * time.Minute
streamAuxiliaryTombstoneTTL = 30 * time.Second
)
// typingEntry wraps a typing stop function with a creation timestamp for TTL eviction.
@@ -82,22 +84,23 @@ type channelWorker struct {
}
type Manager struct {
channels map[string]Channel
workers map[string]*channelWorker
bus *bus.MessageBus
runtimeEvents runtimeevents.Bus
config *config.Config
mediaStore media.MediaStore
dispatchTask *asyncTask
mux *dynamicServeMux
httpServer *http.Server
httpListeners []net.Listener
mu sync.RWMutex
placeholders sync.Map // "channel:chatID" → placeholderID (string)
typingStops sync.Map // "channel:chatID" → func()
reactionUndos sync.Map // "channel:chatID" → reactionEntry
streamActive sync.Map // "channel:chatID" → true (set when streamer.Finalize sent the message)
channelHashes map[string]string // channel name → config hash
channels map[string]Channel
workers map[string]*channelWorker
bus *bus.MessageBus
runtimeEvents runtimeevents.Bus
config *config.Config
mediaStore media.MediaStore
dispatchTask *asyncTask
mux *dynamicServeMux
httpServer *http.Server
httpListeners []net.Listener
mu sync.RWMutex
placeholders sync.Map // "channel:chatID" → placeholderID (string)
typingStops sync.Map // "channel:chatID" → func()
reactionUndos sync.Map // "channel:chatID" → reactionEntry
streamActive sync.Map // streamSuppressionKey → true (set when streamer.Finalize sent the message)
streamAuxiliaryTombstones sync.Map // streamSuppressionKey → time.Time (drops late auxiliary messages after stream final)
channelHashes map[string]string // channel name → config hash
}
type mediaStoreSetter interface {
@@ -166,6 +169,20 @@ func outboundMessageIsToolFeedback(msg bus.OutboundMessage) bool {
return strings.EqualFold(strings.TrimSpace(msg.Context.Raw["message_kind"]), "tool_feedback")
}
func outboundMessageHasAuxiliaryKind(msg bus.OutboundMessage) bool {
if len(msg.Context.Raw) == 0 {
return false
}
return strings.TrimSpace(msg.Context.Raw["message_kind"]) != ""
}
func outboundMessageIsFinal(msg bus.OutboundMessage) bool {
if len(msg.Context.Raw) == 0 {
return false
}
return strings.EqualFold(strings.TrimSpace(msg.Context.Raw["outbound_kind"]), "final")
}
func outboundMessageBypassesPlaceholderEdit(msg bus.OutboundMessage) bool {
if len(msg.Context.Raw) == 0 {
return false
@@ -182,6 +199,14 @@ func outboundMediaChatID(msg bus.OutboundMediaMessage) string {
return msg.ChatID
}
func streamSuppressionKey(channel, chatID, sessionKey string) string {
key := channel + ":" + chatID
if strings.TrimSpace(sessionKey) == "" {
return key
}
return key + ":" + sessionKey
}
func trackedToolFeedbackMessageChatID(ch Channel, chatID string, outboundCtx *bus.InboundContext) string {
if resolver, ok := ch.(toolFeedbackMessageTargetResolver); ok {
if resolved := strings.TrimSpace(resolver.ToolFeedbackMessageChatID(chatID, outboundCtx)); resolved != "" {
@@ -324,6 +349,7 @@ func (m *Manager) RecordReactionUndo(channel, chatID string, undo func()) {
func (m *Manager) preSend(ctx context.Context, name string, msg bus.OutboundMessage, ch Channel) ([]string, bool) {
chatID := outboundMessageChatID(msg)
key := name + ":" + chatID
streamKey := streamSuppressionKey(name, chatID, msg.SessionKey)
// 1. Stop typing
if v, loaded := m.typingStops.LoadAndDelete(key); loaded {
@@ -340,38 +366,58 @@ func (m *Manager) preSend(ctx context.Context, name string, msg bus.OutboundMess
}
isToolFeedback := outboundMessageIsToolFeedback(msg)
isAuxiliaryMessage := outboundMessageHasAuxiliaryKind(msg)
isFinalMessage := outboundMessageIsFinal(msg)
separateToolFeedbackMessages := m.toolFeedbackSeparateMessagesEnabled()
// 3. If a stream already finalized this chat, stale tool feedback must be
// dropped without consuming the final-response marker. Streaming finalization
// bypasses the worker queue, so older queued feedback can arrive before the
// normal final outbound message that cleans up the marker and placeholder.
if isToolFeedback {
if _, loaded := m.streamActive.Load(key); loaded {
// 3. If a stream already finalized this chat, stale auxiliary messages must
// be dropped without consuming the final-response marker. Streaming
// finalization bypasses the worker queue, so older queued feedback/thoughts
// can arrive before the normal final outbound message that cleans up the
// marker and placeholder.
if isAuxiliaryMessage {
if _, loaded := m.streamActive.Load(streamKey); loaded {
return nil, true
}
if m.streamAuxiliaryTombstoneActive(streamKey) {
return nil, true
}
}
// 4. If a stream already finalized this message, delete the placeholder and skip send
if _, loaded := m.streamActive.LoadAndDelete(key); loaded {
if v, loaded := m.placeholders.LoadAndDelete(key); loaded {
if entry, ok := v.(placeholderEntry); ok && entry.id != "" {
// Prefer deleting the placeholder (cleaner UX than editing to same content)
if deleter, ok := ch.(MessageDeleter); ok {
deleter.DeleteMessage(ctx, chatID, entry.id) // best effort
} else if editor, ok := ch.(MessageEditor); ok {
editor.EditMessage(ctx, chatID, entry.id, msg.Content) // fallback
// 4. If a stream already finalized this turn, skip only the duplicate final
// outbound. Earlier queued visible messages must still be delivered.
if isFinalMessage {
if _, loaded := m.streamActive.LoadAndDelete(streamKey); loaded {
if v, loaded := m.placeholders.LoadAndDelete(key); loaded {
if entry, ok := v.(placeholderEntry); ok && entry.id != "" {
// Prefer deleting the placeholder (cleaner UX than editing to same content)
if deleter, ok := ch.(MessageDeleter); ok {
deleter.DeleteMessage(ctx, chatID, entry.id) // best effort
} else if editor, ok := ch.(MessageEditor); ok {
editor.EditMessage(ctx, chatID, entry.id, msg.Content) // fallback
}
}
}
}
if !isToolFeedback {
if separateToolFeedbackMessages {
clearTrackedToolFeedbackMessage(ch, chatID, &msg.Context)
} else {
dismissTrackedToolFeedbackMessage(ctx, ch, chatID, &msg.Context)
if !isToolFeedback {
if separateToolFeedbackMessages {
clearTrackedToolFeedbackMessage(ch, chatID, &msg.Context)
} else {
dismissTrackedToolFeedbackMessage(ctx, ch, chatID, &msg.Context)
}
}
return nil, true
}
return nil, true
}
if _, loaded := m.streamActive.Load(streamKey); loaded {
return nil, false
}
if m.streamActiveForChat(name, chatID) {
return nil, false
}
if !isAuxiliaryMessage {
m.streamAuxiliaryTombstones.Delete(streamKey)
}
if separateToolFeedbackMessages {
@@ -424,6 +470,7 @@ func (m *Manager) preSend(ctx context.Context, name string, msg bus.OutboundMess
func (m *Manager) preSendMedia(ctx context.Context, name string, msg bus.OutboundMediaMessage, ch Channel) {
chatID := outboundMediaChatID(msg)
key := name + ":" + chatID
streamKey := streamSuppressionKey(name, chatID, msg.SessionKey)
// 1. Stop typing
if v, loaded := m.typingStops.LoadAndDelete(key); loaded {
@@ -439,8 +486,9 @@ func (m *Manager) preSendMedia(ctx context.Context, name string, msg bus.Outboun
}
}
// 3. Clear any finalized stream marker for this chat before media delivery.
m.streamActive.LoadAndDelete(key)
// 3. Clear any finalized stream markers for this chat before media delivery.
m.streamActive.LoadAndDelete(streamKey)
m.streamAuxiliaryTombstones.Delete(streamKey)
if m.toolFeedbackSeparateMessagesEnabled() {
clearTrackedToolFeedbackMessage(ch, chatID, &msg.Context)
@@ -507,7 +555,7 @@ func (m *Manager) SetMediaStore(store media.MediaStore) {
// GetStreamer implements bus.StreamDelegate.
// It checks if the named channel supports streaming and returns a Streamer.
func (m *Manager) GetStreamer(ctx context.Context, channelName, chatID string) (bus.Streamer, bool) {
func (m *Manager) GetStreamer(ctx context.Context, channelName, chatID, sessionKey string) (bus.Streamer, bool) {
m.mu.RLock()
ch, exists := m.channels[channelName]
m.mu.RUnlock()
@@ -531,51 +579,295 @@ func (m *Manager) GetStreamer(ctx context.Context, channelName, chatID string) (
}
// Mark streamActive on Finalize so preSend knows to clean up the placeholder
key := channelName + ":" + chatID
return &finalizeHookStreamer{
Streamer: streamer,
onFinalize: func(finalizeCtx context.Context) {
if m.toolFeedbackSeparateMessagesEnabled() {
clearTrackedToolFeedbackMessage(
ch,
chatID,
&bus.InboundContext{
Channel: channelName,
ChatID: chatID,
},
)
} else {
dismissTrackedToolFeedbackMessage(
finalizeCtx,
ch,
chatID,
&bus.InboundContext{
Channel: channelName,
ChatID: chatID,
},
)
// and late auxiliary messages cannot leak after streaming produced a final.
streamKey := streamSuppressionKey(channelName, chatID, sessionKey)
placeholderKey := channelName + ":" + chatID
clearMarker := func() {
m.streamActive.Delete(streamKey)
}
onFinalize := func(finalizeCtx context.Context, finalContent string) {
if m.toolFeedbackSeparateMessagesEnabled() {
clearTrackedToolFeedbackMessage(
ch,
chatID,
&bus.InboundContext{
Channel: channelName,
ChatID: chatID,
},
)
} else {
dismissTrackedToolFeedbackMessage(
finalizeCtx,
ch,
chatID,
&bus.InboundContext{
Channel: channelName,
ChatID: chatID,
},
)
}
if v, loaded := m.placeholders.LoadAndDelete(placeholderKey); loaded {
if entry, ok := v.(placeholderEntry); ok && entry.id != "" {
if deleter, ok := ch.(MessageDeleter); ok {
deleter.DeleteMessage(finalizeCtx, chatID, entry.id) // best effort
} else if editor, ok := ch.(MessageEditor); ok {
editor.EditMessage(finalizeCtx, chatID, entry.id, finalContent) // best effort fallback
}
}
m.streamActive.Store(key, true)
},
}
m.streamActive.Store(streamKey, true)
m.streamAuxiliaryTombstones.Store(streamKey, time.Now())
}
if m.config != nil && m.config.Agents.Defaults.SplitOnMarker {
return &splitMarkerStreamer{
current: streamer,
reasoning: reasoningStreamerFrom(streamer),
begin: func(beginCtx context.Context) (bus.Streamer, error) { return sc.BeginStream(beginCtx, chatID) },
onFinalize: onFinalize,
clearMarker: clearMarker,
}, true
}
return &finalizeHookStreamer{
Streamer: streamer,
clearMarker: clearMarker,
onFinalize: onFinalize,
}, true
}
func reasoningStreamerFrom(streamer bus.Streamer) bus.ReasoningStreamer {
if reasoningStreamer, ok := streamer.(bus.ReasoningStreamer); ok {
return reasoningStreamer
}
return nil
}
// splitMarkerStreamer turns accumulated streaming text containing
// MessageSplitMarker into separate channel stream messages.
type splitMarkerStreamer struct {
mu sync.Mutex
current bus.Streamer
reasoning bus.ReasoningStreamer
begin func(context.Context) (bus.Streamer, error)
completedParts int
finalized bool
onFinalize func(context.Context, string)
clearMarker func()
}
func (s *splitMarkerStreamer) Update(ctx context.Context, content string) error {
s.mu.Lock()
defer s.mu.Unlock()
return s.updateLocked(ctx, content)
}
func (s *splitMarkerStreamer) Finalize(ctx context.Context, content string) error {
return s.FinalizeWithContext(ctx, content, nil)
}
func (s *splitMarkerStreamer) FinalizeWithContext(ctx context.Context, content string, usage *bus.ContextUsage) error {
s.mu.Lock()
defer s.mu.Unlock()
if err := s.finalizeLocked(ctx, content, usage); err != nil {
return err
}
s.runFinalizeHook(ctx, content)
return nil
}
func (s *splitMarkerStreamer) UpdateReasoning(ctx context.Context, content string) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.reasoning == nil {
return nil
}
return s.reasoning.UpdateReasoning(ctx, content)
}
func (s *splitMarkerStreamer) FinalizeReasoning(ctx context.Context, content string) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.reasoning == nil {
return nil
}
return s.reasoning.FinalizeReasoning(ctx, content)
}
func (s *splitMarkerStreamer) Cancel(ctx context.Context) {
s.mu.Lock()
defer s.mu.Unlock()
if s.current != nil {
s.current.Cancel(ctx)
}
}
func (s *splitMarkerStreamer) ClearFinalizedStreamMarker() {
if s.clearMarker != nil {
s.clearMarker()
}
}
func (s *splitMarkerStreamer) updateLocked(ctx context.Context, content string) error {
parts := strings.Split(content, MessageSplitMarker)
completedLimit := len(parts) - 1
if err := s.finalizeCompletedPartsLocked(ctx, parts, completedLimit, nil); err != nil {
return err
}
active := strings.TrimSpace(parts[len(parts)-1])
if active == "" {
return nil
}
if err := s.ensureCurrentLocked(ctx); err != nil {
return err
}
return s.current.Update(ctx, active)
}
func (s *splitMarkerStreamer) finalizeLocked(ctx context.Context, content string, usage *bus.ContextUsage) error {
parts := strings.Split(content, MessageSplitMarker)
return s.finalizeCompletedPartsLocked(ctx, parts, len(parts), usage)
}
func (s *splitMarkerStreamer) finalizeCompletedPartsLocked(
ctx context.Context,
parts []string,
limit int,
usage *bus.ContextUsage,
) error {
for s.completedParts < limit {
content := strings.TrimSpace(parts[s.completedParts])
isLast := s.completedParts == limit-1
if content != "" {
if err := s.ensureCurrentLocked(ctx); err != nil {
return err
}
if isLast && usage != nil {
if contextStreamer, ok := s.current.(bus.ContextUsageStreamer); ok {
if err := contextStreamer.FinalizeWithContext(ctx, content, usage); err != nil {
return err
}
} else if err := s.current.Finalize(ctx, content); err != nil {
return err
}
} else if err := s.current.Finalize(ctx, content); err != nil {
return err
}
s.current = nil
}
s.completedParts++
}
return nil
}
func (s *splitMarkerStreamer) ensureCurrentLocked(ctx context.Context) error {
if s.current != nil {
return nil
}
if s.begin == nil {
return fmt.Errorf("streamer is not initialized")
}
streamer, err := s.begin(ctx)
if err != nil {
return err
}
s.current = streamer
return nil
}
func (s *splitMarkerStreamer) runFinalizeHook(ctx context.Context, content string) {
if s.finalized {
return
}
s.finalized = true
if s.onFinalize != nil {
s.onFinalize(ctx, content)
}
}
func (m *Manager) streamAuxiliaryTombstoneActive(key string) bool {
v, ok := m.streamAuxiliaryTombstones.Load(key)
if !ok {
return false
}
createdAt, ok := v.(time.Time)
if !ok || time.Since(createdAt) > streamAuxiliaryTombstoneTTL {
m.streamAuxiliaryTombstones.Delete(key)
return false
}
return true
}
func (m *Manager) streamActiveForChat(channel, chatID string) bool {
chatKey := streamSuppressionKey(channel, chatID, "")
found := false
m.streamActive.Range(func(key, _ any) bool {
keyString, ok := key.(string)
if !ok {
return true
}
if keyString == chatKey || strings.HasPrefix(keyString, chatKey+":") {
found = true
return false
}
return true
})
return found
}
// finalizeHookStreamer wraps a Streamer to run a hook on Finalize.
type finalizeHookStreamer struct {
Streamer
onFinalize func(context.Context)
onFinalize func(context.Context, string)
clearMarker func()
}
func (s *finalizeHookStreamer) Finalize(ctx context.Context, content string) error {
if err := s.Streamer.Finalize(ctx, content); err != nil {
return err
}
if s.onFinalize != nil {
s.onFinalize(ctx)
s.runFinalizeHook(ctx, content)
return nil
}
func (s *finalizeHookStreamer) FinalizeWithContext(ctx context.Context, content string, usage *bus.ContextUsage) error {
if streamer, ok := s.Streamer.(bus.ContextUsageStreamer); ok {
if err := streamer.FinalizeWithContext(ctx, content, usage); err != nil {
return err
}
} else if err := s.Streamer.Finalize(ctx, content); err != nil {
return err
}
s.runFinalizeHook(ctx, content)
return nil
}
func (s *finalizeHookStreamer) UpdateReasoning(ctx context.Context, content string) error {
if streamer, ok := s.Streamer.(bus.ReasoningStreamer); ok {
return streamer.UpdateReasoning(ctx, content)
}
return nil
}
func (s *finalizeHookStreamer) FinalizeReasoning(ctx context.Context, content string) error {
if streamer, ok := s.Streamer.(bus.ReasoningStreamer); ok {
return streamer.FinalizeReasoning(ctx, content)
}
return nil
}
func (s *finalizeHookStreamer) runFinalizeHook(ctx context.Context, content string) {
if s.onFinalize != nil {
s.onFinalize(ctx, content)
}
}
func (s *finalizeHookStreamer) ClearFinalizedStreamMarker() {
if s.clearMarker != nil {
s.clearMarker()
}
}
// initChannel is a helper that looks up a factory by type name and creates the channel.
// typeName is the channel type used for factory lookup (e.g., "telegram").
// channelName is the config map key used as the channel's runtime name (e.g., "my_telegram").
@@ -1063,7 +1355,11 @@ func (m *Manager) runWorker(ctx context.Context, name string, w *channelWorker)
// Step 1: Try marker-based splitting if enabled.
// Tool feedback must stay a single message, so it skips marker splitting.
if m.config != nil && m.config.Agents.Defaults.SplitOnMarker && !outboundMessageIsToolFeedback(msg) {
// Stream-final duplicate responses must also stay intact so preSend can
// consume the whole final message before any marker chunk leaks.
if m.finalizedStreamActiveForMessage(name, msg) {
chunks = []string{msg.Content}
} else if m.config != nil && m.config.Agents.Defaults.SplitOnMarker && !outboundMessageIsToolFeedback(msg) {
if markerChunks := SplitByMarker(msg.Content); len(markerChunks) > 1 {
for _, chunk := range markerChunks {
chunkMsg := msg
@@ -1090,6 +1386,18 @@ func (m *Manager) runWorker(ctx context.Context, name string, w *channelWorker)
}
}
func (m *Manager) finalizedStreamActiveForMessage(channelName string, msg bus.OutboundMessage) bool {
if m == nil || !outboundMessageIsFinal(msg) {
return false
}
chatID := outboundMessageChatID(msg)
if strings.TrimSpace(channelName) == "" || strings.TrimSpace(chatID) == "" {
return false
}
_, active := m.streamActive.Load(streamSuppressionKey(channelName, chatID, msg.SessionKey))
return active
}
// splitOutboundMessageContent splits regular outbound content by maxLen, but
// keeps tool feedback in a single message by truncating the explanation body.
func splitOutboundMessageContent(msg bus.OutboundMessage, maxLen int) []string {
@@ -1389,9 +1697,9 @@ func (m *Manager) sendMediaWithRetry(
return nil, lastErr
}
// runTTLJanitor periodically scans the typingStops and placeholders maps
// and evicts entries that have exceeded their TTL. This prevents memory
// accumulation when outbound paths fail to trigger preSend (e.g. LLM errors).
// runTTLJanitor periodically scans the typingStops, placeholders, and stream
// tombstone maps and evicts entries that have exceeded their TTL. This prevents
// memory accumulation when outbound paths fail to trigger preSend (e.g. LLM errors).
func (m *Manager) runTTLJanitor(ctx context.Context) {
ticker := time.NewTicker(janitorInterval)
defer ticker.Stop()
@@ -1429,6 +1737,12 @@ func (m *Manager) runTTLJanitor(ctx context.Context) {
}
return true
})
m.streamAuxiliaryTombstones.Range(func(key, value any) bool {
if createdAt, ok := value.(time.Time); !ok || now.Sub(createdAt) > streamAuxiliaryTombstoneTTL {
m.streamAuxiliaryTombstones.Delete(key)
}
return true
})
}
}
}
+631 -6
View File
@@ -104,7 +104,8 @@ func (m *mockDeletingMediaChannel) DismissToolFeedbackMessage(_ context.Context,
}
type mockStreamer struct {
finalizeFn func(context.Context, string) error
finalizeFn func(context.Context, string) error
finalizeWithContextFn func(context.Context, string, *bus.ContextUsage) error
}
func (m *mockStreamer) Update(context.Context, string) error { return nil }
@@ -116,15 +117,68 @@ func (m *mockStreamer) Finalize(ctx context.Context, content string) error {
return nil
}
func (m *mockStreamer) FinalizeWithContext(ctx context.Context, content string, usage *bus.ContextUsage) error {
if m.finalizeWithContextFn != nil {
return m.finalizeWithContextFn(ctx, content, usage)
}
return m.Finalize(ctx, content)
}
func (m *mockStreamer) Cancel(context.Context) {}
type mockReasoningStreamer struct {
mockStreamer
reasoningUpdates []string
reasoningFinal string
}
func (m *mockReasoningStreamer) UpdateReasoning(_ context.Context, content string) error {
m.reasoningUpdates = append(m.reasoningUpdates, content)
return nil
}
func (m *mockReasoningStreamer) FinalizeReasoning(_ context.Context, content string) error {
m.reasoningFinal = content
return nil
}
type recordingStreamSegment struct {
updates []string
finals []string
finalUsage *bus.ContextUsage
canceledCount int
}
func (s *recordingStreamSegment) Update(_ context.Context, content string) error {
s.updates = append(s.updates, content)
return nil
}
func (s *recordingStreamSegment) Finalize(ctx context.Context, content string) error {
return s.FinalizeWithContext(ctx, content, nil)
}
func (s *recordingStreamSegment) FinalizeWithContext(_ context.Context, content string, usage *bus.ContextUsage) error {
s.finals = append(s.finals, content)
s.finalUsage = usage
return nil
}
func (s *recordingStreamSegment) Cancel(context.Context) {
s.canceledCount++
}
type mockStreamingChannel struct {
mockMessageEditor
streamer Streamer
beginStreamFn func(context.Context, string) (Streamer, error)
resolveChatIDFn func(chatID string, outboundCtx *bus.InboundContext) string
}
func (m *mockStreamingChannel) BeginStream(context.Context, string) (Streamer, error) {
func (m *mockStreamingChannel) BeginStream(ctx context.Context, chatID string) (Streamer, error) {
if m.beginStreamFn != nil {
return m.beginStreamFn(ctx, chatID)
}
if m.streamer == nil {
return nil, errors.New("missing streamer")
}
@@ -1476,6 +1530,9 @@ func TestPreSend_StaleToolFeedbackDoesNotConsumeStreamActiveMarker(t *testing.T)
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
Raw: map[string]string{
"outbound_kind": "final",
},
},
})
@@ -1494,6 +1551,206 @@ func TestPreSend_StaleToolFeedbackDoesNotConsumeStreamActiveMarker(t *testing.T)
}
}
func TestPreSend_StaleThoughtDoesNotConsumeStreamActiveMarker(t *testing.T) {
m := newTestManager()
m.streamActive.Store("test:123", true)
m.streamAuxiliaryTombstones.Store("test:123", time.Now())
m.RecordPlaceholder("test", "123", "placeholder-1")
var editedContent string
ch := &mockMessageEditor{
editFn: func(_ context.Context, chatID, messageID, content string) error {
if chatID != "123" || messageID != "placeholder-1" {
t.Fatalf("unexpected edit target: %s/%s", chatID, messageID)
}
editedContent = content
return nil
},
}
thought := testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
Content: "late reasoning",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
Raw: map[string]string{
"message_kind": "thought",
},
},
})
msgIDs, handled := m.preSend(context.Background(), "test", thought, ch)
if !handled {
t.Fatal("expected stale thought to be dropped after stream finalize")
}
if len(msgIDs) != 0 {
t.Fatalf("expected no delivered message IDs for stale thought, got %v", msgIDs)
}
if _, ok := m.streamActive.Load("test:123"); !ok {
t.Fatal("expected streamActive marker to remain for the final outbound message")
}
if _, ok := m.placeholders.Load("test:123"); !ok {
t.Fatal("expected placeholder cleanup to remain deferred to the final outbound message")
}
if ch.editedMessages != 0 {
t.Fatalf("expected no placeholder edit for stale thought, got %d edits", ch.editedMessages)
}
finalMsg := testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
Content: "final streamed reply",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
Raw: map[string]string{
"outbound_kind": "final",
},
},
})
_, handled = m.preSend(context.Background(), "test", finalMsg, ch)
if !handled {
t.Fatal("expected final outbound message to consume streamActive marker")
}
if _, ok := m.streamActive.Load("test:123"); ok {
t.Fatal("expected streamActive marker to be cleared by final outbound message")
}
if _, ok := m.placeholders.Load("test:123"); ok {
t.Fatal("expected placeholder to be cleaned up by final outbound message")
}
if editedContent != "final streamed reply" {
t.Fatalf("editedContent = %q, want final streamed reply", editedContent)
}
lateThought := testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
Content: "later reasoning",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
Raw: map[string]string{
"message_kind": "thought",
},
},
})
msgIDs, handled = m.preSend(context.Background(), "test", lateThought, ch)
if !handled {
t.Fatal("expected tombstone to drop late thought after final outbound was suppressed")
}
if len(msgIDs) != 0 {
t.Fatalf("expected no delivered message IDs for late thought, got %v", msgIDs)
}
}
func TestPreSend_StreamActiveDoesNotConsumeEarlierVisibleMessage(t *testing.T) {
m := newTestManager()
m.streamActive.Store("test:123", true)
m.streamAuxiliaryTombstones.Store("test:123", time.Now())
m.RecordPlaceholder("test", "123", "placeholder-1")
editCalls := 0
ch := &mockMessageEditor{
editFn: func(_ context.Context, chatID, messageID, content string) error {
editCalls++
if chatID != "123" || messageID != "placeholder-1" || content != "final streamed reply" {
t.Fatalf("unexpected placeholder edit for %s/%s: %q", chatID, messageID, content)
}
return nil
},
}
earlierVisible := testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
Content: "earlier visible message",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
},
})
_, handled := m.preSend(context.Background(), "test", earlierVisible, ch)
if handled {
t.Fatal("expected earlier visible message to be delivered normally")
}
if editCalls != 0 {
t.Fatalf("placeholder edits after earlier visible message = %d, want 0", editCalls)
}
if _, ok := m.streamActive.Load("test:123"); !ok {
t.Fatal("expected streamActive marker to remain for final outbound")
}
if _, ok := m.streamAuxiliaryTombstones.Load("test:123"); !ok {
t.Fatal("expected auxiliary tombstone to remain")
}
if _, ok := m.placeholders.Load("test:123"); !ok {
t.Fatal("expected placeholder cleanup to remain deferred to final outbound")
}
finalMsg := testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
Content: "final streamed reply",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
Raw: map[string]string{
"outbound_kind": "final",
},
},
})
_, handled = m.preSend(context.Background(), "test", finalMsg, ch)
if !handled {
t.Fatal("expected final outbound message to consume streamActive marker")
}
if _, ok := m.streamActive.Load("test:123"); ok {
t.Fatal("expected streamActive marker to be cleared by final outbound message")
}
if editCalls != 1 {
t.Fatalf("placeholder edits after final outbound = %d, want 1", editCalls)
}
}
func TestPreSend_StreamActiveDoesNotConsumeOtherSessionFinal(t *testing.T) {
m := newTestManager()
m.streamActive.Store("test:123", true)
m.RecordPlaceholder("test", "123", "placeholder-1")
ch := &mockMessageEditor{
editFn: func(_ context.Context, _, _, _ string) error {
t.Fatal("placeholder edit should remain deferred for the streaming session")
return nil
},
}
otherSessionFinal := testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
SessionKey: "session-other",
Content: "other session final",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
Raw: map[string]string{
"outbound_kind": "final",
},
},
})
_, handled := m.preSend(context.Background(), "test", otherSessionFinal, ch)
if handled {
t.Fatal("expected final outbound from a different session to be delivered normally")
}
if _, ok := m.streamActive.Load("test:123"); !ok {
t.Fatal("expected streaming marker to remain for the streaming session")
}
if _, ok := m.placeholders.Load("test:123"); !ok {
t.Fatal("expected placeholder cleanup to remain deferred to the streaming session")
}
}
func TestPreSendMedia_LeavesTrackedMessageForChannelSend(t *testing.T) {
m := newTestManager()
ch := &mockDeletingMediaChannel{}
@@ -1610,7 +1867,7 @@ func TestGetStreamer_FinalizeDismissesTrackedToolFeedback(t *testing.T) {
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123")
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "")
if !ok {
t.Fatal("expected streamer to be available")
}
@@ -1625,6 +1882,312 @@ func TestGetStreamer_FinalizeDismissesTrackedToolFeedback(t *testing.T) {
}
}
func TestGetStreamer_FinalizeCleansPlaceholderImmediately(t *testing.T) {
m := newTestManager()
m.RecordPlaceholder("test", "123", "placeholder-1")
var editedContent string
editCalls := 0
ch := &mockStreamingChannel{
mockMessageEditor: mockMessageEditor{
editFn: func(_ context.Context, chatID, messageID, content string) error {
if chatID != "123" || messageID != "placeholder-1" {
t.Fatalf("unexpected edit target: %s/%s", chatID, messageID)
}
editCalls++
editedContent = content
return nil
},
},
streamer: &mockStreamer{},
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "")
if !ok {
t.Fatal("expected streamer to be available")
}
if err := streamer.Finalize(context.Background(), "final reply"); err != nil {
t.Fatalf("Finalize() error = %v", err)
}
if editedContent != "final reply" {
t.Fatalf("edited placeholder content = %q, want final reply", editedContent)
}
if _, placeholderExists := m.placeholders.Load("test:123"); placeholderExists {
t.Fatal("expected placeholder to be cleaned up during finalize")
}
if _, streamActiveExists := m.streamActive.Load("test:123"); !streamActiveExists {
t.Fatal("expected streamActive marker to be recorded after finalize")
}
cleaner, ok := streamer.(interface{ ClearFinalizedStreamMarker() })
if !ok {
t.Fatal("expected streamer to expose marker cleanup")
}
cleaner.ClearFinalizedStreamMarker()
if _, streamActiveExists := m.streamActive.Load("test:123"); streamActiveExists {
t.Fatal("expected streamActive marker to be cleared")
}
if _, ok := m.streamAuxiliaryTombstones.Load("test:123"); !ok {
t.Fatal("expected auxiliary tombstone to remain after final marker cleanup")
}
lateThought := testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
Content: "late reasoning",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
Raw: map[string]string{
"message_kind": "thought",
},
},
})
msgIDs, handled := m.preSend(context.Background(), "test", lateThought, ch)
if !handled {
t.Fatal("expected auxiliary tombstone to drop late thought")
}
if len(msgIDs) != 0 {
t.Fatalf("expected no delivered message IDs for late thought, got %v", msgIDs)
}
if editCalls != 1 {
t.Fatalf("expected late thought not to edit placeholder, got %d edits", editCalls)
}
finalOutbound := testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
Content: "visible final reply",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
},
})
_, handled = m.preSend(context.Background(), "test", finalOutbound, ch)
if handled {
t.Fatal("expected cleared final marker to let normal outbound send")
}
if _, ok := m.streamAuxiliaryTombstones.Load("test:123"); ok {
t.Fatal("expected normal outbound to clear auxiliary tombstone")
}
}
func TestGetStreamer_FinalizeCleansPlaceholderWithSessionKey(t *testing.T) {
m := newTestManager()
m.RecordPlaceholder("test", "123", "placeholder-1")
ch := &mockStreamingChannel{
mockMessageEditor: mockMessageEditor{
editFn: func(_ context.Context, chatID, messageID, content string) error {
if chatID != "123" || messageID != "placeholder-1" || content != "final reply" {
t.Fatalf("unexpected edit for %s/%s: %q", chatID, messageID, content)
}
return nil
},
},
streamer: &mockStreamer{},
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "session-1")
if !ok {
t.Fatal("expected streamer to be available")
}
if err := streamer.Finalize(context.Background(), "final reply"); err != nil {
t.Fatalf("Finalize() error = %v", err)
}
if _, placeholderExists := m.placeholders.Load("test:123"); placeholderExists {
t.Fatal("expected placeholder to be cleaned up during finalize")
}
if _, streamActiveExists := m.streamActive.Load("test:123:session-1"); !streamActiveExists {
t.Fatal("expected session streamActive marker to be recorded after finalize")
}
}
func TestGetStreamer_PreservesContextUsageStreamer(t *testing.T) {
m := newTestManager()
var gotUsage *bus.ContextUsage
ch := &mockStreamingChannel{
streamer: &mockStreamer{
finalizeWithContextFn: func(_ context.Context, content string, usage *bus.ContextUsage) error {
if content != "final reply" {
t.Fatalf("unexpected finalize content: %q", content)
}
gotUsage = usage
return nil
},
},
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "")
if !ok {
t.Fatal("expected streamer to be available")
}
contextStreamer, ok := streamer.(bus.ContextUsageStreamer)
if !ok {
t.Fatal("manager-wrapped streamer should preserve ContextUsageStreamer")
}
usage := &bus.ContextUsage{UsedTokens: 10, TotalTokens: 100, CompressAtTokens: 80, UsedPercent: 10}
if err := contextStreamer.FinalizeWithContext(context.Background(), "final reply", usage); err != nil {
t.Fatalf("FinalizeWithContext() error = %v", err)
}
if gotUsage != usage {
t.Fatalf("context usage = %#v, want original usage", gotUsage)
}
if _, ok := m.streamActive.Load("test:123"); !ok {
t.Fatal("expected streamActive marker to be recorded after finalize with context")
}
}
func TestGetStreamer_PreservesReasoningStreamer(t *testing.T) {
m := newTestManager()
inner := &mockReasoningStreamer{}
ch := &mockStreamingChannel{
streamer: inner,
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "")
if !ok {
t.Fatal("expected streamer to be available")
}
reasoningStreamer, ok := streamer.(bus.ReasoningStreamer)
if !ok {
t.Fatal("manager-wrapped streamer should preserve ReasoningStreamer")
}
if err := reasoningStreamer.UpdateReasoning(context.Background(), "thinking"); err != nil {
t.Fatalf("UpdateReasoning() error = %v", err)
}
if err := reasoningStreamer.FinalizeReasoning(context.Background(), "final thought"); err != nil {
t.Fatalf("FinalizeReasoning() error = %v", err)
}
if got := inner.reasoningUpdates; len(got) != 1 || got[0] != "thinking" {
t.Fatalf("reasoning updates = %v, want [thinking]", got)
}
if inner.reasoningFinal != "final thought" {
t.Fatalf("reasoning final = %q, want final thought", inner.reasoningFinal)
}
}
func TestGetStreamer_SplitOnMarkerStreamsSeparateSegments(t *testing.T) {
m := newTestManager()
m.config = &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
SplitOnMarker: true,
},
},
}
var segments []*recordingStreamSegment
ch := &mockStreamingChannel{
beginStreamFn: func(context.Context, string) (Streamer, error) {
segment := &recordingStreamSegment{}
segments = append(segments, segment)
return segment, nil
},
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "session-1")
if !ok {
t.Fatal("expected streamer to be available")
}
contextStreamer, ok := streamer.(bus.ContextUsageStreamer)
if !ok {
t.Fatal("split streamer should preserve ContextUsageStreamer")
}
if err := streamer.Update(context.Background(), "hello"); err != nil {
t.Fatalf("Update(first) error = %v", err)
}
if err := streamer.Update(context.Background(), "hello<|[SPLIT]|>world"); err != nil {
t.Fatalf("Update(split) error = %v", err)
}
if err := streamer.Update(context.Background(), "hello<|[SPLIT]|>world!"); err != nil {
t.Fatalf("Update(second segment) error = %v", err)
}
usage := &bus.ContextUsage{UsedTokens: 10, TotalTokens: 100}
if err := contextStreamer.FinalizeWithContext(
context.Background(),
"hello<|[SPLIT]|>world!",
usage,
); err != nil {
t.Fatalf("FinalizeWithContext() error = %v", err)
}
if len(segments) != 2 {
t.Fatalf("segments = %d, want 2", len(segments))
}
if got := segments[0].updates; len(got) != 1 || got[0] != "hello" {
t.Fatalf("segment 0 updates = %v, want [hello]", got)
}
if got := segments[0].finals; len(got) != 1 || got[0] != "hello" {
t.Fatalf("segment 0 finals = %v, want [hello]", got)
}
if got := segments[1].updates; len(got) != 2 || got[0] != "world" || got[1] != "world!" {
t.Fatalf("segment 1 updates = %v, want [world world!]", got)
}
if got := segments[1].finals; len(got) != 1 || got[0] != "world!" {
t.Fatalf("segment 1 finals = %v, want [world!]", got)
}
if segments[1].finalUsage != usage {
t.Fatalf("final usage = %#v, want original usage", segments[1].finalUsage)
}
if _, ok := m.streamActive.Load("test:123:session-1"); !ok {
t.Fatal("expected streamActive marker to be recorded after split stream finalize")
}
}
func TestGetStreamer_SplitOnMarkerKeepsReasoningOnInitialStreamer(t *testing.T) {
m := newTestManager()
m.config = &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
SplitOnMarker: true,
},
},
}
initial := &mockReasoningStreamer{}
next := &recordingStreamSegment{}
callCount := 0
ch := &mockStreamingChannel{
beginStreamFn: func(context.Context, string) (Streamer, error) {
callCount++
if callCount == 1 {
return initial, nil
}
return next, nil
},
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "")
if !ok {
t.Fatal("expected streamer to be available")
}
if err := streamer.Update(context.Background(), "hello<|[SPLIT]|>world"); err != nil {
t.Fatalf("Update() error = %v", err)
}
reasoningStreamer, ok := streamer.(bus.ReasoningStreamer)
if !ok {
t.Fatal("split streamer should preserve ReasoningStreamer")
}
if err := reasoningStreamer.UpdateReasoning(context.Background(), "thinking"); err != nil {
t.Fatalf("UpdateReasoning() error = %v", err)
}
if err := reasoningStreamer.FinalizeReasoning(context.Background(), "final thought"); err != nil {
t.Fatalf("FinalizeReasoning() error = %v", err)
}
if got := initial.reasoningUpdates; len(got) != 1 || got[0] != "thinking" {
t.Fatalf("initial reasoning updates = %v, want [thinking]", got)
}
if initial.reasoningFinal != "final thought" {
t.Fatalf("initial reasoning final = %q, want final thought", initial.reasoningFinal)
}
}
func TestGetStreamer_FinalizeSeparateMessagesClearsTrackedToolFeedback(t *testing.T) {
m := newTestManager()
m.config = &config.Config{
@@ -1650,7 +2213,7 @@ func TestGetStreamer_FinalizeSeparateMessagesClearsTrackedToolFeedback(t *testin
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123")
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "")
if !ok {
t.Fatal("expected streamer to be available")
}
@@ -1692,7 +2255,7 @@ func TestGetStreamer_FinalizeDismissesResolvedTrackedToolFeedback(t *testing.T)
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "-100123/42")
streamer, ok := m.GetStreamer(context.Background(), "test", "-100123/42", "")
if !ok {
t.Fatal("expected streamer to be available")
}
@@ -1761,7 +2324,7 @@ func TestGetStreamer_FinalizeFailureDoesNotDismissTrackedToolFeedback(t *testing
}
m.channels["test"] = ch
streamer, ok := m.GetStreamer(context.Background(), "test", "123")
streamer, ok := m.GetStreamer(context.Background(), "test", "123", "")
if !ok {
t.Fatal("expected streamer to be available")
}
@@ -1839,6 +2402,68 @@ func TestRunWorker_ToolFeedbackSkipsMarkerSplitting(t *testing.T) {
}
}
func TestRunWorker_FinalizedStreamSuppressesMarkerSplitBeforeSending(t *testing.T) {
m := newTestManager()
m.config = &config.Config{
Agents: config.AgentsConfig{
Defaults: config.AgentDefaults{
SplitOnMarker: true,
},
},
}
var (
mu sync.Mutex
received []string
)
ch := &mockChannel{
sendFn: func(_ context.Context, msg bus.OutboundMessage) error {
mu.Lock()
received = append(received, msg.Content)
mu.Unlock()
return nil
},
}
w := &channelWorker{
ch: ch,
queue: make(chan bus.OutboundMessage, 1),
done: make(chan struct{}),
limiter: rate.NewLimiter(rate.Inf, 1),
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go m.runWorker(ctx, "test", w)
streamKey := streamSuppressionKey("test", "123", "session-1")
m.streamActive.Store(streamKey, true)
w.queue <- testOutboundMessage(bus.OutboundMessage{
Channel: "test",
ChatID: "123",
SessionKey: "session-1",
Content: "streamed full reply<|[SPLIT]|>duplicate chunk",
Context: bus.InboundContext{
Channel: "test",
ChatID: "123",
Raw: map[string]string{
"outbound_kind": "final",
},
},
})
time.Sleep(100 * time.Millisecond)
mu.Lock()
defer mu.Unlock()
if len(received) != 0 {
t.Fatalf("received split duplicate messages = %v, want none", received)
}
if _, ok := m.streamActive.Load(streamKey); ok {
t.Fatal("expected finalized stream marker to be consumed")
}
}
func TestPreSend_PlaceholderEditFails_FallsThrough(t *testing.T) {
m := newTestManager()
+14 -2
View File
@@ -235,6 +235,8 @@ func (c *PicoClientChannel) handleInbound(pc *picoConn, msg PicoMessage) {
case TypeMessageCreate:
// Server sent us a message — treat as inbound
c.handleServerMessage(pc, msg)
case TypeMediaCreate:
c.handleServerMessage(pc, msg)
default:
logger.DebugCF("pico_client", "Ignoring message type", map[string]any{
"type": msg.Type,
@@ -248,7 +250,17 @@ func (c *PicoClientChannel) handleServerMessage(pc *picoConn, msg PicoMessage) {
}
content, _ := msg.Payload[PayloadKeyContent].(string)
if strings.TrimSpace(content) == "" {
media, err := parseInlineImageMedia(msg.Payload)
if err != nil {
logger.WarnCF("pico_client", "Ignoring invalid media payload", map[string]any{
"error": err.Error(),
})
if strings.TrimSpace(content) == "" {
return
}
media = nil
}
if strings.TrimSpace(content) == "" && len(media) == 0 {
return
}
@@ -281,7 +293,7 @@ func (c *PicoClientChannel) handleServerMessage(pc *picoConn, msg PicoMessage) {
},
}
c.HandleInboundContext(c.ctx, chatID, content, nil, inboundCtx, sender)
c.HandleInboundContext(c.ctx, chatID, content, media, inboundCtx, sender)
}
// Send sends a message to the remote server.
+190
View File
@@ -285,6 +285,24 @@ func TestParseInlineImageMedia_Valid(t *testing.T) {
}
}
func TestParseInlineImageMedia_Attachments(t *testing.T) {
imageURL := "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+X2ioAAAAASUVORK5CYII="
media, err := parseInlineImageMedia(map[string]any{
"attachments": []any{
map[string]any{
"type": "image",
"url": imageURL,
},
},
})
if err != nil {
t.Fatalf("parseInlineImageMedia() error = %v", err)
}
if len(media) != 1 || media[0] != imageURL {
t.Fatalf("media = %#v, want attachment image payload", media)
}
}
func TestPicoChannel_HandleMessageSend_AllowsMediaOnly(t *testing.T) {
mb := bus.NewMessageBus()
bc := &config.Channel{Type: "pico", Enabled: true}
@@ -326,6 +344,178 @@ func TestPicoChannel_HandleMessageSend_AllowsMediaOnly(t *testing.T) {
}
}
func newTestPicoClientChannel(t *testing.T) (*PicoClientChannel, *bus.MessageBus) {
t.Helper()
mb := bus.NewMessageBus()
bc := &config.Channel{Type: config.ChannelPicoClient, Enabled: true}
ch, err := NewPicoClientChannel(bc, &config.PicoClientSettings{
URL: "ws://localhost:8080/ws",
}, mb)
if err != nil {
t.Fatalf("NewPicoClientChannel() error = %v", err)
}
ch.ctx = context.Background()
return ch, mb
}
func assertInboundMessage(
t *testing.T,
mb *bus.MessageBus,
wantContent string,
wantMedia []string,
timeoutMessage string,
) {
t.Helper()
select {
case msg := <-mb.InboundChan():
if msg.Content != wantContent {
t.Fatalf("msg.Content = %q, want %s", msg.Content, wantContent)
}
if len(msg.Media) != len(wantMedia) {
t.Fatalf("msg.Media = %#v, want %#v", msg.Media, wantMedia)
}
for i := range wantMedia {
if msg.Media[i] != wantMedia[i] {
t.Fatalf("msg.Media = %#v, want %#v", msg.Media, wantMedia)
}
}
case <-time.After(time.Second):
t.Fatal(timeoutMessage)
}
}
func TestPicoClientChannel_HandleServerMessage_ForwardsMedia(t *testing.T) {
ch, mb := newTestPicoClientChannel(t)
pc := &picoConn{sessionID: "sess-media"}
imageURL := "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+X2ioAAAAASUVORK5CYII="
ch.handleServerMessage(pc, PicoMessage{
Type: TypeMessageCreate,
Payload: map[string]any{
PayloadKeyContent: "describe this",
"attachments": []any{
map[string]any{
"type": "image",
"url": imageURL,
},
},
},
})
assertInboundMessage(
t,
mb,
"describe this",
[]string{imageURL},
"timed out waiting for forwarded media message",
)
}
func TestPicoClientChannel_HandleInbound_ForwardsMediaCreate(t *testing.T) {
ch, mb := newTestPicoClientChannel(t)
pc := &picoConn{sessionID: "sess-media-create"}
imageURL := "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+X2ioAAAAASUVORK5CYII="
ch.handleInbound(pc, PicoMessage{
Type: TypeMediaCreate,
Payload: map[string]any{
PayloadKeyContent: "describe media.create",
"attachments": []any{
map[string]any{
"type": "image",
"url": imageURL,
},
},
},
})
assertInboundMessage(
t,
mb,
"describe media.create",
[]string{imageURL},
"timed out waiting for media.create message",
)
}
func TestPicoClientChannel_HandleServerMessage_ForwardsTextWithDownloadAttachment(t *testing.T) {
mb := bus.NewMessageBus()
bc := &config.Channel{Type: config.ChannelPicoClient, Enabled: true}
ch, err := NewPicoClientChannel(bc, &config.PicoClientSettings{
URL: "ws://localhost:8080/ws",
}, mb)
if err != nil {
t.Fatalf("NewPicoClientChannel() error = %v", err)
}
ch.ctx = context.Background()
pc := &picoConn{sessionID: "sess-download-attachment"}
ch.handleServerMessage(pc, PicoMessage{
Type: TypeMessageCreate,
Payload: map[string]any{
PayloadKeyContent: "see attached",
"attachments": []any{
map[string]any{
"type": "image",
"url": "/pico/media/abc",
"filename": "image.png",
"content_type": "image/png",
},
},
},
})
select {
case msg := <-mb.InboundChan():
if msg.Content != "see attached" {
t.Fatalf("msg.Content = %q, want see attached", msg.Content)
}
if len(msg.Media) != 0 {
t.Fatalf("msg.Media = %#v, want no inline media", msg.Media)
}
case <-time.After(time.Second):
t.Fatal("timed out waiting for text message with download attachment")
}
}
func TestPicoClientChannel_HandleServerMessage_ForwardsTextWithInvalidMediaPayload(t *testing.T) {
mb := bus.NewMessageBus()
bc := &config.Channel{Type: config.ChannelPicoClient, Enabled: true}
ch, err := NewPicoClientChannel(bc, &config.PicoClientSettings{
URL: "ws://localhost:8080/ws",
}, mb)
if err != nil {
t.Fatalf("NewPicoClientChannel() error = %v", err)
}
ch.ctx = context.Background()
pc := &picoConn{sessionID: "sess-invalid-media"}
ch.handleServerMessage(pc, PicoMessage{
Type: TypeMessageCreate,
Payload: map[string]any{
PayloadKeyContent: "hello despite invalid media",
"attachments": "not-an-array",
},
})
select {
case msg := <-mb.InboundChan():
if msg.Content != "hello despite invalid media" {
t.Fatalf("msg.Content = %q, want hello despite invalid media", msg.Content)
}
if len(msg.Media) != 0 {
t.Fatalf("msg.Media = %#v, want no inline media", msg.Media)
}
case <-time.After(time.Second):
t.Fatal("timed out waiting for text message with invalid media payload")
}
}
func TestIsThoughtPayload(t *testing.T) {
tests := []struct {
name string
+249 -5
View File
@@ -464,8 +464,9 @@ func (c *PicoChannel) SendPlaceholder(ctx context.Context, chatID string) (strin
msgID := uuid.New().String()
outMsg := newMessage(TypeMessageCreate, map[string]any{
PayloadKeyContent: text,
"message_id": msgID,
PayloadKeyContent: text,
PayloadKeyPlaceholder: true,
"message_id": msgID,
})
if err := c.broadcastToSession(chatID, outMsg); err != nil {
@@ -475,6 +476,195 @@ func (c *PicoChannel) SendPlaceholder(ctx context.Context, chatID string) (strin
return msgID, nil
}
// BeginStream implements channels.StreamingCapable for Pico WebUI.
func (c *PicoChannel) BeginStream(ctx context.Context, chatID string) (channels.Streamer, error) {
if c == nil || c.config == nil || !c.config.Streaming.Enabled {
return nil, fmt.Errorf("streaming disabled in config")
}
if !c.IsRunning() {
return nil, channels.ErrNotRunning
}
streamCfg := c.config.Streaming.WithDefaults(0, 1)
return &picoStreamer{
channel: c,
chatID: chatID,
throttleInterval: time.Duration(streamCfg.ThrottleSeconds) * time.Second,
minGrowth: streamCfg.MinGrowthChars,
}, nil
}
type picoStreamer struct {
channel *PicoChannel
chatID string
messageID string
reasoningID string
throttleInterval time.Duration
minGrowth int
lastLen int
lastAt time.Time
lastContent string
reasoningLastLen int
reasoningLastAt time.Time
reasoningContent string
mu sync.Mutex
}
func (s *picoStreamer) Update(ctx context.Context, content string) error {
s.mu.Lock()
defer s.mu.Unlock()
return s.updateLocked(ctx, content, false, nil)
}
func (s *picoStreamer) Finalize(ctx context.Context, content string) error {
return s.FinalizeWithContext(ctx, content, nil)
}
func (s *picoStreamer) FinalizeWithContext(ctx context.Context, content string, contextUsage *bus.ContextUsage) error {
s.mu.Lock()
defer s.mu.Unlock()
return s.updateLocked(ctx, content, true, contextUsage)
}
func (s *picoStreamer) UpdateReasoning(ctx context.Context, content string) error {
s.mu.Lock()
defer s.mu.Unlock()
return s.updateReasoningLocked(ctx, content, false)
}
func (s *picoStreamer) FinalizeReasoning(ctx context.Context, content string) error {
s.mu.Lock()
defer s.mu.Unlock()
return s.updateReasoningLocked(ctx, content, true)
}
func (s *picoStreamer) Cancel(ctx context.Context) {
if s == nil {
return
}
s.mu.Lock()
defer s.mu.Unlock()
if s.channel == nil || s.messageID == "" {
if s.channel != nil && s.reasoningID != "" {
_ = s.channel.DeleteMessage(ctx, s.chatID, s.reasoningID)
s.reasoningID = ""
}
return
}
_ = s.channel.DeleteMessage(ctx, s.chatID, s.messageID)
s.messageID = ""
if s.reasoningID != "" {
_ = s.channel.DeleteMessage(ctx, s.chatID, s.reasoningID)
s.reasoningID = ""
}
}
func (s *picoStreamer) updateLocked(
ctx context.Context,
content string,
force bool,
contextUsage *bus.ContextUsage,
) error {
if s == nil || s.channel == nil {
return fmt.Errorf("streamer is not initialized")
}
if strings.TrimSpace(content) == "" && s.messageID == "" {
return nil
}
now := time.Now()
contentLen := len([]rune(content))
if s.messageID != "" && !force {
growth := contentLen - s.lastLen
if now.Sub(s.lastAt) < s.throttleInterval || growth < s.minGrowth {
return nil
}
}
return s.sendLocked(ctx, content, contextUsage)
}
func (s *picoStreamer) updateReasoningLocked(ctx context.Context, content string, force bool) error {
if s == nil || s.channel == nil {
return fmt.Errorf("streamer is not initialized")
}
if strings.TrimSpace(content) == "" && s.reasoningID == "" {
return nil
}
now := time.Now()
contentLen := len([]rune(content))
if s.reasoningID != "" && !force {
growth := contentLen - s.reasoningLastLen
if now.Sub(s.reasoningLastAt) < s.throttleInterval || growth < s.minGrowth {
return nil
}
}
return s.sendReasoningLocked(ctx, content)
}
func (s *picoStreamer) sendLocked(ctx context.Context, content string, contextUsage *bus.ContextUsage) error {
now := time.Now()
contentLen := len([]rune(content))
if s.messageID == "" {
s.messageID = uuid.New().String()
payload := map[string]any{
PayloadKeyContent: content,
"message_id": s.messageID,
}
setContextUsagePayload(payload, contextUsage)
outMsg := newMessage(TypeMessageCreate, payload)
if err := s.channel.broadcastToSession(s.chatID, outMsg); err != nil {
return err
}
} else if content != s.lastContent || contextUsage != nil {
if err := s.channel.editMessage(ctx, s.chatID, s.messageID, content, contextUsage); err != nil {
return err
}
}
s.lastContent = content
s.lastLen = contentLen
s.lastAt = now
return nil
}
func (s *picoStreamer) sendReasoningLocked(ctx context.Context, content string) error {
now := time.Now()
contentLen := len([]rune(content))
if s.reasoningID == "" {
s.reasoningID = uuid.New().String()
payload := map[string]any{
PayloadKeyContent: content,
"message_id": s.reasoningID,
PayloadKeyKind: MessageKindThought,
PayloadKeyThought: true,
}
outMsg := newMessage(TypeMessageCreate, payload)
if err := s.channel.broadcastToSession(s.chatID, outMsg); err != nil {
return err
}
} else if content != s.reasoningContent {
payload := map[string]any{
PayloadKeyContent: content,
"message_id": s.reasoningID,
PayloadKeyKind: MessageKindThought,
PayloadKeyThought: true,
}
outMsg := newMessage(TypeMessageUpdate, payload)
if err := s.channel.broadcastToSession(s.chatID, outMsg); err != nil {
return err
}
}
s.reasoningContent = content
s.reasoningLastLen = contentLen
s.reasoningLastAt = now
return nil
}
// SendMedia implements channels.MediaSender for the Pico web UI.
// Media is delivered as a normal assistant message carrying structured
// attachments plus an authenticated same-origin download URL.
@@ -990,11 +1180,24 @@ func parseInlineImageMedia(payload map[string]any) ([]string, error) {
return nil, nil
}
raw, ok := payload["media"]
if !ok || raw == nil {
return nil, nil
media, err := parseInlineImageValues(payload["media"])
if err != nil {
return nil, err
}
attachments, err := parseInlineImageAttachments(payload["attachments"])
if err != nil {
return nil, err
}
media = append(media, attachments...)
return media, nil
}
func parseInlineImageValues(raw any) ([]string, error) {
if raw == nil {
return nil, nil
}
switch values := raw.(type) {
case []any:
media := make([]string, 0, len(values))
@@ -1030,6 +1233,47 @@ func parseInlineImageMedia(payload map[string]any) ([]string, error) {
}
}
func parseInlineImageAttachments(raw any) ([]string, error) {
if raw == nil {
return nil, nil
}
values, ok := raw.([]any)
if !ok {
return nil, fmt.Errorf("attachments must be an array")
}
media := make([]string, 0, len(values))
for i, item := range values {
attachment, ok := item.(map[string]any)
if !ok {
return nil, fmt.Errorf("attachments[%d]: attachment must be an object", i)
}
attachmentType, _ := attachment["type"].(string)
attachmentType = strings.ToLower(strings.TrimSpace(attachmentType))
if attachmentType != "" && attachmentType != "image" {
continue
}
value, err := inlineImageValue(attachment)
if err != nil {
if attachmentType == "image" {
return nil, fmt.Errorf("attachments[%d]: %w", i, err)
}
continue
}
if !strings.HasPrefix(value, "data:") {
continue
}
if err := validateInlineImageDataURL(value); err != nil {
return nil, fmt.Errorf("attachments[%d]: %w", i, err)
}
media = append(media, value)
}
return media, nil
}
func inlineImageValue(item any) (string, error) {
switch value := item.(type) {
case string:
+296
View File
@@ -226,6 +226,9 @@ func TestSendPlaceholder_EmitsNormalMessageWithoutKind(t *testing.T) {
if got := payload[PayloadKeyContent]; got != "Thinking..." {
t.Fatalf("placeholder content = %#v, want %q", got, "Thinking...")
}
if got := payload[PayloadKeyPlaceholder]; got != true {
t.Fatalf("placeholder marker = %#v, want true", got)
}
if got, ok := payload[PayloadKeyKind]; ok {
t.Fatalf("placeholder kind = %#v, want absent", got)
}
@@ -234,6 +237,279 @@ func TestSendPlaceholder_EmitsNormalMessageWithoutKind(t *testing.T) {
}
}
func TestBeginStream_CreatesAndUpdatesSameMessage(t *testing.T) {
ch := newTestPicoChannel(t)
ch.config.Streaming = config.StreamingConfig{
Enabled: true,
ThrottleSeconds: 1,
MinGrowthChars: 1,
}
if err := ch.Start(context.Background()); err != nil {
t.Fatalf("Start() error = %v", err)
}
defer ch.Stop(context.Background())
clientConn, received, cleanup := newTestPicoWebSocket(t)
defer cleanup()
ch.addConnForTest(&picoConn{id: "conn-1", conn: clientConn, sessionID: "sess-1"})
streamer, err := ch.BeginStream(context.Background(), "pico:sess-1")
if err != nil {
t.Fatalf("BeginStream() error = %v", err)
}
if err := streamer.Update(context.Background(), "hello"); err != nil {
t.Fatalf("Update(first) error = %v", err)
}
first := mustReceivePicoMessage(t, received)
if first.Type != TypeMessageCreate {
t.Fatalf("first type = %q, want %q", first.Type, TypeMessageCreate)
}
msgID, _ := first.Payload["message_id"].(string)
if msgID == "" {
t.Fatalf("first message_id = %#v, want non-empty", first.Payload["message_id"])
}
if got := first.Payload[PayloadKeyContent]; got != "hello" {
t.Fatalf("first content = %#v, want hello", got)
}
rawStreamer := streamer.(*picoStreamer)
rawStreamer.mu.Lock()
rawStreamer.lastAt = time.Now().Add(-2 * time.Second)
rawStreamer.mu.Unlock()
secondContent := "hello world with enough growth to pass the default streaming threshold"
if err := streamer.Update(context.Background(), secondContent); err != nil {
t.Fatalf("Update(second) error = %v", err)
}
second := mustReceivePicoMessage(t, received)
if second.Type != TypeMessageUpdate {
t.Fatalf("second type = %q, want %q", second.Type, TypeMessageUpdate)
}
if got := second.Payload["message_id"]; got != msgID {
t.Fatalf("second message_id = %#v, want %q", got, msgID)
}
if got := second.Payload[PayloadKeyContent]; got != secondContent {
t.Fatalf("second content = %#v, want %q", got, secondContent)
}
}
func TestBeginStream_DefaultStreamingShowsSmallIncrements(t *testing.T) {
ch := newTestPicoChannel(t)
ch.config.Streaming = config.StreamingConfig{Enabled: true}
if err := ch.Start(context.Background()); err != nil {
t.Fatalf("Start() error = %v", err)
}
defer ch.Stop(context.Background())
clientConn, received, cleanup := newTestPicoWebSocket(t)
defer cleanup()
ch.addConnForTest(&picoConn{id: "conn-1", conn: clientConn, sessionID: "sess-1"})
streamer, err := ch.BeginStream(context.Background(), "pico:sess-1")
if err != nil {
t.Fatalf("BeginStream() error = %v", err)
}
if err := streamer.Update(context.Background(), "h"); err != nil {
t.Fatalf("Update(first) error = %v", err)
}
first := mustReceivePicoMessage(t, received)
if first.Type != TypeMessageCreate {
t.Fatalf("first type = %q, want %q", first.Type, TypeMessageCreate)
}
msgID, _ := first.Payload["message_id"].(string)
if msgID == "" {
t.Fatalf("first message_id = %#v, want non-empty", first.Payload["message_id"])
}
if err := streamer.Update(context.Background(), "he"); err != nil {
t.Fatalf("Update(second) error = %v", err)
}
second := mustReceivePicoMessage(t, received)
if second.Type != TypeMessageUpdate {
t.Fatalf("second type = %q, want %q", second.Type, TypeMessageUpdate)
}
if got := second.Payload["message_id"]; got != msgID {
t.Fatalf("second message_id = %#v, want %q", got, msgID)
}
if got := second.Payload[PayloadKeyContent]; got != "he" {
t.Fatalf("second content = %#v, want he", got)
}
}
func TestBeginStream_StreamsReasoningAsThoughtUpdates(t *testing.T) {
ch := newTestPicoChannel(t)
ch.config.Streaming = config.StreamingConfig{Enabled: true}
if err := ch.Start(context.Background()); err != nil {
t.Fatalf("Start() error = %v", err)
}
defer ch.Stop(context.Background())
clientConn, received, cleanup := newTestPicoWebSocket(t)
defer cleanup()
ch.addConnForTest(&picoConn{id: "conn-1", conn: clientConn, sessionID: "sess-1"})
streamer, err := ch.BeginStream(context.Background(), "pico:sess-1")
if err != nil {
t.Fatalf("BeginStream() error = %v", err)
}
reasoningStreamer, ok := streamer.(bus.ReasoningStreamer)
if !ok {
t.Fatal("pico stream should support reasoning updates")
}
if err := reasoningStreamer.UpdateReasoning(context.Background(), "thinking"); err != nil {
t.Fatalf("UpdateReasoning(first) error = %v", err)
}
first := mustReceivePicoMessage(t, received)
if first.Type != TypeMessageCreate {
t.Fatalf("first type = %q, want %q", first.Type, TypeMessageCreate)
}
msgID, _ := first.Payload["message_id"].(string)
if msgID == "" {
t.Fatalf("first message_id = %#v, want non-empty", first.Payload["message_id"])
}
if got := first.Payload[PayloadKeyKind]; got != MessageKindThought {
t.Fatalf("first kind = %#v, want %q", got, MessageKindThought)
}
if got := first.Payload[PayloadKeyContent]; got != "thinking" {
t.Fatalf("first content = %#v, want thinking", got)
}
if err := reasoningStreamer.UpdateReasoning(context.Background(), "thinking more"); err != nil {
t.Fatalf("UpdateReasoning(second) error = %v", err)
}
second := mustReceivePicoMessage(t, received)
if second.Type != TypeMessageUpdate {
t.Fatalf("second type = %q, want %q", second.Type, TypeMessageUpdate)
}
if got := second.Payload["message_id"]; got != msgID {
t.Fatalf("second message_id = %#v, want %q", got, msgID)
}
if got := second.Payload[PayloadKeyKind]; got != MessageKindThought {
t.Fatalf("second kind = %#v, want %q", got, MessageKindThought)
}
if got := second.Payload[PayloadKeyContent]; got != "thinking more" {
t.Fatalf("second content = %#v, want thinking more", got)
}
}
func TestBeginStream_ThrottlesIntermediateUpdatesAndFinalFlushes(t *testing.T) {
ch := newTestPicoChannel(t)
ch.config.Streaming = config.StreamingConfig{
Enabled: true,
ThrottleSeconds: 60,
MinGrowthChars: 100,
}
if err := ch.Start(context.Background()); err != nil {
t.Fatalf("Start() error = %v", err)
}
defer ch.Stop(context.Background())
clientConn, received, cleanup := newTestPicoWebSocket(t)
defer cleanup()
ch.addConnForTest(&picoConn{id: "conn-1", conn: clientConn, sessionID: "sess-1"})
streamer, err := ch.BeginStream(context.Background(), "pico:sess-1")
if err != nil {
t.Fatalf("BeginStream() error = %v", err)
}
if err := streamer.Update(context.Background(), "first"); err != nil {
t.Fatalf("Update(first) error = %v", err)
}
if err := streamer.Update(context.Background(), "first plus short growth"); err != nil {
t.Fatalf("Update(throttled) error = %v", err)
}
if err := streamer.Update(context.Background(), "first"+strings.Repeat("x", 120)); err != nil {
t.Fatalf("Update(enough growth too soon) error = %v", err)
}
first := mustReceivePicoMessage(t, received)
if first.Type != TypeMessageCreate {
t.Fatalf("first type = %q, want %q", first.Type, TypeMessageCreate)
}
msgID, _ := first.Payload["message_id"].(string)
assertNoPicoMessage(t, received)
rawStreamer := streamer.(*picoStreamer)
rawStreamer.mu.Lock()
rawStreamer.lastAt = time.Now().Add(-61 * time.Second)
rawStreamer.mu.Unlock()
if err := streamer.Update(context.Background(), "first plus small growth"); err != nil {
t.Fatalf("Update(enough time too little growth) error = %v", err)
}
assertNoPicoMessage(t, received)
if err := streamer.Finalize(context.Background(), "first plus final text"); err != nil {
t.Fatalf("Finalize() error = %v", err)
}
final := mustReceivePicoMessage(t, received)
if final.Type != TypeMessageUpdate {
t.Fatalf("final type = %q, want %q", final.Type, TypeMessageUpdate)
}
if got := final.Payload["message_id"]; got != msgID {
t.Fatalf("final message_id = %#v, want %q", got, msgID)
}
if got := final.Payload[PayloadKeyContent]; got != "first plus final text" {
t.Fatalf("final content = %#v, want final text", got)
}
assertNoPicoMessage(t, received)
}
func TestBeginStream_FinalizeIncludesContextUsage(t *testing.T) {
ch := newTestPicoChannel(t)
ch.config.Streaming = config.StreamingConfig{
Enabled: true,
ThrottleSeconds: 0,
MinGrowthChars: 0,
}
if err := ch.Start(context.Background()); err != nil {
t.Fatalf("Start() error = %v", err)
}
defer ch.Stop(context.Background())
clientConn, received, cleanup := newTestPicoWebSocket(t)
defer cleanup()
ch.addConnForTest(&picoConn{id: "conn-1", conn: clientConn, sessionID: "sess-1"})
streamer, err := ch.BeginStream(context.Background(), "pico:sess-1")
if err != nil {
t.Fatalf("BeginStream() error = %v", err)
}
if err := streamer.Update(context.Background(), "partial"); err != nil {
t.Fatalf("Update() error = %v", err)
}
first := mustReceivePicoMessage(t, received)
msgID, _ := first.Payload["message_id"].(string)
contextStreamer, ok := streamer.(interface {
FinalizeWithContext(ctx context.Context, content string, usage *bus.ContextUsage) error
})
if !ok {
t.Fatal("streamer should support FinalizeWithContext")
}
if err := contextStreamer.FinalizeWithContext(context.Background(), "final", &bus.ContextUsage{
UsedTokens: 10,
TotalTokens: 100,
CompressAtTokens: 80,
UsedPercent: 10,
}); err != nil {
t.Fatalf("FinalizeWithContext() error = %v", err)
}
final := mustReceivePicoMessage(t, received)
if final.Type != TypeMessageUpdate {
t.Fatalf("final type = %q, want %q", final.Type, TypeMessageUpdate)
}
if got := final.Payload["message_id"]; got != msgID {
t.Fatalf("final message_id = %#v, want %q", got, msgID)
}
rawUsage, ok := final.Payload["context_usage"].(map[string]any)
if !ok {
t.Fatalf("final context_usage = %#v, want map", final.Payload["context_usage"])
}
if got := rawUsage["used_tokens"]; got != float64(10) {
t.Fatalf("used_tokens = %#v, want 10", got)
}
}
func TestCreateAndAddConnection_RespectsMaxConnectionsConcurrently(t *testing.T) {
ch := newTestPicoChannel(t)
@@ -491,6 +767,26 @@ func TestHandleMediaDownload_ServesStoredFile(t *testing.T) {
}
}
func mustReceivePicoMessage(t *testing.T, received <-chan PicoMessage) PicoMessage {
t.Helper()
select {
case msg := <-received:
return msg
case <-time.After(time.Second):
t.Fatal("expected pico message")
}
return PicoMessage{}
}
func assertNoPicoMessage(t *testing.T, received <-chan PicoMessage) {
t.Helper()
select {
case msg := <-received:
t.Fatalf("unexpected pico message: %+v", msg)
case <-time.After(150 * time.Millisecond):
}
}
func (c *PicoChannel) addConnForTest(pc *picoConn) {
c.connsMu.Lock()
defer c.connsMu.Unlock()
+5 -4
View File
@@ -22,10 +22,11 @@ const (
TypeError = "error"
TypePong = "pong"
PayloadKeyContent = "content"
PayloadKeyThought = "thought"
PayloadKeyKind = "kind"
PayloadKeyToolCalls = "tool_calls"
PayloadKeyContent = "content"
PayloadKeyThought = "thought"
PayloadKeyKind = "kind"
PayloadKeyPlaceholder = "placeholder"
PayloadKeyToolCalls = "tool_calls"
MessageKindThought = "thought"
MessageKindToolCalls = "tool_calls"
+2 -2
View File
@@ -191,7 +191,7 @@ func (c *SlackChannel) SendMedia(ctx context.Context, msg bus.OutboundMediaMessa
title = filename
}
_, err = c.api.UploadFileV2Context(ctx, slack.UploadFileV2Parameters{
_, err = c.api.UploadFileContext(ctx, slack.UploadFileParameters{
Channel: channelID,
ThreadTimestamp: threadTS,
File: localPath,
@@ -207,7 +207,7 @@ func (c *SlackChannel) SendMedia(ctx context.Context, msg bus.OutboundMediaMessa
}
}
// UploadFileV2 does not expose the posted message timestamp in its
// UploadFile does not expose the posted message timestamp in its
// response; returning nil avoids conflating file IDs with message IDs.
return nil, nil
}
+30 -7
View File
@@ -1471,7 +1471,7 @@ func (c *TelegramChannel) BeginStream(ctx context.Context, chatID string) (chann
return nil, err
}
streamCfg := c.tgCfg.Streaming
streamCfg := c.tgCfg.Streaming.WithDefaults(3, 200)
return &telegramStreamer{
bot: c.bot,
chatID: cid,
@@ -1483,8 +1483,8 @@ func (c *TelegramChannel) BeginStream(ctx context.Context, chatID string) (chann
}
// telegramStreamer streams partial LLM output via Telegram's sendMessageDraft API.
// On first API error (e.g. bot lacks forum mode), it silently degrades: Update
// becomes a no-op, while Finalize still delivers the final message.
// Draft update failures are returned to the agent, which decides whether the
// stream was already visible enough to keep or should fall back to Chat().
type telegramStreamer struct {
bot *telego.Bot
chatID int64
@@ -1495,6 +1495,7 @@ type telegramStreamer struct {
lastLen int
lastAt time.Time
failed bool
draftTouched bool
mu sync.Mutex
}
@@ -1503,7 +1504,7 @@ func (s *telegramStreamer) Update(ctx context.Context, content string) error {
defer s.mu.Unlock()
if s.failed {
return nil
return fmt.Errorf("telegram streaming disabled after previous draft failure")
}
// Throttle: skip if not enough time or content has passed
@@ -1514,6 +1515,7 @@ func (s *telegramStreamer) Update(ctx context.Context, content string) error {
}
htmlContent := markdownToTelegramHTML(content)
s.draftTouched = true
err := s.bot.SendMessageDraft(ctx, &telego.SendMessageDraftParams{
ChatID: s.chatID,
@@ -1523,12 +1525,11 @@ func (s *telegramStreamer) Update(ctx context.Context, content string) error {
ParseMode: telego.ModeHTML,
})
if err != nil {
// First error → degrade silently (e.g. no forum mode)
logger.WarnCF("telegram", "sendMessageDraft failed, disabling streaming", map[string]any{
"error": err.Error(),
})
s.failed = true
return nil // don't propagate — Finalize will still deliver
return fmt.Errorf("telegram draft update: %w", err)
}
s.lastLen = len(content)
@@ -1554,11 +1555,33 @@ func (s *telegramStreamer) Finalize(ctx context.Context, content string) error {
return fmt.Errorf("telegram finalize: %w", err)
}
}
s.Cancel(ctx)
return nil
}
func (s *telegramStreamer) Cancel(ctx context.Context) {
// Draft auto-expires on Telegram's side; nothing to clean up.
s.mu.Lock()
defer s.mu.Unlock()
s.clearDraft(ctx)
}
func (s *telegramStreamer) clearDraft(ctx context.Context) {
if !s.draftTouched {
return
}
if err := s.bot.SendMessageDraft(ctx, &telego.SendMessageDraftParams{
ChatID: s.chatID,
MessageThreadID: s.threadID,
DraftID: s.draftID,
Text: " ",
}); err != nil {
logger.DebugCF("telegram", "failed to clear streaming draft", map[string]any{
"chat_id": s.chatID,
"error": err.Error(),
})
}
s.lastLen = 0
s.draftTouched = false
}
// cryptoRandInt returns a non-zero random int using crypto/rand.
+114
View File
@@ -762,6 +762,120 @@ func TestBeginStream_UpdateUsesForumThreadID(t *testing.T) {
assert.Equal(t, "partial", params.Text)
}
func TestBeginStream_UsesDefaultThrottleWhenOnlyEnabled(t *testing.T) {
caller := &stubCaller{
callFn: func(ctx context.Context, url string, data *ta.RequestData) (*ta.Response, error) {
return &ta.Response{Ok: true, Result: []byte("true")}, nil
},
}
ch := newTestChannel(t, caller)
ch.tgCfg.Streaming = config.StreamingConfig{Enabled: true}
streamer, err := ch.BeginStream(context.Background(), "12345")
require.NoError(t, err)
require.NoError(t, streamer.Update(context.Background(), "partial"))
require.NoError(t, streamer.Update(context.Background(), "partial plus one"))
require.Len(t, caller.calls, 1, "second small update should be throttled by defaults")
}
func TestBeginStream_UpdateReturnsErrorWhenDraftFails(t *testing.T) {
callCount := 0
caller := &stubCaller{
callFn: func(ctx context.Context, url string, data *ta.RequestData) (*ta.Response, error) {
callCount++
if callCount == 1 {
return nil, errors.New("draft unsupported")
}
return &ta.Response{Ok: true, Result: []byte("true")}, nil
},
}
ch := newTestChannel(t, caller)
ch.tgCfg.Streaming = config.StreamingConfig{Enabled: true}
streamer, err := ch.BeginStream(context.Background(), "12345")
require.NoError(t, err)
err = streamer.Update(context.Background(), "partial")
require.Error(t, err)
assert.Contains(t, err.Error(), "draft unsupported")
streamer.Cancel(context.Background())
require.Len(t, caller.calls, 2)
assert.Contains(t, caller.calls[1].URL, "sendMessageDraft")
var params struct {
ChatID int64 `json:"chat_id"`
DraftID int `json:"draft_id"`
Text string `json:"text"`
}
require.NoError(t, json.Unmarshal(caller.calls[1].Data.BodyRaw, &params))
assert.Equal(t, int64(12345), params.ChatID)
assert.NotZero(t, params.DraftID)
assert.Equal(t, " ", params.Text)
}
func TestBeginStream_CancelClearsExistingDraft(t *testing.T) {
caller := &stubCaller{
callFn: func(ctx context.Context, url string, data *ta.RequestData) (*ta.Response, error) {
return &ta.Response{Ok: true, Result: []byte("true")}, nil
},
}
ch := newTestChannel(t, caller)
ch.tgCfg.Streaming = config.StreamingConfig{Enabled: true}
streamer, err := ch.BeginStream(context.Background(), "12345")
require.NoError(t, err)
require.NoError(t, streamer.Update(context.Background(), "partial"))
streamer.Cancel(context.Background())
require.Len(t, caller.calls, 2)
assert.Contains(t, caller.calls[1].URL, "sendMessageDraft")
var params struct {
ChatID int64 `json:"chat_id"`
DraftID int `json:"draft_id"`
Text string `json:"text"`
}
require.NoError(t, json.Unmarshal(caller.calls[1].Data.BodyRaw, &params))
assert.Equal(t, int64(12345), params.ChatID)
assert.NotZero(t, params.DraftID)
assert.Equal(t, " ", params.Text)
}
func TestBeginStream_FinalizeClearsExistingDraft(t *testing.T) {
caller := &stubCaller{
callFn: func(ctx context.Context, url string, data *ta.RequestData) (*ta.Response, error) {
if strings.Contains(url, "sendMessage") && !strings.Contains(url, "sendMessageDraft") {
return successResponse(t), nil
}
return &ta.Response{Ok: true, Result: []byte("true")}, nil
},
}
ch := newTestChannel(t, caller)
ch.tgCfg.Streaming = config.StreamingConfig{Enabled: true}
streamer, err := ch.BeginStream(context.Background(), "12345")
require.NoError(t, err)
require.NoError(t, streamer.Update(context.Background(), "partial"))
require.NoError(t, streamer.Finalize(context.Background(), "final"))
require.Len(t, caller.calls, 3)
assert.Contains(t, caller.calls[0].URL, "sendMessageDraft")
assert.Contains(t, caller.calls[1].URL, "sendMessage")
assert.Contains(t, caller.calls[2].URL, "sendMessageDraft")
var params struct {
ChatID int64 `json:"chat_id"`
DraftID int `json:"draft_id"`
Text string `json:"text"`
}
require.NoError(t, json.Unmarshal(caller.calls[2].Data.BodyRaw, &params))
assert.Equal(t, int64(12345), params.ChatID)
assert.NotZero(t, params.DraftID)
assert.Equal(t, " ", params.Text)
}
func TestBeginStream_FinalizeUsesForumThreadID(t *testing.T) {
caller := &stubCaller{
callFn: func(ctx context.Context, url string, data *ta.RequestData) (*ta.Response, error) {
+3
View File
@@ -164,6 +164,9 @@ func (c *WeComChannel) Stop(_ context.Context) error {
}
func (c *WeComChannel) BeginStream(_ context.Context, chatID string) (channels.Streamer, error) {
if c == nil || c.config == nil || !c.config.Streaming.Enabled {
return nil, fmt.Errorf("streaming disabled in config")
}
if !c.IsRunning() {
return nil, channels.ErrNotRunning
}
+19
View File
@@ -100,6 +100,7 @@ func TestBeginStream_UpdateAndFinalize(t *testing.T) {
t.Parallel()
ch := newTestWeComChannel(t, bus.NewMessageBus())
ch.config.Streaming.Enabled = true
ch.SetRunning(true)
ch.queueTurn("chat-1", wecomTurn{
ReqID: "req-1",
@@ -158,6 +159,24 @@ func TestBeginStream_UpdateAndFinalize(t *testing.T) {
}
}
func TestBeginStream_RequiresStreamingEnabled(t *testing.T) {
t.Parallel()
ch := newTestWeComChannel(t, bus.NewMessageBus())
ch.SetRunning(true)
ch.queueTurn("chat-1", wecomTurn{
ReqID: "req-1",
ChatID: "chat-1",
ChatType: 1,
StreamID: "stream-1",
CreatedAt: time.Now(),
})
if _, err := ch.BeginStream(context.Background(), "chat-1"); err == nil {
t.Fatal("BeginStream() error = nil, want disabled streaming error")
}
}
func TestSend_StreamFailureFallsBackToActualChatID(t *testing.T) {
t.Parallel()
+77 -33
View File
@@ -468,9 +468,25 @@ func (p *PlaceholderConfig) GetRandomText() string {
}
type StreamingConfig struct {
Enabled bool `json:"enabled,omitempty" env:"PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED"`
ThrottleSeconds int `json:"throttle_seconds,omitempty" env:"PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS"`
MinGrowthChars int `json:"min_growth_chars,omitempty" env:"PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS"`
Enabled bool `json:"enabled,omitempty"`
ThrottleSeconds int `json:"throttle_seconds,omitempty"`
MinGrowthChars int `json:"min_growth_chars,omitempty"`
}
func (c StreamingConfig) IsZero() bool {
return !c.Enabled && c.ThrottleSeconds == 0 && c.MinGrowthChars == 0
}
func (c StreamingConfig) WithDefaults(throttleSeconds, minGrowthChars int) StreamingConfig {
if c.Enabled {
if c.ThrottleSeconds == 0 {
c.ThrottleSeconds = throttleSeconds
}
if c.MinGrowthChars == 0 {
c.MinGrowthChars = minGrowthChars
}
}
return c
}
type WhatsAppSettings struct {
@@ -483,7 +499,7 @@ type TelegramSettings struct {
Token SecureString `json:"token,omitzero" yaml:"token,omitempty" env:"PICOCLAW_CHANNELS_TELEGRAM_TOKEN"`
BaseURL string `json:"base_url" yaml:"-" env:"PICOCLAW_CHANNELS_TELEGRAM_BASE_URL"`
Proxy string `json:"proxy" yaml:"-" env:"PICOCLAW_CHANNELS_TELEGRAM_PROXY"`
Streaming StreamingConfig `json:"streaming,omitempty" yaml:"-"`
Streaming StreamingConfig `json:"streaming,omitzero" yaml:"-"`
UseMarkdownV2 bool `json:"use_markdown_v2" yaml:"-" env:"PICOCLAW_CHANNELS_TELEGRAM_USE_MARKDOWN_V2"`
MediaGroupDelayMS int `json:"media_group_delay_ms" yaml:"-" env:"PICOCLAW_CHANNELS_TELEGRAM_MEDIA_GROUP_DELAY_MS"`
}
@@ -557,10 +573,11 @@ type WeComGroupConfig struct {
}
type WeComSettings struct {
BotID string `json:"bot_id" yaml:"-" env:"BOT_ID"`
Secret SecureString `json:"secret,omitzero" yaml:"secret,omitempty" env:"SECRET"`
WebSocketURL string `json:"websocket_url,omitempty" yaml:"-" env:"WEBSOCKET_URL"`
SendThinkingMessage bool `json:"send_thinking_message" yaml:"-" env:"SEND_THINKING_MESSAGE"`
BotID string `json:"bot_id" yaml:"-" env:"BOT_ID"`
Secret SecureString `json:"secret,omitzero" yaml:"secret,omitempty" env:"SECRET"`
WebSocketURL string `json:"websocket_url,omitempty" yaml:"-" env:"WEBSOCKET_URL"`
SendThinkingMessage bool `json:"send_thinking_message" yaml:"-" env:"SEND_THINKING_MESSAGE"`
Streaming StreamingConfig `json:"streaming,omitzero" yaml:"-"`
}
func (c *WeComSettings) SetSecret(secret string) {
@@ -581,13 +598,14 @@ func (c *WeixinSettings) SetToken(token string) {
}
type PicoSettings struct {
Token SecureString `json:"token,omitzero" yaml:"token,omitempty" env:"PICOCLAW_CHANNELS_PICO_TOKEN"`
AllowTokenQuery bool `json:"allow_token_query,omitempty" yaml:"-"`
AllowOrigins []string `json:"allow_origins,omitempty" yaml:"-"`
PingInterval int `json:"ping_interval,omitempty" yaml:"-"`
ReadTimeout int `json:"read_timeout,omitempty" yaml:"-"`
WriteTimeout int `json:"write_timeout,omitempty" yaml:"-"`
MaxConnections int `json:"max_connections,omitempty" yaml:"-"`
Token SecureString `json:"token,omitzero" yaml:"token,omitempty" env:"PICOCLAW_CHANNELS_PICO_TOKEN"`
AllowTokenQuery bool `json:"allow_token_query,omitempty" yaml:"-"`
AllowOrigins []string `json:"allow_origins,omitempty" yaml:"-"`
Streaming StreamingConfig `json:"streaming,omitzero" yaml:"-"`
PingInterval int `json:"ping_interval,omitempty" yaml:"-"`
ReadTimeout int `json:"read_timeout,omitempty" yaml:"-"`
WriteTimeout int `json:"write_timeout,omitempty" yaml:"-"`
MaxConnections int `json:"max_connections,omitempty" yaml:"-"`
}
// SetToken sets the Pico token and marks it as dirty for security saving
@@ -678,6 +696,14 @@ type VoiceConfig struct {
ElevenLabsAPIKey string `json:"elevenlabs_api_key,omitempty" env:"PICOCLAW_VOICE_ELEVENLABS_API_KEY"`
}
type ModelStreamingConfig struct {
Enabled bool `json:"enabled,omitempty"`
}
func (c ModelStreamingConfig) IsZero() bool {
return !c.Enabled
}
// ModelConfig represents a model-centric provider configuration.
// It allows adding new providers (especially OpenAI-compatible ones) via configuration only.
// The Model field may be either a plain model identifier or a provider-prefixed
@@ -702,13 +728,14 @@ type ModelConfig struct {
Workspace string `json:"workspace,omitempty"` // Workspace path for CLI-based providers
// Optional optimizations
RPM int `json:"rpm,omitempty"` // Requests per minute limit
MaxTokensField string `json:"max_tokens_field,omitempty"` // Field name for max tokens (e.g., "max_completion_tokens")
RequestTimeout int `json:"request_timeout,omitempty"`
ThinkingLevel string `json:"thinking_level,omitempty"` // Extended thinking: off|low|medium|high|xhigh|adaptive
ToolSchemaTransform string `json:"tool_schema_transform,omitempty"` // Optional tool schema compatibility transform (e.g. "simple")
ExtraBody map[string]any `json:"extra_body,omitempty"` // Additional fields to inject into request body
CustomHeaders map[string]string `json:"custom_headers,omitempty"` // Additional headers to inject into every HTTP request
RPM int `json:"rpm,omitempty"` // Requests per minute limit
MaxTokensField string `json:"max_tokens_field,omitempty"` // Field name for max tokens (e.g., "max_completion_tokens")
RequestTimeout int `json:"request_timeout,omitempty"`
ThinkingLevel string `json:"thinking_level,omitempty"` // Extended thinking: off|low|medium|high|xhigh|adaptive
ToolSchemaTransform string `json:"tool_schema_transform,omitempty"` // Optional tool schema compatibility transform (e.g. "simple")
Streaming ModelStreamingConfig `json:"streaming,omitzero"` // Opt-in for provider streaming on this model entry
ExtraBody map[string]any `json:"extra_body,omitempty"` // Additional fields to inject into request body
CustomHeaders map[string]string `json:"custom_headers,omitempty"` // Additional headers to inject into every HTTP request
APIKeys SecureStrings `json:"api_keys,omitzero" yaml:"api_keys,omitempty"` // API authentication keys (multiple keys for failover)
@@ -847,6 +874,13 @@ type SogouConfig struct {
MaxResults int `json:"max_results" env:"PICOCLAW_TOOLS_WEB_SOGOU_MAX_RESULTS"`
}
type GeminiSearchConfig struct {
Enabled bool `json:"enabled" yaml:"-" env:"PICOCLAW_TOOLS_WEB_GEMINI_ENABLED"`
APIKey SecureString `json:"api_key,omitzero" yaml:"api_key,omitempty" env:"PICOCLAW_TOOLS_WEB_GEMINI_API_KEY"`
Model string `json:"model" yaml:"-" env:"PICOCLAW_TOOLS_WEB_GEMINI_MODEL"`
MaxResults int `json:"max_results" yaml:"-" env:"PICOCLAW_TOOLS_WEB_GEMINI_MAX_RESULTS"`
}
type PerplexityConfig struct {
Enabled bool `json:"enabled" yaml:"-" env:"PICOCLAW_TOOLS_WEB_PERPLEXITY_ENABLED"`
APIKeys SecureStrings `json:"api_keys,omitzero" yaml:"api_keys,omitempty" env:"PICOCLAW_TOOLS_WEB_PERPLEXITY_API_KEYS"`
@@ -890,16 +924,17 @@ type BaiduSearchConfig struct {
}
type WebToolsConfig struct {
ToolConfig ` yaml:"-" envPrefix:"PICOCLAW_TOOLS_WEB_"`
Brave BraveConfig `yaml:"brave,omitempty" json:"brave"`
Tavily TavilyConfig `yaml:"tavily,omitempty" json:"tavily"`
Sogou SogouConfig `yaml:"-" json:"sogou"`
DuckDuckGo DuckDuckGoConfig `yaml:"-" json:"duckduckgo"`
Perplexity PerplexityConfig `yaml:"perplexity,omitempty" json:"perplexity"`
SearXNG SearXNGConfig `yaml:"-" json:"searxng"`
GLMSearch GLMSearchConfig `yaml:"glm_search,omitempty" json:"glm_search"`
BaiduSearch BaiduSearchConfig `yaml:"baidu_search,omitempty" json:"baidu_search"`
Provider string `yaml:"-" json:"provider,omitempty" env:"PICOCLAW_TOOLS_WEB_PROVIDER"`
ToolConfig ` yaml:"-" envPrefix:"PICOCLAW_TOOLS_WEB_"`
Brave BraveConfig `yaml:"brave,omitempty" json:"brave"`
Tavily TavilyConfig `yaml:"tavily,omitempty" json:"tavily"`
Sogou SogouConfig `yaml:"-" json:"sogou"`
DuckDuckGo DuckDuckGoConfig `yaml:"-" json:"duckduckgo"`
Gemini GeminiSearchConfig `yaml:"gemini,omitempty" json:"gemini"`
Perplexity PerplexityConfig `yaml:"perplexity,omitempty" json:"perplexity"`
SearXNG SearXNGConfig `yaml:"-" json:"searxng"`
GLMSearch GLMSearchConfig `yaml:"glm_search,omitempty" json:"glm_search"`
BaiduSearch BaiduSearchConfig `yaml:"baidu_search,omitempty" json:"baidu_search"`
Provider string `yaml:"-" json:"provider,omitempty" env:"PICOCLAW_TOOLS_WEB_PROVIDER"`
// PreferNative controls whether to use provider-native web search when
// the active LLM supports it (e.g. OpenAI web_search_preview). When true,
// the client-side web_search tool is hidden to avoid duplicate search surfaces,
@@ -989,6 +1024,7 @@ type ToolsConfig struct {
I2C ToolConfig `json:"i2c" yaml:"-" envPrefix:"PICOCLAW_TOOLS_I2C_"`
InstallSkill ToolConfig `json:"install_skill" yaml:"-" envPrefix:"PICOCLAW_TOOLS_INSTALL_SKILL_"`
ListDir ToolConfig `json:"list_dir" yaml:"-" envPrefix:"PICOCLAW_TOOLS_LIST_DIR_"`
LoadImage ToolConfig `json:"load_image" yaml:"-" envPrefix:"PICOCLAW_TOOLS_LOAD_IMAGE_"`
Message ToolConfig `json:"message" yaml:"-" envPrefix:"PICOCLAW_TOOLS_MESSAGE_"`
ReadFile ReadFileToolConfig `json:"read_file" yaml:"-" envPrefix:"PICOCLAW_TOOLS_READ_FILE_"`
Serial ToolConfig `json:"serial" yaml:"-" envPrefix:"PICOCLAW_TOOLS_SERIAL_"`
@@ -1118,7 +1154,11 @@ type MCPServerConfig struct {
Env map[string]string `json:"env,omitempty"`
// EnvFile is the path to a file containing environment variables (stdio only)
EnvFile string `json:"env_file,omitempty"`
// Type is "stdio", "sse", or "http" (default: stdio if command is set, sse if url is set)
// Type is "stdio", "sse", "http", or "streamable-http".
// "http" and "streamable-http" both select streamable HTTP request-response
// mode, while "sse" keeps the standalone SSE listener enabled for
// server-initiated notifications. Defaults: stdio if command is set, sse if
// url is set.
Type string `json:"type,omitempty"`
// URL is used for SSE/HTTP transport
URL string `json:"url,omitempty"`
@@ -1648,6 +1688,7 @@ func expandMultiKeyModels(models []*ModelConfig) []*ModelConfig {
RequestTimeout: m.RequestTimeout,
ThinkingLevel: m.ThinkingLevel,
ToolSchemaTransform: m.ToolSchemaTransform,
Streaming: m.Streaming,
ExtraBody: m.ExtraBody,
CustomHeaders: m.CustomHeaders,
UserAgent: m.UserAgent,
@@ -1672,6 +1713,7 @@ func expandMultiKeyModels(models []*ModelConfig) []*ModelConfig {
RequestTimeout: m.RequestTimeout,
ThinkingLevel: m.ThinkingLevel,
ToolSchemaTransform: m.ToolSchemaTransform,
Streaming: m.Streaming,
ExtraBody: m.ExtraBody,
CustomHeaders: m.CustomHeaders,
UserAgent: m.UserAgent,
@@ -1715,6 +1757,8 @@ func (t *ToolsConfig) IsToolEnabled(name string) bool {
return t.InstallSkill.Enabled
case "list_dir":
return t.ListDir.Enabled
case "load_image":
return t.LoadImage.Enabled
case "message":
return t.Message.Enabled
case "read_file":
+82
View File
@@ -3,7 +3,9 @@ package config
import (
"encoding/json"
"fmt"
"os"
"reflect"
"strconv"
"strings"
"github.com/caarlos0/env/v11"
@@ -239,6 +241,7 @@ func (b Channel) MarshalJSON() ([]byte, error) {
if err != nil {
return nil, err
}
raw = preserveExplicitDisabledStreaming(raw, b.Settings)
settings = raw
} else {
settings = b.Settings
@@ -252,6 +255,36 @@ func (b Channel) MarshalJSON() ([]byte, error) {
return json.Marshal((*Alias)(&out))
}
func preserveExplicitDisabledStreaming(settings, original RawNode) RawNode {
if len(original) == 0 || len(settings) == 0 {
return settings
}
var originalMap map[string]any
if err := json.Unmarshal(original, &originalMap); err != nil {
return settings
}
originalStreaming, ok := originalMap["streaming"].(map[string]any)
if !ok || originalStreaming["enabled"] != false {
return settings
}
var settingsMap map[string]any
if err := json.Unmarshal(settings, &settingsMap); err != nil {
return settings
}
if _, exists := settingsMap["streaming"]; exists {
return settings
}
settingsMap["streaming"] = map[string]any{"enabled": false}
data, err := json.Marshal(settingsMap)
if err != nil {
return settings
}
return data
}
// MarshalYAML implements yaml.ValueMarshaler for Channel.
// Outputs only secure fields in the Settings YAML (for .security.yml).
// If Decode was called, it serializes from the stored extend (reflecting any
@@ -696,6 +729,10 @@ func InitChannelList(channels ChannelsConfig) error {
if err := env.Parse(target); err != nil {
// Non-fatal: some env vars may not apply
}
applyTelegramStreamingEnvCompat(target)
if err := validateChannelStreamingConfig(name, target); err != nil {
return err
}
}
}
@@ -706,3 +743,48 @@ func InitChannelList(channels ChannelsConfig) error {
return nil
}
func applyTelegramStreamingEnvCompat(target any) {
settings, ok := target.(*TelegramSettings)
if !ok || settings == nil {
return
}
if raw, ok := os.LookupEnv("PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED"); ok {
if value, err := strconv.ParseBool(raw); err == nil {
settings.Streaming.Enabled = value
}
}
if raw, ok := os.LookupEnv("PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS"); ok {
if value, err := strconv.Atoi(raw); err == nil {
settings.Streaming.ThrottleSeconds = value
}
}
if raw, ok := os.LookupEnv("PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS"); ok {
if value, err := strconv.Atoi(raw); err == nil {
settings.Streaming.MinGrowthChars = value
}
}
}
func validateChannelStreamingConfig(channelName string, target any) error {
var streaming StreamingConfig
switch settings := target.(type) {
case *PicoSettings:
streaming = settings.Streaming
case *TelegramSettings:
streaming = settings.Streaming
case *WeComSettings:
streaming = settings.Streaming
default:
return nil
}
if streaming.ThrottleSeconds < 0 {
return fmt.Errorf("channel %q streaming.throttle_seconds must be >= 0", channelName)
}
if streaming.MinGrowthChars < 0 {
return fmt.Errorf("channel %q streaming.min_growth_chars must be >= 0", channelName)
}
return nil
}
+190
View File
@@ -2,6 +2,7 @@ package config
import (
"encoding/json"
"reflect"
"strings"
"testing"
@@ -133,6 +134,179 @@ func TestChannel_JSON_Unmarshal(t *testing.T) {
assert.Equal(t, "", cfg.Token.String())
}
func TestStreamingConfig_IsChannelGeneric(t *testing.T) {
typ := reflect.TypeOf(StreamingConfig{})
for i := 0; i < typ.NumField(); i++ {
field := typ.Field(i)
if got := field.Tag.Get("env"); got != "" {
t.Fatalf("StreamingConfig.%s env tag = %q, want no channel-specific env tag", field.Name, got)
}
}
}
func TestPicoSettings_StreamingConfig(t *testing.T) {
raw := RawNode(`{
"token": "test-token",
"streaming": {
"enabled": true,
"throttle_seconds": 2,
"min_growth_chars": 80
}
}`)
ch := &Channel{
Type: ChannelPico,
Enabled: true,
Settings: raw,
}
ch.SetName("pico")
var picoCfg PicoSettings
if err := ch.Decode(&picoCfg); err != nil {
t.Fatalf("Decode() error = %v", err)
}
assert.True(t, picoCfg.Streaming.Enabled)
assert.Equal(t, 2, picoCfg.Streaming.ThrottleSeconds)
assert.Equal(t, 80, picoCfg.Streaming.MinGrowthChars)
}
func TestWeComSettings_StreamingConfig(t *testing.T) {
raw := RawNode(`{
"bot_id": "bot-1",
"streaming": {
"enabled": true,
"throttle_seconds": 4,
"min_growth_chars": 160
}
}`)
ch := &Channel{
Type: ChannelWeCom,
Enabled: true,
Settings: raw,
}
ch.SetName("wecom")
var wecomCfg WeComSettings
if err := ch.Decode(&wecomCfg); err != nil {
t.Fatalf("Decode() error = %v", err)
}
assert.True(t, wecomCfg.Streaming.Enabled)
assert.Equal(t, 4, wecomCfg.Streaming.ThrottleSeconds)
assert.Equal(t, 160, wecomCfg.Streaming.MinGrowthChars)
}
func TestPicoStreamingConfig_Defaults(t *testing.T) {
cfg := StreamingConfig{Enabled: true}
got := cfg.WithDefaults(1, 40)
assert.Equal(t, 1, got.ThrottleSeconds)
assert.Equal(t, 40, got.MinGrowthChars)
cfg = StreamingConfig{Enabled: true, ThrottleSeconds: 5, MinGrowthChars: 200}
got = cfg.WithDefaults(1, 40)
assert.Equal(t, 5, got.ThrottleSeconds)
assert.Equal(t, 200, got.MinGrowthChars)
cfg = StreamingConfig{}
got = cfg.WithDefaults(1, 40)
assert.Equal(t, 0, got.ThrottleSeconds)
assert.Equal(t, 0, got.MinGrowthChars)
}
func TestInitChannelList_TelegramStreamingEnvCompatibility(t *testing.T) {
t.Setenv("PICOCLAW_CHANNELS_TELEGRAM_STREAMING_ENABLED", "true")
t.Setenv("PICOCLAW_CHANNELS_TELEGRAM_STREAMING_THROTTLE_SECONDS", "3")
t.Setenv("PICOCLAW_CHANNELS_TELEGRAM_STREAMING_MIN_GROWTH_CHARS", "120")
channels := ChannelsConfig{
"telegram": {
Type: ChannelTelegram,
Enabled: true,
Settings: RawNode(`{"token":"telegram-token"}`),
},
"pico": {
Type: ChannelPico,
Enabled: true,
Settings: RawNode(`{"token":"pico-token"}`),
},
}
if err := InitChannelList(channels); err != nil {
t.Fatalf("InitChannelList() error = %v", err)
}
tgDecoded, err := channels["telegram"].GetDecoded()
if err != nil {
t.Fatalf("telegram GetDecoded() error = %v", err)
}
tgCfg := tgDecoded.(*TelegramSettings)
assert.True(t, tgCfg.Streaming.Enabled)
assert.Equal(t, 3, tgCfg.Streaming.ThrottleSeconds)
assert.Equal(t, 120, tgCfg.Streaming.MinGrowthChars)
picoDecoded, err := channels["pico"].GetDecoded()
if err != nil {
t.Fatalf("pico GetDecoded() error = %v", err)
}
picoCfg := picoDecoded.(*PicoSettings)
assert.False(t, picoCfg.Streaming.Enabled)
assert.Equal(t, 0, picoCfg.Streaming.ThrottleSeconds)
assert.Equal(t, 0, picoCfg.Streaming.MinGrowthChars)
}
func TestInitChannelList_RejectsNegativeStreamingDeliveryValues(t *testing.T) {
tests := []struct {
name string
channelType string
settings string
}{
{
name: "pico throttle",
channelType: ChannelPico,
settings: `{"token":"pico-token","streaming":{"enabled":true,"throttle_seconds":-1}}`,
},
{
name: "pico growth",
channelType: ChannelPico,
settings: `{"token":"pico-token","streaming":{"enabled":true,"min_growth_chars":-1}}`,
},
{
name: "telegram throttle",
channelType: ChannelTelegram,
settings: `{"token":"telegram-token","streaming":{"enabled":true,"throttle_seconds":-1}}`,
},
{
name: "telegram growth",
channelType: ChannelTelegram,
settings: `{"token":"telegram-token","streaming":{"enabled":true,"min_growth_chars":-1}}`,
},
{
name: "wecom throttle",
channelType: ChannelWeCom,
settings: `{"bot_id":"bot-1","streaming":{"enabled":true,"throttle_seconds":-1}}`,
},
{
name: "wecom growth",
channelType: ChannelWeCom,
settings: `{"bot_id":"bot-1","streaming":{"enabled":true,"min_growth_chars":-1}}`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
channels := ChannelsConfig{
tt.channelType: {
Type: tt.channelType,
Enabled: true,
Settings: RawNode(tt.settings),
},
}
err := InitChannelList(channels)
if err == nil {
t.Fatal("InitChannelList() error = nil, want validation error")
}
if !strings.Contains(err.Error(), "streaming.") {
t.Fatalf("InitChannelList() error = %v, want streaming field error", err)
}
})
}
}
// ═══════════════════════════════════════════════════
// JSON marshal: secure fields masked as [NOT_HERE]
// ═══════════════════════════════════════════════════
@@ -161,6 +335,22 @@ func TestChannel_JSON_Marshal_SecureMasked(t *testing.T) {
assert.Contains(t, string(data), "proxy")
}
func TestChannel_JSON_Marshal_OmitsUnconfiguredStreaming(t *testing.T) {
ch := Channel{
Enabled: true,
Type: ChannelPico,
name: "pico",
Settings: mustParseRawNode(`{"ping_interval":30}`),
}
var cfg PicoSettings
require.NoError(t, ch.Decode(&cfg))
data, err := json.MarshalIndent(ch, "", " ")
require.NoError(t, err)
assert.NotContains(t, string(data), `"streaming"`)
}
// ═══════════════════════════════════════════════════
// YAML unmarshal: security.yml — only secure data
// ═══════════════════════════════════════════════════
+109
View File
@@ -836,6 +836,42 @@ func TestDefaultConfig_Channels(t *testing.T) {
}
}
func TestDefaultConfig_ChannelStreamingDisabled(t *testing.T) {
cfg := DefaultConfig()
telegram := cfg.Channels.Get(ChannelTelegram)
if telegram == nil {
t.Fatal("DefaultConfig() missing telegram channel")
}
decoded, err := telegram.GetDecoded()
if err != nil {
t.Fatalf("telegram GetDecoded() error = %v", err)
}
settings, ok := decoded.(*TelegramSettings)
if !ok {
t.Fatalf("telegram settings type = %T, want *TelegramSettings", decoded)
}
if settings.Streaming.Enabled {
t.Fatal("DefaultConfig().telegram.settings.streaming.enabled should be false")
}
pico := cfg.Channels.Get(ChannelPico)
if pico == nil {
t.Fatal("DefaultConfig() missing pico channel")
}
decoded, err = pico.GetDecoded()
if err != nil {
t.Fatalf("pico GetDecoded() error = %v", err)
}
picoSettings, ok := decoded.(*PicoSettings)
if !ok {
t.Fatalf("pico settings type = %T, want *PicoSettings", decoded)
}
if !picoSettings.Streaming.Enabled {
t.Fatal("DefaultConfig().pico.settings.streaming.enabled should be true")
}
}
func TestValidateSingletonChannels_RejectsMultipleInstances(t *testing.T) {
channels := ChannelsConfig{
"pico1": &Channel{Enabled: true, Type: ChannelPico},
@@ -975,6 +1011,49 @@ func TestSaveConfig_PreservesDisabledTelegramPlaceholder(t *testing.T) {
}
}
func TestSaveConfig_PreservesExplicitDisabledPicoStreaming(t *testing.T) {
tmpDir := t.TempDir()
path := filepath.Join(tmpDir, "config.json")
cfg := DefaultConfig()
pico := cfg.Channels.Get(ChannelPico)
if pico == nil {
t.Fatal("DefaultConfig() missing pico channel")
}
pico.Settings = RawNode(`{"streaming":{"enabled":false}}`)
if err := SaveConfig(path, cfg); err != nil {
t.Fatalf("SaveConfig failed: %v", err)
}
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile failed: %v", err)
}
if !strings.Contains(string(data), `"streaming"`) || !strings.Contains(string(data), `"enabled": false`) {
t.Fatalf("saved config should preserve explicit disabled pico streaming, got:\n%s", string(data))
}
loaded, err := LoadConfig(path)
if err != nil {
t.Fatalf("LoadConfig failed: %v", err)
}
loadedPico := loaded.Channels.Get(ChannelPico)
if loadedPico == nil {
t.Fatal("loaded config missing pico channel")
}
decoded, err := loadedPico.GetDecoded()
if err != nil {
t.Fatalf("pico GetDecoded() error = %v", err)
}
settings, ok := decoded.(*PicoSettings)
if !ok {
t.Fatalf("pico settings type = %T, want *PicoSettings", decoded)
}
if settings.Streaming.Enabled {
t.Fatal("explicit disabled pico streaming should remain disabled after SaveConfig/LoadConfig round-trip")
}
}
// TestSaveConfig_FiltersVirtualModels verifies that SaveConfig does not write
// virtual models (generated by expandMultiKeyModels) to the config file.
func TestSaveConfig_FiltersVirtualModels(t *testing.T) {
@@ -1247,6 +1326,36 @@ func TestDefaultConfig_FilterMinLength(t *testing.T) {
}
}
func TestDefaultConfig_LoadImageEnabled(t *testing.T) {
cfg := DefaultConfig()
if !cfg.Tools.LoadImage.Enabled {
t.Fatal("DefaultConfig().Tools.LoadImage.Enabled should be true")
}
if !cfg.Tools.IsToolEnabled("load_image") {
t.Fatal("DefaultConfig().Tools.IsToolEnabled(load_image) should be true")
}
}
func TestLoadConfig_LoadImageCanBeDisabled(t *testing.T) {
dir := t.TempDir()
configPath := filepath.Join(dir, "config.json")
raw := "{\n \"version\": 2,\n \"tools\": {\n \"load_image\": {\n \"enabled\": false\n }\n }\n}\n"
if err := os.WriteFile(configPath, []byte(raw), 0o600); err != nil {
t.Fatalf("WriteFile() error: %v", err)
}
cfg, err := LoadConfig(configPath)
if err != nil {
t.Fatalf("LoadConfig() error: %v", err)
}
if cfg.Tools.LoadImage.Enabled {
t.Fatal("LoadConfig().Tools.LoadImage.Enabled should be false")
}
if cfg.Tools.IsToolEnabled("load_image") {
t.Fatal("LoadConfig().Tools.IsToolEnabled(load_image) should be false")
}
}
func TestToolsConfig_GetFilterMinLength(t *testing.T) {
tests := []struct {
name string
+9 -1
View File
@@ -341,6 +341,11 @@ func DefaultConfig() *Config {
Enabled: false,
MaxResults: 5,
},
Gemini: GeminiSearchConfig{
Enabled: false,
Model: "gemini-2.5-flash",
MaxResults: 5,
},
Perplexity: PerplexityConfig{
Enabled: false,
MaxResults: 5,
@@ -439,6 +444,9 @@ func DefaultConfig() *Config {
ListDir: ToolConfig{
Enabled: true,
},
LoadImage: ToolConfig{
Enabled: true,
},
Message: ToolConfig{
Enabled: true,
},
@@ -503,7 +511,6 @@ func defaultChannels() ChannelsConfig {
"typing": map[string]any{"enabled": true},
"placeholder": map[string]any{"enabled": true, "text": []string{"Thinking... 💭"}},
"settings": map[string]any{
"streaming": map[string]any{"enabled": true, "throttle_seconds": 3, "min_growth_chars": 200},
"use_markdown_v2": false,
"media_group_delay_ms": 500,
},
@@ -558,6 +565,7 @@ func defaultChannels() ChannelsConfig {
"read_timeout": 60,
"write_timeout": 10,
"max_connections": 100,
"streaming": map[string]any{"enabled": true},
},
},
"irc": map[string]any{
+32
View File
@@ -0,0 +1,32 @@
package config
import "strings"
// NormalizeMCPTransportType canonicalizes MCP transport names used in config.
// "http" is PicoClaw's streamable HTTP request-response mode, and
// "streamable-http" is accepted as an explicit alias for the same transport.
func NormalizeMCPTransportType(transport string) string {
normalized := strings.ToLower(strings.TrimSpace(transport))
switch normalized {
case "streamable-http", "streamable_http", "streamablehttp":
return "http"
default:
return normalized
}
}
// EffectiveMCPTransportType returns the normalized configured transport, or the
// inferred default when the config leaves Type empty.
func EffectiveMCPTransportType(server MCPServerConfig) string {
if transport := NormalizeMCPTransportType(server.Type); transport != "" {
return transport
}
if server.URL != "" {
return "sse"
}
if server.Command != "" {
return "stdio"
}
return ""
}
+42
View File
@@ -7,6 +7,7 @@ package config
import (
"encoding/json"
"reflect"
"strings"
"sync"
"testing"
@@ -144,6 +145,47 @@ func TestGetModelConfig_Concurrent(t *testing.T) {
}
}
func TestModelConfig_StreamingConfig(t *testing.T) {
t.Run("loads streaming enabled", func(t *testing.T) {
var cfg ModelConfig
err := json.Unmarshal([]byte(`{
"model_name": "stream-model",
"model": "openai/gpt-5.4",
"streaming": {"enabled": true}
}`), &cfg)
if err != nil {
t.Fatalf("Unmarshal() error = %v", err)
}
if !cfg.Streaming.Enabled {
t.Fatal("Streaming.Enabled = false, want true")
}
})
t.Run("defaults disabled", func(t *testing.T) {
var cfg ModelConfig
err := json.Unmarshal([]byte(`{
"model_name": "plain-model",
"model": "openai/gpt-5.4"
}`), &cfg)
if err != nil {
t.Fatalf("Unmarshal() error = %v", err)
}
if cfg.Streaming.Enabled {
t.Fatal("Streaming.Enabled = true, want false by default")
}
})
t.Run("model streaming only has enabled", func(t *testing.T) {
typ := reflect.TypeOf(ModelStreamingConfig{})
if typ.NumField() != 1 {
t.Fatalf("ModelStreamingConfig field count = %d, want 1", typ.NumField())
}
if _, ok := typ.FieldByName("Enabled"); !ok {
t.Fatal("ModelStreamingConfig missing Enabled field")
}
})
}
func TestModelConfig_Validate(t *testing.T) {
tests := []struct {
name string
+7
View File
@@ -197,6 +197,7 @@ func TestExpandMultiKeyModels_PreservesOtherFields(t *testing.T) {
RequestTimeout: 30,
ThinkingLevel: "high",
ToolSchemaTransform: "simple",
Streaming: ModelStreamingConfig{Enabled: true},
}
modelCfg.APIKeys = SimpleSecureStrings("key0", "key1") // Use internal field for multi-key testing
models := []*ModelConfig{modelCfg}
@@ -229,6 +230,9 @@ func TestExpandMultiKeyModels_PreservesOtherFields(t *testing.T) {
if primary.ToolSchemaTransform != "simple" {
t.Errorf("expected tool_schema_transform preserved, got %q", primary.ToolSchemaTransform)
}
if !primary.Streaming.Enabled {
t.Error("expected streaming config preserved on primary")
}
// Check additional entry also preserves fields
additional := result[0]
@@ -244,6 +248,9 @@ func TestExpandMultiKeyModels_PreservesOtherFields(t *testing.T) {
if additional.ToolSchemaTransform != "simple" {
t.Errorf("expected additional tool_schema_transform preserved, got %q", additional.ToolSchemaTransform)
}
if !additional.Streaming.Enabled {
t.Error("expected streaming config preserved on additional")
}
}
func TestExpandMultiKeyModels_IsVirtualFlag(t *testing.T) {
+1 -10
View File
@@ -79,14 +79,5 @@ func setMCPAttrString(attrs map[string]any, key, value string) {
}
func mcpTransportType(cfg config.MCPServerConfig) string {
if cfg.Type != "" {
return cfg.Type
}
if cfg.URL != "" {
return "sse"
}
if cfg.Command != "" {
return "stdio"
}
return ""
return config.EffectiveMCPTransportType(cfg)
}
+7 -14
View File
@@ -342,17 +342,9 @@ func connectServer(
// Create transport based on configuration
// Auto-detect transport type if not explicitly specified
var transport mcp.Transport
transportType := cfg.Type
// Auto-detect: if URL is provided, use SSE; if command is provided, use stdio
transportType := config.EffectiveMCPTransportType(cfg)
if transportType == "" {
if cfg.URL != "" {
transportType = "sse"
} else if cfg.Command != "" {
transportType = "stdio"
} else {
return nil, fmt.Errorf("either URL or command must be provided")
}
return nil, fmt.Errorf("either URL or command must be provided")
}
switch transportType {
@@ -362,12 +354,13 @@ func connectServer(
}
// Configure DisableStandaloneSSE based on transport type.
// - "http": Request-response only mode. Disable the standalone SSE stream
// to avoid compatibility issues with servers that don't support GET /mcp.
// - "http": Streamable HTTP request-response mode. Disable the standalone
// SSE stream to avoid compatibility issues with servers that don't
// support the optional GET listener.
// - "sse": Bidirectional mode. Enable the standalone SSE stream to receive
// server-initiated notifications (e.g., ToolListChangedNotification).
// - Empty or auto-detected: Defaults to "sse" behavior (standalone SSE enabled).
disableStandaloneSSE := (cfg.Type == "http")
disableStandaloneSSE := transportType == "http"
logger.DebugCF("mcp", "Using SSE/HTTP transport",
map[string]any{
@@ -452,7 +445,7 @@ func connectServer(
transport = &isolatedCommandTransport{Command: cmd}
default:
return nil, fmt.Errorf(
"unsupported transport type: %s (supported: stdio, sse, http)",
"unsupported transport type: %s (supported: stdio, sse, http, streamable-http)",
transportType,
)
}
+326
View File
@@ -0,0 +1,326 @@
//go:build integration
package mcp
import (
"bytes"
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp"
"github.com/sipeed/picoclaw/pkg/config"
)
// Run with: go test -tags=integration ./pkg/mcp
func TestIntegration_StreamableHTTPCompatibility(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
tests := []struct {
name string
transportType string
jsonResponse bool
rejectStandaloneGET bool
wantResponseContentType string
}{
{
name: "http/json-only-without-get-listener",
transportType: "http",
jsonResponse: true,
rejectStandaloneGET: true,
wantResponseContentType: "application/json",
},
{
name: "http/streaming-post-responses",
transportType: "http",
jsonResponse: false,
rejectStandaloneGET: false,
wantResponseContentType: "text/event-stream",
},
{
name: "streamable-http-alias/json-only-without-get-listener",
transportType: "streamable-http",
jsonResponse: true,
rejectStandaloneGET: true,
wantResponseContentType: "application/json",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
server, recorder := newRecordedGoSDKStreamableServer(t, tt.jsonResponse, tt.rejectStandaloneGET)
defer server.Close()
mgr := NewManager()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
err := mgr.ConnectServer(ctx, "compat", config.MCPServerConfig{
Enabled: true,
Type: tt.transportType,
URL: server.URL,
Headers: map[string]string{
"Authorization": "Bearer integration-token",
},
})
if err != nil {
t.Fatalf("ConnectServer() error = %v", err)
}
tools := mgr.GetAllTools()
if got := len(tools["compat"]); got != 1 {
t.Fatalf("len(GetAllTools()[\"compat\"]) = %d, want 1", got)
}
result, err := mgr.CallTool(ctx, "compat", "echo", map[string]any{
"message": "hello from integration",
})
if err != nil {
t.Fatalf("CallTool() error = %v", err)
}
if got, want := extractTextResult(t, result), "hello from integration"; got != want {
t.Fatalf("CallTool() text = %q, want %q", got, want)
}
if err := mgr.Close(); err != nil {
t.Fatalf("Manager.Close() error = %v", err)
}
assertRecordedCompatibility(t, recorder.snapshot(), tt.wantResponseContentType)
})
}
}
type recordedRequest struct {
Method string
Path string
JSONRPCMethod string
RequestSessionID string
Authorization string
ResponseStatusCode int
ResponseContentType string
}
type requestRecorder struct {
mu sync.Mutex
requests []recordedRequest
}
func (r *requestRecorder) add(req recordedRequest) {
r.mu.Lock()
defer r.mu.Unlock()
r.requests = append(r.requests, req)
}
func (r *requestRecorder) snapshot() []recordedRequest {
r.mu.Lock()
defer r.mu.Unlock()
out := make([]recordedRequest, len(r.requests))
copy(out, r.requests)
return out
}
func newRecordedGoSDKStreamableServer(
t *testing.T,
jsonResponse bool,
rejectStandaloneGET bool,
) (*httptest.Server, *requestRecorder) {
t.Helper()
server := sdkmcp.NewServer(&sdkmcp.Implementation{
Name: "streamable-integration-server",
Version: "1.0.0",
}, nil)
sdkmcp.AddTool(server, &sdkmcp.Tool{
Name: "echo",
Description: "Echo a message",
}, func(ctx context.Context, req *sdkmcp.CallToolRequest, args map[string]any) (*sdkmcp.CallToolResult, any, error) {
message, _ := args["message"].(string)
return &sdkmcp.CallToolResult{
Content: []sdkmcp.Content{
&sdkmcp.TextContent{Text: message},
},
}, nil, nil
})
recorder := &requestRecorder{}
handler := sdkmcp.NewStreamableHTTPHandler(func(*http.Request) *sdkmcp.Server {
return server
}, &sdkmcp.StreamableHTTPOptions{
JSONResponse: jsonResponse,
})
httpServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if rejectStandaloneGET && r.Method == http.MethodGet {
w.WriteHeader(http.StatusMethodNotAllowed)
recorder.add(recordedRequest{
Method: r.Method,
Path: r.URL.Path,
RequestSessionID: r.Header.Get("Mcp-Session-Id"),
Authorization: r.Header.Get("Authorization"),
ResponseStatusCode: http.StatusMethodNotAllowed,
ResponseContentType: normalizeContentType(w.Header().Get("Content-Type")),
})
return
}
recorded := recordedRequest{
Method: r.Method,
Path: r.URL.Path,
RequestSessionID: r.Header.Get("Mcp-Session-Id"),
Authorization: r.Header.Get("Authorization"),
}
if r.Method == http.MethodPost {
body, err := io.ReadAll(r.Body)
if err != nil {
t.Fatalf("reading request body: %v", err)
}
r.Body = io.NopCloser(bytes.NewReader(body))
var envelope struct {
Method string `json:"method"`
}
if err := json.Unmarshal(body, &envelope); err == nil {
recorded.JSONRPCMethod = envelope.Method
}
}
rw := &recordingResponseWriter{ResponseWriter: w}
handler.ServeHTTP(rw, r)
recorded.ResponseStatusCode = rw.statusCode()
recorded.ResponseContentType = normalizeContentType(rw.Header().Get("Content-Type"))
recorder.add(recorded)
}))
return httpServer, recorder
}
type recordingResponseWriter struct {
http.ResponseWriter
status int
}
func (w *recordingResponseWriter) WriteHeader(status int) {
if w.status == 0 {
w.status = status
}
w.ResponseWriter.WriteHeader(status)
}
func (w *recordingResponseWriter) Write(p []byte) (int, error) {
if w.status == 0 {
w.status = http.StatusOK
}
return w.ResponseWriter.Write(p)
}
func (w *recordingResponseWriter) Flush() {
if flusher, ok := w.ResponseWriter.(http.Flusher); ok {
flusher.Flush()
}
}
func (w *recordingResponseWriter) statusCode() int {
if w.status != 0 {
return w.status
}
return http.StatusOK
}
func extractTextResult(t *testing.T, result *sdkmcp.CallToolResult) string {
t.Helper()
if result == nil || len(result.Content) != 1 {
t.Fatalf("unexpected CallToolResult: %#v", result)
}
text, ok := result.Content[0].(*sdkmcp.TextContent)
if !ok {
t.Fatalf("CallToolResult content type = %T, want *sdkmcp.TextContent", result.Content[0])
}
return text.Text
}
func assertRecordedCompatibility(
t *testing.T,
requests []recordedRequest,
wantResponseContentType string,
) {
t.Helper()
var (
getCount int
deleteCount int
postWithSession int
deleteWithSession int
requestsMissingAuth []string
observedContentTypesByMethod = map[string]string{}
)
for _, req := range requests {
switch req.Method {
case http.MethodGet:
getCount++
case http.MethodPost:
if req.RequestSessionID != "" {
postWithSession++
}
if req.JSONRPCMethod != "" && observedContentTypesByMethod[req.JSONRPCMethod] == "" {
observedContentTypesByMethod[req.JSONRPCMethod] = req.ResponseContentType
}
case http.MethodDelete:
deleteCount++
if req.RequestSessionID != "" {
deleteWithSession++
}
}
if req.Authorization != "Bearer integration-token" {
requestsMissingAuth = append(requestsMissingAuth, req.Method+" "+req.Path)
}
}
if getCount != 0 {
t.Fatalf("expected no standalone GET requests for streamable HTTP mode, saw %d", getCount)
}
if deleteCount != 1 {
t.Fatalf("DELETE count = %d, want 1", deleteCount)
}
if postWithSession == 0 {
t.Fatal("expected at least one POST request with Mcp-Session-Id")
}
if deleteWithSession != 1 {
t.Fatalf("expected exactly one DELETE with Mcp-Session-Id, got %d", deleteWithSession)
}
if len(requestsMissingAuth) > 0 {
t.Fatalf("Authorization header missing on requests: %v", requestsMissingAuth)
}
for _, method := range []string{"initialize", "tools/list", "tools/call"} {
if observedContentTypesByMethod[method] == "" {
t.Fatalf("did not observe POST response for JSON-RPC method %q", method)
}
if observedContentTypesByMethod[method] != wantResponseContentType {
t.Fatalf(
"response content-type for %s = %q, want %q",
method,
observedContentTypesByMethod[method],
wantResponseContentType,
)
}
}
}
func normalizeContentType(value string) string {
return strings.TrimSpace(strings.SplitN(value, ";", 2)[0])
}
@@ -0,0 +1,152 @@
//go:build integration
package mcp
import (
"context"
"encoding/json"
"os"
"strconv"
"strings"
"testing"
"time"
sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp"
"github.com/sipeed/picoclaw/pkg/config"
)
// TestIntegration_RealConfiguredServer is an opt-in smoke test for a real MCP
// server configured via environment variables.
//
// Run with:
//
// go test -tags=integration ./pkg/mcp -run TestIntegration_RealConfiguredServer -v
//
// Minimum configuration:
//
// PICOCLAW_MCP_REAL_SERVER_JSON='{"enabled":true,"type":"http","url":"http://127.0.0.1:8080/mcp"}'
//
// Optional tool invocation:
//
// PICOCLAW_MCP_REAL_TOOL_NAME=echo
// PICOCLAW_MCP_REAL_TOOL_ARGS_JSON='{"message":"hello"}'
// PICOCLAW_MCP_REAL_EXPECT_SUBSTRING=hello
//
// Stdio subprocess example:
//
// PICOCLAW_MCP_REAL_SERVER_JSON='{"enabled":true,"type":"stdio","command":"npx","args":["-y","@modelcontextprotocol/server-filesystem","."]}'
func TestIntegration_RealConfiguredServer(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
serverJSON := strings.TrimSpace(os.Getenv("PICOCLAW_MCP_REAL_SERVER_JSON"))
if serverJSON == "" {
t.Skip("skipping integration test (set PICOCLAW_MCP_REAL_SERVER_JSON to enable)")
}
serverCfg, err := loadRealServerConfig(serverJSON)
if err != nil {
t.Fatalf("loadRealServerConfig() error = %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
mgr := NewManager()
if err := mgr.ConnectServer(ctx, "real", serverCfg); err != nil {
t.Fatalf("ConnectServer() error = %v", err)
}
defer func() {
if err := mgr.Close(); err != nil {
t.Errorf("Manager.Close() error = %v", err)
}
}()
tools := mgr.GetAllTools()["real"]
if len(tools) == 0 {
t.Fatal("expected at least one discovered tool from real MCP server")
}
t.Logf(
"connected to real MCP server via %s with %d tool(s)",
config.EffectiveMCPTransportType(serverCfg),
len(tools),
)
for _, tool := range tools {
if tool != nil {
t.Logf("discovered tool: %s", tool.Name)
}
}
if expectedCountRaw := strings.TrimSpace(os.Getenv("PICOCLAW_MCP_REAL_EXPECT_TOOL_COUNT")); expectedCountRaw != "" {
expectedCount, err := strconv.Atoi(expectedCountRaw)
if err != nil {
t.Fatalf("invalid PICOCLAW_MCP_REAL_EXPECT_TOOL_COUNT %q: %v", expectedCountRaw, err)
}
if len(tools) != expectedCount {
t.Fatalf("tool count = %d, want %d", len(tools), expectedCount)
}
}
toolName := strings.TrimSpace(os.Getenv("PICOCLAW_MCP_REAL_TOOL_NAME"))
if toolName == "" {
return
}
toolArgs, err := loadRealToolArgs(os.Getenv("PICOCLAW_MCP_REAL_TOOL_ARGS_JSON"))
if err != nil {
t.Fatalf("loadRealToolArgs() error = %v", err)
}
result, err := mgr.CallTool(ctx, "real", toolName, toolArgs)
if err != nil {
t.Fatalf("CallTool(%q) error = %v", toolName, err)
}
textPayload := joinTextContents(result)
t.Logf("tool %q returned text payload: %q", toolName, textPayload)
if want := os.Getenv("PICOCLAW_MCP_REAL_EXPECT_SUBSTRING"); want != "" && !strings.Contains(textPayload, want) {
t.Fatalf("tool result %q does not contain expected substring %q", textPayload, want)
}
}
func loadRealServerConfig(raw string) (config.MCPServerConfig, error) {
var cfg config.MCPServerConfig
if err := json.Unmarshal([]byte(raw), &cfg); err != nil {
return config.MCPServerConfig{}, err
}
if !cfg.Enabled {
cfg.Enabled = true
}
return cfg, nil
}
func loadRealToolArgs(raw string) (map[string]any, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return map[string]any{}, nil
}
var args map[string]any
if err := json.Unmarshal([]byte(raw), &args); err != nil {
return nil, err
}
return args, nil
}
func joinTextContents(result *sdkmcp.CallToolResult) string {
if result == nil || len(result.Content) == 0 {
return ""
}
parts := make([]string, 0, len(result.Content))
for _, content := range result.Content {
if text, ok := content.(*sdkmcp.TextContent); ok && text != nil {
parts = append(parts, text.Text)
}
}
return strings.Join(parts, "\n")
}
+129
View File
@@ -5,6 +5,8 @@ import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
@@ -408,6 +410,133 @@ func TestCallTool_ErrorsForClosedOrMissingServer(t *testing.T) {
})
}
func TestConnectServer_StreamableHTTPRequestResponseMode(t *testing.T) {
t.Parallel()
for _, transportType := range []string{"http", "streamable-http"} {
t.Run(transportType, func(t *testing.T) {
t.Parallel()
server := sdkmcp.NewServer(&sdkmcp.Implementation{
Name: "streamable-test-server",
Version: "1.0.0",
}, nil)
sdkmcp.AddTool(server, &sdkmcp.Tool{
Name: "echo",
Description: "Echo test tool",
}, func(ctx context.Context, req *sdkmcp.CallToolRequest, args map[string]any) (*sdkmcp.CallToolResult, any, error) {
return &sdkmcp.CallToolResult{
Content: []sdkmcp.Content{
&sdkmcp.TextContent{Text: "ok"},
},
}, nil, nil
})
type observedRequest struct {
Method string
SessionID string
Authorization string
}
var (
mu sync.Mutex
observed []observedRequest
)
handler := sdkmcp.NewStreamableHTTPHandler(func(*http.Request) *sdkmcp.Server {
return server
}, nil)
httpServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
observed = append(observed, observedRequest{
Method: r.Method,
SessionID: r.Header.Get("Mcp-Session-Id"),
Authorization: r.Header.Get("Authorization"),
})
mu.Unlock()
handler.ServeHTTP(w, r)
}))
defer httpServer.Close()
conn, err := connectServer(context.Background(), "streamable", config.MCPServerConfig{
Enabled: true,
Type: transportType,
URL: httpServer.URL,
Headers: map[string]string{
"Authorization": "Bearer test-token",
},
})
if err != nil {
t.Fatalf("connectServer(%q) error = %v", transportType, err)
}
if got := len(conn.Tools); got != 1 {
t.Fatalf("len(conn.Tools) = %d, want 1", got)
}
if got := conn.Session.ID(); got == "" {
t.Fatal("expected non-empty streamable session ID")
}
if err := conn.Session.Close(); err != nil {
t.Fatalf("Session.Close() error = %v", err)
}
mu.Lock()
defer mu.Unlock()
var (
getCount int
postCount int
deleteCount int
postWithSession bool
deleteWithSession bool
requestsWithAuth int
requestsWithoutAuth []string
)
for _, req := range observed {
switch req.Method {
case http.MethodGet:
getCount++
case http.MethodPost:
postCount++
if req.SessionID != "" {
postWithSession = true
}
case http.MethodDelete:
deleteCount++
if req.SessionID != "" {
deleteWithSession = true
}
}
if req.Authorization == "Bearer test-token" {
requestsWithAuth++
} else {
requestsWithoutAuth = append(requestsWithoutAuth, req.Method)
}
}
if getCount != 0 {
t.Fatalf("expected no standalone GET requests for %q transport, saw %d", transportType, getCount)
}
if postCount == 0 {
t.Fatal("expected POST requests during streamable HTTP handshake")
}
if deleteCount != 1 {
t.Fatalf("DELETE count = %d, want 1", deleteCount)
}
if !postWithSession {
t.Fatal("expected at least one POST request with Mcp-Session-Id")
}
if !deleteWithSession {
t.Fatal("expected DELETE request with Mcp-Session-Id")
}
if requestsWithAuth != len(observed) {
t.Fatalf("Authorization header missing on requests: %v", requestsWithoutAuth)
}
})
}
}
func TestCallTool_ReconnectsWhenHTTPServerLosesSession(t *testing.T) {
originalConnectServerFunc := connectServerFunc
t.Cleanup(func() {

Some files were not shown because too many files have changed in this diff Show More