Compare commits

...

34 Commits

Author SHA1 Message Date
Mauro 4c5adcd78e Merge pull request #3128 from chengzhichao-xydt/codex/web-body-close
fix(web): explicitly ignore resp.Body.Close() errors after io.ReadAll
2026-07-03 20:45:55 +02:00
Mauro cf24f32896 Merge pull request #3156 from loafoe/feat/session-token-usage
feat(pico): emit per-turn LLM token usage on finalized message
2026-07-03 09:14:36 +02:00
Mauro 4ccb6268a4 Merge pull request #3212 from sipeed/dependabot/npm_and_yarn/web/frontend/react-i18next-17.0.7
build(deps): bump react-i18next from 17.0.6 to 17.0.7 in /web/frontend
2026-07-02 23:50:18 +02:00
Mauro 1d9b1c444a Merge pull request #3214 from sipeed/dependabot/npm_and_yarn/web/frontend/shadcn-4.12.0
build(deps): bump shadcn from 4.7.0 to 4.12.0 in /web/frontend
2026-07-02 23:49:35 +02:00
Mauro 0aff1bd7ab Merge pull request #3215 from sipeed/dependabot/npm_and_yarn/web/frontend/typescript-eslint-8.62.1
build(deps-dev): bump typescript-eslint from 8.59.3 to 8.62.1 in /web/frontend
2026-07-02 23:49:10 +02:00
Mauro 1a6dd0efe5 Merge pull request #3216 from sipeed/dependabot/npm_and_yarn/web/frontend/vitejs/plugin-react-6.0.3
build(deps-dev): bump @vitejs/plugin-react from 6.0.1 to 6.0.3 in /web/frontend
2026-07-02 23:48:43 +02:00
Mauro 4b02293516 Merge pull request #3160 from danmobot/fix/launcher-setup-csrf
fix(auth): reject cross-site launcher setup requests
2026-07-02 23:46:41 +02:00
Mauro 883d43b37d Merge pull request #3063 from trufae/deltachan
feat: add deltachat gateway
2026-07-02 23:45:24 +02:00
Mauro 3b24a48a8c Merge pull request #3161 from danmobot/fix/exec-custom-allow-deny
fix(exec): keep deny patterns active for custom allow rules
2026-07-02 21:51:16 +02:00
Mauro 027226997f Merge pull request #3209 from sipeed/dependabot/go_modules/github.com/anthropics/anthropic-sdk-go-1.55.1
build(deps): bump github.com/anthropics/anthropic-sdk-go from 1.50.2 to 1.55.1
2026-07-02 21:44:00 +02:00
Mauro ce5274d179 Merge pull request #3210 from sipeed/dependabot/go_modules/golang.org/x/crypto-0.53.0
build(deps): bump golang.org/x/crypto from 0.51.0 to 0.53.0
2026-07-02 21:42:38 +02:00
Mauro 79ae6bf97f Merge pull request #3158 from danmobot/fix/sandbox-fs-windows-paths
test: cover sandbox fs Windows path handling
2026-07-02 21:39:45 +02:00
dependabot[bot] 93a58e057e build(deps-dev): bump @vitejs/plugin-react in /web/frontend
Bumps [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/tree/HEAD/packages/plugin-react) from 6.0.1 to 6.0.3.
- [Release notes](https://github.com/vitejs/vite-plugin-react/releases)
- [Changelog](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite-plugin-react/commits/plugin-react@6.0.3/packages/plugin-react)

---
updated-dependencies:
- dependency-name: "@vitejs/plugin-react"
  dependency-version: 6.0.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-02 17:21:01 +00:00
dependabot[bot] 70b9cb9ea0 build(deps-dev): bump typescript-eslint in /web/frontend
Bumps [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) from 8.59.3 to 8.62.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.62.1/packages/typescript-eslint)

---
updated-dependencies:
- dependency-name: typescript-eslint
  dependency-version: 8.62.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-02 17:17:28 +00:00
dependabot[bot] 8fd07bcacb build(deps): bump shadcn from 4.7.0 to 4.12.0 in /web/frontend
Bumps [shadcn](https://github.com/shadcn-ui/ui/tree/HEAD/packages/shadcn) from 4.7.0 to 4.12.0.
- [Release notes](https://github.com/shadcn-ui/ui/releases)
- [Changelog](https://github.com/shadcn-ui/ui/blob/main/packages/shadcn/CHANGELOG.md)
- [Commits](https://github.com/shadcn-ui/ui/commits/shadcn@4.12.0/packages/shadcn)

---
updated-dependencies:
- dependency-name: shadcn
  dependency-version: 4.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-02 17:15:41 +00:00
dependabot[bot] e56ab1f16e build(deps): bump react-i18next from 17.0.6 to 17.0.7 in /web/frontend
Bumps [react-i18next](https://github.com/i18next/react-i18next) from 17.0.6 to 17.0.7.
- [Changelog](https://github.com/i18next/react-i18next/blob/master/CHANGELOG.md)
- [Commits](https://github.com/i18next/react-i18next/compare/v17.0.6...v17.0.7)

---
updated-dependencies:
- dependency-name: react-i18next
  dependency-version: 17.0.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-02 17:14:49 +00:00
dependabot[bot] addaef78ad build(deps): bump golang.org/x/crypto from 0.51.0 to 0.53.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.51.0 to 0.53.0.
- [Commits](https://github.com/golang/crypto/compare/v0.51.0...v0.53.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.53.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-02 17:14:16 +00:00
dependabot[bot] ba881f8273 build(deps): bump github.com/anthropics/anthropic-sdk-go
Bumps [github.com/anthropics/anthropic-sdk-go](https://github.com/anthropics/anthropic-sdk-go) from 1.50.2 to 1.55.1.
- [Release notes](https://github.com/anthropics/anthropic-sdk-go/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-go/compare/v1.50.2...v1.55.1)

---
updated-dependencies:
- dependency-name: github.com/anthropics/anthropic-sdk-go
  dependency-version: 1.55.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-02 17:14:13 +00:00
pancake 612e485d4e WIP: Initial support for deltachat gateway
Features:
- Support voice messages
- Optional crossposting
- Automatic account creation
- Custom avatar image
2026-07-02 15:23:35 +02:00
LC 2cf030d2fd fix(providers): surface friendly auth error messages (#3198) 2026-06-30 17:42:07 +08:00
程智超0668000959 c3aa8c088c fix(web): explicitly ignore resp.Body.Close() errors after io.ReadAll 2026-06-25 15:13:29 +08:00
danmobot acc2c5c6aa test(fs): cover root path separator normalization 2026-06-23 20:57:42 +01:00
danmobot ff247b63ac fix(exec): preserve additive custom allow behavior 2026-06-23 20:47:15 +01:00
danmobot 0c404869ae fix: restore openai compat log import 2026-06-23 20:41:28 +01:00
danmobot 263e940825 Merge remote-tracking branch 'upstream/main' into fix/launcher-setup-csrf 2026-06-23 20:38:16 +01:00
danmobot 9721c36e55 fix(exec): keep deny patterns active for custom allow rules 2026-06-23 02:53:14 +01:00
danmobot 1758eea948 test: cover launcher setup csrf guard 2026-06-23 02:24:53 +01:00
danmobot f5b2ce7482 fix: reject cross-site launcher setup requests 2026-06-23 02:23:06 +01:00
danmobot 46ffda264f fix: import log in openai compat provider 2026-06-22 23:42:27 +01:00
danmobot 88bda73db6 Merge remote-tracking branch 'upstream/main' into fix/sandbox-fs-windows-paths 2026-06-22 23:42:04 +01:00
danmobot 29e019ec66 test: cover sandbox fs Windows path handling 2026-06-22 23:01:20 +01:00
Andy Lo-A-Foe cc7b4ca86b fix(pico): deliver per-turn token usage to the streamer
Two bugs prevented the usage block from ever reaching the wire:

1. CallLLM read turnStateFromContext(ctx), but the raw ctx is not seeded
   with the turn state (only turnCtx is), so SetLastUsage/SetLastFinishReason
   were dropped — GetLastUsage() returned nil at finalize. Set them on the
   ts parameter directly, which is also what the streaming publisher reads.

2. The manager wraps the channel streamer in finalizeHookStreamer /
   splitMarkerStreamer, neither of which forwarded SetTurnUsage (it is not
   part of the bus.Streamer interface), so the type assertion in the
   publisher's Finalize failed silently. Mirror the existing SetModelName
   forwarding: add a turnUsageStreamer interface + setStreamerTurnUsage
   helper and SetTurnUsage methods on both wrappers (splitMarker also stores
   and re-applies usage to each freshly-begun part streamer).

Adds regression tests asserting both wrappers forward SetTurnUsage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:38:46 +02:00
Andy Lo-A-Foe 697e94fe8c feat(pico): emit real per-turn token usage on finalized message 2026-06-22 13:15:35 +02:00
Andy Lo-A-Foe 052c742fe7 feat(pico): add SetTurnUsage and usage payload helper 2026-06-22 13:07:48 +02:00
41 changed files with 5082 additions and 852 deletions
+1
View File
@@ -494,6 +494,7 @@ Talk to your PicoClaw through 19+ messaging platforms:
| **QQ** | Easy (AppID + AppSecret) | WebSocket | [Guide](docs/channels/qq/README.md) |
| **Slack** | Easy (bot + app token) | Socket Mode | [Guide](docs/channels/slack/README.md) |
| **Matrix** | Medium (homeserver + token) | Sync API | [Guide](docs/channels/matrix/README.md) |
| **Delta Chat** | Easy (account script or email/password) | JSON-RPC (email/E2EE) | [Guide](docs/channels/deltachat/README.md) |
| **DingTalk** | Medium (client credentials) | Stream | [Guide](docs/channels/dingtalk/README.md) |
| **Feishu / Lark** | Medium (App ID + Secret) | WebSocket/SDK | [Guide](docs/channels/feishu/README.md) |
| **LINE** | Medium (credentials + webhook) | Webhook | [Guide](docs/channels/line/README.md) |
+148
View File
@@ -0,0 +1,148 @@
> Back to [README](../../../README.md)
# Delta Chat Channel
PicoClaw can run as a Delta Chat bot by launching a local
`deltachat-rpc-server` process and talking to it over JSON-RPC. The RPC server
handles the email account, IMAP/SMTP connection, message store, and encryption
keys.
## Install
Install the Delta Chat RPC server. If `deltachat-rpc-server` is on `PATH`,
PicoClaw can find it automatically; otherwise set `rpc_server_path` to the
exact binary path.
```bash
pip install deltachat-rpc-server
which deltachat-rpc-server
```
Prebuilt binaries are also available from the
[Delta Chat core releases](https://github.com/deltachat/deltachat-core-rust/releases).
## Configure
The easiest setup is to let PicoClaw create a chatmail account in Delta
Chat's local account store. Put a relay marker in `email` using an empty local
part, for example `@nine.testrun.org`:
```json
{
"channel_list": {
"deltachat": {
"enabled": true,
"type": "deltachat",
"allow_from": ["friend@example.org"],
"group_trigger": {
"mention_only": true
},
"settings": {
"email": "@nine.testrun.org",
"display_name": "PicoClaw Bot",
"avatar_image": "/home/me/bot-avatar.png"
}
}
}
}
```
On startup, PicoClaw creates the account through `deltachat-rpc-server`, then
stops with an error that contains the generated address. Replace the relay
marker with that full email address and run PicoClaw again:
```json
{
"email": "bot123@nine.testrun.org",
"display_name": "PicoClaw Bot",
"avatar_image": "/home/me/bot-avatar.png"
}
```
If `email` is missing, the startup error lists the built-in relay choices copied
from Parla. You can use one of those relay markers, or a custom chatmail relay
with the same `@server.name` form.
`password` is not needed for PicoClaw-created chatmail accounts. Omit it when
`email` points to an already configured account in `data_dir`; the JSON-RPC
server owns the mailbox password. The legacy password-based path remains only
for classic email accounts that PicoClaw must configure itself. In that mode,
`password` is a secure field; on first config load it is moved to
`~/.picoclaw/.security.yml`, and it can also be set with
`PICOCLAW_CHANNELS_DELTACHAT_PASSWORD`.
`display_name` and `avatar_image` are optional profile settings. When present,
PicoClaw applies them on every startup, so changing the avatar path in config is
enough to update the bot profile.
| Field | Required | Description |
|-------|----------|-------------|
| `email` | Yes | Full bot mailbox address, or first-run relay marker such as `@nine.testrun.org` |
| `rpc_server_path` | No | Path to `deltachat-rpc-server`; only needed when it is not on `PATH` |
| `password` | No | Legacy only; required when PicoClaw must configure/reconfigure a classic mailbox itself |
| `display_name` | No | Startup-applied profile name shown to contacts and used for group mention detection |
| `avatar_image` | No | Startup-applied profile avatar image path; `~` is expanded. Missing files are warned and ignored |
| `data_dir` | No | Account database directory. Default: `~/.picoclaw/deltachat/<channel-name>` |
| `invite_link` | No | Delta Chat invite link to join on startup |
| `allow_crosspost` | No | Default `false`. When `true`, senders allowed by `allow_from` may use `message` tool targets outside the current chat, or resolve recipients by email/contact/chat name |
| `imap_server`, `imap_port` | No | Manual IMAP override for password-based configuration |
| `smtp_server`, `smtp_port` | No | Manual SMTP override for password-based configuration |
Standard channel fields such as `allow_from`, `group_trigger`, and
`reasoning_channel_id` also apply.
## First Run
With `email` set to `@server`, PicoClaw creates the chatmail account, prints
the generated full email in the startup error, and exits. Update `email` to that
full address and run PicoClaw again. On later runs, PicoClaw selects the
configured account by `email`, applies optional profile settings, marks it as a
bot, and starts IO.
With a new `data_dir` plus legacy `password`, PicoClaw can still configure a
classic email account and validate the mailbox credentials; after that, the
account is reused from the local data directory.
Delta Chat requires peers to learn the bot's encryption key before messaging
it. On startup PicoClaw prints the bot invite link and QR code. Add the bot from
Delta Chat with that invite, not by typing the bare email address.
## Behavior
- Direct chats always respond after `allow_from` passes.
- Group chats follow `group_trigger`; without one, every group message is
handled.
- Messages from the bot itself, device chats, and info/system messages are
ignored.
- Accepted inbound messages are marked seen after the allow-list check.
- Incoming attachments (images, audio, video, documents) are registered with
the media store and handed to the agent, so it can view images or operate on
the files directly. If no media store is available, the path is appended
inline as `[attachment: /path]` instead.
- Outbound attachments are supported: when the agent emits media, each file is
sent as a Delta Chat message (with the caption as text). Delta Chat infers the
view type from the file, so images, GIFs, and videos render natively.
- Crosspost recipient lookup is disabled by default. The agent can always reply
to the current numeric chat ID, but sending to another numeric chat ID or
resolving an email/contact/chat name requires `allow_crosspost: true`
and the current sender must pass `allow_from`; `allow_from: ["*"]` allows this
for any sender.
- Voice is supported in both directions when voice providers are configured:
incoming voice notes are transcribed by the agent's ASR and the transcript is
passed to the model; the agent can reply with synthesized speech, which is
delivered as a native Delta Chat voice message (`send_tts`). This requires an
ASR and/or TTS provider under `voice` — it is not Delta Chat-specific config.
## Troubleshooting
| Symptom | Fix |
|---------|-----|
| `deltachat-rpc-server not found on PATH` or `rpc_server_path ... not found` | Install the RPC server on PATH, or set `rpc_server_path` to an absolute path |
| `email is required` | Choose one listed chatmail server, set `email` to a first-run marker such as `@nine.testrun.org`, run `picoclaw g`, then replace it with the generated full email |
| `created chatmail account ...` | Replace the `@server` marker in `email` with the generated full email and run PicoClaw again |
| `account ... is not configured in data_dir` | Point `data_dir` at the existing JSON-RPC account store, or use `email="@server"` to create one |
| `configure (check email/password/server)` | Check credentials, app password requirements, or IMAP/SMTP overrides |
| Bot does not answer in a group | Check `group_trigger`; mention `display_name` or use a configured prefix |
| Bot ignores a sender | Add the sender email to `allow_from`, or use `["*"]` for open access |
| Sender cannot message the bot | Re-add the bot with the startup QR/invite so Delta Chat can establish encryption |
| Agent cannot send to an email/name/other chat ID | Enable `settings.allow_crosspost` and allow the controlling sender in `allow_from`; this capability is disabled by default for privacy |
+3 -3
View File
@@ -8,7 +8,7 @@ require (
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.0
github.com/SevereCloud/vksdk/v3 v3.3.1
github.com/adhocore/gronx v1.20.0
github.com/anthropics/anthropic-sdk-go v1.50.2
github.com/anthropics/anthropic-sdk-go v1.55.1
github.com/atc0005/go-teams-notify/v2 v2.14.0
github.com/aws/aws-sdk-go-v2 v1.42.0
github.com/aws/aws-sdk-go-v2/config v1.32.25
@@ -119,7 +119,7 @@ require (
go.opentelemetry.io/otel/trace v1.35.0 // indirect
go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
golang.org/x/text v0.37.0 // indirect
golang.org/x/text v0.38.0 // indirect
modernc.org/libc v1.73.4 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
@@ -148,7 +148,7 @@ require (
github.com/valyala/fastjson v1.6.10 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
golang.org/x/arch v0.24.0 // indirect
golang.org/x/crypto v0.51.0
golang.org/x/crypto v0.53.0
golang.org/x/net v0.55.0
golang.org/x/sync v0.21.0
golang.org/x/sys v0.46.0
+6 -6
View File
@@ -29,8 +29,8 @@ github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883 h1:bvNMNQO63//z+xNg
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/anthropics/anthropic-sdk-go v1.50.2 h1:K+YJWWzeN2h5MAbh9xeUWY8yAB2oOMp2xLLAODrVBXA=
github.com/anthropics/anthropic-sdk-go v1.50.2/go.mod h1:3EfIfmFqxH6rbiLcIP4tPFyXL/IHakx2wDG4OU+TIEI=
github.com/anthropics/anthropic-sdk-go v1.55.1 h1:GxukHUVou6AFIngxa/Aw1z79hmwg13Hmn++KE9werbM=
github.com/anthropics/anthropic-sdk-go v1.55.1/go.mod h1:3EfIfmFqxH6rbiLcIP4tPFyXL/IHakx2wDG4OU+TIEI=
github.com/atc0005/go-teams-notify/v2 v2.14.0 h1:7N+xw+COnYANLREaAveQ65rsNQ12nIZJED9nMLyscCo=
github.com/atc0005/go-teams-notify/v2 v2.14.0/go.mod h1:EECsWM2b0Hvoz7O+QdlsvyN2KCUOFQCGj8bUBXv3A3Q=
github.com/aws/aws-sdk-go-v2 v1.42.0 h1:XvXMJTkFQtpBKIWZnmr9ZEOc2InWM2yldjXEJ/bymhA=
@@ -362,8 +362,8 @@ golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWP
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.16.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
@@ -438,8 +438,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+7 -2
View File
@@ -100,10 +100,15 @@ func resolveMediaRefs(
localPath, meta, err := store.ResolveWithMeta(ref)
if err != nil {
logger.WarnCF("agent", "Failed to resolve media ref", map[string]any{
fields := map[string]any{
"ref": ref,
"error": err.Error(),
})
}
if idx < currentTurnStart {
logger.DebugCF("agent", "Skipped stale historical media ref", fields)
} else {
logger.WarnCF("agent", "Failed to resolve media ref", fields)
}
continue
}
+1 -2
View File
@@ -6,7 +6,6 @@ import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
@@ -21,7 +20,7 @@ func (al *AgentLoop) maybePublishError(ctx context.Context, channel, chatID, ses
if errors.Is(err, context.Canceled) {
return false
}
al.PublishResponseIfNeeded(ctx, channel, chatID, sessionKey, fmt.Sprintf("Error processing message: %v", err))
al.PublishResponseIfNeeded(ctx, channel, chatID, sessionKey, formatProcessingError(err))
return true
}
+36
View File
@@ -0,0 +1,36 @@
package agent
import (
"fmt"
"github.com/sipeed/picoclaw/pkg/providers"
)
func formatProcessingError(err error) string {
if err == nil {
return ""
}
if kind, ok := providers.ClassifyAuthError(err); ok {
return fmt.Sprintf(
"Error processing message: %s\n\nOriginal error:\n%s",
authErrorFriendlyMessage(kind),
err.Error(),
)
}
return fmt.Sprintf("Error processing message: %v", err)
}
func authErrorFriendlyMessage(kind providers.AuthErrorKind) string {
switch kind {
case providers.AuthErrorInvalidAPIKey:
return "Authentication failed: the API key appears to be invalid. Check the API key configured for this model or provider."
case providers.AuthErrorMissingAPIKey:
return "Authentication failed: no API key is configured for this model or provider. Add an API key in the model settings or config."
case providers.AuthErrorExpiredToken:
return "Authentication failed: the saved login or token appears to be expired. Re-authenticate the provider."
default:
return "Authentication failed: check the API key, token, OAuth login, or provider permissions for this model."
}
}
+52
View File
@@ -0,0 +1,52 @@
package agent
import (
"errors"
"strings"
"testing"
"github.com/sipeed/picoclaw/pkg/providers/common"
)
func TestFormatProcessingError_InvalidAPIKey(t *testing.T) {
err := errors.New(
`LLM call failed after retries: API request failed: Status: 401 Body: {"error":{"message":"Incorrect API key provided"}}`,
)
got := formatProcessingError(err)
if !strings.Contains(got, "API key appears to be invalid") {
t.Fatalf("formatted error missing friendly API key hint: %q", got)
}
if !strings.Contains(got, "Original error:") {
t.Fatalf("formatted error missing original error label: %q", got)
}
if !strings.Contains(got, err.Error()) {
t.Fatalf("formatted error missing original error: %q", got)
}
}
func TestFormatProcessingError_GenericAuthHTTPError(t *testing.T) {
err := &common.HTTPError{
StatusCode: 401,
BodyPreview: `{"error":"unauthorized"}`,
ContentType: "application/json",
APIBase: "https://api.example.com",
}
got := formatProcessingError(err)
if !strings.Contains(got, "check the API key, token, OAuth login, or provider permissions") {
t.Fatalf("formatted error missing generic auth hint: %q", got)
}
if !strings.Contains(got, "Original error:") {
t.Fatalf("formatted error missing original error: %q", got)
}
}
func TestFormatProcessingError_NonAuth(t *testing.T) {
err := errors.New("connection reset by peer")
got := formatProcessingError(err)
want := "Error processing message: connection reset by peer"
if got != want {
t.Fatalf("formatted error = %q, want %q", got, want)
}
}
+9 -4
View File
@@ -495,11 +495,16 @@ func (p *Pipeline) CallLLM(
}
}
// Save finishReason to turnState for SubTurn truncation detection
if innerTS := turnStateFromContext(ctx); innerTS != nil {
innerTS.SetLastFinishReason(exec.response.FinishReason)
// Save finishReason and usage on the turn state. Use ts directly (the
// authoritative turn state for this call) rather than a context lookup:
// the raw ctx passed to CallLLM is not seeded with turnState (only turnCtx
// is), so turnStateFromContext(ctx) returns nil here and silently dropped
// both the finish reason and the per-turn token usage. ts is also exactly
// what the streaming publisher reads via GetLastUsage at finalize.
if ts != nil {
ts.SetLastFinishReason(exec.response.FinishReason)
if exec.response.Usage != nil {
innerTS.SetLastUsage(exec.response.Usage)
ts.SetLastUsage(exec.response.Usage)
}
}
+7
View File
@@ -54,6 +54,7 @@ func (p *Pipeline) tryConfiguredStreamingLLM(
channel: ts.channel,
chatID: ts.chatID,
modelName: exec.llmModelName,
ts: ts,
}
logger.DebugCF("agent", "configured streaming enabled", map[string]any{
@@ -376,6 +377,7 @@ type streamingChunkPublisher struct {
published bool
reasoningPublished bool
err error
ts *turnState
}
func (p *streamingChunkPublisher) Update(ctx context.Context, accumulated string) {
@@ -445,6 +447,11 @@ func (p *streamingChunkPublisher) Finalize(ctx context.Context, content string,
if setter, ok := p.streamer.(interface{ SetModelName(modelName string) }); ok {
setter.SetModelName(p.modelName)
}
if usage := p.ts.GetLastUsage(); usage != nil {
if setter, ok := p.streamer.(interface{ SetTurnUsage(in, out int) }); ok {
setter.SetTurnUsage(usage.PromptTokens, usage.CompletionTokens)
}
}
var err error
if streamer, ok := p.streamer.(bus.ContextUsageStreamer); ok {
err = streamer.FinalizeWithContext(ctx, content, contextUsage)
+7 -5
View File
@@ -266,7 +266,7 @@ func (c *BaseChannel) HandleMessageWithContext(
media []string,
inboundCtx bus.InboundContext,
senderOpts ...bus.SenderInfo,
) {
) error {
// Use SenderInfo-based allow check when available, else fall back to string
var sender bus.SenderInfo
if len(senderOpts) > 0 {
@@ -275,11 +275,11 @@ func (c *BaseChannel) HandleMessageWithContext(
senderID := strings.TrimSpace(inboundCtx.SenderID)
if sender.CanonicalID != "" || sender.PlatformID != "" {
if !c.IsAllowedSender(sender) {
return
return nil
}
} else {
if !c.IsAllowed(senderID) {
return
return nil
}
}
@@ -350,7 +350,9 @@ func (c *BaseChannel) HandleMessageWithContext(
"chat_id": deliveryChatID,
"error": err.Error(),
})
return err
}
return nil
}
// HandleInboundContext publishes a normalized inbound message using only the
@@ -361,8 +363,8 @@ func (c *BaseChannel) HandleInboundContext(
media []string,
inboundCtx bus.InboundContext,
senderOpts ...bus.SenderInfo,
) {
c.HandleMessageWithContext(ctx, deliveryChatID, content, media, inboundCtx, senderOpts...)
) error {
return c.HandleMessageWithContext(ctx, deliveryChatID, content, media, inboundCtx, senderOpts...)
}
func (c *BaseChannel) SetRunning(running bool) {
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+398
View File
@@ -0,0 +1,398 @@
package deltachat
import (
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"unicode"
"github.com/google/uuid"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/channels"
"github.com/sipeed/picoclaw/pkg/config"
"github.com/sipeed/picoclaw/pkg/identity"
"github.com/sipeed/picoclaw/pkg/logger"
"github.com/sipeed/picoclaw/pkg/media"
"github.com/sipeed/picoclaw/pkg/utils"
)
// listen is the inbound message loop. It blocks on wait_next_msgs and feeds
// each new message into the PicoClaw inbound pipeline.
func (c *DeltaChatChannel) listen() {
logger.InfoCF("deltachat", "Listening for messages", map[string]any{
"account_id": c.accountID,
"email": c.selfAddr,
})
for c.IsRunning() && c.ctx.Err() == nil {
raw, err := c.rpc.call(c.ctx, "wait_next_msgs", c.accountID)
if err != nil {
if c.ctx.Err() != nil || !c.IsRunning() {
return
}
logger.ErrorCF("deltachat", "wait_next_msgs failed", map[string]any{"error": err.Error()})
time.Sleep(time.Second)
continue
}
var messageIDs []int64
if err := json.Unmarshal(raw, &messageIDs); err != nil {
continue
}
if len(messageIDs) > 0 {
logger.DebugCF("deltachat", "Received message batch", map[string]any{
"count": len(messageIDs),
})
}
for _, messageID := range messageIDs {
c.handleMessage(messageID)
}
}
}
// handleMessage fetches one message, applies inbound filtering, and publishes it.
func (c *DeltaChatChannel) handleMessage(messageID int64) {
msg, err := c.getMessage(messageID)
if err != nil {
logger.DebugCF("deltachat", "get_message failed", map[string]any{
"message_id": messageID,
"error": err.Error(),
})
return
}
if msg.IsInfo || (strings.TrimSpace(msg.Text) == "" && msg.File == "") {
return
}
senderAddr := ""
if msg.Sender != nil {
senderAddr = msg.Sender.Address
}
if senderAddr != "" && strings.EqualFold(senderAddr, c.selfAddr) {
logger.DebugCF("deltachat", "Drop: own message", map[string]any{"message_id": messageID})
return
}
chat, err := c.getFullChat(msg.ChatID)
if err != nil {
logger.DebugCF("deltachat", "get_full_chat_by_id failed", map[string]any{
"chat_id": msg.ChatID,
"error": err.Error(),
})
return
}
// Device messages are core-generated notices, not real conversations.
if chat.IsDeviceChat {
logger.DebugCF("deltachat", "Drop: device message", map[string]any{"chat_id": msg.ChatID})
return
}
isGroup := chat.ChatType != chatTypeSingle
logger.DebugCF("deltachat", "Inbound message", map[string]any{
"message_id": messageID,
"chat_id": msg.ChatID,
"from": senderAddr,
"is_group": isGroup,
"has_file": msg.File != "",
"text_len": len(strings.TrimSpace(msg.Text)),
})
senderName := senderAddr
if msg.Sender != nil {
if msg.Sender.DisplayName != "" {
senderName = msg.Sender.DisplayName
} else if msg.Sender.Name != "" {
senderName = msg.Sender.Name
}
}
if senderName == "" {
senderName = "unknown"
}
chatID := strconv.FormatInt(msg.ChatID, 10)
messageIDStr := strconv.FormatInt(msg.ID, 10)
content := strings.TrimSpace(msg.Text)
// Register any attachment with the media store so the agent pipeline can
// view images and operate on files. The ref is scoped to the same key the
// BaseChannel derives for this message, so it is released with the turn.
var mediaRefs []string
if msg.File != "" {
scope := channels.BuildMediaScope(c.Name(), chatID, messageIDStr)
if ref := c.registerInboundFile(scope, msg); ref != "" {
mediaRefs = append(mediaRefs, ref)
} else {
// Fallback when no media store is available: surface the path inline
// so the attachment is not silently lost.
annotation := fmt.Sprintf("[attachment: %s]", msg.File)
if content == "" {
content = annotation
} else {
content = content + "\n" + annotation
}
}
}
// A file with no caption still warrants a turn; give the agent a minimal
// placeholder so the message survives the empty-content guard below. Audio
// gets a "[voice]" tag specifically, so the agent's transcription step can
// substitute the transcript in place rather than appending it.
if content == "" && len(mediaRefs) > 0 {
if utils.IsAudioFile(msg.FileName, msg.FileMime) {
content = "[voice]"
} else {
content = "[media]"
}
}
sender := bus.SenderInfo{
Platform: config.ChannelDeltaChat,
PlatformID: senderAddr,
CanonicalID: identity.BuildCanonicalID(config.ChannelDeltaChat, senderAddr),
Username: senderAddr,
DisplayName: senderName,
}
if !c.IsAllowedSender(sender) {
logger.DebugCF("deltachat", "Drop: sender not in allow_from", map[string]any{
"from": senderAddr,
})
return
}
isMentioned := false
if isGroup {
botName := c.config.DisplayName
if botName == "" {
botName = c.selfAddr
}
isMentioned = mentionsBot(content, botName, c.selfAddr)
respond, cleaned := c.ShouldRespondInGroup(isMentioned, content)
if !respond {
logger.DebugCF("deltachat", "Drop: group trigger not satisfied", map[string]any{
"chat_id": msg.ChatID,
"mentioned": isMentioned,
})
return
}
content = cleaned
}
if strings.TrimSpace(content) == "" {
return
}
metadata := map[string]string{
"platform": config.ChannelDeltaChat,
"chat_name": chat.Name,
}
if msg.File != "" {
metadata["file"] = msg.File
metadata["file_name"] = msg.FileName
metadata["file_mime"] = msg.FileMime
}
inboundCtx := bus.InboundContext{
Channel: config.ChannelDeltaChat,
ChatID: chatID,
SenderID: senderAddr,
MessageID: messageIDStr,
Mentioned: isMentioned,
Raw: metadata,
}
if isGroup {
inboundCtx.ChatType = "group"
} else {
inboundCtx.ChatType = "direct"
}
logger.DebugCF("deltachat", "Dispatching to agent", map[string]any{
"chat_id": chatID,
"chat_type": inboundCtx.ChatType,
"from": senderAddr,
})
if err := c.HandleInboundContext(c.ctx, chatID, content, mediaRefs, inboundCtx, sender); err != nil {
logger.ErrorCF("deltachat", "Dispatch failed; leaving message unseen", map[string]any{
"message_id": messageID,
"chat_id": chatID,
"error": err.Error(),
})
return
}
if _, err := c.rpc.call(c.ctx, "markseen_msgs", c.accountID, []int64{messageID}); err != nil {
logger.WarnCF("deltachat", "Failed to mark message seen", map[string]any{
"message_id": messageID,
"chat_id": chatID,
"error": err.Error(),
})
}
}
// registerInboundFile records an inbound attachment with the media store under
// the given scope and returns its media:// ref. Returns "" when there is no
// media store or registration fails, letting the caller fall back to an inline
// path annotation.
//
// Delta Chat stores attachments inside the account directory, next to the
// credential database — a location tools are intentionally NOT allowed to read.
// We therefore copy the single attachment out into the shared media temp dir
// (which read_file/load_image are permitted to access) and register that copy,
// so the agent can actually open the file. The copy is store-managed and deleted
// when the turn's scope is released.
func (c *DeltaChatChannel) registerInboundFile(scope string, msg *dcMessage) string {
store := c.GetMediaStore()
if store == nil {
return ""
}
filename := msg.FileName
if filename == "" {
filename = filepath.Base(msg.File)
}
localPath, err := copyToMediaTemp(msg.File, filename)
if err != nil {
logger.WarnCF("deltachat", "Failed to copy attachment into media dir", map[string]any{
"file": msg.File,
"error": err.Error(),
})
return ""
}
ref, err := store.Store(localPath, media.MediaMeta{
Filename: filename,
ContentType: msg.FileMime,
Source: config.ChannelDeltaChat,
CleanupPolicy: media.CleanupPolicyDeleteOnCleanup,
}, scope)
if err != nil {
logger.WarnCF("deltachat", "Failed to register attachment with media store", map[string]any{
"file": localPath,
"error": err.Error(),
})
_ = os.Remove(localPath)
return ""
}
return ref
}
// copyToMediaTemp copies srcPath into the shared media temp directory under a
// unique name and returns the destination path. The media temp dir is the
// location the read_file/load_image tools are permitted to read, so copying here
// makes the attachment readable without exposing Delta Chat's account directory.
func copyToMediaTemp(srcPath, filename string) (string, error) {
if err := os.MkdirAll(media.TempDir(), 0o700); err != nil {
return "", err
}
safe := utils.SanitizeFilename(filename)
if safe == "" {
safe = filepath.Base(srcPath)
}
dstPath := filepath.Join(media.TempDir(), uuid.NewString()[:8]+"_"+safe)
src, err := os.Open(srcPath)
if err != nil {
return "", err
}
defer src.Close()
dst, err := os.Create(dstPath)
if err != nil {
return "", err
}
if _, err := io.Copy(dst, src); err != nil {
dst.Close()
_ = os.Remove(dstPath)
return "", err
}
if err := dst.Close(); err != nil {
_ = os.Remove(dstPath)
return "", err
}
return dstPath, nil
}
func (c *DeltaChatChannel) getMessage(messageID int64) (*dcMessage, error) {
raw, err := c.rpc.call(c.ctx, "get_message", c.accountID, messageID)
if err != nil {
return nil, err
}
var msg dcMessage
if err := json.Unmarshal(raw, &msg); err != nil {
return nil, err
}
return &msg, nil
}
func (c *DeltaChatChannel) getFullChat(chatID int64) (*dcChat, error) {
raw, err := c.rpc.call(c.ctx, "get_full_chat_by_id", c.accountID, chatID)
if err != nil {
return nil, err
}
var chat dcChat
if err := json.Unmarshal(raw, &chat); err != nil {
return nil, err
}
return &chat, nil
}
// mentionsBot reports whether the message references the bot by display name or
// the local-part of its email address (a common addressing convention).
func mentionsBot(content, displayName, email string) bool {
if containsMentionToken(content, displayName) {
return true
}
if local, _, ok := strings.Cut(email, "@"); ok && local != "" {
if containsMentionToken(content, "@"+local) {
return true
}
}
return false
}
func containsMentionToken(content, token string) bool {
token = strings.TrimSpace(token)
if token == "" {
return false
}
contentRunes := []rune(strings.ToLower(content))
tokenRunes := []rune(strings.ToLower(token))
if len(tokenRunes) == 0 || len(tokenRunes) > len(contentRunes) {
return false
}
for i := 0; i <= len(contentRunes)-len(tokenRunes); i++ {
if !sameRunes(contentRunes[i:i+len(tokenRunes)], tokenRunes) {
continue
}
before := i == 0 || !isMentionWordRune(contentRunes[i-1])
afterIdx := i + len(tokenRunes)
after := afterIdx >= len(contentRunes) || !isMentionWordRune(contentRunes[afterIdx])
if before && after {
return true
}
}
return false
}
func sameRunes(a, b []rune) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
func isMentionWordRune(r rune) bool {
return unicode.IsLetter(r) || unicode.IsDigit(r) || r == '_'
}
+35
View File
@@ -0,0 +1,35 @@
package deltachat
import (
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/channels"
"github.com/sipeed/picoclaw/pkg/config"
)
func init() {
channels.RegisterFactory(
config.ChannelDeltaChat,
func(channelName, channelType string, cfg *config.Config, b *bus.MessageBus) (channels.Channel, error) {
bc := cfg.Channels[channelName]
if bc == nil || !bc.Enabled {
return nil, nil
}
decoded, err := bc.GetDecoded()
if err != nil {
return nil, err
}
c, ok := decoded.(*config.DeltaChatSettings)
if !ok {
return nil, channels.ErrSendFailed
}
ch, err := NewDeltaChatChannel(bc, c, b)
if err != nil {
return nil, err
}
if channelName != config.ChannelDeltaChat {
ch.SetName(channelName)
}
return ch, nil
},
)
}
+188
View File
@@ -0,0 +1,188 @@
package deltachat
import (
"bufio"
"context"
"encoding/json"
"fmt"
"io"
"os/exec"
"sync"
"github.com/sipeed/picoclaw/pkg/logger"
)
// rpcRequest is a single JSON-RPC 2.0 request. Delta Chat uses positional
// (array) parameters.
type rpcRequest struct {
JSONRPC string `json:"jsonrpc"`
ID uint64 `json:"id"`
Method string `json:"method"`
Params []any `json:"params,omitempty"`
}
// rpcResponse is a single JSON-RPC 2.0 response.
type rpcResponse struct {
ID uint64 `json:"id"`
Result json.RawMessage `json:"result"`
Error *rpcError `json:"error"`
}
type rpcError struct {
Code int `json:"code"`
Message string `json:"message"`
}
func (e *rpcError) Error() string {
return fmt.Sprintf("deltachat rpc error %d: %s", e.Code, e.Message)
}
// rpcClient drives a `deltachat-rpc-server` child process over stdio using
// newline-delimited JSON-RPC 2.0. The server answers requests asynchronously
// and out of order, so every call is correlated back to its caller by id.
type rpcClient struct {
cmd *exec.Cmd
stdin io.WriteCloser
stdout io.ReadCloser
mu sync.Mutex
nextID uint64
pending map[uint64]chan rpcResponse
closed bool
}
// startRPC spawns the RPC server with DC_ACCOUNTS_PATH pointing at dataDir and
// begins the background read loop.
func startRPC(serverPath, dataDir string) (*rpcClient, error) {
cmd := exec.Command(serverPath)
cmd.Env = append(cmd.Environ(), "DC_ACCOUNTS_PATH="+dataDir)
stdin, err := cmd.StdinPipe()
if err != nil {
return nil, fmt.Errorf("deltachat rpc stdin: %w", err)
}
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("deltachat rpc stdout: %w", err)
}
// Let the server's logs flow to our stderr for easy diagnostics.
cmd.Stderr = logWriter{}
if err := cmd.Start(); err != nil {
return nil, fmt.Errorf("start deltachat-rpc-server (%s): %w", serverPath, err)
}
c := &rpcClient{
cmd: cmd,
stdin: stdin,
stdout: stdout,
pending: make(map[uint64]chan rpcResponse),
}
go c.readLoop()
return c, nil
}
// readLoop reads newline-delimited responses and dispatches them to waiters.
func (c *rpcClient) readLoop() {
reader := bufio.NewReader(c.stdout)
for {
line, err := reader.ReadBytes('\n')
if len(line) > 0 {
var resp rpcResponse
if jsonErr := json.Unmarshal(line, &resp); jsonErr == nil && resp.ID != 0 {
c.mu.Lock()
ch := c.pending[resp.ID]
delete(c.pending, resp.ID)
c.mu.Unlock()
if ch != nil {
ch <- resp
}
}
}
if err != nil {
c.failAll(err)
return
}
}
}
// failAll wakes every pending caller with an error (used on EOF / shutdown).
func (c *rpcClient) failAll(err error) {
c.mu.Lock()
defer c.mu.Unlock()
c.closed = true
for id, ch := range c.pending {
ch <- rpcResponse{Error: &rpcError{Code: -1, Message: "rpc closed: " + err.Error()}}
delete(c.pending, id)
}
}
// call issues one request and blocks until the matching response arrives, the
// context is canceled, or the server goes away.
func (c *rpcClient) call(ctx context.Context, method string, params ...any) (json.RawMessage, error) {
c.mu.Lock()
if c.closed {
c.mu.Unlock()
return nil, fmt.Errorf("deltachat rpc: connection closed")
}
c.nextID++
id := c.nextID
ch := make(chan rpcResponse, 1)
c.pending[id] = ch
c.mu.Unlock()
req := rpcRequest{JSONRPC: "2.0", ID: id, Method: method, Params: params}
data, err := json.Marshal(req)
if err != nil {
c.clearPending(id)
return nil, err
}
data = append(data, '\n')
c.mu.Lock()
_, err = c.stdin.Write(data)
c.mu.Unlock()
if err != nil {
c.clearPending(id)
return nil, fmt.Errorf("deltachat rpc write %s: %w", method, err)
}
select {
case <-ctx.Done():
c.clearPending(id)
return nil, ctx.Err()
case resp := <-ch:
if resp.Error != nil {
return nil, resp.Error
}
return resp.Result, nil
}
}
func (c *rpcClient) clearPending(id uint64) {
c.mu.Lock()
delete(c.pending, id)
c.mu.Unlock()
}
// close terminates the RPC server process.
func (c *rpcClient) close() {
c.mu.Lock()
c.closed = true
c.mu.Unlock()
if c.stdin != nil {
_ = c.stdin.Close()
}
if c.cmd != nil && c.cmd.Process != nil {
_ = c.cmd.Process.Kill()
_ = c.cmd.Wait()
}
}
// logWriter forwards deltachat-rpc-server stderr lines into the logger.
type logWriter struct{}
func (logWriter) Write(p []byte) (int, error) {
logger.DebugC("deltachat", string(p))
return len(p), nil
}
+38 -9
View File
@@ -699,18 +699,34 @@ func setStreamerModelName(streamer any, modelName string) {
setter.SetModelName(modelName)
}
type turnUsageStreamer interface {
SetTurnUsage(inputTokens, outputTokens int)
}
// setStreamerTurnUsage forwards real per-turn token usage to a streamer that
// supports it, transparently unwrapping the manager's streamer wrappers.
func setStreamerTurnUsage(streamer any, inputTokens, outputTokens int) {
setter, ok := streamer.(turnUsageStreamer)
if !ok {
return
}
setter.SetTurnUsage(inputTokens, outputTokens)
}
// splitMarkerStreamer turns accumulated streaming text containing
// MessageSplitMarker into separate channel stream messages.
type splitMarkerStreamer struct {
mu sync.Mutex
current bus.Streamer
reasoning bus.ReasoningStreamer
begin func(context.Context) (bus.Streamer, error)
completedParts int
finalized bool
onFinalize func(context.Context, string)
clearMarker func()
modelName string
mu sync.Mutex
current bus.Streamer
reasoning bus.ReasoningStreamer
begin func(context.Context) (bus.Streamer, error)
completedParts int
finalized bool
onFinalize func(context.Context, string)
clearMarker func()
modelName string
turnInputTokens int
turnOutputTokens int
}
func (s *splitMarkerStreamer) Update(ctx context.Context, content string) error {
@@ -761,6 +777,14 @@ func (s *splitMarkerStreamer) SetModelName(modelName string) {
setStreamerModelName(s.reasoning, s.modelName)
}
func (s *splitMarkerStreamer) SetTurnUsage(inputTokens, outputTokens int) {
s.mu.Lock()
defer s.mu.Unlock()
s.turnInputTokens = inputTokens
s.turnOutputTokens = outputTokens
setStreamerTurnUsage(s.current, s.turnInputTokens, s.turnOutputTokens)
}
func (s *splitMarkerStreamer) Cancel(ctx context.Context) {
s.mu.Lock()
defer s.mu.Unlock()
@@ -840,6 +864,7 @@ func (s *splitMarkerStreamer) ensureCurrentLocked(ctx context.Context) error {
}
s.current = streamer
setStreamerModelName(s.current, s.modelName)
setStreamerTurnUsage(s.current, s.turnInputTokens, s.turnOutputTokens)
return nil
}
@@ -928,6 +953,10 @@ func (s *finalizeHookStreamer) SetModelName(modelName string) {
setStreamerModelName(s.Streamer, strings.TrimSpace(modelName))
}
func (s *finalizeHookStreamer) SetTurnUsage(inputTokens, outputTokens int) {
setStreamerTurnUsage(s.Streamer, inputTokens, outputTokens)
}
func (s *finalizeHookStreamer) runFinalizeHook(ctx context.Context, content string) {
if s.onFinalize != nil {
s.onFinalize(ctx, content)
+53
View File
@@ -3383,3 +3383,56 @@ func TestManager_SendPlaceholder(t *testing.T) {
t.Error("expected SendPlaceholder to fail for unknown channel")
}
}
// turnUsageTrackingStreamer is a mockStreamer that records SetTurnUsage calls,
// used to verify the manager's streamer wrappers forward per-turn token usage
// to the inner streamer (regression: the wrappers previously dropped it because
// SetTurnUsage is not part of the bus.Streamer interface).
type turnUsageTrackingStreamer struct {
mockStreamer
inputTokens int
outputTokens int
usageCalls int
}
func (m *turnUsageTrackingStreamer) SetTurnUsage(inputTokens, outputTokens int) {
m.usageCalls++
m.inputTokens = inputTokens
m.outputTokens = outputTokens
}
func TestFinalizeHookStreamerForwardsTurnUsage(t *testing.T) {
inner := &turnUsageTrackingStreamer{}
wrapper := &finalizeHookStreamer{Streamer: inner}
setter, ok := any(wrapper).(turnUsageStreamer)
if !ok {
t.Fatal("finalizeHookStreamer does not satisfy turnUsageStreamer")
}
setter.SetTurnUsage(1234, 567)
if inner.usageCalls != 1 {
t.Fatalf("inner SetTurnUsage calls = %d, want 1", inner.usageCalls)
}
if inner.inputTokens != 1234 || inner.outputTokens != 567 {
t.Errorf("inner usage = (%d, %d), want (1234, 567)", inner.inputTokens, inner.outputTokens)
}
}
func TestSplitMarkerStreamerForwardsTurnUsage(t *testing.T) {
inner := &turnUsageTrackingStreamer{}
wrapper := &splitMarkerStreamer{current: inner}
setter, ok := any(wrapper).(turnUsageStreamer)
if !ok {
t.Fatal("splitMarkerStreamer does not satisfy turnUsageStreamer")
}
setter.SetTurnUsage(1234, 567)
if inner.usageCalls != 1 {
t.Fatalf("inner SetTurnUsage calls = %d, want 1", inner.usageCalls)
}
if inner.inputTokens != 1234 || inner.outputTokens != 567 {
t.Errorf("inner usage = (%d, %d), want (1234, 567)", inner.inputTokens, inner.outputTokens)
}
}
+40 -1
View File
@@ -105,6 +105,8 @@ type PicoChannel struct {
cancel context.CancelFunc
progress *channels.ToolFeedbackAnimator
deleteMessageFn func(context.Context, string, string) error
// broadcastFn lets tests intercept outbound broadcasts. nil → broadcastToSession.
broadcastFn func(chatID string, msg PicoMessage) error
}
// NewPicoChannel creates a new Pico Protocol channel.
@@ -531,6 +533,8 @@ type picoStreamer struct {
channel *PicoChannel
chatID string
modelName string
turnInputTokens int
turnOutputTokens int
messageID string
reasoningID string
throttleInterval time.Duration
@@ -553,6 +557,17 @@ func (s *picoStreamer) SetModelName(modelName string) {
s.modelName = strings.TrimSpace(modelName)
}
// SetTurnUsage records the real per-turn LLM token usage to emit on finalize.
func (s *picoStreamer) SetTurnUsage(inputTokens, outputTokens int) {
if s == nil {
return
}
s.mu.Lock()
defer s.mu.Unlock()
s.turnInputTokens = inputTokens
s.turnOutputTokens = outputTokens
}
func (s *picoStreamer) Update(ctx context.Context, content string) error {
s.mu.Lock()
defer s.mu.Unlock()
@@ -661,8 +676,9 @@ func (s *picoStreamer) sendLocked(ctx context.Context, content string, contextUs
payload[PayloadKeyModelName] = s.modelName
}
setContextUsagePayload(payload, contextUsage)
setTurnUsagePayload(payload, s.turnInputTokens, s.turnOutputTokens)
outMsg := newMessage(TypeMessageCreate, payload)
if err := s.channel.broadcastToSession(s.chatID, outMsg); err != nil {
if err := s.channel.broadcast(s.chatID, outMsg); err != nil {
return err
}
} else if content != s.lastContent || contextUsage != nil {
@@ -673,6 +689,7 @@ func (s *picoStreamer) sendLocked(ctx context.Context, content string, contextUs
if s.modelName != "" {
payload[PayloadKeyModelName] = s.modelName
}
setTurnUsagePayload(payload, s.turnInputTokens, s.turnOutputTokens)
if err := s.channel.editMessagePayload(ctx, s.chatID, s.messageID, payload, contextUsage); err != nil {
return err
}
@@ -932,6 +949,14 @@ func (c *PicoChannel) handleMediaDownload(w http.ResponseWriter, r *http.Request
http.ServeContent(w, r, filename, info.ModTime(), file)
}
// broadcast routes through broadcastFn when set (tests), else broadcastToSession.
func (c *PicoChannel) broadcast(chatID string, msg PicoMessage) error {
if c.broadcastFn != nil {
return c.broadcastFn(chatID, msg)
}
return c.broadcastToSession(chatID, msg)
}
// broadcastToSession sends a message to all connections with a matching session.
func (c *PicoChannel) broadcastToSession(chatID string, msg PicoMessage) error {
// chatID format: "pico:<sessionID>"
@@ -1403,6 +1428,20 @@ func setContextUsagePayload(payload map[string]any, u *bus.ContextUsage) {
}
}
// setTurnUsagePayload attaches real per-turn LLM token usage to the payload.
// Input and output are kept separate (billed at different rates); total is a
// convenience sum. Omitted entirely when both counts are zero.
func setTurnUsagePayload(payload map[string]any, inputTokens, outputTokens int) {
if inputTokens <= 0 && outputTokens <= 0 {
return
}
payload[PayloadKeyUsage] = map[string]any{
"input_tokens": inputTokens,
"output_tokens": outputTokens,
"total_tokens": inputTokens + outputTokens,
}
}
func picoToolCallsPayload(msg bus.OutboundMessage) ([]utils.VisibleToolCall, bool) {
raw := strings.TrimSpace(msg.Context.Raw[PayloadKeyToolCalls])
if raw == "" {
+69
View File
@@ -0,0 +1,69 @@
package pico
import (
"context"
"testing"
)
func TestSetTurnUsagePayload(t *testing.T) {
t.Run("populates usage block when counts present", func(t *testing.T) {
payload := map[string]any{PayloadKeyContent: "hi"}
setTurnUsagePayload(payload, 1234, 567)
raw, ok := payload[PayloadKeyUsage]
if !ok {
t.Fatalf("expected %q key in payload", PayloadKeyUsage)
}
usage, ok := raw.(map[string]any)
if !ok {
t.Fatalf("usage block is not a map: %T", raw)
}
if usage["input_tokens"] != 1234 {
t.Errorf("input_tokens = %v, want 1234", usage["input_tokens"])
}
if usage["output_tokens"] != 567 {
t.Errorf("output_tokens = %v, want 567", usage["output_tokens"])
}
if usage["total_tokens"] != 1801 {
t.Errorf("total_tokens = %v, want 1801", usage["total_tokens"])
}
})
t.Run("omits usage block when both counts zero", func(t *testing.T) {
payload := map[string]any{PayloadKeyContent: "hi"}
setTurnUsagePayload(payload, 0, 0)
if _, ok := payload[PayloadKeyUsage]; ok {
t.Errorf("expected no %q key when counts are zero", PayloadKeyUsage)
}
})
}
// newCaptureStreamer returns a streamer whose broadcasts are captured into the
// returned map pointer, so tests need no live websocket.
func newCaptureStreamer() (*picoStreamer, *map[string]any) {
var last map[string]any
ch := &PicoChannel{}
ch.broadcastFn = func(chatID string, msg PicoMessage) error {
last = msg.Payload
return nil
}
s := &picoStreamer{channel: ch, chatID: "c1"}
return s, &last
}
func TestStreamerEmitsUsageOnFinalize(t *testing.T) {
s, last := newCaptureStreamer()
s.SetTurnUsage(100, 40)
// sendLocked with empty messageID takes the create branch, which attaches
// usage from the streamer's stored counts.
s.mu.Lock()
err := s.sendLocked(context.Background(), "answer", nil)
s.mu.Unlock()
if err != nil {
t.Fatalf("sendLocked: %v", err)
}
if _, ok := (*last)[PayloadKeyUsage]; !ok {
t.Fatalf("expected usage in payload, got %+v", *last)
}
}
+1
View File
@@ -28,6 +28,7 @@ const (
PayloadKeyPlaceholder = "placeholder"
PayloadKeyToolCalls = "tool_calls"
PayloadKeyModelName = "model_name"
PayloadKeyUsage = "usage"
MessageKindThought = "thought"
MessageKindToolCalls = "tool_calls"
+25
View File
@@ -599,6 +599,31 @@ type MatrixSettings struct {
CryptoPassphrase string `json:"crypto_passphrase,omitempty" yaml:"-"`
}
// DeltaChatSettings configures the Delta Chat channel. Delta Chat is an
// email-based, end-to-end encrypted messenger; PicoClaw talks to a local
// `deltachat-rpc-server` process over JSON-RPC (stdio).
//
// Email is the only required setting. A full address selects an already
// configured account in DataDir; a first-run marker such as "@nine.testrun.org"
// creates a chatmail account and tells the user which full email to save.
// Mailbox credentials stay in the Delta Chat account store. DisplayName and
// AvatarImage are optional profile settings applied on startup. Password remains
// only for legacy PicoClaw-managed email configuration.
type DeltaChatSettings struct {
Email string `json:"email" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_EMAIL"`
Password SecureString `json:"password,omitzero" yaml:"password,omitempty" env:"PICOCLAW_CHANNELS_DELTACHAT_PASSWORD"`
DisplayName string `json:"display_name,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_DISPLAY_NAME"`
AvatarImage string `json:"avatar_image,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_AVATAR_IMAGE"`
DataDir string `json:"data_dir,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_DATA_DIR"`
RPCServerPath string `json:"rpc_server_path,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_RPC_SERVER_PATH"`
InviteLink string `json:"invite_link,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_INVITE_LINK"`
AllowCrosspost bool `json:"allow_crosspost,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_ALLOW_CROSSPOST"`
IMAPServer string `json:"imap_server,omitempty" yaml:"-"`
IMAPPort int `json:"imap_port,omitempty" yaml:"-"`
SMTPServer string `json:"smtp_server,omitempty" yaml:"-"`
SMTPPort int `json:"smtp_port,omitempty" yaml:"-"`
}
type LINESettings struct {
ChannelSecret SecureString `json:"channel_secret,omitzero" yaml:"channel_secret,omitempty" env:"PICOCLAW_CHANNELS_LINE_CHANNEL_SECRET"`
ChannelAccessToken SecureString `json:"channel_access_token,omitzero" yaml:"channel_access_token,omitempty" env:"PICOCLAW_CHANNELS_LINE_CHANNEL_ACCESS_TOKEN"`
+2
View File
@@ -27,6 +27,7 @@ const (
ChannelDingTalk = "dingtalk"
ChannelSlack = "slack"
ChannelMatrix = "matrix"
ChannelDeltaChat = "deltachat"
ChannelLINE = "line"
ChannelOneBot = "onebot"
ChannelQQ = "qq"
@@ -669,6 +670,7 @@ var channelSettingsFactory = map[string]any{
ChannelDingTalk: (DingTalkSettings{}),
ChannelSlack: (SlackSettings{}),
ChannelMatrix: (MatrixSettings{}),
ChannelDeltaChat: (DeltaChatSettings{}),
ChannelLINE: (LINESettings{}),
ChannelOneBot: (OneBotSettings{}),
ChannelQQ: (QQSettings{}),
+32
View File
@@ -1019,6 +1019,38 @@ func TestDefaultConfig_ChannelStreamingDisabled(t *testing.T) {
}
}
func TestDefaultConfig_DeltaChatExample(t *testing.T) {
cfg := DefaultConfig()
deltachat := cfg.Channels.Get(ChannelDeltaChat)
if deltachat == nil {
t.Fatal("DefaultConfig() missing deltachat channel")
}
if deltachat.Enabled {
t.Fatal("DefaultConfig().deltachat should be disabled")
}
if !deltachat.GroupTrigger.MentionOnly {
t.Fatal("DefaultConfig().deltachat should use mention-only group trigger")
}
decoded, err := deltachat.GetDecoded()
if err != nil {
t.Fatalf("deltachat GetDecoded() error = %v", err)
}
settings, ok := decoded.(*DeltaChatSettings)
if !ok {
t.Fatalf("deltachat settings type = %T, want *DeltaChatSettings", decoded)
}
if settings.Email != "@nine.testrun.org" {
t.Fatalf("DefaultConfig().deltachat.settings.email = %q, want @nine.testrun.org", settings.Email)
}
if settings.Password.String() != "" {
t.Fatal("DefaultConfig().deltachat.settings.password should be empty")
}
if settings.DisplayName == "" {
t.Fatal("DefaultConfig().deltachat.settings.display_name should be populated")
}
}
func TestValidateSingletonChannels_RejectsMultipleInstances(t *testing.T) {
channels := ChannelsConfig{
"pico1": &Channel{Enabled: true, Type: ChannelPico},
+7
View File
@@ -549,6 +549,13 @@ func defaultChannels() ChannelsConfig {
"join_on_invite": true,
},
},
"deltachat": map[string]any{
"group_trigger": map[string]any{"mention_only": true},
"settings": map[string]any{
"email": "@nine.testrun.org",
"display_name": "PicoClaw Bot",
},
},
"line": map[string]any{
"group_trigger": map[string]any{"mention_only": true},
"settings": map[string]any{
+1
View File
@@ -19,6 +19,7 @@ import (
"github.com/sipeed/picoclaw/pkg/audio/tts"
"github.com/sipeed/picoclaw/pkg/bus"
"github.com/sipeed/picoclaw/pkg/channels"
_ "github.com/sipeed/picoclaw/pkg/channels/deltachat"
_ "github.com/sipeed/picoclaw/pkg/channels/dingtalk"
_ "github.com/sipeed/picoclaw/pkg/channels/discord"
_ "github.com/sipeed/picoclaw/pkg/channels/feishu"
+142
View File
@@ -0,0 +1,142 @@
package providers
import (
"errors"
"regexp"
"strings"
"github.com/sipeed/picoclaw/pkg/providers/common"
)
type AuthErrorKind string
const (
AuthErrorInvalidAPIKey AuthErrorKind = "invalid_api_key"
AuthErrorMissingAPIKey AuthErrorKind = "missing_api_key"
AuthErrorExpiredToken AuthErrorKind = "expired_token"
AuthErrorGeneric AuthErrorKind = "auth"
)
var (
invalidAPIKeyPattern = regexp.MustCompile(
`(?i)\b(?:invalid|incorrect|malformed|wrong)[-_\s]+(?:api[-_\s]*)?key\b|\b(?:api[-_\s]*)?key[-_\s]+(?:is[-_\s]+)?(?:invalid|incorrect|malformed|wrong)\b|\binvalid[-_]?api[-_]?key\b`,
)
missingAPIKeyPattern = regexp.MustCompile(
`(?i)\b(?:missing|no|not[-_\s]+configured)[-_\s]+(?:api[-_\s]*)?key\b|\bno[-_\s]+credentials[-_\s]+found\b|\bapi[-_\s]+key[-_\s]+not[-_\s]+configured\b`,
)
expiredTokenPattern = regexp.MustCompile(
`(?i)\b(?:oauth[-_\s]+token[-_\s]+refresh[-_\s]+failed|re-authenticate|token[-_\s]+(?:has[-_\s]+)?expired|expired[-_\s]+token)\b`,
)
genericAuthPattern = regexp.MustCompile(
`(?i)\b(?:unauthorized|forbidden|authentication[-_\s]+(?:failed|required)|access[-_\s]+denied)\b`,
)
)
func ClassifyAuthError(err error) (AuthErrorKind, bool) {
if err == nil {
return "", false
}
var exhausted *FallbackExhaustedError
if errors.As(err, &exhausted) && exhausted != nil {
return classifyFallbackExhaustedAuthError(exhausted)
}
msg := authErrorText(err)
if missingAPIKeyPattern.MatchString(msg) {
return AuthErrorMissingAPIKey, true
}
if expiredTokenPattern.MatchString(msg) {
return AuthErrorExpiredToken, true
}
if invalidAPIKeyPattern.MatchString(msg) {
return AuthErrorInvalidAPIKey, true
}
if hasStructuredAuthError(err) || genericAuthPattern.MatchString(msg) {
return AuthErrorGeneric, true
}
return "", false
}
func authErrorText(err error) string {
var parts []string
if err != nil {
parts = append(parts, err.Error())
}
var httpErr *common.HTTPError
if errors.As(err, &httpErr) && httpErr != nil {
parts = append(parts, httpErr.BodyPreview)
}
return strings.Join(parts, "\n")
}
func hasStructuredAuthError(err error) bool {
var failErr *FailoverError
if errors.As(err, &failErr) && failErr != nil && failErr.Reason == FailoverAuth {
return true
}
var httpErr *common.HTTPError
if errors.As(err, &httpErr) && httpErr != nil {
return httpErr.StatusCode == 401 || httpErr.StatusCode == 403
}
return false
}
func classifyFallbackExhaustedAuthError(err *FallbackExhaustedError) (AuthErrorKind, bool) {
if err == nil {
return "", false
}
var authMessages []string
nonSkipped := 0
for _, attempt := range err.Attempts {
if attempt.Skipped {
continue
}
nonSkipped++
if !attemptIsAuthFailure(attempt) {
return "", false
}
if attempt.Error != nil {
authMessages = append(authMessages, authErrorText(attempt.Error))
}
}
if nonSkipped == 0 {
return "", false
}
msg := strings.Join(authMessages, "\n")
if missingAPIKeyPattern.MatchString(msg) {
return AuthErrorMissingAPIKey, true
}
if expiredTokenPattern.MatchString(msg) {
return AuthErrorExpiredToken, true
}
if invalidAPIKeyPattern.MatchString(msg) {
return AuthErrorInvalidAPIKey, true
}
return AuthErrorGeneric, true
}
func attemptIsAuthFailure(attempt FallbackAttempt) bool {
if attempt.Reason == FailoverAuth {
return true
}
if attempt.Error == nil {
return false
}
var failErr *FailoverError
if errors.As(attempt.Error, &failErr) && failErr != nil && failErr.Reason == FailoverAuth {
return true
}
var httpErr *common.HTTPError
if errors.As(attempt.Error, &httpErr) && httpErr != nil {
return httpErr.StatusCode == 401 || httpErr.StatusCode == 403
}
return false
}
+133
View File
@@ -0,0 +1,133 @@
package providers
import (
"errors"
"fmt"
"testing"
"github.com/sipeed/picoclaw/pkg/providers/common"
)
func TestClassifyError_HTTPErrorStatus(t *testing.T) {
err := fmt.Errorf("provider request: %w", &common.HTTPError{
StatusCode: httpStatusUnauthorized,
BodyPreview: `{"error":"unauthorized"}`,
ContentType: "application/json",
APIBase: "https://api.example.com",
})
result := ClassifyError(err, "openai", "gpt-4")
if result == nil {
t.Fatal("expected classified error")
}
if result.Reason != FailoverAuth {
t.Fatalf("reason = %q, want %q", result.Reason, FailoverAuth)
}
if result.Status != httpStatusUnauthorized {
t.Fatalf("status = %d, want %d", result.Status, httpStatusUnauthorized)
}
}
func TestClassifyAuthError(t *testing.T) {
tests := []struct {
name string
err error
want AuthErrorKind
}{
{
name: "invalid api key",
err: errors.New(
`API request failed: Status: 401 Body: {"error":{"message":"Incorrect API key provided"}}`,
),
want: AuthErrorInvalidAPIKey,
},
{
name: "missing api key",
err: errors.New("API key not configured"),
want: AuthErrorMissingAPIKey,
},
{
name: "expired token",
err: errors.New("oauth token refresh failed: token has expired"),
want: AuthErrorExpiredToken,
},
{
name: "structured generic auth",
err: &common.HTTPError{
StatusCode: httpStatusUnauthorized,
BodyPreview: `{"error":"unauthorized"}`,
ContentType: "application/json",
APIBase: "https://api.example.com",
},
want: AuthErrorGeneric,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, ok := ClassifyAuthError(tt.err)
if !ok {
t.Fatal("expected auth classification")
}
if got != tt.want {
t.Fatalf("kind = %q, want %q", got, tt.want)
}
})
}
}
func TestClassifyAuthError_FallbackExhaustedAllAuth(t *testing.T) {
err := &FallbackExhaustedError{
Attempts: []FallbackAttempt{
{
Reason: FailoverAuth,
Error: &FailoverError{
Reason: FailoverAuth,
Wrapped: errors.New("invalid api key"),
},
},
{
Reason: FailoverAuth,
Error: &FailoverError{
Reason: FailoverAuth,
Wrapped: errors.New("unauthorized"),
},
},
},
}
got, ok := ClassifyAuthError(err)
if !ok {
t.Fatal("expected auth classification")
}
if got != AuthErrorInvalidAPIKey {
t.Fatalf("kind = %q, want %q", got, AuthErrorInvalidAPIKey)
}
}
func TestClassifyAuthError_FallbackExhaustedMixedFailures(t *testing.T) {
err := &FallbackExhaustedError{
Attempts: []FallbackAttempt{
{
Reason: FailoverAuth,
Error: &FailoverError{
Reason: FailoverAuth,
Wrapped: errors.New("invalid api key"),
},
},
{
Reason: FailoverRateLimit,
Error: &FailoverError{
Reason: FailoverRateLimit,
Wrapped: errors.New("rate limit exceeded"),
},
},
},
}
if got, ok := ClassifyAuthError(err); ok {
t.Fatalf("kind = %q, want no auth classification for mixed failures", got)
}
}
const httpStatusUnauthorized = 401
+43 -13
View File
@@ -371,6 +371,36 @@ func DecodeToolCallArguments(raw json.RawMessage, name string) map[string]any {
// --- HTTP response helpers ---
// HTTPError captures structured details for non-OK provider HTTP responses.
// Error preserves the previous user/log-facing text format for compatibility.
type HTTPError struct {
StatusCode int
BodyPreview string
ContentType string
APIBase string
IsHTML bool
}
func (e *HTTPError) Error() string {
if e == nil {
return "API request failed"
}
if e.IsHTML {
return fmt.Sprintf(
"API request failed: %s returned HTML instead of JSON (content-type: %s); check api_base or proxy configuration.\n Status: %d\n Body: %s",
e.APIBase,
e.ContentType,
e.StatusCode,
e.BodyPreview,
)
}
return fmt.Sprintf(
"API request failed:\n Status: %d\n Body: %s",
e.StatusCode,
e.BodyPreview,
)
}
// HandleErrorResponse reads a non-200 response body and returns an appropriate error.
func HandleErrorResponse(resp *http.Response, apiBase string) error {
contentType := resp.Header.Get("Content-Type")
@@ -381,11 +411,12 @@ func HandleErrorResponse(resp *http.Response, apiBase string) error {
if LooksLikeHTML(body, contentType) {
return WrapHTMLResponseError(resp.StatusCode, body, contentType, apiBase)
}
return fmt.Errorf(
"API request failed:\n Status: %d\n Body: %s",
resp.StatusCode,
ResponsePreview(body, 128),
)
return &HTTPError{
StatusCode: resp.StatusCode,
BodyPreview: ResponsePreview(body, 128),
ContentType: contentType,
APIBase: apiBase,
}
}
// ReadAndParseResponse peeks at the response body to detect HTML errors,
@@ -422,14 +453,13 @@ func LooksLikeHTML(body []byte, contentType string) bool {
// WrapHTMLResponseError creates a descriptive error for HTML responses.
func WrapHTMLResponseError(statusCode int, body []byte, contentType, apiBase string) error {
respPreview := ResponsePreview(body, 128)
return fmt.Errorf(
"API request failed: %s returned HTML instead of JSON (content-type: %s); check api_base or proxy configuration.\n Status: %d\n Body: %s",
apiBase,
contentType,
statusCode,
respPreview,
)
return &HTTPError{
StatusCode: statusCode,
BodyPreview: ResponsePreview(body, 128),
ContentType: contentType,
APIBase: apiBase,
IsHTML: true,
}
}
// ResponsePreview returns a truncated preview of response body for error messages.
+59
View File
@@ -0,0 +1,59 @@
package common
import (
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestHandleErrorResponse_ReturnsHTTPError(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusUnauthorized)
_, _ = w.Write([]byte(`{"error":"unauthorized"}`))
}))
defer server.Close()
resp, err := http.Get(server.URL)
if err != nil {
t.Fatalf("http.Get() error = %v", err)
}
defer resp.Body.Close()
err = HandleErrorResponse(resp, server.URL)
var httpErr *HTTPError
if !errors.As(err, &httpErr) {
t.Fatalf("HandleErrorResponse() error = %T, want *HTTPError", err)
}
if httpErr.StatusCode != http.StatusUnauthorized {
t.Fatalf("StatusCode = %d, want %d", httpErr.StatusCode, http.StatusUnauthorized)
}
if httpErr.BodyPreview != `{"error":"unauthorized"}` {
t.Fatalf("BodyPreview = %q", httpErr.BodyPreview)
}
if !strings.Contains(err.Error(), "Status: 401") {
t.Fatalf("Error() should preserve status text, got %q", err.Error())
}
}
func TestWrapHTMLResponseError_ReturnsHTTPError(t *testing.T) {
err := WrapHTMLResponseError(
http.StatusBadGateway,
[]byte("<html>bad</html>"),
"text/html",
"https://api.example.com",
)
var httpErr *HTTPError
if !errors.As(err, &httpErr) {
t.Fatalf("WrapHTMLResponseError() error = %T, want *HTTPError", err)
}
if !httpErr.IsHTML {
t.Fatal("expected IsHTML")
}
if !strings.Contains(err.Error(), "HTML instead of JSON") {
t.Fatalf("Error() should preserve HTML message, got %q", err.Error())
}
}
+16
View File
@@ -8,6 +8,8 @@ import (
"regexp"
"strings"
"syscall"
"github.com/sipeed/picoclaw/pkg/providers/common"
)
// Common patterns in Go HTTP error messages
@@ -88,6 +90,8 @@ var (
}
authPatterns = []errorPattern{
rxp(`\b(?:invalid|incorrect|malformed|wrong)[-_\s]+(?:api[-_\s]*)?key\b`),
rxp(`\b(?:api[-_\s]*)?key[-_\s]+(?:is[-_\s]+)?(?:invalid|incorrect|malformed|wrong)\b`),
rxp(`invalid[_ ]?api[_ ]?key`),
substr("incorrect api key"),
substr("invalid token"),
@@ -188,6 +192,18 @@ func ClassifyError(err error, provider, model string) *FailoverError {
}
// Try HTTP status code extraction first.
var httpErr *common.HTTPError
if errors.As(err, &httpErr) && httpErr != nil {
if reason := classifyByStatus(httpErr.StatusCode); reason != "" {
return &FailoverError{
Reason: reason,
Provider: provider,
Model: model,
Status: httpErr.StatusCode,
Wrapped: err,
}
}
}
if status := extractHTTPStatus(msg); status > 0 {
if reason := classifyByStatus(status); reason != "" {
return &FailoverError{
+1
View File
@@ -7,6 +7,7 @@ import (
"encoding/json"
"fmt"
"io"
"log"
"maps"
"net/http"
"net/url"
+13 -2
View File
@@ -1064,8 +1064,8 @@ func (r *sandboxFs) execute(path string, fn func(root *os.Root, relPath string)
return err
}
// os.Root api on windows only accept forward slashes (/)
relPath = filepath.ToSlash(relPath)
// os.Root API on Windows only accepts forward slashes (/).
relPath = normalizeRootRelPath(relPath)
return fn(root, relPath)
}
@@ -1230,6 +1230,17 @@ func buildFs(workspace string, restrict bool, patterns []*regexp.Regexp) fileSys
return sandbox
}
func normalizeRootRelPath(relPath string) string {
return normalizeRootRelPathForSeparator(relPath, os.PathSeparator)
}
func normalizeRootRelPathForSeparator(relPath string, sep rune) string {
if sep == '\\' {
return strings.ReplaceAll(relPath, `\`, `/`)
}
return relPath
}
// Helper to get a safe relative path for os.Root usage
func getSafeRelPath(workspace, path string) (string, error) {
if workspace == "" {
@@ -0,0 +1,21 @@
package fstools
import "testing"
func TestNormalizeRootRelPathForWindowsSeparator(t *testing.T) {
got := normalizeRootRelPathForSeparator(`aaa\bbb\file.txt`, '\\')
want := "aaa/bbb/file.txt"
if got != want {
t.Fatalf("normalizeRootRelPathForSeparator() = %q, want %q", got, want)
}
}
func TestNormalizeRootRelPathForUnixSeparatorLeavesBackslashUnchanged(t *testing.T) {
input := `aaa\bbb\file.txt`
got := normalizeRootRelPathForSeparator(input, '/')
if got != input {
t.Fatalf("normalizeRootRelPathForSeparator() = %q, want %q", got, input)
}
}
+4 -4
View File
@@ -315,7 +315,7 @@ func (p *BraveSearchProvider) Search(
}
body, err := io.ReadAll(resp.Body)
resp.Body.Close()
_ = resp.Body.Close()
if err != nil {
lastErr = fmt.Errorf("failed to read response: %w", err)
@@ -448,7 +448,7 @@ func (p *TavilySearchProvider) Search(
}
body, err := io.ReadAll(resp.Body)
resp.Body.Close()
_ = resp.Body.Close()
if err != nil {
lastErr = fmt.Errorf("failed to read response: %w", err)
@@ -906,7 +906,7 @@ func (p *SogouSearchProvider) Search(
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
_ = resp.Body.Close()
if err != nil {
return "", fmt.Errorf("failed to read response: %w", err)
}
@@ -1147,7 +1147,7 @@ func (p *PerplexitySearchProvider) Search(
}
body, err := io.ReadAll(resp.Body)
resp.Body.Close()
_ = resp.Body.Close()
if err != nil {
lastErr = fmt.Errorf("failed to read response: %w", err)
+20 -21
View File
@@ -1135,36 +1135,35 @@ func expandPowerShellEnvVars(cmd string) string {
})
}
func (t *ExecTool) commandMatchesAllowPattern(lower string) bool {
for _, pattern := range t.allowPatterns {
if pattern.MatchString(lower) {
return true
}
}
for _, pattern := range t.customAllowPatterns {
if pattern.MatchString(lower) {
return true
}
}
return false
}
func (t *ExecTool) guardCommand(command, cwd string) string {
cmd := strings.TrimSpace(command)
lower := strings.ToLower(cmd)
// Custom allow patterns exempt a command from deny checks.
explicitlyAllowed := false
for _, pattern := range t.customAllowPatterns {
// Deny patterns always apply, even when a command matches a custom allow rule.
// Custom allow rules can permit a command, but must not disable secret-safety
// deny rules such as jq env access checks (#3079).
for _, pattern := range t.denyPatterns {
if pattern.MatchString(lower) {
explicitlyAllowed = true
break
}
}
if !explicitlyAllowed {
for _, pattern := range t.denyPatterns {
if pattern.MatchString(lower) {
return "Command blocked by safety guard (dangerous pattern detected)"
}
return "Command blocked by safety guard (dangerous pattern detected)"
}
}
if len(t.allowPatterns) > 0 {
allowed := false
for _, pattern := range t.allowPatterns {
if pattern.MatchString(lower) {
allowed = true
break
}
}
if !allowed {
if !t.commandMatchesAllowPattern(lower) {
return "Command blocked by safety guard (not in allowlist)"
}
}
+51
View File
@@ -1936,3 +1936,54 @@ func TestShellTool_SchemelessURLDetection(t *testing.T) {
}
}
}
func TestShellTool_CustomAllowDoesNotBypassDenyPatterns(t *testing.T) {
cfg := &config.Config{}
cfg.Tools.Exec.EnableDenyPatterns = true
cfg.Tools.Exec.CustomAllowPatterns = []string{`^jq\b`}
cfg.Tools.Exec.CustomDenyPatterns = []string{`\$env\b`, `(^|[^.$a-z0-9_])env([^a-z0-9_]|$)`}
tool, err := NewExecToolWithConfig(t.TempDir(), false, cfg)
if err != nil {
t.Fatalf("NewExecToolWithConfig() error: %v", err)
}
got := tool.guardCommand(`jq -n '$ENV.PICOCLAW_VARIANT_CANARY'`, t.TempDir())
if !strings.Contains(got, "dangerous pattern detected") {
t.Fatalf("custom allow should not bypass deny patterns, got: %q", got)
}
}
func TestShellTool_CustomAllowStillPermitsSafeMatch(t *testing.T) {
cfg := &config.Config{}
cfg.Tools.Exec.EnableDenyPatterns = true
cfg.Tools.Exec.CustomAllowPatterns = []string{`^jq\b`}
cfg.Tools.Exec.CustomDenyPatterns = []string{`\$env\b`, `(^|[^.$a-z0-9_])env([^a-z0-9_]|$)`}
tool, err := NewExecToolWithConfig(t.TempDir(), false, cfg)
if err != nil {
t.Fatalf("NewExecToolWithConfig() error: %v", err)
}
got := tool.guardCommand(`jq -n '"ok"'`, t.TempDir())
if got != "" {
t.Fatalf("safe custom-allowed command should pass guard, got: %q", got)
}
}
func TestShellTool_CustomAllowDoesNotBecomeStrictAllowlist(t *testing.T) {
cfg := &config.Config{}
cfg.Tools.Exec.EnableDenyPatterns = true
cfg.Tools.Exec.CustomAllowPatterns = []string{`^jq\b`}
cfg.Tools.Exec.CustomDenyPatterns = []string{`\$env\b`, `(^|[^.$a-z0-9_])env([^a-z0-9_]|$)`}
tool, err := NewExecToolWithConfig(t.TempDir(), false, cfg)
if err != nil {
t.Fatalf("NewExecToolWithConfig() error: %v", err)
}
got := tool.guardCommand("ls", t.TempDir())
if got != "" {
t.Fatalf("custom allow patterns should not become a strict allowlist, got: %q", got)
}
}
+61
View File
@@ -7,6 +7,7 @@ import (
"fmt"
"io"
"net/http"
"net/url"
"strings"
"github.com/sipeed/picoclaw/web/backend/middleware"
@@ -89,6 +90,60 @@ func (h *launcherAuthHandlers) isStoreInitialized(ctx context.Context) (bool, er
return h.store.IsInitialized(ctx)
}
func launcherSetupCrossSite(r *http.Request) bool {
fetchSite := strings.ToLower(strings.TrimSpace(r.Header.Get("Sec-Fetch-Site")))
if fetchSite == "cross-site" {
return true
}
if origin := strings.TrimSpace(r.Header.Get("Origin")); origin != "" {
return !sameLauncherRequestOrigin(r, origin)
}
if referer := strings.TrimSpace(r.Header.Get("Referer")); referer != "" {
return !sameLauncherRequestOrigin(r, referer)
}
return false
}
func sameLauncherRequestOrigin(r *http.Request, raw string) bool {
if strings.ContainsAny(raw, " \t\r\n") {
return false
}
u, err := url.Parse(raw)
if err != nil || u.Scheme == "" || u.Host == "" {
return false
}
wantScheme := launcherRequestScheme(r)
wantHost := r.Host
if wantHost == "" {
wantHost = r.URL.Host
}
return strings.EqualFold(u.Scheme, wantScheme) && strings.EqualFold(u.Host, wantHost)
}
func launcherRequestScheme(r *http.Request) string {
if proto := strings.TrimSpace(r.Header.Get("X-Forwarded-Proto")); proto != "" {
if i := strings.IndexByte(proto, ','); i >= 0 {
proto = proto[:i]
}
proto = strings.ToLower(strings.TrimSpace(proto))
if proto == "http" || proto == "https" {
return proto
}
}
if r.TLS != nil {
return "https"
}
if r.URL != nil && r.URL.Scheme != "" {
return r.URL.Scheme
}
return "http"
}
func (h *launcherAuthHandlers) handleLogin(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var body launcherAuthLoginBody
@@ -198,6 +253,12 @@ func (h *launcherAuthHandlers) handleStatus(w http.ResponseWriter, r *http.Reque
func (h *launcherAuthHandlers) handleSetup(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
if launcherSetupCrossSite(r) {
w.WriteHeader(http.StatusForbidden)
_, _ = w.Write([]byte(`{"error":"cross-site setup request rejected"}`))
return
}
if h.store == nil {
w.WriteHeader(http.StatusServiceUnavailable)
if h.storeErr != nil {
+63
View File
@@ -0,0 +1,63 @@
package api
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestLauncherAuthSetupRejectsCrossSiteFirstRun(t *testing.T) {
store := &fakePasswordStore{}
mux := http.NewServeMux()
RegisterLauncherAuthRoutes(mux, LauncherAuthRouteOpts{
SessionCookie: "session-cookie-value",
PasswordStore: store,
})
rec := httptest.NewRecorder()
req := httptest.NewRequest(
http.MethodPost,
"http://127.0.0.1:18800/api/auth/setup",
strings.NewReader(`{"password":"CrossSitePwn123!","confirm":"CrossSitePwn123!"}`),
)
req.Header.Set("Origin", "https://evil.example")
req.Header.Set("Referer", "https://evil.example/attack")
req.Header.Set("Sec-Fetch-Site", "cross-site")
req.Header.Set("Content-Type", "application/json")
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusForbidden {
t.Fatalf("cross-site setup code = %d body=%s", rec.Code, rec.Body.String())
}
if store.initialized || store.password != "" {
t.Fatalf("cross-site setup mutated store: initialized=%v password=%q", store.initialized, store.password)
}
}
func TestLauncherAuthSetupAllowsSameOriginFirstRun(t *testing.T) {
store := &fakePasswordStore{}
mux := http.NewServeMux()
RegisterLauncherAuthRoutes(mux, LauncherAuthRouteOpts{
SessionCookie: "session-cookie-value",
PasswordStore: store,
})
rec := httptest.NewRecorder()
req := httptest.NewRequest(
http.MethodPost,
"http://127.0.0.1:18800/api/auth/setup",
strings.NewReader(`{"password":"LocalSetup123!","confirm":"LocalSetup123!"}`),
)
req.Header.Set("Origin", "http://127.0.0.1:18800")
req.Header.Set("Sec-Fetch-Site", "same-origin")
req.Header.Set("Content-Type", "application/json")
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("same-origin setup code = %d body=%s", rec.Code, rec.Body.String())
}
if !store.initialized || store.password != "LocalSetup123!" {
t.Fatalf("same-origin setup store: initialized=%v password=%q", store.initialized, store.password)
}
}
+4 -4
View File
@@ -35,14 +35,14 @@
"radix-ui": "^1.4.3",
"react": "19.2.5",
"react-dom": "19.2.5",
"react-i18next": "^17.0.6",
"react-i18next": "^17.0.7",
"react-markdown": "^10.1.0",
"react-textarea-autosize": "^8.5.9",
"rehype-highlight": "^7.0.2",
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"remark-gfm": "^4.0.1",
"shadcn": "^4.7.0",
"shadcn": "^4.12.0",
"sonner": "^2.0.7",
"tailwind-merge": "^3.5.0",
"tailwindcss": "^4.3.0",
@@ -58,7 +58,7 @@
"@types/react": "^19.2.7",
"@types/react-dom": "^19.2.3",
"@typescript-eslint/eslint-plugin": "^8.58.2",
"@vitejs/plugin-react": "^6.0.1",
"@vitejs/plugin-react": "^6.0.3",
"eslint": "^10.4.1",
"eslint-config-prettier": "^10.1.8",
"eslint-plugin-react-hooks": "^7.1.1",
@@ -67,7 +67,7 @@
"prettier": "^3.8.3",
"prettier-plugin-tailwindcss": "^0.8.0",
"typescript": "~5.9.3",
"typescript-eslint": "^8.59.3",
"typescript-eslint": "^8.62.1",
"vite": "^8.0.16"
}
}
+701 -776
View File
File diff suppressed because it is too large Load Diff