mirror of
https://github.com/sipeed/picoclaw.git
synced 2026-07-28 01:27:58 +00:00
Compare commits
34 Commits
52320f4875
...
4c5adcd78e
| Author | SHA1 | Date | |
|---|---|---|---|
| 4c5adcd78e | |||
| cf24f32896 | |||
| 4ccb6268a4 | |||
| 1d9b1c444a | |||
| 0aff1bd7ab | |||
| 1a6dd0efe5 | |||
| 4b02293516 | |||
| 883d43b37d | |||
| 3b24a48a8c | |||
| 027226997f | |||
| ce5274d179 | |||
| 79ae6bf97f | |||
| 93a58e057e | |||
| 70b9cb9ea0 | |||
| 8fd07bcacb | |||
| e56ab1f16e | |||
| addaef78ad | |||
| ba881f8273 | |||
| 612e485d4e | |||
| 2cf030d2fd | |||
| c3aa8c088c | |||
| acc2c5c6aa | |||
| ff247b63ac | |||
| 0c404869ae | |||
| 263e940825 | |||
| 9721c36e55 | |||
| 1758eea948 | |||
| f5b2ce7482 | |||
| 46ffda264f | |||
| 88bda73db6 | |||
| 29e019ec66 | |||
| cc7b4ca86b | |||
| 697e94fe8c | |||
| 052c742fe7 |
@@ -494,6 +494,7 @@ Talk to your PicoClaw through 19+ messaging platforms:
|
||||
| **QQ** | Easy (AppID + AppSecret) | WebSocket | [Guide](docs/channels/qq/README.md) |
|
||||
| **Slack** | Easy (bot + app token) | Socket Mode | [Guide](docs/channels/slack/README.md) |
|
||||
| **Matrix** | Medium (homeserver + token) | Sync API | [Guide](docs/channels/matrix/README.md) |
|
||||
| **Delta Chat** | Easy (account script or email/password) | JSON-RPC (email/E2EE) | [Guide](docs/channels/deltachat/README.md) |
|
||||
| **DingTalk** | Medium (client credentials) | Stream | [Guide](docs/channels/dingtalk/README.md) |
|
||||
| **Feishu / Lark** | Medium (App ID + Secret) | WebSocket/SDK | [Guide](docs/channels/feishu/README.md) |
|
||||
| **LINE** | Medium (credentials + webhook) | Webhook | [Guide](docs/channels/line/README.md) |
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
> Back to [README](../../../README.md)
|
||||
|
||||
# Delta Chat Channel
|
||||
|
||||
PicoClaw can run as a Delta Chat bot by launching a local
|
||||
`deltachat-rpc-server` process and talking to it over JSON-RPC. The RPC server
|
||||
handles the email account, IMAP/SMTP connection, message store, and encryption
|
||||
keys.
|
||||
|
||||
## Install
|
||||
|
||||
Install the Delta Chat RPC server. If `deltachat-rpc-server` is on `PATH`,
|
||||
PicoClaw can find it automatically; otherwise set `rpc_server_path` to the
|
||||
exact binary path.
|
||||
|
||||
```bash
|
||||
pip install deltachat-rpc-server
|
||||
which deltachat-rpc-server
|
||||
```
|
||||
|
||||
Prebuilt binaries are also available from the
|
||||
[Delta Chat core releases](https://github.com/deltachat/deltachat-core-rust/releases).
|
||||
|
||||
## Configure
|
||||
|
||||
The easiest setup is to let PicoClaw create a chatmail account in Delta
|
||||
Chat's local account store. Put a relay marker in `email` using an empty local
|
||||
part, for example `@nine.testrun.org`:
|
||||
|
||||
```json
|
||||
{
|
||||
"channel_list": {
|
||||
"deltachat": {
|
||||
"enabled": true,
|
||||
"type": "deltachat",
|
||||
"allow_from": ["friend@example.org"],
|
||||
"group_trigger": {
|
||||
"mention_only": true
|
||||
},
|
||||
"settings": {
|
||||
"email": "@nine.testrun.org",
|
||||
"display_name": "PicoClaw Bot",
|
||||
"avatar_image": "/home/me/bot-avatar.png"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
On startup, PicoClaw creates the account through `deltachat-rpc-server`, then
|
||||
stops with an error that contains the generated address. Replace the relay
|
||||
marker with that full email address and run PicoClaw again:
|
||||
|
||||
```json
|
||||
{
|
||||
"email": "bot123@nine.testrun.org",
|
||||
"display_name": "PicoClaw Bot",
|
||||
"avatar_image": "/home/me/bot-avatar.png"
|
||||
}
|
||||
```
|
||||
|
||||
If `email` is missing, the startup error lists the built-in relay choices copied
|
||||
from Parla. You can use one of those relay markers, or a custom chatmail relay
|
||||
with the same `@server.name` form.
|
||||
|
||||
`password` is not needed for PicoClaw-created chatmail accounts. Omit it when
|
||||
`email` points to an already configured account in `data_dir`; the JSON-RPC
|
||||
server owns the mailbox password. The legacy password-based path remains only
|
||||
for classic email accounts that PicoClaw must configure itself. In that mode,
|
||||
`password` is a secure field; on first config load it is moved to
|
||||
`~/.picoclaw/.security.yml`, and it can also be set with
|
||||
`PICOCLAW_CHANNELS_DELTACHAT_PASSWORD`.
|
||||
|
||||
`display_name` and `avatar_image` are optional profile settings. When present,
|
||||
PicoClaw applies them on every startup, so changing the avatar path in config is
|
||||
enough to update the bot profile.
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `email` | Yes | Full bot mailbox address, or first-run relay marker such as `@nine.testrun.org` |
|
||||
| `rpc_server_path` | No | Path to `deltachat-rpc-server`; only needed when it is not on `PATH` |
|
||||
| `password` | No | Legacy only; required when PicoClaw must configure/reconfigure a classic mailbox itself |
|
||||
| `display_name` | No | Startup-applied profile name shown to contacts and used for group mention detection |
|
||||
| `avatar_image` | No | Startup-applied profile avatar image path; `~` is expanded. Missing files are warned and ignored |
|
||||
| `data_dir` | No | Account database directory. Default: `~/.picoclaw/deltachat/<channel-name>` |
|
||||
| `invite_link` | No | Delta Chat invite link to join on startup |
|
||||
| `allow_crosspost` | No | Default `false`. When `true`, senders allowed by `allow_from` may use `message` tool targets outside the current chat, or resolve recipients by email/contact/chat name |
|
||||
| `imap_server`, `imap_port` | No | Manual IMAP override for password-based configuration |
|
||||
| `smtp_server`, `smtp_port` | No | Manual SMTP override for password-based configuration |
|
||||
|
||||
Standard channel fields such as `allow_from`, `group_trigger`, and
|
||||
`reasoning_channel_id` also apply.
|
||||
|
||||
## First Run
|
||||
|
||||
With `email` set to `@server`, PicoClaw creates the chatmail account, prints
|
||||
the generated full email in the startup error, and exits. Update `email` to that
|
||||
full address and run PicoClaw again. On later runs, PicoClaw selects the
|
||||
configured account by `email`, applies optional profile settings, marks it as a
|
||||
bot, and starts IO.
|
||||
|
||||
With a new `data_dir` plus legacy `password`, PicoClaw can still configure a
|
||||
classic email account and validate the mailbox credentials; after that, the
|
||||
account is reused from the local data directory.
|
||||
|
||||
Delta Chat requires peers to learn the bot's encryption key before messaging
|
||||
it. On startup PicoClaw prints the bot invite link and QR code. Add the bot from
|
||||
Delta Chat with that invite, not by typing the bare email address.
|
||||
|
||||
## Behavior
|
||||
|
||||
- Direct chats always respond after `allow_from` passes.
|
||||
- Group chats follow `group_trigger`; without one, every group message is
|
||||
handled.
|
||||
- Messages from the bot itself, device chats, and info/system messages are
|
||||
ignored.
|
||||
- Accepted inbound messages are marked seen after the allow-list check.
|
||||
- Incoming attachments (images, audio, video, documents) are registered with
|
||||
the media store and handed to the agent, so it can view images or operate on
|
||||
the files directly. If no media store is available, the path is appended
|
||||
inline as `[attachment: /path]` instead.
|
||||
- Outbound attachments are supported: when the agent emits media, each file is
|
||||
sent as a Delta Chat message (with the caption as text). Delta Chat infers the
|
||||
view type from the file, so images, GIFs, and videos render natively.
|
||||
- Crosspost recipient lookup is disabled by default. The agent can always reply
|
||||
to the current numeric chat ID, but sending to another numeric chat ID or
|
||||
resolving an email/contact/chat name requires `allow_crosspost: true`
|
||||
and the current sender must pass `allow_from`; `allow_from: ["*"]` allows this
|
||||
for any sender.
|
||||
- Voice is supported in both directions when voice providers are configured:
|
||||
incoming voice notes are transcribed by the agent's ASR and the transcript is
|
||||
passed to the model; the agent can reply with synthesized speech, which is
|
||||
delivered as a native Delta Chat voice message (`send_tts`). This requires an
|
||||
ASR and/or TTS provider under `voice` — it is not Delta Chat-specific config.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Fix |
|
||||
|---------|-----|
|
||||
| `deltachat-rpc-server not found on PATH` or `rpc_server_path ... not found` | Install the RPC server on PATH, or set `rpc_server_path` to an absolute path |
|
||||
| `email is required` | Choose one listed chatmail server, set `email` to a first-run marker such as `@nine.testrun.org`, run `picoclaw g`, then replace it with the generated full email |
|
||||
| `created chatmail account ...` | Replace the `@server` marker in `email` with the generated full email and run PicoClaw again |
|
||||
| `account ... is not configured in data_dir` | Point `data_dir` at the existing JSON-RPC account store, or use `email="@server"` to create one |
|
||||
| `configure (check email/password/server)` | Check credentials, app password requirements, or IMAP/SMTP overrides |
|
||||
| Bot does not answer in a group | Check `group_trigger`; mention `display_name` or use a configured prefix |
|
||||
| Bot ignores a sender | Add the sender email to `allow_from`, or use `["*"]` for open access |
|
||||
| Sender cannot message the bot | Re-add the bot with the startup QR/invite so Delta Chat can establish encryption |
|
||||
| Agent cannot send to an email/name/other chat ID | Enable `settings.allow_crosspost` and allow the controlling sender in `allow_from`; this capability is disabled by default for privacy |
|
||||
@@ -8,7 +8,7 @@ require (
|
||||
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.0
|
||||
github.com/SevereCloud/vksdk/v3 v3.3.1
|
||||
github.com/adhocore/gronx v1.20.0
|
||||
github.com/anthropics/anthropic-sdk-go v1.50.2
|
||||
github.com/anthropics/anthropic-sdk-go v1.55.1
|
||||
github.com/atc0005/go-teams-notify/v2 v2.14.0
|
||||
github.com/aws/aws-sdk-go-v2 v1.42.0
|
||||
github.com/aws/aws-sdk-go-v2/config v1.32.25
|
||||
@@ -119,7 +119,7 @@ require (
|
||||
go.opentelemetry.io/otel/trace v1.35.0 // indirect
|
||||
go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect
|
||||
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
|
||||
golang.org/x/text v0.37.0 // indirect
|
||||
golang.org/x/text v0.38.0 // indirect
|
||||
modernc.org/libc v1.73.4 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
@@ -148,7 +148,7 @@ require (
|
||||
github.com/valyala/fastjson v1.6.10 // indirect
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
|
||||
golang.org/x/arch v0.24.0 // indirect
|
||||
golang.org/x/crypto v0.51.0
|
||||
golang.org/x/crypto v0.53.0
|
||||
golang.org/x/net v0.55.0
|
||||
golang.org/x/sync v0.21.0
|
||||
golang.org/x/sys v0.46.0
|
||||
|
||||
@@ -29,8 +29,8 @@ github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883 h1:bvNMNQO63//z+xNg
|
||||
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
|
||||
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
|
||||
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
|
||||
github.com/anthropics/anthropic-sdk-go v1.50.2 h1:K+YJWWzeN2h5MAbh9xeUWY8yAB2oOMp2xLLAODrVBXA=
|
||||
github.com/anthropics/anthropic-sdk-go v1.50.2/go.mod h1:3EfIfmFqxH6rbiLcIP4tPFyXL/IHakx2wDG4OU+TIEI=
|
||||
github.com/anthropics/anthropic-sdk-go v1.55.1 h1:GxukHUVou6AFIngxa/Aw1z79hmwg13Hmn++KE9werbM=
|
||||
github.com/anthropics/anthropic-sdk-go v1.55.1/go.mod h1:3EfIfmFqxH6rbiLcIP4tPFyXL/IHakx2wDG4OU+TIEI=
|
||||
github.com/atc0005/go-teams-notify/v2 v2.14.0 h1:7N+xw+COnYANLREaAveQ65rsNQ12nIZJED9nMLyscCo=
|
||||
github.com/atc0005/go-teams-notify/v2 v2.14.0/go.mod h1:EECsWM2b0Hvoz7O+QdlsvyN2KCUOFQCGj8bUBXv3A3Q=
|
||||
github.com/aws/aws-sdk-go-v2 v1.42.0 h1:XvXMJTkFQtpBKIWZnmr9ZEOc2InWM2yldjXEJ/bymhA=
|
||||
@@ -362,8 +362,8 @@ golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWP
|
||||
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
|
||||
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
|
||||
golang.org/x/crypto v0.16.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
|
||||
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
|
||||
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
|
||||
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
|
||||
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
|
||||
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
|
||||
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
|
||||
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
@@ -438,8 +438,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
|
||||
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
|
||||
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
|
||||
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
|
||||
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
|
||||
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
|
||||
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
|
||||
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
|
||||
@@ -100,10 +100,15 @@ func resolveMediaRefs(
|
||||
|
||||
localPath, meta, err := store.ResolveWithMeta(ref)
|
||||
if err != nil {
|
||||
logger.WarnCF("agent", "Failed to resolve media ref", map[string]any{
|
||||
fields := map[string]any{
|
||||
"ref": ref,
|
||||
"error": err.Error(),
|
||||
})
|
||||
}
|
||||
if idx < currentTurnStart {
|
||||
logger.DebugCF("agent", "Skipped stale historical media ref", fields)
|
||||
} else {
|
||||
logger.WarnCF("agent", "Failed to resolve media ref", fields)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -21,7 +20,7 @@ func (al *AgentLoop) maybePublishError(ctx context.Context, channel, chatID, ses
|
||||
if errors.Is(err, context.Canceled) {
|
||||
return false
|
||||
}
|
||||
al.PublishResponseIfNeeded(ctx, channel, chatID, sessionKey, fmt.Sprintf("Error processing message: %v", err))
|
||||
al.PublishResponseIfNeeded(ctx, channel, chatID, sessionKey, formatProcessingError(err))
|
||||
return true
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/providers"
|
||||
)
|
||||
|
||||
func formatProcessingError(err error) string {
|
||||
if err == nil {
|
||||
return ""
|
||||
}
|
||||
|
||||
if kind, ok := providers.ClassifyAuthError(err); ok {
|
||||
return fmt.Sprintf(
|
||||
"Error processing message: %s\n\nOriginal error:\n%s",
|
||||
authErrorFriendlyMessage(kind),
|
||||
err.Error(),
|
||||
)
|
||||
}
|
||||
|
||||
return fmt.Sprintf("Error processing message: %v", err)
|
||||
}
|
||||
|
||||
func authErrorFriendlyMessage(kind providers.AuthErrorKind) string {
|
||||
switch kind {
|
||||
case providers.AuthErrorInvalidAPIKey:
|
||||
return "Authentication failed: the API key appears to be invalid. Check the API key configured for this model or provider."
|
||||
case providers.AuthErrorMissingAPIKey:
|
||||
return "Authentication failed: no API key is configured for this model or provider. Add an API key in the model settings or config."
|
||||
case providers.AuthErrorExpiredToken:
|
||||
return "Authentication failed: the saved login or token appears to be expired. Re-authenticate the provider."
|
||||
default:
|
||||
return "Authentication failed: check the API key, token, OAuth login, or provider permissions for this model."
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/providers/common"
|
||||
)
|
||||
|
||||
func TestFormatProcessingError_InvalidAPIKey(t *testing.T) {
|
||||
err := errors.New(
|
||||
`LLM call failed after retries: API request failed: Status: 401 Body: {"error":{"message":"Incorrect API key provided"}}`,
|
||||
)
|
||||
|
||||
got := formatProcessingError(err)
|
||||
if !strings.Contains(got, "API key appears to be invalid") {
|
||||
t.Fatalf("formatted error missing friendly API key hint: %q", got)
|
||||
}
|
||||
if !strings.Contains(got, "Original error:") {
|
||||
t.Fatalf("formatted error missing original error label: %q", got)
|
||||
}
|
||||
if !strings.Contains(got, err.Error()) {
|
||||
t.Fatalf("formatted error missing original error: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatProcessingError_GenericAuthHTTPError(t *testing.T) {
|
||||
err := &common.HTTPError{
|
||||
StatusCode: 401,
|
||||
BodyPreview: `{"error":"unauthorized"}`,
|
||||
ContentType: "application/json",
|
||||
APIBase: "https://api.example.com",
|
||||
}
|
||||
|
||||
got := formatProcessingError(err)
|
||||
if !strings.Contains(got, "check the API key, token, OAuth login, or provider permissions") {
|
||||
t.Fatalf("formatted error missing generic auth hint: %q", got)
|
||||
}
|
||||
if !strings.Contains(got, "Original error:") {
|
||||
t.Fatalf("formatted error missing original error: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatProcessingError_NonAuth(t *testing.T) {
|
||||
err := errors.New("connection reset by peer")
|
||||
got := formatProcessingError(err)
|
||||
want := "Error processing message: connection reset by peer"
|
||||
if got != want {
|
||||
t.Fatalf("formatted error = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
@@ -495,11 +495,16 @@ func (p *Pipeline) CallLLM(
|
||||
}
|
||||
}
|
||||
|
||||
// Save finishReason to turnState for SubTurn truncation detection
|
||||
if innerTS := turnStateFromContext(ctx); innerTS != nil {
|
||||
innerTS.SetLastFinishReason(exec.response.FinishReason)
|
||||
// Save finishReason and usage on the turn state. Use ts directly (the
|
||||
// authoritative turn state for this call) rather than a context lookup:
|
||||
// the raw ctx passed to CallLLM is not seeded with turnState (only turnCtx
|
||||
// is), so turnStateFromContext(ctx) returns nil here and silently dropped
|
||||
// both the finish reason and the per-turn token usage. ts is also exactly
|
||||
// what the streaming publisher reads via GetLastUsage at finalize.
|
||||
if ts != nil {
|
||||
ts.SetLastFinishReason(exec.response.FinishReason)
|
||||
if exec.response.Usage != nil {
|
||||
innerTS.SetLastUsage(exec.response.Usage)
|
||||
ts.SetLastUsage(exec.response.Usage)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -54,6 +54,7 @@ func (p *Pipeline) tryConfiguredStreamingLLM(
|
||||
channel: ts.channel,
|
||||
chatID: ts.chatID,
|
||||
modelName: exec.llmModelName,
|
||||
ts: ts,
|
||||
}
|
||||
|
||||
logger.DebugCF("agent", "configured streaming enabled", map[string]any{
|
||||
@@ -376,6 +377,7 @@ type streamingChunkPublisher struct {
|
||||
published bool
|
||||
reasoningPublished bool
|
||||
err error
|
||||
ts *turnState
|
||||
}
|
||||
|
||||
func (p *streamingChunkPublisher) Update(ctx context.Context, accumulated string) {
|
||||
@@ -445,6 +447,11 @@ func (p *streamingChunkPublisher) Finalize(ctx context.Context, content string,
|
||||
if setter, ok := p.streamer.(interface{ SetModelName(modelName string) }); ok {
|
||||
setter.SetModelName(p.modelName)
|
||||
}
|
||||
if usage := p.ts.GetLastUsage(); usage != nil {
|
||||
if setter, ok := p.streamer.(interface{ SetTurnUsage(in, out int) }); ok {
|
||||
setter.SetTurnUsage(usage.PromptTokens, usage.CompletionTokens)
|
||||
}
|
||||
}
|
||||
var err error
|
||||
if streamer, ok := p.streamer.(bus.ContextUsageStreamer); ok {
|
||||
err = streamer.FinalizeWithContext(ctx, content, contextUsage)
|
||||
|
||||
@@ -266,7 +266,7 @@ func (c *BaseChannel) HandleMessageWithContext(
|
||||
media []string,
|
||||
inboundCtx bus.InboundContext,
|
||||
senderOpts ...bus.SenderInfo,
|
||||
) {
|
||||
) error {
|
||||
// Use SenderInfo-based allow check when available, else fall back to string
|
||||
var sender bus.SenderInfo
|
||||
if len(senderOpts) > 0 {
|
||||
@@ -275,11 +275,11 @@ func (c *BaseChannel) HandleMessageWithContext(
|
||||
senderID := strings.TrimSpace(inboundCtx.SenderID)
|
||||
if sender.CanonicalID != "" || sender.PlatformID != "" {
|
||||
if !c.IsAllowedSender(sender) {
|
||||
return
|
||||
return nil
|
||||
}
|
||||
} else {
|
||||
if !c.IsAllowed(senderID) {
|
||||
return
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
@@ -350,7 +350,9 @@ func (c *BaseChannel) HandleMessageWithContext(
|
||||
"chat_id": deliveryChatID,
|
||||
"error": err.Error(),
|
||||
})
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// HandleInboundContext publishes a normalized inbound message using only the
|
||||
@@ -361,8 +363,8 @@ func (c *BaseChannel) HandleInboundContext(
|
||||
media []string,
|
||||
inboundCtx bus.InboundContext,
|
||||
senderOpts ...bus.SenderInfo,
|
||||
) {
|
||||
c.HandleMessageWithContext(ctx, deliveryChatID, content, media, inboundCtx, senderOpts...)
|
||||
) error {
|
||||
return c.HandleMessageWithContext(ctx, deliveryChatID, content, media, inboundCtx, senderOpts...)
|
||||
}
|
||||
|
||||
func (c *BaseChannel) SetRunning(running bool) {
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,398 @@
|
||||
package deltachat
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/bus"
|
||||
"github.com/sipeed/picoclaw/pkg/channels"
|
||||
"github.com/sipeed/picoclaw/pkg/config"
|
||||
"github.com/sipeed/picoclaw/pkg/identity"
|
||||
"github.com/sipeed/picoclaw/pkg/logger"
|
||||
"github.com/sipeed/picoclaw/pkg/media"
|
||||
"github.com/sipeed/picoclaw/pkg/utils"
|
||||
)
|
||||
|
||||
// listen is the inbound message loop. It blocks on wait_next_msgs and feeds
|
||||
// each new message into the PicoClaw inbound pipeline.
|
||||
func (c *DeltaChatChannel) listen() {
|
||||
logger.InfoCF("deltachat", "Listening for messages", map[string]any{
|
||||
"account_id": c.accountID,
|
||||
"email": c.selfAddr,
|
||||
})
|
||||
for c.IsRunning() && c.ctx.Err() == nil {
|
||||
raw, err := c.rpc.call(c.ctx, "wait_next_msgs", c.accountID)
|
||||
if err != nil {
|
||||
if c.ctx.Err() != nil || !c.IsRunning() {
|
||||
return
|
||||
}
|
||||
logger.ErrorCF("deltachat", "wait_next_msgs failed", map[string]any{"error": err.Error()})
|
||||
time.Sleep(time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
var messageIDs []int64
|
||||
if err := json.Unmarshal(raw, &messageIDs); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
if len(messageIDs) > 0 {
|
||||
logger.DebugCF("deltachat", "Received message batch", map[string]any{
|
||||
"count": len(messageIDs),
|
||||
})
|
||||
}
|
||||
for _, messageID := range messageIDs {
|
||||
c.handleMessage(messageID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// handleMessage fetches one message, applies inbound filtering, and publishes it.
|
||||
func (c *DeltaChatChannel) handleMessage(messageID int64) {
|
||||
msg, err := c.getMessage(messageID)
|
||||
if err != nil {
|
||||
logger.DebugCF("deltachat", "get_message failed", map[string]any{
|
||||
"message_id": messageID,
|
||||
"error": err.Error(),
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
if msg.IsInfo || (strings.TrimSpace(msg.Text) == "" && msg.File == "") {
|
||||
return
|
||||
}
|
||||
|
||||
senderAddr := ""
|
||||
if msg.Sender != nil {
|
||||
senderAddr = msg.Sender.Address
|
||||
}
|
||||
if senderAddr != "" && strings.EqualFold(senderAddr, c.selfAddr) {
|
||||
logger.DebugCF("deltachat", "Drop: own message", map[string]any{"message_id": messageID})
|
||||
return
|
||||
}
|
||||
|
||||
chat, err := c.getFullChat(msg.ChatID)
|
||||
if err != nil {
|
||||
logger.DebugCF("deltachat", "get_full_chat_by_id failed", map[string]any{
|
||||
"chat_id": msg.ChatID,
|
||||
"error": err.Error(),
|
||||
})
|
||||
return
|
||||
}
|
||||
// Device messages are core-generated notices, not real conversations.
|
||||
if chat.IsDeviceChat {
|
||||
logger.DebugCF("deltachat", "Drop: device message", map[string]any{"chat_id": msg.ChatID})
|
||||
return
|
||||
}
|
||||
isGroup := chat.ChatType != chatTypeSingle
|
||||
|
||||
logger.DebugCF("deltachat", "Inbound message", map[string]any{
|
||||
"message_id": messageID,
|
||||
"chat_id": msg.ChatID,
|
||||
"from": senderAddr,
|
||||
"is_group": isGroup,
|
||||
"has_file": msg.File != "",
|
||||
"text_len": len(strings.TrimSpace(msg.Text)),
|
||||
})
|
||||
|
||||
senderName := senderAddr
|
||||
if msg.Sender != nil {
|
||||
if msg.Sender.DisplayName != "" {
|
||||
senderName = msg.Sender.DisplayName
|
||||
} else if msg.Sender.Name != "" {
|
||||
senderName = msg.Sender.Name
|
||||
}
|
||||
}
|
||||
if senderName == "" {
|
||||
senderName = "unknown"
|
||||
}
|
||||
|
||||
chatID := strconv.FormatInt(msg.ChatID, 10)
|
||||
messageIDStr := strconv.FormatInt(msg.ID, 10)
|
||||
|
||||
content := strings.TrimSpace(msg.Text)
|
||||
|
||||
// Register any attachment with the media store so the agent pipeline can
|
||||
// view images and operate on files. The ref is scoped to the same key the
|
||||
// BaseChannel derives for this message, so it is released with the turn.
|
||||
var mediaRefs []string
|
||||
if msg.File != "" {
|
||||
scope := channels.BuildMediaScope(c.Name(), chatID, messageIDStr)
|
||||
if ref := c.registerInboundFile(scope, msg); ref != "" {
|
||||
mediaRefs = append(mediaRefs, ref)
|
||||
} else {
|
||||
// Fallback when no media store is available: surface the path inline
|
||||
// so the attachment is not silently lost.
|
||||
annotation := fmt.Sprintf("[attachment: %s]", msg.File)
|
||||
if content == "" {
|
||||
content = annotation
|
||||
} else {
|
||||
content = content + "\n" + annotation
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A file with no caption still warrants a turn; give the agent a minimal
|
||||
// placeholder so the message survives the empty-content guard below. Audio
|
||||
// gets a "[voice]" tag specifically, so the agent's transcription step can
|
||||
// substitute the transcript in place rather than appending it.
|
||||
if content == "" && len(mediaRefs) > 0 {
|
||||
if utils.IsAudioFile(msg.FileName, msg.FileMime) {
|
||||
content = "[voice]"
|
||||
} else {
|
||||
content = "[media]"
|
||||
}
|
||||
}
|
||||
|
||||
sender := bus.SenderInfo{
|
||||
Platform: config.ChannelDeltaChat,
|
||||
PlatformID: senderAddr,
|
||||
CanonicalID: identity.BuildCanonicalID(config.ChannelDeltaChat, senderAddr),
|
||||
Username: senderAddr,
|
||||
DisplayName: senderName,
|
||||
}
|
||||
|
||||
if !c.IsAllowedSender(sender) {
|
||||
logger.DebugCF("deltachat", "Drop: sender not in allow_from", map[string]any{
|
||||
"from": senderAddr,
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
isMentioned := false
|
||||
if isGroup {
|
||||
botName := c.config.DisplayName
|
||||
if botName == "" {
|
||||
botName = c.selfAddr
|
||||
}
|
||||
isMentioned = mentionsBot(content, botName, c.selfAddr)
|
||||
respond, cleaned := c.ShouldRespondInGroup(isMentioned, content)
|
||||
if !respond {
|
||||
logger.DebugCF("deltachat", "Drop: group trigger not satisfied", map[string]any{
|
||||
"chat_id": msg.ChatID,
|
||||
"mentioned": isMentioned,
|
||||
})
|
||||
return
|
||||
}
|
||||
content = cleaned
|
||||
}
|
||||
|
||||
if strings.TrimSpace(content) == "" {
|
||||
return
|
||||
}
|
||||
|
||||
metadata := map[string]string{
|
||||
"platform": config.ChannelDeltaChat,
|
||||
"chat_name": chat.Name,
|
||||
}
|
||||
if msg.File != "" {
|
||||
metadata["file"] = msg.File
|
||||
metadata["file_name"] = msg.FileName
|
||||
metadata["file_mime"] = msg.FileMime
|
||||
}
|
||||
|
||||
inboundCtx := bus.InboundContext{
|
||||
Channel: config.ChannelDeltaChat,
|
||||
ChatID: chatID,
|
||||
SenderID: senderAddr,
|
||||
MessageID: messageIDStr,
|
||||
Mentioned: isMentioned,
|
||||
Raw: metadata,
|
||||
}
|
||||
if isGroup {
|
||||
inboundCtx.ChatType = "group"
|
||||
} else {
|
||||
inboundCtx.ChatType = "direct"
|
||||
}
|
||||
|
||||
logger.DebugCF("deltachat", "Dispatching to agent", map[string]any{
|
||||
"chat_id": chatID,
|
||||
"chat_type": inboundCtx.ChatType,
|
||||
"from": senderAddr,
|
||||
})
|
||||
if err := c.HandleInboundContext(c.ctx, chatID, content, mediaRefs, inboundCtx, sender); err != nil {
|
||||
logger.ErrorCF("deltachat", "Dispatch failed; leaving message unseen", map[string]any{
|
||||
"message_id": messageID,
|
||||
"chat_id": chatID,
|
||||
"error": err.Error(),
|
||||
})
|
||||
return
|
||||
}
|
||||
if _, err := c.rpc.call(c.ctx, "markseen_msgs", c.accountID, []int64{messageID}); err != nil {
|
||||
logger.WarnCF("deltachat", "Failed to mark message seen", map[string]any{
|
||||
"message_id": messageID,
|
||||
"chat_id": chatID,
|
||||
"error": err.Error(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// registerInboundFile records an inbound attachment with the media store under
|
||||
// the given scope and returns its media:// ref. Returns "" when there is no
|
||||
// media store or registration fails, letting the caller fall back to an inline
|
||||
// path annotation.
|
||||
//
|
||||
// Delta Chat stores attachments inside the account directory, next to the
|
||||
// credential database — a location tools are intentionally NOT allowed to read.
|
||||
// We therefore copy the single attachment out into the shared media temp dir
|
||||
// (which read_file/load_image are permitted to access) and register that copy,
|
||||
// so the agent can actually open the file. The copy is store-managed and deleted
|
||||
// when the turn's scope is released.
|
||||
func (c *DeltaChatChannel) registerInboundFile(scope string, msg *dcMessage) string {
|
||||
store := c.GetMediaStore()
|
||||
if store == nil {
|
||||
return ""
|
||||
}
|
||||
filename := msg.FileName
|
||||
if filename == "" {
|
||||
filename = filepath.Base(msg.File)
|
||||
}
|
||||
|
||||
localPath, err := copyToMediaTemp(msg.File, filename)
|
||||
if err != nil {
|
||||
logger.WarnCF("deltachat", "Failed to copy attachment into media dir", map[string]any{
|
||||
"file": msg.File,
|
||||
"error": err.Error(),
|
||||
})
|
||||
return ""
|
||||
}
|
||||
|
||||
ref, err := store.Store(localPath, media.MediaMeta{
|
||||
Filename: filename,
|
||||
ContentType: msg.FileMime,
|
||||
Source: config.ChannelDeltaChat,
|
||||
CleanupPolicy: media.CleanupPolicyDeleteOnCleanup,
|
||||
}, scope)
|
||||
if err != nil {
|
||||
logger.WarnCF("deltachat", "Failed to register attachment with media store", map[string]any{
|
||||
"file": localPath,
|
||||
"error": err.Error(),
|
||||
})
|
||||
_ = os.Remove(localPath)
|
||||
return ""
|
||||
}
|
||||
return ref
|
||||
}
|
||||
|
||||
// copyToMediaTemp copies srcPath into the shared media temp directory under a
|
||||
// unique name and returns the destination path. The media temp dir is the
|
||||
// location the read_file/load_image tools are permitted to read, so copying here
|
||||
// makes the attachment readable without exposing Delta Chat's account directory.
|
||||
func copyToMediaTemp(srcPath, filename string) (string, error) {
|
||||
if err := os.MkdirAll(media.TempDir(), 0o700); err != nil {
|
||||
return "", err
|
||||
}
|
||||
safe := utils.SanitizeFilename(filename)
|
||||
if safe == "" {
|
||||
safe = filepath.Base(srcPath)
|
||||
}
|
||||
dstPath := filepath.Join(media.TempDir(), uuid.NewString()[:8]+"_"+safe)
|
||||
|
||||
src, err := os.Open(srcPath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer src.Close()
|
||||
|
||||
dst, err := os.Create(dstPath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if _, err := io.Copy(dst, src); err != nil {
|
||||
dst.Close()
|
||||
_ = os.Remove(dstPath)
|
||||
return "", err
|
||||
}
|
||||
if err := dst.Close(); err != nil {
|
||||
_ = os.Remove(dstPath)
|
||||
return "", err
|
||||
}
|
||||
return dstPath, nil
|
||||
}
|
||||
|
||||
func (c *DeltaChatChannel) getMessage(messageID int64) (*dcMessage, error) {
|
||||
raw, err := c.rpc.call(c.ctx, "get_message", c.accountID, messageID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var msg dcMessage
|
||||
if err := json.Unmarshal(raw, &msg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &msg, nil
|
||||
}
|
||||
|
||||
func (c *DeltaChatChannel) getFullChat(chatID int64) (*dcChat, error) {
|
||||
raw, err := c.rpc.call(c.ctx, "get_full_chat_by_id", c.accountID, chatID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var chat dcChat
|
||||
if err := json.Unmarshal(raw, &chat); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &chat, nil
|
||||
}
|
||||
|
||||
// mentionsBot reports whether the message references the bot by display name or
|
||||
// the local-part of its email address (a common addressing convention).
|
||||
func mentionsBot(content, displayName, email string) bool {
|
||||
if containsMentionToken(content, displayName) {
|
||||
return true
|
||||
}
|
||||
if local, _, ok := strings.Cut(email, "@"); ok && local != "" {
|
||||
if containsMentionToken(content, "@"+local) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func containsMentionToken(content, token string) bool {
|
||||
token = strings.TrimSpace(token)
|
||||
if token == "" {
|
||||
return false
|
||||
}
|
||||
contentRunes := []rune(strings.ToLower(content))
|
||||
tokenRunes := []rune(strings.ToLower(token))
|
||||
if len(tokenRunes) == 0 || len(tokenRunes) > len(contentRunes) {
|
||||
return false
|
||||
}
|
||||
for i := 0; i <= len(contentRunes)-len(tokenRunes); i++ {
|
||||
if !sameRunes(contentRunes[i:i+len(tokenRunes)], tokenRunes) {
|
||||
continue
|
||||
}
|
||||
before := i == 0 || !isMentionWordRune(contentRunes[i-1])
|
||||
afterIdx := i + len(tokenRunes)
|
||||
after := afterIdx >= len(contentRunes) || !isMentionWordRune(contentRunes[afterIdx])
|
||||
if before && after {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func sameRunes(a, b []rune) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for i := range a {
|
||||
if a[i] != b[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func isMentionWordRune(r rune) bool {
|
||||
return unicode.IsLetter(r) || unicode.IsDigit(r) || r == '_'
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
package deltachat
|
||||
|
||||
import (
|
||||
"github.com/sipeed/picoclaw/pkg/bus"
|
||||
"github.com/sipeed/picoclaw/pkg/channels"
|
||||
"github.com/sipeed/picoclaw/pkg/config"
|
||||
)
|
||||
|
||||
func init() {
|
||||
channels.RegisterFactory(
|
||||
config.ChannelDeltaChat,
|
||||
func(channelName, channelType string, cfg *config.Config, b *bus.MessageBus) (channels.Channel, error) {
|
||||
bc := cfg.Channels[channelName]
|
||||
if bc == nil || !bc.Enabled {
|
||||
return nil, nil
|
||||
}
|
||||
decoded, err := bc.GetDecoded()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c, ok := decoded.(*config.DeltaChatSettings)
|
||||
if !ok {
|
||||
return nil, channels.ErrSendFailed
|
||||
}
|
||||
ch, err := NewDeltaChatChannel(bc, c, b)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if channelName != config.ChannelDeltaChat {
|
||||
ch.SetName(channelName)
|
||||
}
|
||||
return ch, nil
|
||||
},
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
package deltachat
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"os/exec"
|
||||
"sync"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/logger"
|
||||
)
|
||||
|
||||
// rpcRequest is a single JSON-RPC 2.0 request. Delta Chat uses positional
|
||||
// (array) parameters.
|
||||
type rpcRequest struct {
|
||||
JSONRPC string `json:"jsonrpc"`
|
||||
ID uint64 `json:"id"`
|
||||
Method string `json:"method"`
|
||||
Params []any `json:"params,omitempty"`
|
||||
}
|
||||
|
||||
// rpcResponse is a single JSON-RPC 2.0 response.
|
||||
type rpcResponse struct {
|
||||
ID uint64 `json:"id"`
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error *rpcError `json:"error"`
|
||||
}
|
||||
|
||||
type rpcError struct {
|
||||
Code int `json:"code"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
func (e *rpcError) Error() string {
|
||||
return fmt.Sprintf("deltachat rpc error %d: %s", e.Code, e.Message)
|
||||
}
|
||||
|
||||
// rpcClient drives a `deltachat-rpc-server` child process over stdio using
|
||||
// newline-delimited JSON-RPC 2.0. The server answers requests asynchronously
|
||||
// and out of order, so every call is correlated back to its caller by id.
|
||||
type rpcClient struct {
|
||||
cmd *exec.Cmd
|
||||
stdin io.WriteCloser
|
||||
stdout io.ReadCloser
|
||||
|
||||
mu sync.Mutex
|
||||
nextID uint64
|
||||
pending map[uint64]chan rpcResponse
|
||||
closed bool
|
||||
}
|
||||
|
||||
// startRPC spawns the RPC server with DC_ACCOUNTS_PATH pointing at dataDir and
|
||||
// begins the background read loop.
|
||||
func startRPC(serverPath, dataDir string) (*rpcClient, error) {
|
||||
cmd := exec.Command(serverPath)
|
||||
cmd.Env = append(cmd.Environ(), "DC_ACCOUNTS_PATH="+dataDir)
|
||||
|
||||
stdin, err := cmd.StdinPipe()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("deltachat rpc stdin: %w", err)
|
||||
}
|
||||
stdout, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("deltachat rpc stdout: %w", err)
|
||||
}
|
||||
// Let the server's logs flow to our stderr for easy diagnostics.
|
||||
cmd.Stderr = logWriter{}
|
||||
|
||||
if err := cmd.Start(); err != nil {
|
||||
return nil, fmt.Errorf("start deltachat-rpc-server (%s): %w", serverPath, err)
|
||||
}
|
||||
|
||||
c := &rpcClient{
|
||||
cmd: cmd,
|
||||
stdin: stdin,
|
||||
stdout: stdout,
|
||||
pending: make(map[uint64]chan rpcResponse),
|
||||
}
|
||||
go c.readLoop()
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// readLoop reads newline-delimited responses and dispatches them to waiters.
|
||||
func (c *rpcClient) readLoop() {
|
||||
reader := bufio.NewReader(c.stdout)
|
||||
for {
|
||||
line, err := reader.ReadBytes('\n')
|
||||
if len(line) > 0 {
|
||||
var resp rpcResponse
|
||||
if jsonErr := json.Unmarshal(line, &resp); jsonErr == nil && resp.ID != 0 {
|
||||
c.mu.Lock()
|
||||
ch := c.pending[resp.ID]
|
||||
delete(c.pending, resp.ID)
|
||||
c.mu.Unlock()
|
||||
if ch != nil {
|
||||
ch <- resp
|
||||
}
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
c.failAll(err)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// failAll wakes every pending caller with an error (used on EOF / shutdown).
|
||||
func (c *rpcClient) failAll(err error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.closed = true
|
||||
for id, ch := range c.pending {
|
||||
ch <- rpcResponse{Error: &rpcError{Code: -1, Message: "rpc closed: " + err.Error()}}
|
||||
delete(c.pending, id)
|
||||
}
|
||||
}
|
||||
|
||||
// call issues one request and blocks until the matching response arrives, the
|
||||
// context is canceled, or the server goes away.
|
||||
func (c *rpcClient) call(ctx context.Context, method string, params ...any) (json.RawMessage, error) {
|
||||
c.mu.Lock()
|
||||
if c.closed {
|
||||
c.mu.Unlock()
|
||||
return nil, fmt.Errorf("deltachat rpc: connection closed")
|
||||
}
|
||||
c.nextID++
|
||||
id := c.nextID
|
||||
ch := make(chan rpcResponse, 1)
|
||||
c.pending[id] = ch
|
||||
c.mu.Unlock()
|
||||
|
||||
req := rpcRequest{JSONRPC: "2.0", ID: id, Method: method, Params: params}
|
||||
data, err := json.Marshal(req)
|
||||
if err != nil {
|
||||
c.clearPending(id)
|
||||
return nil, err
|
||||
}
|
||||
data = append(data, '\n')
|
||||
|
||||
c.mu.Lock()
|
||||
_, err = c.stdin.Write(data)
|
||||
c.mu.Unlock()
|
||||
if err != nil {
|
||||
c.clearPending(id)
|
||||
return nil, fmt.Errorf("deltachat rpc write %s: %w", method, err)
|
||||
}
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
c.clearPending(id)
|
||||
return nil, ctx.Err()
|
||||
case resp := <-ch:
|
||||
if resp.Error != nil {
|
||||
return nil, resp.Error
|
||||
}
|
||||
return resp.Result, nil
|
||||
}
|
||||
}
|
||||
|
||||
func (c *rpcClient) clearPending(id uint64) {
|
||||
c.mu.Lock()
|
||||
delete(c.pending, id)
|
||||
c.mu.Unlock()
|
||||
}
|
||||
|
||||
// close terminates the RPC server process.
|
||||
func (c *rpcClient) close() {
|
||||
c.mu.Lock()
|
||||
c.closed = true
|
||||
c.mu.Unlock()
|
||||
if c.stdin != nil {
|
||||
_ = c.stdin.Close()
|
||||
}
|
||||
if c.cmd != nil && c.cmd.Process != nil {
|
||||
_ = c.cmd.Process.Kill()
|
||||
_ = c.cmd.Wait()
|
||||
}
|
||||
}
|
||||
|
||||
// logWriter forwards deltachat-rpc-server stderr lines into the logger.
|
||||
type logWriter struct{}
|
||||
|
||||
func (logWriter) Write(p []byte) (int, error) {
|
||||
logger.DebugC("deltachat", string(p))
|
||||
return len(p), nil
|
||||
}
|
||||
+38
-9
@@ -699,18 +699,34 @@ func setStreamerModelName(streamer any, modelName string) {
|
||||
setter.SetModelName(modelName)
|
||||
}
|
||||
|
||||
type turnUsageStreamer interface {
|
||||
SetTurnUsage(inputTokens, outputTokens int)
|
||||
}
|
||||
|
||||
// setStreamerTurnUsage forwards real per-turn token usage to a streamer that
|
||||
// supports it, transparently unwrapping the manager's streamer wrappers.
|
||||
func setStreamerTurnUsage(streamer any, inputTokens, outputTokens int) {
|
||||
setter, ok := streamer.(turnUsageStreamer)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
setter.SetTurnUsage(inputTokens, outputTokens)
|
||||
}
|
||||
|
||||
// splitMarkerStreamer turns accumulated streaming text containing
|
||||
// MessageSplitMarker into separate channel stream messages.
|
||||
type splitMarkerStreamer struct {
|
||||
mu sync.Mutex
|
||||
current bus.Streamer
|
||||
reasoning bus.ReasoningStreamer
|
||||
begin func(context.Context) (bus.Streamer, error)
|
||||
completedParts int
|
||||
finalized bool
|
||||
onFinalize func(context.Context, string)
|
||||
clearMarker func()
|
||||
modelName string
|
||||
mu sync.Mutex
|
||||
current bus.Streamer
|
||||
reasoning bus.ReasoningStreamer
|
||||
begin func(context.Context) (bus.Streamer, error)
|
||||
completedParts int
|
||||
finalized bool
|
||||
onFinalize func(context.Context, string)
|
||||
clearMarker func()
|
||||
modelName string
|
||||
turnInputTokens int
|
||||
turnOutputTokens int
|
||||
}
|
||||
|
||||
func (s *splitMarkerStreamer) Update(ctx context.Context, content string) error {
|
||||
@@ -761,6 +777,14 @@ func (s *splitMarkerStreamer) SetModelName(modelName string) {
|
||||
setStreamerModelName(s.reasoning, s.modelName)
|
||||
}
|
||||
|
||||
func (s *splitMarkerStreamer) SetTurnUsage(inputTokens, outputTokens int) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.turnInputTokens = inputTokens
|
||||
s.turnOutputTokens = outputTokens
|
||||
setStreamerTurnUsage(s.current, s.turnInputTokens, s.turnOutputTokens)
|
||||
}
|
||||
|
||||
func (s *splitMarkerStreamer) Cancel(ctx context.Context) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
@@ -840,6 +864,7 @@ func (s *splitMarkerStreamer) ensureCurrentLocked(ctx context.Context) error {
|
||||
}
|
||||
s.current = streamer
|
||||
setStreamerModelName(s.current, s.modelName)
|
||||
setStreamerTurnUsage(s.current, s.turnInputTokens, s.turnOutputTokens)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -928,6 +953,10 @@ func (s *finalizeHookStreamer) SetModelName(modelName string) {
|
||||
setStreamerModelName(s.Streamer, strings.TrimSpace(modelName))
|
||||
}
|
||||
|
||||
func (s *finalizeHookStreamer) SetTurnUsage(inputTokens, outputTokens int) {
|
||||
setStreamerTurnUsage(s.Streamer, inputTokens, outputTokens)
|
||||
}
|
||||
|
||||
func (s *finalizeHookStreamer) runFinalizeHook(ctx context.Context, content string) {
|
||||
if s.onFinalize != nil {
|
||||
s.onFinalize(ctx, content)
|
||||
|
||||
@@ -3383,3 +3383,56 @@ func TestManager_SendPlaceholder(t *testing.T) {
|
||||
t.Error("expected SendPlaceholder to fail for unknown channel")
|
||||
}
|
||||
}
|
||||
|
||||
// turnUsageTrackingStreamer is a mockStreamer that records SetTurnUsage calls,
|
||||
// used to verify the manager's streamer wrappers forward per-turn token usage
|
||||
// to the inner streamer (regression: the wrappers previously dropped it because
|
||||
// SetTurnUsage is not part of the bus.Streamer interface).
|
||||
type turnUsageTrackingStreamer struct {
|
||||
mockStreamer
|
||||
inputTokens int
|
||||
outputTokens int
|
||||
usageCalls int
|
||||
}
|
||||
|
||||
func (m *turnUsageTrackingStreamer) SetTurnUsage(inputTokens, outputTokens int) {
|
||||
m.usageCalls++
|
||||
m.inputTokens = inputTokens
|
||||
m.outputTokens = outputTokens
|
||||
}
|
||||
|
||||
func TestFinalizeHookStreamerForwardsTurnUsage(t *testing.T) {
|
||||
inner := &turnUsageTrackingStreamer{}
|
||||
wrapper := &finalizeHookStreamer{Streamer: inner}
|
||||
|
||||
setter, ok := any(wrapper).(turnUsageStreamer)
|
||||
if !ok {
|
||||
t.Fatal("finalizeHookStreamer does not satisfy turnUsageStreamer")
|
||||
}
|
||||
setter.SetTurnUsage(1234, 567)
|
||||
|
||||
if inner.usageCalls != 1 {
|
||||
t.Fatalf("inner SetTurnUsage calls = %d, want 1", inner.usageCalls)
|
||||
}
|
||||
if inner.inputTokens != 1234 || inner.outputTokens != 567 {
|
||||
t.Errorf("inner usage = (%d, %d), want (1234, 567)", inner.inputTokens, inner.outputTokens)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitMarkerStreamerForwardsTurnUsage(t *testing.T) {
|
||||
inner := &turnUsageTrackingStreamer{}
|
||||
wrapper := &splitMarkerStreamer{current: inner}
|
||||
|
||||
setter, ok := any(wrapper).(turnUsageStreamer)
|
||||
if !ok {
|
||||
t.Fatal("splitMarkerStreamer does not satisfy turnUsageStreamer")
|
||||
}
|
||||
setter.SetTurnUsage(1234, 567)
|
||||
|
||||
if inner.usageCalls != 1 {
|
||||
t.Fatalf("inner SetTurnUsage calls = %d, want 1", inner.usageCalls)
|
||||
}
|
||||
if inner.inputTokens != 1234 || inner.outputTokens != 567 {
|
||||
t.Errorf("inner usage = (%d, %d), want (1234, 567)", inner.inputTokens, inner.outputTokens)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -105,6 +105,8 @@ type PicoChannel struct {
|
||||
cancel context.CancelFunc
|
||||
progress *channels.ToolFeedbackAnimator
|
||||
deleteMessageFn func(context.Context, string, string) error
|
||||
// broadcastFn lets tests intercept outbound broadcasts. nil → broadcastToSession.
|
||||
broadcastFn func(chatID string, msg PicoMessage) error
|
||||
}
|
||||
|
||||
// NewPicoChannel creates a new Pico Protocol channel.
|
||||
@@ -531,6 +533,8 @@ type picoStreamer struct {
|
||||
channel *PicoChannel
|
||||
chatID string
|
||||
modelName string
|
||||
turnInputTokens int
|
||||
turnOutputTokens int
|
||||
messageID string
|
||||
reasoningID string
|
||||
throttleInterval time.Duration
|
||||
@@ -553,6 +557,17 @@ func (s *picoStreamer) SetModelName(modelName string) {
|
||||
s.modelName = strings.TrimSpace(modelName)
|
||||
}
|
||||
|
||||
// SetTurnUsage records the real per-turn LLM token usage to emit on finalize.
|
||||
func (s *picoStreamer) SetTurnUsage(inputTokens, outputTokens int) {
|
||||
if s == nil {
|
||||
return
|
||||
}
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.turnInputTokens = inputTokens
|
||||
s.turnOutputTokens = outputTokens
|
||||
}
|
||||
|
||||
func (s *picoStreamer) Update(ctx context.Context, content string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
@@ -661,8 +676,9 @@ func (s *picoStreamer) sendLocked(ctx context.Context, content string, contextUs
|
||||
payload[PayloadKeyModelName] = s.modelName
|
||||
}
|
||||
setContextUsagePayload(payload, contextUsage)
|
||||
setTurnUsagePayload(payload, s.turnInputTokens, s.turnOutputTokens)
|
||||
outMsg := newMessage(TypeMessageCreate, payload)
|
||||
if err := s.channel.broadcastToSession(s.chatID, outMsg); err != nil {
|
||||
if err := s.channel.broadcast(s.chatID, outMsg); err != nil {
|
||||
return err
|
||||
}
|
||||
} else if content != s.lastContent || contextUsage != nil {
|
||||
@@ -673,6 +689,7 @@ func (s *picoStreamer) sendLocked(ctx context.Context, content string, contextUs
|
||||
if s.modelName != "" {
|
||||
payload[PayloadKeyModelName] = s.modelName
|
||||
}
|
||||
setTurnUsagePayload(payload, s.turnInputTokens, s.turnOutputTokens)
|
||||
if err := s.channel.editMessagePayload(ctx, s.chatID, s.messageID, payload, contextUsage); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -932,6 +949,14 @@ func (c *PicoChannel) handleMediaDownload(w http.ResponseWriter, r *http.Request
|
||||
http.ServeContent(w, r, filename, info.ModTime(), file)
|
||||
}
|
||||
|
||||
// broadcast routes through broadcastFn when set (tests), else broadcastToSession.
|
||||
func (c *PicoChannel) broadcast(chatID string, msg PicoMessage) error {
|
||||
if c.broadcastFn != nil {
|
||||
return c.broadcastFn(chatID, msg)
|
||||
}
|
||||
return c.broadcastToSession(chatID, msg)
|
||||
}
|
||||
|
||||
// broadcastToSession sends a message to all connections with a matching session.
|
||||
func (c *PicoChannel) broadcastToSession(chatID string, msg PicoMessage) error {
|
||||
// chatID format: "pico:<sessionID>"
|
||||
@@ -1403,6 +1428,20 @@ func setContextUsagePayload(payload map[string]any, u *bus.ContextUsage) {
|
||||
}
|
||||
}
|
||||
|
||||
// setTurnUsagePayload attaches real per-turn LLM token usage to the payload.
|
||||
// Input and output are kept separate (billed at different rates); total is a
|
||||
// convenience sum. Omitted entirely when both counts are zero.
|
||||
func setTurnUsagePayload(payload map[string]any, inputTokens, outputTokens int) {
|
||||
if inputTokens <= 0 && outputTokens <= 0 {
|
||||
return
|
||||
}
|
||||
payload[PayloadKeyUsage] = map[string]any{
|
||||
"input_tokens": inputTokens,
|
||||
"output_tokens": outputTokens,
|
||||
"total_tokens": inputTokens + outputTokens,
|
||||
}
|
||||
}
|
||||
|
||||
func picoToolCallsPayload(msg bus.OutboundMessage) ([]utils.VisibleToolCall, bool) {
|
||||
raw := strings.TrimSpace(msg.Context.Raw[PayloadKeyToolCalls])
|
||||
if raw == "" {
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
package pico
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSetTurnUsagePayload(t *testing.T) {
|
||||
t.Run("populates usage block when counts present", func(t *testing.T) {
|
||||
payload := map[string]any{PayloadKeyContent: "hi"}
|
||||
setTurnUsagePayload(payload, 1234, 567)
|
||||
|
||||
raw, ok := payload[PayloadKeyUsage]
|
||||
if !ok {
|
||||
t.Fatalf("expected %q key in payload", PayloadKeyUsage)
|
||||
}
|
||||
usage, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("usage block is not a map: %T", raw)
|
||||
}
|
||||
if usage["input_tokens"] != 1234 {
|
||||
t.Errorf("input_tokens = %v, want 1234", usage["input_tokens"])
|
||||
}
|
||||
if usage["output_tokens"] != 567 {
|
||||
t.Errorf("output_tokens = %v, want 567", usage["output_tokens"])
|
||||
}
|
||||
if usage["total_tokens"] != 1801 {
|
||||
t.Errorf("total_tokens = %v, want 1801", usage["total_tokens"])
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("omits usage block when both counts zero", func(t *testing.T) {
|
||||
payload := map[string]any{PayloadKeyContent: "hi"}
|
||||
setTurnUsagePayload(payload, 0, 0)
|
||||
if _, ok := payload[PayloadKeyUsage]; ok {
|
||||
t.Errorf("expected no %q key when counts are zero", PayloadKeyUsage)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// newCaptureStreamer returns a streamer whose broadcasts are captured into the
|
||||
// returned map pointer, so tests need no live websocket.
|
||||
func newCaptureStreamer() (*picoStreamer, *map[string]any) {
|
||||
var last map[string]any
|
||||
ch := &PicoChannel{}
|
||||
ch.broadcastFn = func(chatID string, msg PicoMessage) error {
|
||||
last = msg.Payload
|
||||
return nil
|
||||
}
|
||||
s := &picoStreamer{channel: ch, chatID: "c1"}
|
||||
return s, &last
|
||||
}
|
||||
|
||||
func TestStreamerEmitsUsageOnFinalize(t *testing.T) {
|
||||
s, last := newCaptureStreamer()
|
||||
s.SetTurnUsage(100, 40)
|
||||
|
||||
// sendLocked with empty messageID takes the create branch, which attaches
|
||||
// usage from the streamer's stored counts.
|
||||
s.mu.Lock()
|
||||
err := s.sendLocked(context.Background(), "answer", nil)
|
||||
s.mu.Unlock()
|
||||
if err != nil {
|
||||
t.Fatalf("sendLocked: %v", err)
|
||||
}
|
||||
if _, ok := (*last)[PayloadKeyUsage]; !ok {
|
||||
t.Fatalf("expected usage in payload, got %+v", *last)
|
||||
}
|
||||
}
|
||||
@@ -28,6 +28,7 @@ const (
|
||||
PayloadKeyPlaceholder = "placeholder"
|
||||
PayloadKeyToolCalls = "tool_calls"
|
||||
PayloadKeyModelName = "model_name"
|
||||
PayloadKeyUsage = "usage"
|
||||
|
||||
MessageKindThought = "thought"
|
||||
MessageKindToolCalls = "tool_calls"
|
||||
|
||||
@@ -599,6 +599,31 @@ type MatrixSettings struct {
|
||||
CryptoPassphrase string `json:"crypto_passphrase,omitempty" yaml:"-"`
|
||||
}
|
||||
|
||||
// DeltaChatSettings configures the Delta Chat channel. Delta Chat is an
|
||||
// email-based, end-to-end encrypted messenger; PicoClaw talks to a local
|
||||
// `deltachat-rpc-server` process over JSON-RPC (stdio).
|
||||
//
|
||||
// Email is the only required setting. A full address selects an already
|
||||
// configured account in DataDir; a first-run marker such as "@nine.testrun.org"
|
||||
// creates a chatmail account and tells the user which full email to save.
|
||||
// Mailbox credentials stay in the Delta Chat account store. DisplayName and
|
||||
// AvatarImage are optional profile settings applied on startup. Password remains
|
||||
// only for legacy PicoClaw-managed email configuration.
|
||||
type DeltaChatSettings struct {
|
||||
Email string `json:"email" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_EMAIL"`
|
||||
Password SecureString `json:"password,omitzero" yaml:"password,omitempty" env:"PICOCLAW_CHANNELS_DELTACHAT_PASSWORD"`
|
||||
DisplayName string `json:"display_name,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_DISPLAY_NAME"`
|
||||
AvatarImage string `json:"avatar_image,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_AVATAR_IMAGE"`
|
||||
DataDir string `json:"data_dir,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_DATA_DIR"`
|
||||
RPCServerPath string `json:"rpc_server_path,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_RPC_SERVER_PATH"`
|
||||
InviteLink string `json:"invite_link,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_INVITE_LINK"`
|
||||
AllowCrosspost bool `json:"allow_crosspost,omitempty" yaml:"-" env:"PICOCLAW_CHANNELS_DELTACHAT_ALLOW_CROSSPOST"`
|
||||
IMAPServer string `json:"imap_server,omitempty" yaml:"-"`
|
||||
IMAPPort int `json:"imap_port,omitempty" yaml:"-"`
|
||||
SMTPServer string `json:"smtp_server,omitempty" yaml:"-"`
|
||||
SMTPPort int `json:"smtp_port,omitempty" yaml:"-"`
|
||||
}
|
||||
|
||||
type LINESettings struct {
|
||||
ChannelSecret SecureString `json:"channel_secret,omitzero" yaml:"channel_secret,omitempty" env:"PICOCLAW_CHANNELS_LINE_CHANNEL_SECRET"`
|
||||
ChannelAccessToken SecureString `json:"channel_access_token,omitzero" yaml:"channel_access_token,omitempty" env:"PICOCLAW_CHANNELS_LINE_CHANNEL_ACCESS_TOKEN"`
|
||||
|
||||
@@ -27,6 +27,7 @@ const (
|
||||
ChannelDingTalk = "dingtalk"
|
||||
ChannelSlack = "slack"
|
||||
ChannelMatrix = "matrix"
|
||||
ChannelDeltaChat = "deltachat"
|
||||
ChannelLINE = "line"
|
||||
ChannelOneBot = "onebot"
|
||||
ChannelQQ = "qq"
|
||||
@@ -669,6 +670,7 @@ var channelSettingsFactory = map[string]any{
|
||||
ChannelDingTalk: (DingTalkSettings{}),
|
||||
ChannelSlack: (SlackSettings{}),
|
||||
ChannelMatrix: (MatrixSettings{}),
|
||||
ChannelDeltaChat: (DeltaChatSettings{}),
|
||||
ChannelLINE: (LINESettings{}),
|
||||
ChannelOneBot: (OneBotSettings{}),
|
||||
ChannelQQ: (QQSettings{}),
|
||||
|
||||
@@ -1019,6 +1019,38 @@ func TestDefaultConfig_ChannelStreamingDisabled(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestDefaultConfig_DeltaChatExample(t *testing.T) {
|
||||
cfg := DefaultConfig()
|
||||
|
||||
deltachat := cfg.Channels.Get(ChannelDeltaChat)
|
||||
if deltachat == nil {
|
||||
t.Fatal("DefaultConfig() missing deltachat channel")
|
||||
}
|
||||
if deltachat.Enabled {
|
||||
t.Fatal("DefaultConfig().deltachat should be disabled")
|
||||
}
|
||||
if !deltachat.GroupTrigger.MentionOnly {
|
||||
t.Fatal("DefaultConfig().deltachat should use mention-only group trigger")
|
||||
}
|
||||
decoded, err := deltachat.GetDecoded()
|
||||
if err != nil {
|
||||
t.Fatalf("deltachat GetDecoded() error = %v", err)
|
||||
}
|
||||
settings, ok := decoded.(*DeltaChatSettings)
|
||||
if !ok {
|
||||
t.Fatalf("deltachat settings type = %T, want *DeltaChatSettings", decoded)
|
||||
}
|
||||
if settings.Email != "@nine.testrun.org" {
|
||||
t.Fatalf("DefaultConfig().deltachat.settings.email = %q, want @nine.testrun.org", settings.Email)
|
||||
}
|
||||
if settings.Password.String() != "" {
|
||||
t.Fatal("DefaultConfig().deltachat.settings.password should be empty")
|
||||
}
|
||||
if settings.DisplayName == "" {
|
||||
t.Fatal("DefaultConfig().deltachat.settings.display_name should be populated")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateSingletonChannels_RejectsMultipleInstances(t *testing.T) {
|
||||
channels := ChannelsConfig{
|
||||
"pico1": &Channel{Enabled: true, Type: ChannelPico},
|
||||
|
||||
@@ -549,6 +549,13 @@ func defaultChannels() ChannelsConfig {
|
||||
"join_on_invite": true,
|
||||
},
|
||||
},
|
||||
"deltachat": map[string]any{
|
||||
"group_trigger": map[string]any{"mention_only": true},
|
||||
"settings": map[string]any{
|
||||
"email": "@nine.testrun.org",
|
||||
"display_name": "PicoClaw Bot",
|
||||
},
|
||||
},
|
||||
"line": map[string]any{
|
||||
"group_trigger": map[string]any{"mention_only": true},
|
||||
"settings": map[string]any{
|
||||
|
||||
@@ -19,6 +19,7 @@ import (
|
||||
"github.com/sipeed/picoclaw/pkg/audio/tts"
|
||||
"github.com/sipeed/picoclaw/pkg/bus"
|
||||
"github.com/sipeed/picoclaw/pkg/channels"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/deltachat"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/dingtalk"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/discord"
|
||||
_ "github.com/sipeed/picoclaw/pkg/channels/feishu"
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
package providers
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/providers/common"
|
||||
)
|
||||
|
||||
type AuthErrorKind string
|
||||
|
||||
const (
|
||||
AuthErrorInvalidAPIKey AuthErrorKind = "invalid_api_key"
|
||||
AuthErrorMissingAPIKey AuthErrorKind = "missing_api_key"
|
||||
AuthErrorExpiredToken AuthErrorKind = "expired_token"
|
||||
AuthErrorGeneric AuthErrorKind = "auth"
|
||||
)
|
||||
|
||||
var (
|
||||
invalidAPIKeyPattern = regexp.MustCompile(
|
||||
`(?i)\b(?:invalid|incorrect|malformed|wrong)[-_\s]+(?:api[-_\s]*)?key\b|\b(?:api[-_\s]*)?key[-_\s]+(?:is[-_\s]+)?(?:invalid|incorrect|malformed|wrong)\b|\binvalid[-_]?api[-_]?key\b`,
|
||||
)
|
||||
missingAPIKeyPattern = regexp.MustCompile(
|
||||
`(?i)\b(?:missing|no|not[-_\s]+configured)[-_\s]+(?:api[-_\s]*)?key\b|\bno[-_\s]+credentials[-_\s]+found\b|\bapi[-_\s]+key[-_\s]+not[-_\s]+configured\b`,
|
||||
)
|
||||
expiredTokenPattern = regexp.MustCompile(
|
||||
`(?i)\b(?:oauth[-_\s]+token[-_\s]+refresh[-_\s]+failed|re-authenticate|token[-_\s]+(?:has[-_\s]+)?expired|expired[-_\s]+token)\b`,
|
||||
)
|
||||
genericAuthPattern = regexp.MustCompile(
|
||||
`(?i)\b(?:unauthorized|forbidden|authentication[-_\s]+(?:failed|required)|access[-_\s]+denied)\b`,
|
||||
)
|
||||
)
|
||||
|
||||
func ClassifyAuthError(err error) (AuthErrorKind, bool) {
|
||||
if err == nil {
|
||||
return "", false
|
||||
}
|
||||
|
||||
var exhausted *FallbackExhaustedError
|
||||
if errors.As(err, &exhausted) && exhausted != nil {
|
||||
return classifyFallbackExhaustedAuthError(exhausted)
|
||||
}
|
||||
|
||||
msg := authErrorText(err)
|
||||
if missingAPIKeyPattern.MatchString(msg) {
|
||||
return AuthErrorMissingAPIKey, true
|
||||
}
|
||||
if expiredTokenPattern.MatchString(msg) {
|
||||
return AuthErrorExpiredToken, true
|
||||
}
|
||||
if invalidAPIKeyPattern.MatchString(msg) {
|
||||
return AuthErrorInvalidAPIKey, true
|
||||
}
|
||||
|
||||
if hasStructuredAuthError(err) || genericAuthPattern.MatchString(msg) {
|
||||
return AuthErrorGeneric, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
func authErrorText(err error) string {
|
||||
var parts []string
|
||||
if err != nil {
|
||||
parts = append(parts, err.Error())
|
||||
}
|
||||
|
||||
var httpErr *common.HTTPError
|
||||
if errors.As(err, &httpErr) && httpErr != nil {
|
||||
parts = append(parts, httpErr.BodyPreview)
|
||||
}
|
||||
|
||||
return strings.Join(parts, "\n")
|
||||
}
|
||||
|
||||
func hasStructuredAuthError(err error) bool {
|
||||
var failErr *FailoverError
|
||||
if errors.As(err, &failErr) && failErr != nil && failErr.Reason == FailoverAuth {
|
||||
return true
|
||||
}
|
||||
|
||||
var httpErr *common.HTTPError
|
||||
if errors.As(err, &httpErr) && httpErr != nil {
|
||||
return httpErr.StatusCode == 401 || httpErr.StatusCode == 403
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
func classifyFallbackExhaustedAuthError(err *FallbackExhaustedError) (AuthErrorKind, bool) {
|
||||
if err == nil {
|
||||
return "", false
|
||||
}
|
||||
|
||||
var authMessages []string
|
||||
nonSkipped := 0
|
||||
for _, attempt := range err.Attempts {
|
||||
if attempt.Skipped {
|
||||
continue
|
||||
}
|
||||
nonSkipped++
|
||||
if !attemptIsAuthFailure(attempt) {
|
||||
return "", false
|
||||
}
|
||||
if attempt.Error != nil {
|
||||
authMessages = append(authMessages, authErrorText(attempt.Error))
|
||||
}
|
||||
}
|
||||
if nonSkipped == 0 {
|
||||
return "", false
|
||||
}
|
||||
|
||||
msg := strings.Join(authMessages, "\n")
|
||||
if missingAPIKeyPattern.MatchString(msg) {
|
||||
return AuthErrorMissingAPIKey, true
|
||||
}
|
||||
if expiredTokenPattern.MatchString(msg) {
|
||||
return AuthErrorExpiredToken, true
|
||||
}
|
||||
if invalidAPIKeyPattern.MatchString(msg) {
|
||||
return AuthErrorInvalidAPIKey, true
|
||||
}
|
||||
return AuthErrorGeneric, true
|
||||
}
|
||||
|
||||
func attemptIsAuthFailure(attempt FallbackAttempt) bool {
|
||||
if attempt.Reason == FailoverAuth {
|
||||
return true
|
||||
}
|
||||
if attempt.Error == nil {
|
||||
return false
|
||||
}
|
||||
var failErr *FailoverError
|
||||
if errors.As(attempt.Error, &failErr) && failErr != nil && failErr.Reason == FailoverAuth {
|
||||
return true
|
||||
}
|
||||
var httpErr *common.HTTPError
|
||||
if errors.As(attempt.Error, &httpErr) && httpErr != nil {
|
||||
return httpErr.StatusCode == 401 || httpErr.StatusCode == 403
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package providers
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/providers/common"
|
||||
)
|
||||
|
||||
func TestClassifyError_HTTPErrorStatus(t *testing.T) {
|
||||
err := fmt.Errorf("provider request: %w", &common.HTTPError{
|
||||
StatusCode: httpStatusUnauthorized,
|
||||
BodyPreview: `{"error":"unauthorized"}`,
|
||||
ContentType: "application/json",
|
||||
APIBase: "https://api.example.com",
|
||||
})
|
||||
|
||||
result := ClassifyError(err, "openai", "gpt-4")
|
||||
if result == nil {
|
||||
t.Fatal("expected classified error")
|
||||
}
|
||||
if result.Reason != FailoverAuth {
|
||||
t.Fatalf("reason = %q, want %q", result.Reason, FailoverAuth)
|
||||
}
|
||||
if result.Status != httpStatusUnauthorized {
|
||||
t.Fatalf("status = %d, want %d", result.Status, httpStatusUnauthorized)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyAuthError(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
err error
|
||||
want AuthErrorKind
|
||||
}{
|
||||
{
|
||||
name: "invalid api key",
|
||||
err: errors.New(
|
||||
`API request failed: Status: 401 Body: {"error":{"message":"Incorrect API key provided"}}`,
|
||||
),
|
||||
want: AuthErrorInvalidAPIKey,
|
||||
},
|
||||
{
|
||||
name: "missing api key",
|
||||
err: errors.New("API key not configured"),
|
||||
want: AuthErrorMissingAPIKey,
|
||||
},
|
||||
{
|
||||
name: "expired token",
|
||||
err: errors.New("oauth token refresh failed: token has expired"),
|
||||
want: AuthErrorExpiredToken,
|
||||
},
|
||||
{
|
||||
name: "structured generic auth",
|
||||
err: &common.HTTPError{
|
||||
StatusCode: httpStatusUnauthorized,
|
||||
BodyPreview: `{"error":"unauthorized"}`,
|
||||
ContentType: "application/json",
|
||||
APIBase: "https://api.example.com",
|
||||
},
|
||||
want: AuthErrorGeneric,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, ok := ClassifyAuthError(tt.err)
|
||||
if !ok {
|
||||
t.Fatal("expected auth classification")
|
||||
}
|
||||
if got != tt.want {
|
||||
t.Fatalf("kind = %q, want %q", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyAuthError_FallbackExhaustedAllAuth(t *testing.T) {
|
||||
err := &FallbackExhaustedError{
|
||||
Attempts: []FallbackAttempt{
|
||||
{
|
||||
Reason: FailoverAuth,
|
||||
Error: &FailoverError{
|
||||
Reason: FailoverAuth,
|
||||
Wrapped: errors.New("invalid api key"),
|
||||
},
|
||||
},
|
||||
{
|
||||
Reason: FailoverAuth,
|
||||
Error: &FailoverError{
|
||||
Reason: FailoverAuth,
|
||||
Wrapped: errors.New("unauthorized"),
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
got, ok := ClassifyAuthError(err)
|
||||
if !ok {
|
||||
t.Fatal("expected auth classification")
|
||||
}
|
||||
if got != AuthErrorInvalidAPIKey {
|
||||
t.Fatalf("kind = %q, want %q", got, AuthErrorInvalidAPIKey)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyAuthError_FallbackExhaustedMixedFailures(t *testing.T) {
|
||||
err := &FallbackExhaustedError{
|
||||
Attempts: []FallbackAttempt{
|
||||
{
|
||||
Reason: FailoverAuth,
|
||||
Error: &FailoverError{
|
||||
Reason: FailoverAuth,
|
||||
Wrapped: errors.New("invalid api key"),
|
||||
},
|
||||
},
|
||||
{
|
||||
Reason: FailoverRateLimit,
|
||||
Error: &FailoverError{
|
||||
Reason: FailoverRateLimit,
|
||||
Wrapped: errors.New("rate limit exceeded"),
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
if got, ok := ClassifyAuthError(err); ok {
|
||||
t.Fatalf("kind = %q, want no auth classification for mixed failures", got)
|
||||
}
|
||||
}
|
||||
|
||||
const httpStatusUnauthorized = 401
|
||||
@@ -371,6 +371,36 @@ func DecodeToolCallArguments(raw json.RawMessage, name string) map[string]any {
|
||||
|
||||
// --- HTTP response helpers ---
|
||||
|
||||
// HTTPError captures structured details for non-OK provider HTTP responses.
|
||||
// Error preserves the previous user/log-facing text format for compatibility.
|
||||
type HTTPError struct {
|
||||
StatusCode int
|
||||
BodyPreview string
|
||||
ContentType string
|
||||
APIBase string
|
||||
IsHTML bool
|
||||
}
|
||||
|
||||
func (e *HTTPError) Error() string {
|
||||
if e == nil {
|
||||
return "API request failed"
|
||||
}
|
||||
if e.IsHTML {
|
||||
return fmt.Sprintf(
|
||||
"API request failed: %s returned HTML instead of JSON (content-type: %s); check api_base or proxy configuration.\n Status: %d\n Body: %s",
|
||||
e.APIBase,
|
||||
e.ContentType,
|
||||
e.StatusCode,
|
||||
e.BodyPreview,
|
||||
)
|
||||
}
|
||||
return fmt.Sprintf(
|
||||
"API request failed:\n Status: %d\n Body: %s",
|
||||
e.StatusCode,
|
||||
e.BodyPreview,
|
||||
)
|
||||
}
|
||||
|
||||
// HandleErrorResponse reads a non-200 response body and returns an appropriate error.
|
||||
func HandleErrorResponse(resp *http.Response, apiBase string) error {
|
||||
contentType := resp.Header.Get("Content-Type")
|
||||
@@ -381,11 +411,12 @@ func HandleErrorResponse(resp *http.Response, apiBase string) error {
|
||||
if LooksLikeHTML(body, contentType) {
|
||||
return WrapHTMLResponseError(resp.StatusCode, body, contentType, apiBase)
|
||||
}
|
||||
return fmt.Errorf(
|
||||
"API request failed:\n Status: %d\n Body: %s",
|
||||
resp.StatusCode,
|
||||
ResponsePreview(body, 128),
|
||||
)
|
||||
return &HTTPError{
|
||||
StatusCode: resp.StatusCode,
|
||||
BodyPreview: ResponsePreview(body, 128),
|
||||
ContentType: contentType,
|
||||
APIBase: apiBase,
|
||||
}
|
||||
}
|
||||
|
||||
// ReadAndParseResponse peeks at the response body to detect HTML errors,
|
||||
@@ -422,14 +453,13 @@ func LooksLikeHTML(body []byte, contentType string) bool {
|
||||
|
||||
// WrapHTMLResponseError creates a descriptive error for HTML responses.
|
||||
func WrapHTMLResponseError(statusCode int, body []byte, contentType, apiBase string) error {
|
||||
respPreview := ResponsePreview(body, 128)
|
||||
return fmt.Errorf(
|
||||
"API request failed: %s returned HTML instead of JSON (content-type: %s); check api_base or proxy configuration.\n Status: %d\n Body: %s",
|
||||
apiBase,
|
||||
contentType,
|
||||
statusCode,
|
||||
respPreview,
|
||||
)
|
||||
return &HTTPError{
|
||||
StatusCode: statusCode,
|
||||
BodyPreview: ResponsePreview(body, 128),
|
||||
ContentType: contentType,
|
||||
APIBase: apiBase,
|
||||
IsHTML: true,
|
||||
}
|
||||
}
|
||||
|
||||
// ResponsePreview returns a truncated preview of response body for error messages.
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
package common
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestHandleErrorResponse_ReturnsHTTPError(t *testing.T) {
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusUnauthorized)
|
||||
_, _ = w.Write([]byte(`{"error":"unauthorized"}`))
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
resp, err := http.Get(server.URL)
|
||||
if err != nil {
|
||||
t.Fatalf("http.Get() error = %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
err = HandleErrorResponse(resp, server.URL)
|
||||
var httpErr *HTTPError
|
||||
if !errors.As(err, &httpErr) {
|
||||
t.Fatalf("HandleErrorResponse() error = %T, want *HTTPError", err)
|
||||
}
|
||||
if httpErr.StatusCode != http.StatusUnauthorized {
|
||||
t.Fatalf("StatusCode = %d, want %d", httpErr.StatusCode, http.StatusUnauthorized)
|
||||
}
|
||||
if httpErr.BodyPreview != `{"error":"unauthorized"}` {
|
||||
t.Fatalf("BodyPreview = %q", httpErr.BodyPreview)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "Status: 401") {
|
||||
t.Fatalf("Error() should preserve status text, got %q", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func TestWrapHTMLResponseError_ReturnsHTTPError(t *testing.T) {
|
||||
err := WrapHTMLResponseError(
|
||||
http.StatusBadGateway,
|
||||
[]byte("<html>bad</html>"),
|
||||
"text/html",
|
||||
"https://api.example.com",
|
||||
)
|
||||
|
||||
var httpErr *HTTPError
|
||||
if !errors.As(err, &httpErr) {
|
||||
t.Fatalf("WrapHTMLResponseError() error = %T, want *HTTPError", err)
|
||||
}
|
||||
if !httpErr.IsHTML {
|
||||
t.Fatal("expected IsHTML")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "HTML instead of JSON") {
|
||||
t.Fatalf("Error() should preserve HTML message, got %q", err.Error())
|
||||
}
|
||||
}
|
||||
@@ -8,6 +8,8 @@ import (
|
||||
"regexp"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
"github.com/sipeed/picoclaw/pkg/providers/common"
|
||||
)
|
||||
|
||||
// Common patterns in Go HTTP error messages
|
||||
@@ -88,6 +90,8 @@ var (
|
||||
}
|
||||
|
||||
authPatterns = []errorPattern{
|
||||
rxp(`\b(?:invalid|incorrect|malformed|wrong)[-_\s]+(?:api[-_\s]*)?key\b`),
|
||||
rxp(`\b(?:api[-_\s]*)?key[-_\s]+(?:is[-_\s]+)?(?:invalid|incorrect|malformed|wrong)\b`),
|
||||
rxp(`invalid[_ ]?api[_ ]?key`),
|
||||
substr("incorrect api key"),
|
||||
substr("invalid token"),
|
||||
@@ -188,6 +192,18 @@ func ClassifyError(err error, provider, model string) *FailoverError {
|
||||
}
|
||||
|
||||
// Try HTTP status code extraction first.
|
||||
var httpErr *common.HTTPError
|
||||
if errors.As(err, &httpErr) && httpErr != nil {
|
||||
if reason := classifyByStatus(httpErr.StatusCode); reason != "" {
|
||||
return &FailoverError{
|
||||
Reason: reason,
|
||||
Provider: provider,
|
||||
Model: model,
|
||||
Status: httpErr.StatusCode,
|
||||
Wrapped: err,
|
||||
}
|
||||
}
|
||||
}
|
||||
if status := extractHTTPStatus(msg); status > 0 {
|
||||
if reason := classifyByStatus(status); reason != "" {
|
||||
return &FailoverError{
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"maps"
|
||||
"net/http"
|
||||
"net/url"
|
||||
|
||||
@@ -1064,8 +1064,8 @@ func (r *sandboxFs) execute(path string, fn func(root *os.Root, relPath string)
|
||||
return err
|
||||
}
|
||||
|
||||
// os.Root api on windows only accept forward slashes (/)
|
||||
relPath = filepath.ToSlash(relPath)
|
||||
// os.Root API on Windows only accepts forward slashes (/).
|
||||
relPath = normalizeRootRelPath(relPath)
|
||||
|
||||
return fn(root, relPath)
|
||||
}
|
||||
@@ -1230,6 +1230,17 @@ func buildFs(workspace string, restrict bool, patterns []*regexp.Regexp) fileSys
|
||||
return sandbox
|
||||
}
|
||||
|
||||
func normalizeRootRelPath(relPath string) string {
|
||||
return normalizeRootRelPathForSeparator(relPath, os.PathSeparator)
|
||||
}
|
||||
|
||||
func normalizeRootRelPathForSeparator(relPath string, sep rune) string {
|
||||
if sep == '\\' {
|
||||
return strings.ReplaceAll(relPath, `\`, `/`)
|
||||
}
|
||||
return relPath
|
||||
}
|
||||
|
||||
// Helper to get a safe relative path for os.Root usage
|
||||
func getSafeRelPath(workspace, path string) (string, error) {
|
||||
if workspace == "" {
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
package fstools
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestNormalizeRootRelPathForWindowsSeparator(t *testing.T) {
|
||||
got := normalizeRootRelPathForSeparator(`aaa\bbb\file.txt`, '\\')
|
||||
want := "aaa/bbb/file.txt"
|
||||
|
||||
if got != want {
|
||||
t.Fatalf("normalizeRootRelPathForSeparator() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeRootRelPathForUnixSeparatorLeavesBackslashUnchanged(t *testing.T) {
|
||||
input := `aaa\bbb\file.txt`
|
||||
got := normalizeRootRelPathForSeparator(input, '/')
|
||||
|
||||
if got != input {
|
||||
t.Fatalf("normalizeRootRelPathForSeparator() = %q, want %q", got, input)
|
||||
}
|
||||
}
|
||||
@@ -315,7 +315,7 @@ func (p *BraveSearchProvider) Search(
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
_ = resp.Body.Close()
|
||||
|
||||
if err != nil {
|
||||
lastErr = fmt.Errorf("failed to read response: %w", err)
|
||||
@@ -448,7 +448,7 @@ func (p *TavilySearchProvider) Search(
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
_ = resp.Body.Close()
|
||||
|
||||
if err != nil {
|
||||
lastErr = fmt.Errorf("failed to read response: %w", err)
|
||||
@@ -906,7 +906,7 @@ func (p *SogouSearchProvider) Search(
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
resp.Body.Close()
|
||||
_ = resp.Body.Close()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to read response: %w", err)
|
||||
}
|
||||
@@ -1147,7 +1147,7 @@ func (p *PerplexitySearchProvider) Search(
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
_ = resp.Body.Close()
|
||||
|
||||
if err != nil {
|
||||
lastErr = fmt.Errorf("failed to read response: %w", err)
|
||||
|
||||
+20
-21
@@ -1135,36 +1135,35 @@ func expandPowerShellEnvVars(cmd string) string {
|
||||
})
|
||||
}
|
||||
|
||||
func (t *ExecTool) commandMatchesAllowPattern(lower string) bool {
|
||||
for _, pattern := range t.allowPatterns {
|
||||
if pattern.MatchString(lower) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
for _, pattern := range t.customAllowPatterns {
|
||||
if pattern.MatchString(lower) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (t *ExecTool) guardCommand(command, cwd string) string {
|
||||
cmd := strings.TrimSpace(command)
|
||||
lower := strings.ToLower(cmd)
|
||||
|
||||
// Custom allow patterns exempt a command from deny checks.
|
||||
explicitlyAllowed := false
|
||||
for _, pattern := range t.customAllowPatterns {
|
||||
// Deny patterns always apply, even when a command matches a custom allow rule.
|
||||
// Custom allow rules can permit a command, but must not disable secret-safety
|
||||
// deny rules such as jq env access checks (#3079).
|
||||
for _, pattern := range t.denyPatterns {
|
||||
if pattern.MatchString(lower) {
|
||||
explicitlyAllowed = true
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if !explicitlyAllowed {
|
||||
for _, pattern := range t.denyPatterns {
|
||||
if pattern.MatchString(lower) {
|
||||
return "Command blocked by safety guard (dangerous pattern detected)"
|
||||
}
|
||||
return "Command blocked by safety guard (dangerous pattern detected)"
|
||||
}
|
||||
}
|
||||
|
||||
if len(t.allowPatterns) > 0 {
|
||||
allowed := false
|
||||
for _, pattern := range t.allowPatterns {
|
||||
if pattern.MatchString(lower) {
|
||||
allowed = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !allowed {
|
||||
if !t.commandMatchesAllowPattern(lower) {
|
||||
return "Command blocked by safety guard (not in allowlist)"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1936,3 +1936,54 @@ func TestShellTool_SchemelessURLDetection(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestShellTool_CustomAllowDoesNotBypassDenyPatterns(t *testing.T) {
|
||||
cfg := &config.Config{}
|
||||
cfg.Tools.Exec.EnableDenyPatterns = true
|
||||
cfg.Tools.Exec.CustomAllowPatterns = []string{`^jq\b`}
|
||||
cfg.Tools.Exec.CustomDenyPatterns = []string{`\$env\b`, `(^|[^.$a-z0-9_])env([^a-z0-9_]|$)`}
|
||||
|
||||
tool, err := NewExecToolWithConfig(t.TempDir(), false, cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("NewExecToolWithConfig() error: %v", err)
|
||||
}
|
||||
|
||||
got := tool.guardCommand(`jq -n '$ENV.PICOCLAW_VARIANT_CANARY'`, t.TempDir())
|
||||
if !strings.Contains(got, "dangerous pattern detected") {
|
||||
t.Fatalf("custom allow should not bypass deny patterns, got: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestShellTool_CustomAllowStillPermitsSafeMatch(t *testing.T) {
|
||||
cfg := &config.Config{}
|
||||
cfg.Tools.Exec.EnableDenyPatterns = true
|
||||
cfg.Tools.Exec.CustomAllowPatterns = []string{`^jq\b`}
|
||||
cfg.Tools.Exec.CustomDenyPatterns = []string{`\$env\b`, `(^|[^.$a-z0-9_])env([^a-z0-9_]|$)`}
|
||||
|
||||
tool, err := NewExecToolWithConfig(t.TempDir(), false, cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("NewExecToolWithConfig() error: %v", err)
|
||||
}
|
||||
|
||||
got := tool.guardCommand(`jq -n '"ok"'`, t.TempDir())
|
||||
if got != "" {
|
||||
t.Fatalf("safe custom-allowed command should pass guard, got: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestShellTool_CustomAllowDoesNotBecomeStrictAllowlist(t *testing.T) {
|
||||
cfg := &config.Config{}
|
||||
cfg.Tools.Exec.EnableDenyPatterns = true
|
||||
cfg.Tools.Exec.CustomAllowPatterns = []string{`^jq\b`}
|
||||
cfg.Tools.Exec.CustomDenyPatterns = []string{`\$env\b`, `(^|[^.$a-z0-9_])env([^a-z0-9_]|$)`}
|
||||
|
||||
tool, err := NewExecToolWithConfig(t.TempDir(), false, cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("NewExecToolWithConfig() error: %v", err)
|
||||
}
|
||||
|
||||
got := tool.guardCommand("ls", t.TempDir())
|
||||
if got != "" {
|
||||
t.Fatalf("custom allow patterns should not become a strict allowlist, got: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/sipeed/picoclaw/web/backend/middleware"
|
||||
@@ -89,6 +90,60 @@ func (h *launcherAuthHandlers) isStoreInitialized(ctx context.Context) (bool, er
|
||||
return h.store.IsInitialized(ctx)
|
||||
}
|
||||
|
||||
func launcherSetupCrossSite(r *http.Request) bool {
|
||||
fetchSite := strings.ToLower(strings.TrimSpace(r.Header.Get("Sec-Fetch-Site")))
|
||||
if fetchSite == "cross-site" {
|
||||
return true
|
||||
}
|
||||
|
||||
if origin := strings.TrimSpace(r.Header.Get("Origin")); origin != "" {
|
||||
return !sameLauncherRequestOrigin(r, origin)
|
||||
}
|
||||
|
||||
if referer := strings.TrimSpace(r.Header.Get("Referer")); referer != "" {
|
||||
return !sameLauncherRequestOrigin(r, referer)
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
func sameLauncherRequestOrigin(r *http.Request, raw string) bool {
|
||||
if strings.ContainsAny(raw, " \t\r\n") {
|
||||
return false
|
||||
}
|
||||
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil || u.Scheme == "" || u.Host == "" {
|
||||
return false
|
||||
}
|
||||
|
||||
wantScheme := launcherRequestScheme(r)
|
||||
wantHost := r.Host
|
||||
if wantHost == "" {
|
||||
wantHost = r.URL.Host
|
||||
}
|
||||
return strings.EqualFold(u.Scheme, wantScheme) && strings.EqualFold(u.Host, wantHost)
|
||||
}
|
||||
|
||||
func launcherRequestScheme(r *http.Request) string {
|
||||
if proto := strings.TrimSpace(r.Header.Get("X-Forwarded-Proto")); proto != "" {
|
||||
if i := strings.IndexByte(proto, ','); i >= 0 {
|
||||
proto = proto[:i]
|
||||
}
|
||||
proto = strings.ToLower(strings.TrimSpace(proto))
|
||||
if proto == "http" || proto == "https" {
|
||||
return proto
|
||||
}
|
||||
}
|
||||
if r.TLS != nil {
|
||||
return "https"
|
||||
}
|
||||
if r.URL != nil && r.URL.Scheme != "" {
|
||||
return r.URL.Scheme
|
||||
}
|
||||
return "http"
|
||||
}
|
||||
|
||||
func (h *launcherAuthHandlers) handleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
var body launcherAuthLoginBody
|
||||
@@ -198,6 +253,12 @@ func (h *launcherAuthHandlers) handleStatus(w http.ResponseWriter, r *http.Reque
|
||||
func (h *launcherAuthHandlers) handleSetup(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
|
||||
if launcherSetupCrossSite(r) {
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
_, _ = w.Write([]byte(`{"error":"cross-site setup request rejected"}`))
|
||||
return
|
||||
}
|
||||
|
||||
if h.store == nil {
|
||||
w.WriteHeader(http.StatusServiceUnavailable)
|
||||
if h.storeErr != nil {
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestLauncherAuthSetupRejectsCrossSiteFirstRun(t *testing.T) {
|
||||
store := &fakePasswordStore{}
|
||||
mux := http.NewServeMux()
|
||||
RegisterLauncherAuthRoutes(mux, LauncherAuthRouteOpts{
|
||||
SessionCookie: "session-cookie-value",
|
||||
PasswordStore: store,
|
||||
})
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest(
|
||||
http.MethodPost,
|
||||
"http://127.0.0.1:18800/api/auth/setup",
|
||||
strings.NewReader(`{"password":"CrossSitePwn123!","confirm":"CrossSitePwn123!"}`),
|
||||
)
|
||||
req.Header.Set("Origin", "https://evil.example")
|
||||
req.Header.Set("Referer", "https://evil.example/attack")
|
||||
req.Header.Set("Sec-Fetch-Site", "cross-site")
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
mux.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Fatalf("cross-site setup code = %d body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if store.initialized || store.password != "" {
|
||||
t.Fatalf("cross-site setup mutated store: initialized=%v password=%q", store.initialized, store.password)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLauncherAuthSetupAllowsSameOriginFirstRun(t *testing.T) {
|
||||
store := &fakePasswordStore{}
|
||||
mux := http.NewServeMux()
|
||||
RegisterLauncherAuthRoutes(mux, LauncherAuthRouteOpts{
|
||||
SessionCookie: "session-cookie-value",
|
||||
PasswordStore: store,
|
||||
})
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest(
|
||||
http.MethodPost,
|
||||
"http://127.0.0.1:18800/api/auth/setup",
|
||||
strings.NewReader(`{"password":"LocalSetup123!","confirm":"LocalSetup123!"}`),
|
||||
)
|
||||
req.Header.Set("Origin", "http://127.0.0.1:18800")
|
||||
req.Header.Set("Sec-Fetch-Site", "same-origin")
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
mux.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("same-origin setup code = %d body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if !store.initialized || store.password != "LocalSetup123!" {
|
||||
t.Fatalf("same-origin setup store: initialized=%v password=%q", store.initialized, store.password)
|
||||
}
|
||||
}
|
||||
@@ -35,14 +35,14 @@
|
||||
"radix-ui": "^1.4.3",
|
||||
"react": "19.2.5",
|
||||
"react-dom": "19.2.5",
|
||||
"react-i18next": "^17.0.6",
|
||||
"react-i18next": "^17.0.7",
|
||||
"react-markdown": "^10.1.0",
|
||||
"react-textarea-autosize": "^8.5.9",
|
||||
"rehype-highlight": "^7.0.2",
|
||||
"rehype-raw": "^7.0.0",
|
||||
"rehype-sanitize": "^6.0.0",
|
||||
"remark-gfm": "^4.0.1",
|
||||
"shadcn": "^4.7.0",
|
||||
"shadcn": "^4.12.0",
|
||||
"sonner": "^2.0.7",
|
||||
"tailwind-merge": "^3.5.0",
|
||||
"tailwindcss": "^4.3.0",
|
||||
@@ -58,7 +58,7 @@
|
||||
"@types/react": "^19.2.7",
|
||||
"@types/react-dom": "^19.2.3",
|
||||
"@typescript-eslint/eslint-plugin": "^8.58.2",
|
||||
"@vitejs/plugin-react": "^6.0.1",
|
||||
"@vitejs/plugin-react": "^6.0.3",
|
||||
"eslint": "^10.4.1",
|
||||
"eslint-config-prettier": "^10.1.8",
|
||||
"eslint-plugin-react-hooks": "^7.1.1",
|
||||
@@ -67,7 +67,7 @@
|
||||
"prettier": "^3.8.3",
|
||||
"prettier-plugin-tailwindcss": "^0.8.0",
|
||||
"typescript": "~5.9.3",
|
||||
"typescript-eslint": "^8.59.3",
|
||||
"typescript-eslint": "^8.62.1",
|
||||
"vite": "^8.0.16"
|
||||
}
|
||||
}
|
||||
|
||||
Generated
+701
-776
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user