Commit Graph

105 Commits

Author SHA1 Message Date
uiyzzi 16d23d8cdc feat(security): add sensitive data filtering for tool results sent to
LLM

Prevent LLM from seeing its own credentials (API keys, tokens, secrets)
by filtering sensitive values from tool call results before sending to
the
model. Values are collected from .security.yml and replaced with
[FILTERED] using an efficient strings.Replacer (O(n+m)).

- Add FilterSensitiveData and FilterMinLength to ToolsConfig
- Implement SensitiveDataReplacer() with sync.Once caching in
  SecurityConfig
- Use reflection to collect all sensitive values (Model API keys,
  channel
  tokens, web tool API keys, skills tokens)
- Apply filtering in agent loop at 4 tool result locations
- Add comprehensive tests covering all token types
2026-03-23 20:55:41 +08:00
uiyzzi 79df938696 Use getter/setter methods for API key access in ModelConfig 2026-03-23 16:39:43 +08:00
uiyzzi f2985b8bee feat(providers): add extra_body config to inject custom fields into request body
Allow configuring provider-specific fields like reasoning_split for minimax via
the model config's extra_body map. These fields are merged into the request
body last, giving them precedence over default values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 16:39:42 +08:00
Cytown 36f9d20de1 Merge branch 'main' into version 2026-03-23 15:00:18 +08:00
Kunal Karmakar 40279c8dde chore(config): move loglevel settings under gateway (#1912)
* Move log level config to gateway property

* Fix unit test

* Fix linting

* Fix linting

* Add comment for log level
2026-03-23 06:14:53 +01:00
Cytown 5a8aab8143 Merge branch 'main' into version 2026-03-23 11:41:36 +08:00
Cytown 7bf4831059 Merge branch 'main' into version 2026-03-23 10:54:08 +08:00
Caize Wu 3a61892313 Merge pull request #1875 from BeaconCat/docs/readme-restructure-v2
docs: restructure README with Quick Start Guide, i18n, and Weixin channel
2026-03-23 10:30:05 +08:00
Mauro 3500080abb Merge pull request #1891 from RussellLuo/audio-transcription
feat(voice): add audio-model transcription support
2026-03-23 00:23:30 +01:00
BeaconCat 60a7098fd3 feat(search): add Baidu Qianfan AI Search provider with i18n docs
- Add BaiduSearchConfig struct and register in WebToolsConfig/defaults
- Insert Baidu Search in priority chain: DuckDuckGo > Baidu > GLM Search
- Use perplexityTimeout (30s) — Qianfan is LLM-based
- Fix response parsing: use references[] field per API spec
- Add baidu_search block to config.example.json

docs: sync configuration.md and README Documentation table across all languages

- Complete truncated configuration.md for fr/ja/pt-br/vi/zh: add Spawn
  async flow diagram, Providers table, Model Configuration (all vendors,
  examples, load balancing, migration), Provider Architecture, Scheduled
  Tasks, and Advanced Topics links
- Add Hooks/Steering/SubTurn entries to Documentation table in all 8
  READMEs (en/zh/fr/id/it/ja/pt-br/vi), ordered before Troubleshooting
- Add Baidu Search row to web search table in all 8 READMEs and
  tools_configuration.md (en + 5 i18n); zh README reorders search
  engines with China-friendly options first
- Add Matrix channel docs translations (fr/ja/pt-br/vi)
- Add Weixin channel to chat-apps.md and all README Channels tables

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 00:51:27 +08:00
yinwm c48954d32d merge: sync main into refactor/agent 2026-03-22 21:44:17 +08:00
RussellLuo 8ad4b9b497 feat(voice): add audio-model transcription support
- Add `AudioModelTranscriber` for model-based audio transcription via LLM providers
- Support selecting a transcription model with `voice.model_name` in config
- Keep Groq transcription as a fallback and move it into dedicated files with focused tests
- Serialize `data:audio/...` media as input_audio for OpenAI-compatible providers
- Improve transcription logging by rendering error fields as strings
- Add coverage for transcriber detection, audio-model behavior, provider audio serialization, and Groq transcription

Fixes #1890.
2026-03-22 20:07:22 +08:00
Cytown 284ced1f5c Merge branch 'main' into version 2026-03-22 19:58:33 +08:00
Administrator f7f27e237a merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
Hua Audio dd82794255 Feat/weixin openclaw port (#1873)
* init

* fix lint

* fix go test

* update docs

* incorporate pr review

* Update pkg/channels/weixin/weixin.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat(weixin): add media sync and typing support

* test(weixin): cover media and sync helpers

---------

Co-authored-by: zhangmikoto <i@electromaster.me>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Hoshina <hoshina@evaz.org>
2026-03-22 14:23:39 +08:00
Administrator 88d754b172 merge main 2026-03-22 13:47:14 +08:00
Cytown 7c854fe6d7 Merge branch 'main' into version 2026-03-22 02:53:55 +08:00
Cytown e455eb5e67 refactor: seperate security.yml for store keys 2026-03-22 01:55:00 +08:00
Administrator 1bd144ac13 Merge branch 'upstream-main' into feat/subturn-poc 2026-03-21 17:13:26 +08:00
Kunal Karmakar 647071d342 Add default value for config 2026-03-21 06:54:49 +00:00
Amir Mamaghani 71134babb9 feat(telegram): stream LLM responses via sendMessageDraft (#1101)
* feat(telegram): stream LLM responses in real-time via sendMessageDraft

Implements real-time token streaming to Telegram using the sendMessageDraft
API (telego v1.6.0). Instead of showing only a "Thinking..." placeholder
until the full response arrives, users now see partial LLM output appear
in the chat as it's generated.

The streaming pipeline threads through all layers:

- StreamingProvider interface (providers/types.go): opt-in ChatStream()
  method that receives an onChunk callback with accumulated text
- OpenAI-compatible SSE streaming (openai_compat/provider.go): parses
  SSE events with stream:true, handles text deltas and tool call assembly
- Anthropic native streaming (anthropic/provider.go): uses SDK's
  NewStreaming() for direct Anthropic API connections
- HTTPProvider delegation (http_provider.go): delegates ChatStream to
  the underlying openai_compat provider
- StreamingCapable + Streamer interfaces (channels/interfaces.go):
  opt-in channel capability like TypingCapable/PlaceholderCapable
- Telegram streamer (telegram/telegram.go): BeginStream returns a
  telegramStreamer that throttles sendMessageDraft calls (3s/200 chars)
  with graceful degradation on API errors
- StreamDelegate bridge (bus/bus.go): decouples agent loop from channel
  manager without tight imports
- Manager integration (manager.go): implements StreamDelegate, tracks
  streamActive state, coordinates with placeholder editing
- Agent loop (loop.go): uses ChatStream when both provider and channel
  support streaming, cancels stream on tool calls, skips PublishOutbound
  when Finalize already delivered the message

Graceful degradation:
- Bots without forum/topics mode: first sendMessageDraft error sets
  failed=true, subsequent Updates become no-ops, Finalize still delivers
  via SendMessage. User sees normal non-streaming behavior.
- Non-streaming providers: type assertion fails, falls back to Chat()
- Config opt-out: streaming.enabled (default true) in telegram config

Closes #1098

* fix(telegram): delete placeholder message when streaming delivers response

When streaming was active, the "Thinking..." placeholder message stayed
in the chat because preSend only deleted the tracking entry without
removing the actual Telegram message. Now preSend deletes the placeholder
via the new MessageDeleter interface when streamActive is set.

* refactor(streaming): remove dead code and simplify streaming wiring

- Delete unused Anthropic ChatStream/parseStream (-131 lines) — factory
  creates HTTPProvider for all OpenAI-compat providers including OpenRouter
- Simplify runLLMIteration from 4 to 3 return values (remove unused
  streamed bool)
- Replace managerStreamer struct with finalizeHookStreamer using embedding
  (Update/Cancel promoted, only Finalize overridden)

* fix(streaming): skip streamer acquisition when SendResponse is false

Heartbeat messages set SendResponse=false but the streaming path
was unconditionally acquiring a streamer, causing HEARTBEAT_OK to
leak to Telegram via streamer.Finalize().

* fix(streaming): guard streamer for non-sendable messages, add streaming config

Skip streamer acquisition for heartbeat (NoHistory=true), preventing
HEARTBEAT_OK from leaking to Telegram via streamer.Finalize().

Add streaming.enabled to Telegram defaults and example config.

* feat(telegram): stream LLM responses in real-time via sendMessageDraft

Implements real-time token streaming to Telegram using the sendMessageDraft
API (telego v1.6.0). Instead of showing only a "Thinking..." placeholder
until the full response arrives, users now see partial LLM output appear
in the chat as it's generated.

The streaming pipeline threads through all layers:

- StreamingProvider interface (providers/types.go): opt-in ChatStream()
  method that receives an onChunk callback with accumulated text
- OpenAI-compatible SSE streaming (openai_compat/provider.go): parses
  SSE events with stream:true, handles text deltas and tool call assembly
- Anthropic native streaming (anthropic/provider.go): uses SDK's
  NewStreaming() for direct Anthropic API connections
- HTTPProvider delegation (http_provider.go): delegates ChatStream to
  the underlying openai_compat provider
- StreamingCapable + Streamer interfaces (channels/interfaces.go):
  opt-in channel capability like TypingCapable/PlaceholderCapable
- Telegram streamer (telegram/telegram.go): BeginStream returns a
  telegramStreamer that throttles sendMessageDraft calls (3s/200 chars)
  with graceful degradation on API errors
- StreamDelegate bridge (bus/bus.go): decouples agent loop from channel
  manager without tight imports
- Manager integration (manager.go): implements StreamDelegate, tracks
  streamActive state, coordinates with placeholder editing
- Agent loop (loop.go): uses ChatStream when both provider and channel
  support streaming, cancels stream on tool calls, skips PublishOutbound
  when Finalize already delivered the message

Graceful degradation:
- Bots without forum/topics mode: first sendMessageDraft error sets
  failed=true, subsequent Updates become no-ops, Finalize still delivers
  via SendMessage. User sees normal non-streaming behavior.
- Non-streaming providers: type assertion fails, falls back to Chat()
- Config opt-out: streaming.enabled (default true) in telegram config

Closes #1098

* fix(telegram): delete placeholder message when streaming delivers response

When streaming was active, the "Thinking..." placeholder message stayed
in the chat because preSend only deleted the tracking entry without
removing the actual Telegram message. Now preSend deletes the placeholder
via the new MessageDeleter interface when streamActive is set.

* refactor(streaming): remove dead code and simplify streaming wiring

- Delete unused Anthropic ChatStream/parseStream (-131 lines) — factory
  creates HTTPProvider for all OpenAI-compat providers including OpenRouter
- Simplify runLLMIteration from 4 to 3 return values (remove unused
  streamed bool)
- Replace managerStreamer struct with finalizeHookStreamer using embedding
  (Update/Cancel promoted, only Finalize overridden)

* fix(streaming): skip streamer acquisition when SendResponse is false

Heartbeat messages set SendResponse=false but the streaming path
was unconditionally acquiring a streamer, causing HEARTBEAT_OK to
leak to Telegram via streamer.Finalize().

* fix(streaming): guard streamer for non-sendable messages, add streaming config

Skip streamer acquisition for heartbeat (NoHistory=true), preventing
HEARTBEAT_OK from leaking to Telegram via streamer.Finalize().

Add streaming.enabled to Telegram defaults and example config.

* fix(picoclaw): add missing closing brace for StreamingProvider interface

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve golangci-lint formatting issues

Fix gci import ordering in telegram and anthropic provider, and break
long function signature in openai_compat provider to satisfy golines.

* fix: address code review feedback on streaming PR

- Deduplicate Streamer interface: alias channels.Streamer to bus.Streamer
  to prevent type drift across packages
- Increase SSE scanner buffer to 10MB max to handle large single-line
  responses that exceed bufio.Scanner's 64KB default
- Switch draftID generation from math/rand to crypto/rand for
  collision-resistant random IDs
- Add context cancellation check in SSE parsing loop so cancelled
  streams stop processing immediately
- Log Finalize failures with chat_id and content length for debugging
  silent message delivery failures

* feat: make streaming throttle interval and min growth configurable

Move hardcoded streamThrottleInterval (3s) and streamMinGrowth (200)
into StreamingConfig so they can be tuned per deployment via config
or environment variables.

* fix(telegram): use parseTelegramChatID in DeleteMessage and BeginStream

These two functions called undefined parseChatID. Use
parseTelegramChatID with _ for the unused threadID instead of adding
a wrapper function. Fixes all three CI checks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(streaming): set streamActive only after successful Finalize

Move onFinalize hook to run after Streamer.Finalize succeeds, so that
if Finalize fails the streamActive flag stays false and the regular
placeholder fallback path remains available.

Addresses review feedback from @alexhoshina.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 21:04:14 +08:00
Administrator 4f646ef2b8 Merge branch 'main' into feat/subturn-poc 2026-03-20 11:51:25 +08:00
美電球 75d86721a3 Feat/wecom aibot processing message config (#1785)
* feat(wecom_aibot): make processing message configurable

* docs(wecom): document ai bot processing message

* test(wecom_aibot): adapt webhook tests to channel interface

* fix: lint err
2026-03-20 00:23:40 +08:00
Administrator 583c586db6 Merge branch 'main' into feat/subturn-poc 2026-03-19 20:20:31 +08:00
Cytown 94fcb25039 Merge branch 'main' into version 2026-03-19 18:16:15 +08:00
Mauro 7673b626b3 feat(tool): debug tool usage via channels (#1332)
* feat(tool): debug usage via channel

* set defaults

* fix conflicts
2026-03-19 18:08:50 +08:00
Cytown cfd3a1b441 Merge branch 'main' into version 2026-03-19 18:04:58 +08:00
Administrator 532ea4b56c Merge branch 'upstream-main' into feat/subturn-poc 2026-03-19 16:47:35 +08:00
美電球 828971d549 Feat/qq local file upload (#1722)
* feat(qq): support media uploads and inbound attachments

* docs(qq): document media size limit settings

* chore(web): add QQ media size limit hints

* fix(qq): demote botgo heartbeat logs

* style(qq): fix lint issues
2026-03-19 16:27:34 +08:00
Administrator 431a53cbb1 Merge branch 'upstream-main' into feat/subturn-poc 2026-03-18 22:57:01 +08:00
Alexander 12f4029610 feat: telegram use parse mode ModeMarkdownV2 instead of ModeHTML (#1018)
* feat: telegram use parse mode ModeMarkdownV2 instead of ModeHTML

* handle expandable block quotation starts, add test for all md2 formats

* fix: linter issue

* feat: added flag use_markdown_v2, corrected config, updated
documentation

* move parseChatID to parser_markdown_to_html

* fix: tests and linter issues

* fix: case with ~

* test: fixed Test_markdownToTelegramMarkdownV2

* fix: regex block-quote line  >

* fix: linter issues

* fix: send chunk param mismatched, in edit msg use HTML parse mode too

* fix: remove from .gitignore redundant comment
2026-03-18 21:29:21 +08:00
dev-miro26 c07f5c948f refactor: centralize environment variable key constants (#1730)
* refactor: centralize environment variable key constants

* refactor: update environment variable constants and usage in gateway
2026-03-18 18:03:24 +08:00
dataCenter430 f79469c19d Add model-native search (prefer_native) for OpenAI/Codex (#1618)
* config: add prefer_native and NativeSearchCapable for model-native search

* providers: implement native web search for OpenAI and Codex

* agent: use provider-native search when prefer_native and supported

* tests: add coverage for model-native search

* fix: Golang lint errors

* fix: update the code based on the review

* fix: update codex_provider_test
2026-03-18 11:55:30 +08:00
Mauro 3791f06faf Merge branch 'main' into feat/markdown-output-format-web-fetch 2026-03-17 16:37:22 +01:00
Administrator e00a3d9017 Merge upstream/main into feat/subturn-poc
Includes JSONL session persistence (#1170), spawn_status tool, Azure provider,
credential encryption, and various fixes. SubTurn features preserved and
integrated with new spawn_status functionality.
2026-03-17 21:55:20 +08:00
wenjie 8a44410e37 feat: add web gateway hot reload and polling state sync (#1684)
* feat(gateway): support hot reload and empty startup

- extract gateway runtime into pkg/gateway
- add gateway.hot_reload config with default and example values
- allow starting the gateway without a default model via --allow-empty
- stop treating missing enabled channels as a startup error
- update related tests

* feat: replace gateway SSE updates with polling-based state sync

- remove gateway SSE broadcasting and event endpoint
- add polling-based gateway status refresh with stopping state handling
- detect when gateway restart is required after default model changes
- resolve gateway health and websocket proxy targets from configured host
- update gateway UI labels and add backend/frontend test coverage
2026-03-17 18:46:00 +08:00
Desmond Foo b402888bfa feat(tools): add SpawnStatusTool for reporting subagent statuses (#1540)
* feat(tools): add SpawnStatusTool for reporting subagent statuses

* feat(tools): enhance SpawnStatusTool to restrict task visibility by conversation context

* feat(tests): add Unicode result truncation and channel filtering tests for SpawnStatusTool

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* feat(tools): enhance SpawnStatusTool with task ID validation and sorting by creation timestamp

* feat(tools): update SpawnStatusTool description and parameter documentation for clarity

* refactor(tests): improve comments for clarity in ChannelFiltering test case

* fix(tools): update no subagents message for clarity and remove unnecessary locking in runTask

* fix(tools): improve description clarity for SpawnStatusTool regarding task context

* feat(tools): add spawn_status tool configuration and registration

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(agent): improve subagent management for spawn and spawn_status tools

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(tests): update ResultTruncation_Unicode test to use valid CJK character

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: lxowalle <83055338+lxowalle@users.noreply.github.com>
2026-03-17 14:41:43 +08:00
wenjie fcb69860c4 feat(web): add configurable cron command execution settings (#1647)
- add tools.cron.allow_command config with a default value of true
- require command_confirm only when cron command execution is disabled
- expose cron command permission and timeout settings in the config UI
- add backend tests and update i18n strings
2026-03-17 09:44:32 +08:00
afjcjsbx d5c2bc538a feat(tool): markdown format in output web_fetch tool 2026-03-15 22:12:03 +01:00
Mauro 021aa7d6d5 feat(agent): steering (#1517)
* feat(agent): steering

* fix loop

* fix lint

* fix lint
2026-03-16 00:08:16 +08:00
Kunal Karmakar 5fb4b3bedf feat(provider): add support for azure openai provider (#1422)
* Add support for azure openai provider

* Add checks for deployment model name

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Addressing @Copilot suggestion to remove the init() function which seemed redundant

* Fix readme

* Fix linting checks

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-14 22:52:34 +08:00
dataCenter430 b811e9186c feat(provider): add ModelScope as OpenAI-compatible provider (#1486)
* feat(provider): add ModelScope as OpenAI-compatible provider

* test(provider): add ModelScope provider and migration tests

* docs: add ModelScope to README provider tables and free tier sections

* chore: add ModelScope to example config and env template
2026-03-13 14:02:23 +08:00
Guoguo 1e024321c0 refactor: update model name and add VolcEngine coding plan (#1412)
* docs: swap header logo to webp, move meme logo to bottom

Replace header logo with assets/logo.webp across all 6 README
language variants and move the original meme logo (logo.jpg)
to the bottom of each file.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update GPT model names to gpt-5.4 and refine provider descriptions

Update all 6 language README variants:
- Correct GPT model references from gpt-5.2/gpt4 to gpt-5.4
- Refine provider descriptions in API Key comparison tables

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update default model to gpt-5.4, codex to gpt-5.3-codex

Update OpenAI default model references from gpt-5.2 to gpt-5.4
across source code, config examples, tests, and docs. Set Codex
default model to gpt-5.3-codex.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 16:10:29 +08:00
Cytown 1c123e0162 refactor Config to add Version and migratable 2026-03-12 13:52:55 +08:00
LeaderOnePro 9222351871 feat(providers): add LongCat model provider support (#1317)
* feat(providers): add LongCat model provider support

Add LongCat as an OpenAI-compatible provider with base URL
https://api.longcat.chat/openai and default model LongCat-Flash-Thinking.
Includes provider config, migration, factory routing, example config,
tests, and README entries for all 6 locales.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(providers): address LongCat review feedback

- Add dedicated factory routing test for LongCat provider
- Add longcat to DefaultAPIBase test coverage
- Set default api_base in example config providers section

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(providers): add ResolveProviderSelection tests for LongCat

Add two test cases to TestResolveProviderSelection:
- Explicit provider selection with api_base default and proxy wiring
- Fallback inference from model name with api_base default

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-12 02:34:42 +08:00
wenjie 8949a2575b Add exec allow_remote config support in web settings (#1363)
- default tools.exec.allow_remote to true when omitted in config loading
- preserve allow_remote in OpenClaw config migration and API updates
- expose allow_remote in the web config form with i18n strings
- add backend and config tests covering the new default behavior
2026-03-11 19:57:59 +08:00
wenjie 8c2a9332c6 fix(security): harden unauthenticated tool-exec paths (#1360)
* fix(security): harden unauthenticated tool-exec paths (GHSA-pv8c-p6jf-3fpp)

- Exec tool: channel-based access control (default deny remote)
- Cron tool: command scheduling restricted to internal channels
- Web fetch: SSRF defense-in-depth (pre-flight + dial-time + redirect checks)
- File permissions: session/state dirs 0700, files 0600
- Registry: inject __channel/__chat_id into tool args (replaces racy SetContext)

28 new security regression tests.

(cherry picked from commit 191446ae19021604d3d5b0d9376b9655ab749105)

* fix(exec): revalidate working_dir before command start

* test(web): allow local oversized payload fixture

---------

Co-authored-by: xj <gh-xj@users.noreply.github.com>
2026-03-11 19:22:20 +08:00
afjcjsbx 08cc09e091 resolve conflicts 2026-03-11 00:17:10 +01:00
lxowalle 680e845d61 feat:Modify the location where version is obtained, and insert version information into the context (#1300)
* feat:migrate version info from internal package to pkg/config

* * fix lint issue
2026-03-10 17:42:05 +08:00
yanhool 95716b106b feat(web_search): add load balance and failover for api keys (#982)
* feat(web_search): add load balance and failover for api keys

* feat(web_search): add load balance and failover for api keys

* lint

* new iter to get api key

* deleted conflicts
2026-03-10 19:34:11 +11:00