Files
picoclaw/pkg/channels/feishu/common.go
T
ywj 009a8d702b Feat/feishu card parsing (#1534)
* feat(feishu): add interactive card message parsing

Add support for parsing inbound Feishu interactive card messages.
When a user sends a card message, the text content is now extracted
and passed to the LLM for processing.

- Add extractCardText() to recursively extract text from card JSON
- Support both JSON 1.0 (legacy) and JSON 2.0 schema formats
- Handle nested elements: header, body, actions, columns
- Extract text from markdown, lark_md, and plain_text elements
- Add comprehensive unit tests for card parsing

Fixes #<issue_number>

💘 Generated with Crush

Assisted-by: GLM-5 via Crush <crush@charm.land>

* feat(feishu): extract and download images from interactive cards

When receiving interactive card messages, extract embedded images
(img_key, src, icon_key) and download them for LLM processing.

- Add extractCardImageKeys() to recursively extract image keys from card JSON
- Support img elements (img_key, src) and icon elements (icon_key)
- Update downloadInboundMedia() to handle MsgTypeInteractive
- Add comprehensive unit tests for image extraction

Images are downloaded and stored via MediaStore, then appended to
the message content as [image: photo] tags for LLM visibility.

💘 Generated with Crush

Assisted-by: GLM-5 via Crush <crush@charm.land>

* fix(feishu): simplify card parsing - pass raw JSON, only extract images

Address review feedback: text extraction cannot exhaustively handle all
card formats (i18n_elements, div.fields, etc.). Pass raw JSON to LLM
instead - same approach as MsgTypePost. Only image extraction remains
as images must be downloaded for LLM to process.

- Remove extractCardText() and helper functions
- extractContent() now returns raw JSON for MsgTypeInteractive
- Keep extractCardImageKeys() for downloading embedded images
- Update tests to expect raw JSON for interactive cards

* fix(feishu): don't append media tags to interactive card JSON

Appending media tags like "[attachment]" to raw JSON content produces
invalid JSON format. For interactive cards, the JSON already contains
image information and media refs are downloaded separately.

- Skip appendMediaTags for MsgTypeInteractive to preserve valid JSON
- Add test case for interactive card with images

* fix(feishu): filter out external URLs from card image extraction

Only Feishu-hosted image keys (img_xxx, icon_xxx) can be downloaded via
the Feishu API. External URLs in src field (https://...) should be
filtered out to avoid download failures.

- Add isFeishuImageKey() to detect Feishu-hosted keys vs external URLs
- Update extractImageKeysRecursive to skip external URLs in src field
- Add tests for external URL filtering and mixed scenarios

* feat(feishu): support downloading external images from interactive cards

Previously only Feishu-hosted images (img_key, icon_key) could be
downloaded. Now external URLs in src field are also downloaded via
HTTP and made available to the LLM.

- extractCardImageKeys now returns two slices: Feishu keys and external URLs
- Add downloadExternalImage to download images from HTTP URLs
- Update downloadInboundMedia to handle both Feishu API and HTTP downloads
- Update tests for new function signature

* fix(feishu): use HTTP client with timeout for external image downloads

Replaced http.DefaultClient with a client that has a 30-second timeout
to prevent hanging on unresponsive external URLs.

Generated with Crush

Assisted-by: GLM-5 via Crush <crush@charm.land>

* fix(feishu): resolve lint errors for shadow and formatting

- Rename err variables to avoid shadowing in downloadExternalImage
- Fix struct field alignment in TestExtractCardImageKeys

Generated with Crush

Assisted-by: GLM-5 via Crush <crush@charm.land>

* refactor(feishu): pass external image URLs to LLM instead of downloading

Instead of downloading external images from interactive cards, pass
the URLs directly to LLM. This reduces network overhead and lets
vision-capable models fetch images as needed.

- Remove downloadExternalImage function
- Append external URLs to card content for LLM processing
- Only download Feishu-hosted images via API

💘 Generated with Crush

Assisted-by: GLM-5 via Crush <crush@charm.land>

* fix(feishu): add blank line between functions for gci formatting

* fix(feishu): keep interactive card content as valid JSON
2026-03-20 12:59:43 +08:00

148 lines
4.5 KiB
Go

package feishu
import (
"encoding/json"
"regexp"
"strings"
larkim "github.com/larksuite/oapi-sdk-go/v3/service/im/v1"
)
// mentionPlaceholderRegex matches @_user_N placeholders inserted by Feishu for mentions.
var mentionPlaceholderRegex = regexp.MustCompile(`@_user_\d+`)
// stringValue safely dereferences a *string pointer.
func stringValue(v *string) string {
if v == nil {
return ""
}
return *v
}
// buildMarkdownCard builds a Feishu Interactive Card JSON 2.0 string with markdown content.
// JSON 2.0 cards support full CommonMark standard markdown syntax.
func buildMarkdownCard(content string) (string, error) {
card := map[string]any{
"schema": "2.0",
"body": map[string]any{
"elements": []map[string]any{
{
"tag": "markdown",
"content": content,
},
},
},
}
data, err := json.Marshal(card)
if err != nil {
return "", err
}
return string(data), nil
}
// extractJSONStringField unmarshals content as JSON and returns the value of the given string field.
// Returns "" if the content is invalid JSON or the field is missing/empty.
func extractJSONStringField(content, field string) string {
var m map[string]json.RawMessage
if err := json.Unmarshal([]byte(content), &m); err != nil {
return ""
}
raw, ok := m[field]
if !ok {
return ""
}
var s string
if err := json.Unmarshal(raw, &s); err != nil {
return ""
}
return s
}
// extractImageKey extracts the image_key from a Feishu image message content JSON.
// Format: {"image_key": "img_xxx"}
func extractImageKey(content string) string { return extractJSONStringField(content, "image_key") }
// extractFileKey extracts the file_key from a Feishu file/audio message content JSON.
// Format: {"file_key": "file_xxx", "file_name": "...", ...}
func extractFileKey(content string) string { return extractJSONStringField(content, "file_key") }
// extractFileName extracts the file_name from a Feishu file message content JSON.
func extractFileName(content string) string { return extractJSONStringField(content, "file_name") }
// stripMentionPlaceholders removes @_user_N placeholders from the text content.
// These are inserted by Feishu when users @mention someone in a message.
func stripMentionPlaceholders(content string, mentions []*larkim.MentionEvent) string {
if len(mentions) == 0 {
return content
}
for _, m := range mentions {
if m.Key != nil && *m.Key != "" {
content = strings.ReplaceAll(content, *m.Key, "")
}
}
// Also clean up any remaining @_user_N patterns
content = mentionPlaceholderRegex.ReplaceAllString(content, "")
return strings.TrimSpace(content)
}
// extractCardImageKeys recursively extracts all image keys from a Feishu interactive card.
// Image keys are used to download images from Feishu API.
// Returns two slices: Feishu-hosted keys and external URLs.
func extractCardImageKeys(rawContent string) (feishuKeys []string, externalURLs []string) {
if rawContent == "" {
return nil, nil
}
var card map[string]any
if err := json.Unmarshal([]byte(rawContent), &card); err != nil {
return nil, nil
}
extractImageKeysRecursive(card, &feishuKeys, &externalURLs)
return feishuKeys, externalURLs
}
// isExternalURL returns true if the string is an external HTTP/HTTPS URL.
func isExternalURL(s string) bool {
return strings.HasPrefix(s, "http://") || strings.HasPrefix(s, "https://")
}
// extractImageKeysRecursive traverses card structure to find all image keys.
// Collects both Feishu-hosted keys and external URLs separately.
func extractImageKeysRecursive(v any, feishuKeys, externalURLs *[]string) {
switch val := v.(type) {
case map[string]any:
// Check if this is an img element
if tag, ok := val["tag"].(string); ok {
switch tag {
case "img":
// Try img_key first (always Feishu-hosted)
if imgKey, ok := val["img_key"].(string); ok && imgKey != "" {
*feishuKeys = append(*feishuKeys, imgKey)
}
// Check src - could be Feishu key or external URL
if src, ok := val["src"].(string); ok && src != "" {
if isExternalURL(src) {
*externalURLs = append(*externalURLs, src)
} else {
*feishuKeys = append(*feishuKeys, src)
}
}
case "icon":
// Icon elements use icon_key
if iconKey, ok := val["icon_key"].(string); ok && iconKey != "" {
*feishuKeys = append(*feishuKeys, iconKey)
}
}
}
// Recurse into all nested structures
for _, child := range val {
extractImageKeysRecursive(child, feishuKeys, externalURLs)
}
case []any:
for _, item := range val {
extractImageKeysRecursive(item, feishuKeys, externalURLs)
}
}
}