* refactor: introduce DCP pipeline layer for unified context assembly
Introduce a Deterministic Context Pipeline (DCP) inspired by Cahciua,
providing event-driven context assembly for LLM conversations.
- Add `internal/pipeline/` package with Canonical Event types, Projection
(reduce), Rendering (XML RC), Pipeline manager, and EventStore persistence
- Change user message format from YAML front-matter to XML `<message>` tags
with self-contained attributes (sender, channel, conversation, type)
- Merge CLI/Web dual API into single `/local/` endpoint, remove CLI handler
- Add `bot_session_events` table for event persistence and cold-start replay
- Add `discuss` session type (reserved for future Cahciua-style mode)
- Wire pipeline into HandleInbound: adapt → persist → push on every message
- Lazy cold-start replay: load events from DB on first session access
* feat: implement discuss mode with reactive driver and probe gate
Add discuss session mode where the bot autonomously decides when to speak
in group chats via tool-gated output (send tool only, no direct text reply).
- Add discuss driver (per-session goroutine, RC watch, step loop via
agent.Generate, TR persistence, late-binding prompt with mention hints)
- Add system_discuss.md prompt template ("text = inner monologue, send = speak")
- Add context composition (MergeContext, ComposeContext, TrimContext) for
RC + assistant/tool message interleaving by timestamp
- Add probe gate: when discuss_probe_model_id is set, cheap model pre-filters
group messages; no tool calls = silence, tool calls = activate primary
- Add /new [chat|discuss] command: explicit mode selection, defaults to
discuss in groups, chat in DMs, chat-only for WebUI
- Add ResolveRunConfig on flow.Resolver for discuss driver to reuse
model/tools/system-prompt resolution without reimplementing
- Fix send tool for discuss mode: same-conversation sends now go through
SendDirect (channel adapter) instead of the local emitter shortcut
- Add target attribute to XML message format (reply_target for routing)
- Add discuss_probe_model_id to bots table settings
- Remove pipeline compaction (SetCompactCursor) — reuse existing compaction.Service
- Persist full SDK messages (including tool calls) in discuss mode
* refactor: unify DCP event layer, fix persistence and local channel
- Fix bot_session_events dedup index to include event_kind so that
message + edit events for the same external_message_id coexist.
- Change CreateSessionEvent from :one to :exec so ON CONFLICT DO NOTHING
does not produce spurious errors on duplicate delivery.
- Move ACL evaluation before event ingest; denied messages no longer
enter bot_session_events or the in-memory pipeline.
- Let chat mode consume RenderedContext from the DCP pipeline when
available, sharing the same event-driven context assembly as discuss.
- Collapse local WebSocket handler to route through HandleInbound
instead of directly calling StreamChatWS, eliminating the dual
business entry point.
- Extract buildBaseRunConfig shared builder so resolve() and
ResolveRunConfig() no longer duplicate model/credentials/skills setup.
- Add StoreRound to RunConfigResolver interface so discuss driver
persists assistant output with full metadata, usage, and memory
extraction (same quality as chat mode).
- Fix discuss driver context: use context.Background() instead of the
short-lived HTTP request context that was getting cancelled.
- Fix model ID passed to StoreRound: return database UUID from
ResolveRunConfig instead of SDK model name.
- Remove dead CLIAdapter/CLIType and update legacy web/cli references
in tests and comments.
* fix: stop idle discuss goroutines after 10min timeout
Discuss session goroutines were never cleaned up when a session became
inactive (e.g. after /new). Add a 10-minute idle timer that auto-exits
the goroutine and removes it from the sessions map when no new RC
arrives.
* refactor: pipeline details — event types, structured reply, display content
- Remove [User sent N attachments] placeholder text from buildInboundQuery;
attachment info is now expressed via pipeline <attachment> tags.
- Unify in-reply-to as structured ReplyRef (Sender/Preview fields) across
Telegram, Discord, Feishu, and Matrix adapters instead of prepending
[Reply to ...] text into the message body. Remove now-unused
buildTelegramQuotedText, buildDiscordQuotedText, buildMatrixQuotedText.
- Make AdaptInbound return CanonicalEvent interface and dispatch to
adaptMessage/adaptEdit/adaptService based on metadata["event_type"].
- Add event_id column to bot_history_messages (migration 0059) so user
messages can reference their canonical pipeline event.
- PersistEvent now returns the event UUID; HandleInbound passes it through
to both persistPassiveMessage and ChatRequest.EventID for storeRound.
- Add FillDisplayContent to message service: extracts plain text from
event_data for clean frontend display.
- Frontend extractMessageText prefers display_content when available,
falling back to legacy strip logic for old messages.
- Fix: always generate headerifiedQuery for storage even when usePipeline
is true, so user messages are persisted via storeRound in chat mode.
* fix: use json.Marshal for pipeline context content serialization
The manual string escaping in buildMessagesFromPipeline only handled
double quotes but not newlines, backslashes, and other JSON special
characters, producing invalid json.RawMessage values. The LLM then
received empty/malformed context and complained about having no history.
* fix: restore WebSocket handler to use StreamChatWS directly
The previous refactoring replaced the WS handler with HandleInbound +
RouteHub subscription, which broke streaming because RouteHub events
use a different format (channel.StreamEvent) than what the frontend
expects (flow.WSStreamEvent with text_delta, tool_call_start, etc.).
Restore the original direct StreamChatWS call path so WebUI streaming
works again. The WS handler now matches the pre-refactoring behavior
while all other changes (pipeline, ACL, event types, etc.) are kept.
* feat: store display_text directly in bot_history_messages
Instead of computing display content at API response time by querying
bot_session_events via event_id, store the raw user text in a dedicated
display_text column at write time. This works for all paths including
the WebSocket handler which does not go through the pipeline/event layer.
- Migration 0060: add display_text TEXT column
- PersistInput gains DisplayText; filled from trimmedText (passive) and
req.Query (storeRound)
- toMessageFields reads display_text into DisplayContent
- Remove FillDisplayContent runtime query and ListSessionEventsByEventID
- Frontend already prefers display_content when available (no change)
* fix: display_text should contain raw user text, not XML-wrapped query
req.Query gets overwritten to headerifiedQuery (with XML <message> tags)
before storeRound runs. Add RawQuery field to ChatRequest to preserve
the original user text, and use it for display_text in storeMessages.
* fix(web): show discuss sessions
* refactor: introduce DCP pipeline layer for unified context assembly
Introduce a Deterministic Context Pipeline (DCP) inspired by Cahciua,
providing event-driven context assembly for LLM conversations.
- Add `internal/pipeline/` package with Canonical Event types, Projection
(reduce), Rendering (XML RC), Pipeline manager, and EventStore persistence
- Change user message format from YAML front-matter to XML `<message>` tags
with self-contained attributes (sender, channel, conversation, type)
- Merge CLI/Web dual API into single `/local/` endpoint, remove CLI handler
- Add `bot_session_events` table for event persistence and cold-start replay
- Add `discuss` session type (reserved for future Cahciua-style mode)
- Wire pipeline into HandleInbound: adapt → persist → push on every message
- Lazy cold-start replay: load events from DB on first session access
* feat: implement discuss mode with reactive driver and probe gate
Add discuss session mode where the bot autonomously decides when to speak
in group chats via tool-gated output (send tool only, no direct text reply).
- Add discuss driver (per-session goroutine, RC watch, step loop via
agent.Generate, TR persistence, late-binding prompt with mention hints)
- Add system_discuss.md prompt template ("text = inner monologue, send = speak")
- Add context composition (MergeContext, ComposeContext, TrimContext) for
RC + assistant/tool message interleaving by timestamp
- Add probe gate: when discuss_probe_model_id is set, cheap model pre-filters
group messages; no tool calls = silence, tool calls = activate primary
- Add /new [chat|discuss] command: explicit mode selection, defaults to
discuss in groups, chat in DMs, chat-only for WebUI
- Add ResolveRunConfig on flow.Resolver for discuss driver to reuse
model/tools/system-prompt resolution without reimplementing
- Fix send tool for discuss mode: same-conversation sends now go through
SendDirect (channel adapter) instead of the local emitter shortcut
- Add target attribute to XML message format (reply_target for routing)
- Add discuss_probe_model_id to bots table settings
- Remove pipeline compaction (SetCompactCursor) — reuse existing compaction.Service
- Persist full SDK messages (including tool calls) in discuss mode
* refactor: unify DCP event layer, fix persistence and local channel
- Fix bot_session_events dedup index to include event_kind so that
message + edit events for the same external_message_id coexist.
- Change CreateSessionEvent from :one to :exec so ON CONFLICT DO NOTHING
does not produce spurious errors on duplicate delivery.
- Move ACL evaluation before event ingest; denied messages no longer
enter bot_session_events or the in-memory pipeline.
- Let chat mode consume RenderedContext from the DCP pipeline when
available, sharing the same event-driven context assembly as discuss.
- Collapse local WebSocket handler to route through HandleInbound
instead of directly calling StreamChatWS, eliminating the dual
business entry point.
- Extract buildBaseRunConfig shared builder so resolve() and
ResolveRunConfig() no longer duplicate model/credentials/skills setup.
- Add StoreRound to RunConfigResolver interface so discuss driver
persists assistant output with full metadata, usage, and memory
extraction (same quality as chat mode).
- Fix discuss driver context: use context.Background() instead of the
short-lived HTTP request context that was getting cancelled.
- Fix model ID passed to StoreRound: return database UUID from
ResolveRunConfig instead of SDK model name.
- Remove dead CLIAdapter/CLIType and update legacy web/cli references
in tests and comments.
* fix: stop idle discuss goroutines after 10min timeout
Discuss session goroutines were never cleaned up when a session became
inactive (e.g. after /new). Add a 10-minute idle timer that auto-exits
the goroutine and removes it from the sessions map when no new RC
arrives.
* refactor: pipeline details — event types, structured reply, display content
- Remove [User sent N attachments] placeholder text from buildInboundQuery;
attachment info is now expressed via pipeline <attachment> tags.
- Unify in-reply-to as structured ReplyRef (Sender/Preview fields) across
Telegram, Discord, Feishu, and Matrix adapters instead of prepending
[Reply to ...] text into the message body. Remove now-unused
buildTelegramQuotedText, buildDiscordQuotedText, buildMatrixQuotedText.
- Make AdaptInbound return CanonicalEvent interface and dispatch to
adaptMessage/adaptEdit/adaptService based on metadata["event_type"].
- Add event_id column to bot_history_messages (migration 0059) so user
messages can reference their canonical pipeline event.
- PersistEvent now returns the event UUID; HandleInbound passes it through
to both persistPassiveMessage and ChatRequest.EventID for storeRound.
- Add FillDisplayContent to message service: extracts plain text from
event_data for clean frontend display.
- Frontend extractMessageText prefers display_content when available,
falling back to legacy strip logic for old messages.
- Fix: always generate headerifiedQuery for storage even when usePipeline
is true, so user messages are persisted via storeRound in chat mode.
* fix: use json.Marshal for pipeline context content serialization
The manual string escaping in buildMessagesFromPipeline only handled
double quotes but not newlines, backslashes, and other JSON special
characters, producing invalid json.RawMessage values. The LLM then
received empty/malformed context and complained about having no history.
* fix: restore WebSocket handler to use StreamChatWS directly
The previous refactoring replaced the WS handler with HandleInbound +
RouteHub subscription, which broke streaming because RouteHub events
use a different format (channel.StreamEvent) than what the frontend
expects (flow.WSStreamEvent with text_delta, tool_call_start, etc.).
Restore the original direct StreamChatWS call path so WebUI streaming
works again. The WS handler now matches the pre-refactoring behavior
while all other changes (pipeline, ACL, event types, etc.) are kept.
* feat: store display_text directly in bot_history_messages
Instead of computing display content at API response time by querying
bot_session_events via event_id, store the raw user text in a dedicated
display_text column at write time. This works for all paths including
the WebSocket handler which does not go through the pipeline/event layer.
- Migration 0060: add display_text TEXT column
- PersistInput gains DisplayText; filled from trimmedText (passive) and
req.Query (storeRound)
- toMessageFields reads display_text into DisplayContent
- Remove FillDisplayContent runtime query and ListSessionEventsByEventID
- Frontend already prefers display_content when available (no change)
* fix: display_text should contain raw user text, not XML-wrapped query
req.Query gets overwritten to headerifiedQuery (with XML <message> tags)
before storeRound runs. Add RawQuery field to ChatRequest to preserve
the original user text, and use it for display_text in storeMessages.
* fix(web): show discuss sessions
* chore(feishu): change discuss output to stream card
* fix(channel): unify discuss/chat send path and card markdown delivery
* feat(discuss): switch to stream execution with RouteHub broadcasting
* refactor(pipeline): remove context trimming from ComposeContext
The pipeline path should not trim context by token budget — the
upstream IC/RC already bounds the event window. Remove TrimContext,
FindWorkingWindowCursor, EstimateTokens, FormatLastProcessedMs (all
unused or only used for trimming), the maxTokens parameter from
ComposeContext, and MaxContextTokens from DiscussSessionConfig.
---------
Co-authored-by: 晨苒 <16112591+chen-ran@users.noreply.github.com>
* feat(channel): add Matrix adapter support
* fix(channel): prevent reasoning leaks in Matrix replies
* fix(channel): persist Matrix sync cursors
* fix(channel): improve Matrix markdown rendering
* fix(channel): support Matrix attachments and multimodal history
* fix(channel): expand Matrix reply media context
* fix(handlers): allow media downloads for chat-access bots
* fix(channel): classify Matrix DMs as direct chats
* fix(channel): auto-join Matrix room invites
* fix(channel): resolve Matrix room aliases for outbound send
* fix(web): use Matrix brand icon in channel badges
Replace the generic Matrix hashtag badge with the official brand asset so channel badges feel recognizable and fit the circular mask cleanly.
* fix(channel): add Matrix room whitelist controls
Let Matrix bots decide whether to auto-join invites and restrict inbound activity to allowed rooms or aliases. Expose the new controls in the web settings UI with line-based whitelist input so access rules stay explicit.
* fix(channel): stabilize Matrix multimodal follow-ups and settings
* fix(flow): avoid gosec panic on byte decoding
* fix: fix golangci-lint
* fix(channel): remove Matrix built-in ACL
* fix(channel): preserve Matrix image captions
* fix(channel): validate Matrix homeserver and sync access
Fail Matrix connections early when the homeserver, access token, or /sync capability is misconfigured so bot health checks surface actionable errors.
* fix(channel): preserve optional toggles and relax Matrix startup validation
* fix(channel): tighten Matrix mention fallback parsing
* fix(flow): skip structured assistant tool-call outputs
* fix(flow): resolve merged resolver duplication
Keep the internal agent resolver implementation after merging main so split helper files do not redeclare flow symbols. Restore user message normalization in sanitize and persistence paths to keep flow tests and command packages building.
* fix(flow): remove unused merged resolver helper
Drop the leftover truncate helper and import from the resolver merge fix so golangci-lint passes again without affecting flow behavior.
---------
Co-authored-by: Acbox Liu <acbox0328@gmail.com>
* fix(text): resolve emoji shown in telegram stream mode
* chore(text): removing "reasoing" types in plain msg.
* feat(conversation): add function to check for tool call content in assistant outputs
- In group chats, only process slash commands when the message is
directed at this bot (via @mention or reply-to-bot), preventing
all bots from responding to the same command.
- Use raw_text metadata (before quote/forward context prepending)
for command detection so quoted content like "/fs" doesn't
accidentally match a command.
- Fix isTelegramBotMentioned text_mention entity check to verify
the mentioned bot matches the current bot, not just any bot.
Wire SetCommandHandler into ChannelInboundProcessor so slash commands
are intercepted before reaching the LLM. Also apply lint fixes across
command package (strconv.Itoa, comment formatting, unused code removal)
and remove obsolete tool-call-browser.vue component.
Prepend replied-to message text and attachments into the user query so
the LLM can see what is being replied to, matching the existing Telegram
behavior. Also set is_reply_to_bot metadata for Feishu reply-to-bot
detection in group chats.
Migrate the imported WeCom adapter to current channel interfaces and stabilize stream delivery by preventing heartbeat/reply ACK timeout regressions and post-final overwrite updates.
- Extract ContainsMarkdown to shared channel package
- Auto-detect markdown in normalizeOutboundMessage and MCP send tool
- Apply markdown-to-HTML conversion during streaming deltas, not just
on the final message
- Remove resolveTelegramParseMode which incorrectly returned Telegram's
native "Markdown" mode instead of converting to HTML
- Fix all 14 Telegram send/edit paths for consistent parse mode handling
- Reset parseMode for plain-text error messages to avoid HTML corruption
Use rune-aware truncation for user-facing text and log previews so multibyte content is not corrupted in memory context, Telegram messages, or diagnostics.
- Extract parseTelegramTarget helper to consolidate duplicated @username
vs numeric chat ID parsing from 6+ locations (builder functions,
sendTelegramTextReturnMessage, sendTelegramAttachmentImpl)
- Extract Config.baseURL() to eliminate duplicate base URL resolution
between apiEndpoint() and fileEndpoint()
- Refactor stream.go Push method: extract resetStreamState(),
deliverFinalText(), and per-event-type sub-methods (pushDelta,
pushFinal, pushToolCallStart, pushAttachment, pushPhaseEnd,
pushError), reducing the 200-line switch-case to a clean dispatcher
- Use pushFinal's existing getBot() instead of duplicating parseConfig +
getOrCreateBot
- Replace sort.SliceStable with slices.SortStableFunc + cmp.Compare
- Replace strings.Index + manual slicing with strings.Cut in
decodeDataURLBytes, ResolveAttachment, and parseTelegramUserInput
* feat(channel): add qq adapter and outbound delivery
* feat(channel): ingest inbound qq messages
* feat(web): expose qq channel in management ui
* feat(channel): support qq attachment ingestion
* fix(mcp): fail read raw immediately for missing files
* fix(agent): parse inline image data into native image parts
* test(agent): align read_media tool tests with SDK options
* fix(channel): harden qq image delivery and reconnect loop
Avoid data URLs for qq channel images, reset reconnect backoff after healthy sessions, and fall back gracefully for malformed public image URLs.
* fix(channel): restore qq media delivery and target resolution
* fix(qq,mcp,agent): fix message/qq regressions and pass go lint
* fix(qq,agent): validate inline base64 and sync heartbeat seq
* fix(qq): validate remote voice mime for upload checks
* fix(qq): fall back intents and restore adapter wiring
* fix(qq): prevent final text leakage and dedupe persisted inbound query
Split long AI responses into multiple platform messages during streaming
instead of truncating them. The manager counts accumulated delta runes
and opens a new stream when approaching the platform's TextChunkLimit.
Uses a soft/hard limit strategy that prefers splitting at sentence ends
or line breaks over cutting mid-sentence.
- Add pushDelta with soft (75%) / hard (100%) limit and natural break
point detection
- Add splitStream, pushFinalAfterSplit, pushFinalWithChunking helpers
- Fix Discord adapter to use RuneCount Message Length
- Add tests for delta splitting, natural breaks, and final handling
* feat(telegram): use sendMessageDraft for streaming in private chats
Use Telegram Bot API 9.3's sendMessageDraft to stream partial messages
with smooth animation in private chats, replacing the sendMessage +
editMessageText approach. Group/channel chats keep the existing
edit-based streaming.
- Add sendTelegramDraft() for the sendMessageDraft API
- Detect private chats via conversation_type metadata in OpenStream
- Use 300ms throttle for drafts (vs 5s for edits)
- Send permanent messages at tool call boundaries and on final event
- Reset buffer atomically in StreamEventFinal to prevent duplicate
messages when multiple final events fire (one per assistant output)
* test(telegram): improve draft mode test assertions
Add sendTextForTest hook for sendTelegramTextReturnMessage to enable
direct assertion of send calls. Clean up residual unused variables
and replace indirect assertions with explicit mock-based verification.
Allow configuring a custom Telegram Bot API base URL (`apiBaseURL`) per
channel, enabling users behind restricted networks to route requests
through a reverse proxy (e.g. Nginx, Cloudflare Workers).
Both API calls and file downloads respect the configured endpoint.
When omitted, the official https://api.telegram.org is used.
Closes#159
Add ProcessingStatusNotifier implementation: show 👀 reaction while
processing, remove on completion/failure. Fix isReplyToBot to match
only the current bot (bot.Self.ID) instead of any bot, preventing
multiple bots from responding to the same reply. Add rune-safe text
truncation for Telegram message length limit.
Strip invalid UTF-8 byte sequences in sendTelegramTextReturnMessage and
editTelegramMessageText to prevent "text must be encoded in UTF-8" errors
that abort the stream mid-response.
Increase edit throttle from 250ms to 1500ms to respect Telegram per-chat limits,
remove newline bypass that effectively disabled throttling, add dedicated
editStreamMessageFinal with retry for final message delivery, and simplify
editTelegramMessageText by removing blocking sleep in favor of caller-level retry.
- Treat 400 "message is not modified" as success to avoid user-facing error
- On 429: sleep retry_after and retry once in editTelegramMessageText;
stream backs off lastEditedAt and returns nil
- Require error code 400 for message-not-modified check; add production
error string to unit tests
- Lower base throttle to 250ms; add test hooks and tests for 429 retry
and stream backoff
Merge upstream fx refactor and adapt all services to use go.uber.org/fx
for dependency injection. Resolve conflicts in main.go, server.go,
and service constructors while preserving our domain model changes.
- Fix telegram adapter panic on shutdown (double close channel)
- Fix feishu adapter processing messages after stop
- Increase directory lookup timeout from 2s to 5s
Acbox Liu