mirror of
https://github.com/memohai/Memoh.git
synced 2026-04-27 07:16:19 +09:00
48da4e026e6cc34e2467ebecb851433fa80ce6b7
14 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Acbox Liu
|
43c4153938 |
feat: introduce DCP pipeline layer for unified context assembly (#329)
* refactor: introduce DCP pipeline layer for unified context assembly
Introduce a Deterministic Context Pipeline (DCP) inspired by Cahciua,
providing event-driven context assembly for LLM conversations.
- Add `internal/pipeline/` package with Canonical Event types, Projection
(reduce), Rendering (XML RC), Pipeline manager, and EventStore persistence
- Change user message format from YAML front-matter to XML `<message>` tags
with self-contained attributes (sender, channel, conversation, type)
- Merge CLI/Web dual API into single `/local/` endpoint, remove CLI handler
- Add `bot_session_events` table for event persistence and cold-start replay
- Add `discuss` session type (reserved for future Cahciua-style mode)
- Wire pipeline into HandleInbound: adapt → persist → push on every message
- Lazy cold-start replay: load events from DB on first session access
* feat: implement discuss mode with reactive driver and probe gate
Add discuss session mode where the bot autonomously decides when to speak
in group chats via tool-gated output (send tool only, no direct text reply).
- Add discuss driver (per-session goroutine, RC watch, step loop via
agent.Generate, TR persistence, late-binding prompt with mention hints)
- Add system_discuss.md prompt template ("text = inner monologue, send = speak")
- Add context composition (MergeContext, ComposeContext, TrimContext) for
RC + assistant/tool message interleaving by timestamp
- Add probe gate: when discuss_probe_model_id is set, cheap model pre-filters
group messages; no tool calls = silence, tool calls = activate primary
- Add /new [chat|discuss] command: explicit mode selection, defaults to
discuss in groups, chat in DMs, chat-only for WebUI
- Add ResolveRunConfig on flow.Resolver for discuss driver to reuse
model/tools/system-prompt resolution without reimplementing
- Fix send tool for discuss mode: same-conversation sends now go through
SendDirect (channel adapter) instead of the local emitter shortcut
- Add target attribute to XML message format (reply_target for routing)
- Add discuss_probe_model_id to bots table settings
- Remove pipeline compaction (SetCompactCursor) — reuse existing compaction.Service
- Persist full SDK messages (including tool calls) in discuss mode
* refactor: unify DCP event layer, fix persistence and local channel
- Fix bot_session_events dedup index to include event_kind so that
message + edit events for the same external_message_id coexist.
- Change CreateSessionEvent from :one to :exec so ON CONFLICT DO NOTHING
does not produce spurious errors on duplicate delivery.
- Move ACL evaluation before event ingest; denied messages no longer
enter bot_session_events or the in-memory pipeline.
- Let chat mode consume RenderedContext from the DCP pipeline when
available, sharing the same event-driven context assembly as discuss.
- Collapse local WebSocket handler to route through HandleInbound
instead of directly calling StreamChatWS, eliminating the dual
business entry point.
- Extract buildBaseRunConfig shared builder so resolve() and
ResolveRunConfig() no longer duplicate model/credentials/skills setup.
- Add StoreRound to RunConfigResolver interface so discuss driver
persists assistant output with full metadata, usage, and memory
extraction (same quality as chat mode).
- Fix discuss driver context: use context.Background() instead of the
short-lived HTTP request context that was getting cancelled.
- Fix model ID passed to StoreRound: return database UUID from
ResolveRunConfig instead of SDK model name.
- Remove dead CLIAdapter/CLIType and update legacy web/cli references
in tests and comments.
* fix: stop idle discuss goroutines after 10min timeout
Discuss session goroutines were never cleaned up when a session became
inactive (e.g. after /new). Add a 10-minute idle timer that auto-exits
the goroutine and removes it from the sessions map when no new RC
arrives.
* refactor: pipeline details — event types, structured reply, display content
- Remove [User sent N attachments] placeholder text from buildInboundQuery;
attachment info is now expressed via pipeline <attachment> tags.
- Unify in-reply-to as structured ReplyRef (Sender/Preview fields) across
Telegram, Discord, Feishu, and Matrix adapters instead of prepending
[Reply to ...] text into the message body. Remove now-unused
buildTelegramQuotedText, buildDiscordQuotedText, buildMatrixQuotedText.
- Make AdaptInbound return CanonicalEvent interface and dispatch to
adaptMessage/adaptEdit/adaptService based on metadata["event_type"].
- Add event_id column to bot_history_messages (migration 0059) so user
messages can reference their canonical pipeline event.
- PersistEvent now returns the event UUID; HandleInbound passes it through
to both persistPassiveMessage and ChatRequest.EventID for storeRound.
- Add FillDisplayContent to message service: extracts plain text from
event_data for clean frontend display.
- Frontend extractMessageText prefers display_content when available,
falling back to legacy strip logic for old messages.
- Fix: always generate headerifiedQuery for storage even when usePipeline
is true, so user messages are persisted via storeRound in chat mode.
* fix: use json.Marshal for pipeline context content serialization
The manual string escaping in buildMessagesFromPipeline only handled
double quotes but not newlines, backslashes, and other JSON special
characters, producing invalid json.RawMessage values. The LLM then
received empty/malformed context and complained about having no history.
* fix: restore WebSocket handler to use StreamChatWS directly
The previous refactoring replaced the WS handler with HandleInbound +
RouteHub subscription, which broke streaming because RouteHub events
use a different format (channel.StreamEvent) than what the frontend
expects (flow.WSStreamEvent with text_delta, tool_call_start, etc.).
Restore the original direct StreamChatWS call path so WebUI streaming
works again. The WS handler now matches the pre-refactoring behavior
while all other changes (pipeline, ACL, event types, etc.) are kept.
* feat: store display_text directly in bot_history_messages
Instead of computing display content at API response time by querying
bot_session_events via event_id, store the raw user text in a dedicated
display_text column at write time. This works for all paths including
the WebSocket handler which does not go through the pipeline/event layer.
- Migration 0060: add display_text TEXT column
- PersistInput gains DisplayText; filled from trimmedText (passive) and
req.Query (storeRound)
- toMessageFields reads display_text into DisplayContent
- Remove FillDisplayContent runtime query and ListSessionEventsByEventID
- Frontend already prefers display_content when available (no change)
* fix: display_text should contain raw user text, not XML-wrapped query
req.Query gets overwritten to headerifiedQuery (with XML <message> tags)
before storeRound runs. Add RawQuery field to ChatRequest to preserve
the original user text, and use it for display_text in storeMessages.
* fix(web): show discuss sessions
* refactor: introduce DCP pipeline layer for unified context assembly
Introduce a Deterministic Context Pipeline (DCP) inspired by Cahciua,
providing event-driven context assembly for LLM conversations.
- Add `internal/pipeline/` package with Canonical Event types, Projection
(reduce), Rendering (XML RC), Pipeline manager, and EventStore persistence
- Change user message format from YAML front-matter to XML `<message>` tags
with self-contained attributes (sender, channel, conversation, type)
- Merge CLI/Web dual API into single `/local/` endpoint, remove CLI handler
- Add `bot_session_events` table for event persistence and cold-start replay
- Add `discuss` session type (reserved for future Cahciua-style mode)
- Wire pipeline into HandleInbound: adapt → persist → push on every message
- Lazy cold-start replay: load events from DB on first session access
* feat: implement discuss mode with reactive driver and probe gate
Add discuss session mode where the bot autonomously decides when to speak
in group chats via tool-gated output (send tool only, no direct text reply).
- Add discuss driver (per-session goroutine, RC watch, step loop via
agent.Generate, TR persistence, late-binding prompt with mention hints)
- Add system_discuss.md prompt template ("text = inner monologue, send = speak")
- Add context composition (MergeContext, ComposeContext, TrimContext) for
RC + assistant/tool message interleaving by timestamp
- Add probe gate: when discuss_probe_model_id is set, cheap model pre-filters
group messages; no tool calls = silence, tool calls = activate primary
- Add /new [chat|discuss] command: explicit mode selection, defaults to
discuss in groups, chat in DMs, chat-only for WebUI
- Add ResolveRunConfig on flow.Resolver for discuss driver to reuse
model/tools/system-prompt resolution without reimplementing
- Fix send tool for discuss mode: same-conversation sends now go through
SendDirect (channel adapter) instead of the local emitter shortcut
- Add target attribute to XML message format (reply_target for routing)
- Add discuss_probe_model_id to bots table settings
- Remove pipeline compaction (SetCompactCursor) — reuse existing compaction.Service
- Persist full SDK messages (including tool calls) in discuss mode
* refactor: unify DCP event layer, fix persistence and local channel
- Fix bot_session_events dedup index to include event_kind so that
message + edit events for the same external_message_id coexist.
- Change CreateSessionEvent from :one to :exec so ON CONFLICT DO NOTHING
does not produce spurious errors on duplicate delivery.
- Move ACL evaluation before event ingest; denied messages no longer
enter bot_session_events or the in-memory pipeline.
- Let chat mode consume RenderedContext from the DCP pipeline when
available, sharing the same event-driven context assembly as discuss.
- Collapse local WebSocket handler to route through HandleInbound
instead of directly calling StreamChatWS, eliminating the dual
business entry point.
- Extract buildBaseRunConfig shared builder so resolve() and
ResolveRunConfig() no longer duplicate model/credentials/skills setup.
- Add StoreRound to RunConfigResolver interface so discuss driver
persists assistant output with full metadata, usage, and memory
extraction (same quality as chat mode).
- Fix discuss driver context: use context.Background() instead of the
short-lived HTTP request context that was getting cancelled.
- Fix model ID passed to StoreRound: return database UUID from
ResolveRunConfig instead of SDK model name.
- Remove dead CLIAdapter/CLIType and update legacy web/cli references
in tests and comments.
* fix: stop idle discuss goroutines after 10min timeout
Discuss session goroutines were never cleaned up when a session became
inactive (e.g. after /new). Add a 10-minute idle timer that auto-exits
the goroutine and removes it from the sessions map when no new RC
arrives.
* refactor: pipeline details — event types, structured reply, display content
- Remove [User sent N attachments] placeholder text from buildInboundQuery;
attachment info is now expressed via pipeline <attachment> tags.
- Unify in-reply-to as structured ReplyRef (Sender/Preview fields) across
Telegram, Discord, Feishu, and Matrix adapters instead of prepending
[Reply to ...] text into the message body. Remove now-unused
buildTelegramQuotedText, buildDiscordQuotedText, buildMatrixQuotedText.
- Make AdaptInbound return CanonicalEvent interface and dispatch to
adaptMessage/adaptEdit/adaptService based on metadata["event_type"].
- Add event_id column to bot_history_messages (migration 0059) so user
messages can reference their canonical pipeline event.
- PersistEvent now returns the event UUID; HandleInbound passes it through
to both persistPassiveMessage and ChatRequest.EventID for storeRound.
- Add FillDisplayContent to message service: extracts plain text from
event_data for clean frontend display.
- Frontend extractMessageText prefers display_content when available,
falling back to legacy strip logic for old messages.
- Fix: always generate headerifiedQuery for storage even when usePipeline
is true, so user messages are persisted via storeRound in chat mode.
* fix: use json.Marshal for pipeline context content serialization
The manual string escaping in buildMessagesFromPipeline only handled
double quotes but not newlines, backslashes, and other JSON special
characters, producing invalid json.RawMessage values. The LLM then
received empty/malformed context and complained about having no history.
* fix: restore WebSocket handler to use StreamChatWS directly
The previous refactoring replaced the WS handler with HandleInbound +
RouteHub subscription, which broke streaming because RouteHub events
use a different format (channel.StreamEvent) than what the frontend
expects (flow.WSStreamEvent with text_delta, tool_call_start, etc.).
Restore the original direct StreamChatWS call path so WebUI streaming
works again. The WS handler now matches the pre-refactoring behavior
while all other changes (pipeline, ACL, event types, etc.) are kept.
* feat: store display_text directly in bot_history_messages
Instead of computing display content at API response time by querying
bot_session_events via event_id, store the raw user text in a dedicated
display_text column at write time. This works for all paths including
the WebSocket handler which does not go through the pipeline/event layer.
- Migration 0060: add display_text TEXT column
- PersistInput gains DisplayText; filled from trimmedText (passive) and
req.Query (storeRound)
- toMessageFields reads display_text into DisplayContent
- Remove FillDisplayContent runtime query and ListSessionEventsByEventID
- Frontend already prefers display_content when available (no change)
* fix: display_text should contain raw user text, not XML-wrapped query
req.Query gets overwritten to headerifiedQuery (with XML <message> tags)
before storeRound runs. Add RawQuery field to ChatRequest to preserve
the original user text, and use it for display_text in storeMessages.
* fix(web): show discuss sessions
* chore(feishu): change discuss output to stream card
* fix(channel): unify discuss/chat send path and card markdown delivery
* feat(discuss): switch to stream execution with RouteHub broadcasting
* refactor(pipeline): remove context trimming from ComposeContext
The pipeline path should not trim context by token budget — the
upstream IC/RC already bounds the event window. Remove TrimContext,
FindWorkingWindowCursor, EstimateTokens, FormatLastProcessedMs (all
unused or only used for trimming), the maxTokens parameter from
ComposeContext, and MaxContextTokens from DiscussSessionConfig.
---------
Co-authored-by: 晨苒 <16112591+chen-ran@users.noreply.github.com>
|
||
|
AlexMa233
|
609ca49cf5 |
feat: matrix support (part 1) (#242)
* feat(channel): add Matrix adapter support * fix(channel): prevent reasoning leaks in Matrix replies * fix(channel): persist Matrix sync cursors * fix(channel): improve Matrix markdown rendering * fix(channel): support Matrix attachments and multimodal history * fix(channel): expand Matrix reply media context * fix(handlers): allow media downloads for chat-access bots * fix(channel): classify Matrix DMs as direct chats * fix(channel): auto-join Matrix room invites * fix(channel): resolve Matrix room aliases for outbound send * fix(web): use Matrix brand icon in channel badges Replace the generic Matrix hashtag badge with the official brand asset so channel badges feel recognizable and fit the circular mask cleanly. * fix(channel): add Matrix room whitelist controls Let Matrix bots decide whether to auto-join invites and restrict inbound activity to allowed rooms or aliases. Expose the new controls in the web settings UI with line-based whitelist input so access rules stay explicit. * fix(channel): stabilize Matrix multimodal follow-ups and settings * fix(flow): avoid gosec panic on byte decoding * fix: fix golangci-lint * fix(channel): remove Matrix built-in ACL * fix(channel): preserve Matrix image captions * fix(channel): validate Matrix homeserver and sync access Fail Matrix connections early when the homeserver, access token, or /sync capability is misconfigured so bot health checks surface actionable errors. * fix(channel): preserve optional toggles and relax Matrix startup validation * fix(channel): tighten Matrix mention fallback parsing * fix(flow): skip structured assistant tool-call outputs * fix(flow): resolve merged resolver duplication Keep the internal agent resolver implementation after merging main so split helper files do not redeclare flow symbols. Restore user message normalization in sanitize and persistence paths to keep flow tests and command packages building. * fix(flow): remove unused merged resolver helper Drop the leftover truncate helper and import from the resolver merge fix so golangci-lint passes again without affecting flow behavior. --------- Co-authored-by: Acbox Liu <acbox0328@gmail.com> |
||
|
BBQ
|
839e63acda | feat(access): add guest chat ACL (#235) | ||
|
Fodesu
|
b46e494d3a |
feat(tts): introduce TTS system (#195)
|
||
|
Acbox
|
1da251885d |
feat(agent): add extensible tag interception system and inline reactions
Refactor the attachment tag extraction into a generic TagResolver/StreamTagExtractor system that supports multiple custom tags. Implement <reactions> tag allowing the agent to embed emoji reactions directly in text responses, dispatched as side-effects through the channel reactor interface. - Add TagResolver interface and StreamTagExtractor streaming state machine - Refactor AttachmentsStreamExtractor as backward-compatible wrapper - Add reactionsResolver and ReactionDeltaAction stream event - Wire reaction dispatch in Go channel inbound processor - Fix .gitignore to scope compiled binary patterns to repo root |
||
|
BBQ
|
bc374fe8cd |
refactor: content-addressed assets, cross-channel multimodal, infra simplification (#63)
* refactor(attachment): multimodal attachment refactor with snapshot schema and storage layer - Add snapshot schema migration (0008) and update init/versions/snapshots - Add internal/attachment and internal/channel normalize for unified attachment handling - Move containerfs provider from internal/media to internal/storage - Update agent types, channel adapters (Telegram/Feishu), inbound and handlers - Add containerd snapshot lineage and local_channel tests - Regenerate sqlc, swagger and SDK * refactor(media): content-addressed asset system with unified naming - Replace asset_id foreign key with content_hash as sole identifier for bot_history_message_assets (pure soft-link model) - Remove mime, size_bytes, storage_key from DB; derive at read time via media.Resolve from actual storage - Merge migrations 0008/0009 into single 0008; keep 0001 as canonical schema - Add Docker initdb script for deterministic migration execution order - Fix cross-channel real-time image display (Telegram → WebUI SSE) - Fix message disappearing on refresh (null assets fallback) - Fix file icon instead of image preview (mime derivation from storage) - Unify AssetID → ContentHash naming across Go, Agent, and Frontend - Change storage key prefix from 4-char to 2-char for directory sharding - Add server-entrypoint.sh for Docker deployment migration handling * refactor(infra): embedded migrations, Docker simplification, and config consolidation - Embed SQL migrations into Go binary, removing shell-based migration scripts - Consolidate config files into conf/ directory (app.example.toml, app.docker.toml, app.dev.toml) - Simplify Docker setup: remove initdb.d scripts, streamline nginx config and entrypoint - Remove legacy CLI, feishu-echo commands, and obsolete incremental migration files - Update install script and docs to require sudo for one-click install - Add mise tasks for dev environment orchestration * chore: recover migrations --------- Co-authored-by: Acbox <acbox0328@gmail.com> |
||
|
BBQ
|
df7876a30c | feat: add media asset system, channel lifecycle refactor, and chat attachments (#54) | ||
|
Acbox
|
38753ef054 | refactor: channel tools | ||
|
Ran
|
7817ec8147 |
fix(web): channel switch failure
Also add webui memory page |
||
|
Ran
|
6acdd191c7 |
Squashed commit of the following:
commit bcdb026ae43e4f95d0b2c4f9bd440a2df9d6b514 Author: Ran <16112591+chen-ran@users.noreply.github.com> Date: Thu Feb 12 17:10:32 2026 +0800 chore: update DEVELOPMENT.md commit |
||
|
BBQ
|
29e6ddd1f9 |
refactor: replace global channel registry with instance-based Registry and interface-driven adapters
- Replace global channelRegistry singleton with explicit *Registry passed via dependency injection - Split monolithic manager.go into connection.go (lifecycle), inbound.go (dispatch), outbound.go (pipeline) - Introduce optional adapter interfaces: ConfigNormalizer, TargetResolver, BindingMatcher - Move Descriptor() to Adapter interface, remove init()-based registration - Relocate SessionHub to adapters/local package - Extract shared UUID/time helpers to internal/db/uuid.go - Decompose ConfigStore into fine-grained interfaces: ConfigLister, ConfigResolver, BindingStore, SessionStore |
||
|
BBQ
|
a246b79a4f |
refactor: restructure channel gateway and chat module architecture
- Refactor channel adapters (feishu, telegram, local) with enhanced descriptor and config - Restructure channel manager, service, types, and outbound messaging - Simplify chat module by removing normalize.go and chat.go, consolidating into resolver and types - Update router channel handlers and tests - Sync swagger documentation |
||
|
BBQ
|
5a35ef34ac |
feat: channel gateway implementation and multi-bot refactor
- Refactor channel manager with support for Sender/Receiver interfaces and hot-swappable adapters. - Implement identity routing and pre-authentication logic for inbound messages. - Update database schema to support bot pre-auth keys and extended channel session metadata. - Add Telegram and Feishu channel configuration and adapter enhancements. - Update Swagger documentation and internal handlers for channel management. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
BBQ
|
6aebbe9279 |
feat: refactor User/Bot architecture and implement multi-channel gateway
Major changes: 1. Core Architecture: Decoupled Bots from Users. Bots now have independent lifecycles, member management (bot_members), and dedicated configurations. 2. Channel Gateway: - Implemented a unified Channel Manager supporting Feishu, Telegram, and Local (Web/CLI) adapters. - Added message processing pipeline to normalize interactions across different platforms. - Introduced a Contact system for identity binding and guest access policies. 3. Database & Tooling: - Consolidated all migrations into 0001_init with updated schema for bots, channels, and contacts. - Optimized sqlc.yaml to automatically track the migrations directory. 4. Agent Enhancements: - Introduced ToolContext to provide Agents with platform-aware execution capabilities (e.g., messaging, contact lookups). - Added tool logging and fallback mechanisms for toolChoice execution. 5. UI & Docs: Updated frontend stores, UI components, and Swagger documentation to align with the new Bot-centric model. |