Commit Graph

15 Commits

Author SHA1 Message Date
BBQ 21999b49f4 feat(container): add explicit data workflows and snapshot rollback (#193)
* feat(container): add explicit data workflows and snapshot rollback

Make container upgrades and recreation data-safe by adding explicit preserve, export, import, restore, and rollback flows across the backend, SDK, and web UI.

* fix(container): resolve go lint issues

Fix formatting and lint violations introduced by the container data workflow changes so the Go CI lint job passes cleanly.
2026-03-06 17:57:48 +08:00
BBQ 3feb03aca7 ci: add go lint and race test workflow (#187) 2026-03-05 11:25:33 +08:00
BBQ 9ceabf68c4 feat(mcp): replace bind-mount+exec with in-container gRPC service (#179)
Replace the host bind-mount + containerd exec approach with a per-bot
in-container gRPC server (ContainerService, port 9090). All file I/O,
exec, and MCP stdio sessions now go through gRPC instead of running
shell commands or reading host-mounted directories.

Architecture changes:
- cmd/mcp: rewritten as a gRPC server (ContainerService) with full
  file and exec API (ReadFile, WriteFile, ListDir, ReadRaw, WriteRaw,
  Exec, Stat, Mkdir, Rename, DeleteFile)
- internal/mcp/mcpcontainer: protobuf definitions and generated stubs
- internal/mcp/mcpclient: gRPC client wrapper with connection pool
  (Pool) and Provider interface for dependency injection
- mcp.Manager: add per-bot IP cache, gRPC connection pool, and
  SetContainerIP/MCPClient methods; remove DataDir/Exec helpers
- containerd.Service: remove ExecTask/ExecTaskStreaming; network setup
  now returns NetworkResult{IP} for pool routing
- internal/fs/service.go: deleted (replaced by mcpclient)
- handlers/fs.go: deleted; MCP stdio session logic moved to mcp_stdio.go
- container provider Executor: all tools (read/write/list/edit/exec)
  now call gRPC client instead of running shell via exec
- storefs, containerfs, media, skills, memory: all I/O ported to
  mcpclient.Provider

Database:
- migration 0022: drop host_path column from containers table

One-time data migration:
- migrateBindMountData: on first Start() after upgrade, copies old
  bind-mount data into the container via gRPC, then renames src dir
  to prevent re-migration; runs in background goroutine

Bug fixes:
- mcp_stdio: callRaw now returns full JSON-RPC envelope
  {"jsonrpc","id","result"|"error"} matching protocol spec;
  explicit "initialize" call now advances session init state to
  prevent duplicate handshake on next non-initialize call
- mcpclient Pool: properly evict stale gRPC connection after snapshot
  replace (container process recreated); use SetContainerIP instead
  of direct map write so IP changes always evict pool entry
- migrateBindMountData: walkErr on directories now counted as failure
  so partially-walked trees don't get incorrectly marked as migrated
- cmd/mcp/Dockerfile: removed dead file (docker/Dockerfile.mcp is the
  canonical production build)

Tests:
- provider_test.go: restored with bufconn in-process gRPC mock
  (fakeContainerService + staticProvider), 14 cases covering all 5
  tools plus edge cases
- mcp_session_test.go: new, covers JSON-RPC envelope, init state
  machine, pending cleanup on cancel/close, readLoop cancel
- storefs/service_test.go: restored (pure function roundtrip tests)
2026-03-04 21:50:08 +08:00
BBQ ee587b8ef5 fix(mcp): fix snapshot management and encapsulate locking (#169)
- Fix DeleteContainer FAILED_PRECONDITION by cleaning up stopped task
  entries before container deletion
- Fix CreateSnapshot leaving container in broken state: commit turns
  the active snapshot read-only, so the full cycle (stop → commit →
  prepare → delete → recreate → start) is now applied consistently
- Use context.WithoutCancel for atomic container replacement sequences
  to prevent cancelled HTTP requests from corrupting container state
- Use dctx for DB operations (recordSnapshotVersion/insertEvent) to
  avoid orphan snapshots in containerd without matching DB records
- Restart task + network after snapshot replacement, fixing Exec after
  CreateVersion where the container had no running task
- Extract replaceContainerSnapshot helper to deduplicate the prepare →
  delete → recreate → start pattern across three call sites
- Move snapshot list data fetching into Manager.ListBotSnapshotData to
  encapsulate per-container locking; remove exported LockBot method
- Use UnixNano for snapshot names to avoid second-precision collisions
2026-03-03 15:59:57 +08:00
Ran 5e12b5a53f fix: ensure unifying on hardcoded /data mount path 2026-02-24 03:35:27 +08:00
Ran 65c4d6f793 feat(container): support for apple container 2026-02-23 22:40:46 +08:00
BBQ 4fc0ca5110 fix(container): propagate host timezone to all containers
Replace TZ env var with /etc/localtime bind-mount in docker-compose
and inject timezone spec opts into containerd bot containers.
2026-02-20 03:22:31 +08:00
BBQ bc374fe8cd refactor: content-addressed assets, cross-channel multimodal, infra simplification (#63)
* refactor(attachment): multimodal attachment refactor with snapshot schema and storage layer

- Add snapshot schema migration (0008) and update init/versions/snapshots
- Add internal/attachment and internal/channel normalize for unified attachment handling
- Move containerfs provider from internal/media to internal/storage
- Update agent types, channel adapters (Telegram/Feishu), inbound and handlers
- Add containerd snapshot lineage and local_channel tests
- Regenerate sqlc, swagger and SDK

* refactor(media): content-addressed asset system with unified naming

- Replace asset_id foreign key with content_hash as sole identifier
  for bot_history_message_assets (pure soft-link model)
- Remove mime, size_bytes, storage_key from DB; derive at read time
  via media.Resolve from actual storage
- Merge migrations 0008/0009 into single 0008; keep 0001 as canonical schema
- Add Docker initdb script for deterministic migration execution order
- Fix cross-channel real-time image display (Telegram → WebUI SSE)
- Fix message disappearing on refresh (null assets fallback)
- Fix file icon instead of image preview (mime derivation from storage)
- Unify AssetID → ContentHash naming across Go, Agent, and Frontend
- Change storage key prefix from 4-char to 2-char for directory sharding
- Add server-entrypoint.sh for Docker deployment migration handling

* refactor(infra): embedded migrations, Docker simplification, and config consolidation

- Embed SQL migrations into Go binary, removing shell-based migration scripts
- Consolidate config files into conf/ directory (app.example.toml, app.docker.toml, app.dev.toml)
- Simplify Docker setup: remove initdb.d scripts, streamline nginx config and entrypoint
- Remove legacy CLI, feishu-echo commands, and obsolete incremental migration files
- Update install script and docs to require sudo for one-click install
- Add mise tasks for dev environment orchestration

* chore: recover migrations

---------

Co-authored-by: Acbox <acbox0328@gmail.com>
2026-02-19 00:20:27 +08:00
Ran 6acdd191c7 Squashed commit of the following:
commit bcdb026ae43e4f95d0b2c4f9bd440a2df9d6b514
Author: Ran <16112591+chen-ran@users.noreply.github.com>
Date:   Thu Feb 12 17:10:32 2026 +0800

    chore: update DEVELOPMENT.md

commit 30281742ef
Merge: ca5c6a1 5b05f13
Author: BBQ <bbq@BBQdeMacBook-Air.local>
Date:   Thu Feb 12 15:49:17 2026 +0800

    merge(github/main): integrate fx dependency injection framework

    Merge upstream fx refactor and adapt all services to use go.uber.org/fx
    for dependency injection. Resolve conflicts in main.go, server.go,
    and service constructors while preserving our domain model changes.

    - Fix telegram adapter panic on shutdown (double close channel)
    - Fix feishu adapter processing messages after stop
    - Increase directory lookup timeout from 2s to 5s

commit ca5c6a1866
Author: BBQ <bbq@BBQdeMacBook-Air.local>
Date:   Thu Feb 12 15:33:09 2026 +0800

    refactor(core): restructure conversation, channel and message domains

    - Rename chat module to conversation with flow-based architecture
    - Move channelidentities into channel/identities subpackage
    - Add channel/route for routing logic
    - Add message service with event hub
    - Add MCP providers: container, directory, schedule
    - Refactor Feishu/Telegram adapters with directory and stream support
    - Add platform management page and channel badges in web UI
    - Update database schema for conversations, messages and channel routes
    - Add @memoh/shared package for cross-package type definitions

commit 75e2ef0467
Merge: d99ba38 01cb6c8
Author: BBQ <bbq@BBQdeMacBook-Air.local>
Date:   Thu Feb 12 14:45:49 2026 +0800

    merge(github): merge github/main, resolve index.ts URL conflict

    Keep our defensive absolute-URL check in createAuthFetcher.

commit d99ba38b7d
Merge: 860e20f 35ce7d1
Author: BBQ <bbq@BBQdeMacBook-Air.local>
Date:   Thu Feb 12 05:20:18 2026 +0800

    merge(github): merge github/main, keep our code and docs/spec

commit 860e20fe70
Author: BBQ <bbq@BBQdeMacBook-Air.local>
Date:   Wed Feb 11 22:13:27 2026 +0800

    docs(docs): add concepts and style guides for VitePress site

    - Add concepts: identity-and-binding, index (en/zh)
    - Add style: terminology (en/zh)
    - Update index and zh/index
    - Update .vitepress/config.ts

commit a75fdb8040
Author: BBQ <bbq@BBQdeMacBook-Air.local>
Date:   Wed Feb 11 17:37:16 2026 +0800

    refactor(mcp): standardize unified tool gateway on go-sdk

    Split business executors from federation sources and migrate unified tool/federation transports to the official go-sdk for stricter MCP compliance and safer session lifecycle handling. Add targeted regression tests for accept compatibility, initialization retries, pending cleanup, and include updated swagger artifacts.

commit 02b33c8e85
Author: BBQ <bbq@BBQdeMacBook-Air.local>
Date:   Wed Feb 11 15:42:21 2026 +0800

    refactor(core): finalize user-centric identity and policy cleanup

    Unify auth and chat identity semantics around user_id, enforce personal-bot owner-only authorization, and remove legacy compatibility branches in integration tests.

commit 06e8619a37
Author: BBQ <bbq@BBQdeMacBook-Air.local>
Date:   Wed Feb 11 14:47:03 2026 +0800

    refactor(core): migrate channel identity and binding across app

    Align channel identity and bind flow across backend and app-facing layers, including generated swagger artifacts and package lock updates while excluding docs content changes.
2026-02-12 17:13:03 +08:00
Ran 26dd8651b7 feat: go cni lifecycle manage 2026-02-08 21:39:34 +08:00
BBQ 5a35ef34ac feat: channel gateway implementation and multi-bot refactor
- Refactor channel manager with support for Sender/Receiver interfaces and hot-swappable adapters.
- Implement identity routing and pre-authentication logic for inbound messages.
- Update database schema to support bot pre-auth keys and extended channel session metadata.
- Add Telegram and Feishu channel configuration and adapter enhancements.
- Update Swagger documentation and internal handlers for channel management.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:41:54 +08:00
Ran cb36b68ee4 feat: Atomic update mcp image 2026-02-05 02:40:10 +08:00
BBQ 6aebbe9279 feat: refactor User/Bot architecture and implement multi-channel gateway
Major changes:
1. Core Architecture: Decoupled Bots from Users. Bots now have independent lifecycles, member management (bot_members), and dedicated configurations.
2. Channel Gateway:
   - Implemented a unified Channel Manager supporting Feishu, Telegram, and Local (Web/CLI) adapters.
   - Added message processing pipeline to normalize interactions across different platforms.
   - Introduced a Contact system for identity binding and guest access policies.
3. Database & Tooling:
   - Consolidated all migrations into 0001_init with updated schema for bots, channels, and contacts.
   - Optimized sqlc.yaml to automatically track the migrations directory.
4. Agent Enhancements:
   - Introduced ToolContext to provide Agents with platform-aware execution capabilities (e.g., messaging, contact lookups).
   - Added tool logging and fallback mechanisms for toolChoice execution.
5. UI & Docs: Updated frontend stores, UI components, and Swagger documentation to align with the new Bot-centric model.
2026-02-04 23:49:50 +08:00
Ran 0edaba4e74 fix: update go dependencies 2026-01-20 23:23:07 +07:00
Ran d40cc581d2 refactor: initial go service 2026-01-20 00:04:23 +07:00