35 Commits

Author SHA1 Message Date
Acbox Liu c8728ffc2c refactor(browser): split browser cores via build ARG, add core selector (#237)
* refactor(browser): split browser cores via build ARG, add core selector

- Replace playwright official image with ubuntu:noble base in both
  docker/Dockerfile.browser and devenv/Dockerfile.browser; install
  browsers at build time driven by ARG/ENV BROWSER_CORES
- Add GET /cores endpoint to Browser Gateway reporting available cores
- Proxy GET /browser-contexts/cores in Go handler to Browser Gateway
- Add `core` field to BrowserContextConfigModel and GatewayBrowserContext;
  context creation selects the appropriate browser instance by core
- Frontend context-setting page fetches available cores and renders a
  core selector; saves core as part of the config JSON
- install.sh prompts for browser core selection and writes BROWSER_CORES
  to .env; builds the browser image locally before docker compose up
- Regenerate OpenAPI spec and TypeScript SDK

* fix: lint
2026-03-14 12:37:20 +08:00
晨苒 627b673a5c refactor: multi-provider memory adapters with scan-based builtin (#227)
* refactor: restructure memory into multi-provider adapters, remove manifest.json dependency

- Rename internal/memory/provider to internal/memory/adapters with per-provider subdirectories (builtin, mem0, openviking)
- Replace manifest.json-based delete/update with scan-based index from daily files
- Add mem0 and openviking provider adapters with HTTP client, chat hooks, MCP tools, and CRUD
- Wire provider lifecycle into registry (auto-instantiate on create, evict on update/delete)
- Split docker-compose into base stack + optional overlays (qdrant, browser, mem0, openviking)
- Update admin UI to support dynamic provider config schema rendering

* chore(lint): fix all golangci-lint issues for clean CI

* refactor(docker): replace compose overlay files with profiles

* feat(memory): add built-in memory multi modes

* fix(ci): golangci lint

* feat(memory): edit built-in memory sparse design
2026-03-14 06:04:13 +08:00
Acbox Liu 23d49a1c7b feat: message abort and web socket support (#222)
* feat: message abort and web socket support

* fix(web): chat end

* fix: lint

* fix: lint
2026-03-09 23:27:50 +08:00
Menci 8ce5243eb3 fix(mcp): use dumb-init as PID 1 to reap zombie processes
MCP containers spawning child processes (npx, uvx, etc.) left zombies
because the Go binary running as PID 1 does not reap orphaned children.
Add dumb-init as the entrypoint init process.
2026-03-09 12:41:23 +08:00
BBQ abbb14c59f fix(containerd): prevent silent network failures from leaving containers unreachable (#202)
* fix(containerd): prevent silent network failures from leaving containers unreachable

Container network setup failures were silently swallowed at multiple
points in the call chain, leaving containers in a "running but
unreachable" ghost state. This patch closes every silent-failure path:

- setupCNINetwork: return error when CNI yields no usable IP
- Manager.Start: roll back container when IP is empty instead of
  returning success
- ensureContainerAndTask: extract setupNetworkOrFail with 1 retry,
  propagate error to callers
- ReconcileContainers: stop reporting "healthy" when network setup fails
- recoverContainerIP: retry up to 2 times with backoff for transient
  CNI failures (IPAM lock contention, etc.)
- gRPC Pool: evict connections stuck in Connecting state for >30s

* fix(containerd): clean stale cni0 bridge on startup to prevent MAC error

After a Docker container restart, the cni0 bridge interface can linger
with a zeroed MAC (00:00:00:00:00:00) and DOWN state. The CNI bridge
plugin then fails with "could not set bridge's mac: invalid argument",
making all MCP containers unreachable.

Two-layer fix:
- Entrypoint: delete cni0 and flush IPAM state before starting containerd
- Go: detect bridge MAC errors in setupCNINetwork and auto-delete cni0
  before retrying, as defense-in-depth for runtime recovery

* fix(containerd): use exec.CommandContext to satisfy noctx linter
2026-03-07 17:50:01 +08:00
Acbox 635d2847be fix(docker): run browser gateway with bun 2026-03-07 15:49:27 +08:00
Acbox 32f331011d fix(docker): web 2026-03-07 15:23:34 +08:00
Acbox Liu bafd327b6b feat: agent browser (#200)
* feat: agent browser

* chore: complete docker and action config

* feat: more actions

* feat: browser tab switch

* fix: browser build

* fix: lint

* fix: migrations
2026-03-07 15:06:00 +08:00
Acbox 47a425baf5 chore: move /agent to /apps/agent 2026-03-06 17:32:12 +08:00
BBQ a8b960db80 fix(agent): replace jsdom with linkedom to fix Docker EISDIR error (#186)
jsdom relies on Node.js-specific APIs that Bun cannot properly resolve
when running a bundled artifact. This caused an EISDIR error in Docker
containers (Bun tried to read the jsdom directory as a file).

Replace jsdom with linkedom, a lightweight pure-JS DOM implementation
that is fully compatible with Bun and @mozilla/readability. Also remove
the --external jsdom build flag since linkedom bundles cleanly.

Closes #181
2026-03-04 22:11:08 +08:00
BBQ 9ceabf68c4 feat(mcp): replace bind-mount+exec with in-container gRPC service (#179)
Replace the host bind-mount + containerd exec approach with a per-bot
in-container gRPC server (ContainerService, port 9090). All file I/O,
exec, and MCP stdio sessions now go through gRPC instead of running
shell commands or reading host-mounted directories.

Architecture changes:
- cmd/mcp: rewritten as a gRPC server (ContainerService) with full
  file and exec API (ReadFile, WriteFile, ListDir, ReadRaw, WriteRaw,
  Exec, Stat, Mkdir, Rename, DeleteFile)
- internal/mcp/mcpcontainer: protobuf definitions and generated stubs
- internal/mcp/mcpclient: gRPC client wrapper with connection pool
  (Pool) and Provider interface for dependency injection
- mcp.Manager: add per-bot IP cache, gRPC connection pool, and
  SetContainerIP/MCPClient methods; remove DataDir/Exec helpers
- containerd.Service: remove ExecTask/ExecTaskStreaming; network setup
  now returns NetworkResult{IP} for pool routing
- internal/fs/service.go: deleted (replaced by mcpclient)
- handlers/fs.go: deleted; MCP stdio session logic moved to mcp_stdio.go
- container provider Executor: all tools (read/write/list/edit/exec)
  now call gRPC client instead of running shell via exec
- storefs, containerfs, media, skills, memory: all I/O ported to
  mcpclient.Provider

Database:
- migration 0022: drop host_path column from containers table

One-time data migration:
- migrateBindMountData: on first Start() after upgrade, copies old
  bind-mount data into the container via gRPC, then renames src dir
  to prevent re-migration; runs in background goroutine

Bug fixes:
- mcp_stdio: callRaw now returns full JSON-RPC envelope
  {"jsonrpc","id","result"|"error"} matching protocol spec;
  explicit "initialize" call now advances session init state to
  prevent duplicate handshake on next non-initialize call
- mcpclient Pool: properly evict stale gRPC connection after snapshot
  replace (container process recreated); use SetContainerIP instead
  of direct map write so IP changes always evict pool entry
- migrateBindMountData: walkErr on directories now counted as failure
  so partially-walked trees don't get incorrectly marked as migrated
- cmd/mcp/Dockerfile: removed dead file (docker/Dockerfile.mcp is the
  canonical production build)

Tests:
- provider_test.go: restored with bufconn in-process gRPC mock
  (fakeContainerService + staticProvider), 14 cases covering all 5
  tools plus edge cases
- mcp_session_test.go: new, covers JSON-RPC envelope, init state
  machine, pending cleanup on cancel/close, readLoop cancel
- storefs/service_test.go: restored (pure function roundtrip tests)
2026-03-04 21:50:08 +08:00
BBQ 7730096696 fix(containerd): restore CNI MASQUERADE after container restart (#167)
cni.Remove() failure on stale iptables state blocked the retry
cni.Setup(), leaving bot containers without SNAT/MASQUERADE.

- Ignore cni.Remove() error so retry Setup always runs
- Add global MASQUERADE rule in entrypoints as belt-and-suspenders

Closes #161
2026-03-03 16:00:46 +08:00
BBQ 04bce702b7 feat(devenv): MCP dev hot-reload with image-based approach (#145)
Add mcp-build.sh that compiles the MCP binary and packages it as an
OCI image layer on top of the base rootfs, imported directly into
containerd. Air triggers rebuild on code changes, cleaning stale
containers automatically.

Consolidate dev-only files (Dockerfiles, entrypoint, config, build
script) into devenv/ to separate dev tooling from production artifacts.
Skip image pull when already imported to speed up dev startup.
2026-03-02 14:59:48 +08:00
BBQ d6aebf654f feat(devenv): add containerized development environment (#116)
* feat(devenv): add containerized development environment

Replace local-process dev workflow with a fully containerized stack
using docker compose. This enables consistent development across
machines without requiring local Go/Node toolchains or containerd.

- Add Dockerfile.server.dev with containerd + CNI networking support
- Add Dockerfile.web.dev for frontend dev server
- Add server-dev-entrypoint.sh for containerd lifecycle management
- Expand devenv/docker-compose.yml with server, agent, web, migrate
  and deps services with proper health checks and dependency ordering
- Update app.dev.toml to use container service names instead of localhost
- Refactor mise.toml dev tasks to drive docker compose workflow
- Support agent_gateway.server_addr in config package for inter-container
  communication

* feat(devenv): add hot-reload and registry mirror support

- Add air for Go server hot-reload in dev containers
- Fix agent_gateway host in dev config (0.0.0.0 -> agent)
- Add configurable registry mirror for China mainland users
- Unify MCP image refs via MCPConfig.ImageRef()

* feat(scripts): add China mainland mirror option to install script

Prompt users to opt-in to memoh.cn mirror during installation,
which applies docker-compose.cn.yml overlay and sets registry
in config.toml for MCP image pulls.
2026-02-26 17:32:19 +08:00
Ran a9680587c6 fix: change docker/docker-compose.yml to override 2026-02-25 16:32:52 +08:00
Ran 2b75d955e3 fix(script): update install.sh 2026-02-25 16:23:28 +08:00
斬風千雪 a5b11dddc2 fix(mcp): adapt tool list checker to underscore name format (#113)
* fix(mcp): adapt tool list checker to underscore name format

---------

Co-authored-by: Ran <16112591+chen-ran@users.noreply.github.com>
2026-02-24 17:00:41 +08:00
Ran 0239d395e2 chore: update docker compose image 2026-02-24 00:08:36 +08:00
Ringo.Typowriter 08daa83178 fix(agent): resolve workspace dependency @memoh/agent in Docker build (#94) 2026-02-22 13:32:31 +08:00
BBQ bc374fe8cd refactor: content-addressed assets, cross-channel multimodal, infra simplification (#63)
* refactor(attachment): multimodal attachment refactor with snapshot schema and storage layer

- Add snapshot schema migration (0008) and update init/versions/snapshots
- Add internal/attachment and internal/channel normalize for unified attachment handling
- Move containerfs provider from internal/media to internal/storage
- Update agent types, channel adapters (Telegram/Feishu), inbound and handlers
- Add containerd snapshot lineage and local_channel tests
- Regenerate sqlc, swagger and SDK

* refactor(media): content-addressed asset system with unified naming

- Replace asset_id foreign key with content_hash as sole identifier
  for bot_history_message_assets (pure soft-link model)
- Remove mime, size_bytes, storage_key from DB; derive at read time
  via media.Resolve from actual storage
- Merge migrations 0008/0009 into single 0008; keep 0001 as canonical schema
- Add Docker initdb script for deterministic migration execution order
- Fix cross-channel real-time image display (Telegram → WebUI SSE)
- Fix message disappearing on refresh (null assets fallback)
- Fix file icon instead of image preview (mime derivation from storage)
- Unify AssetID → ContentHash naming across Go, Agent, and Frontend
- Change storage key prefix from 4-char to 2-char for directory sharding
- Add server-entrypoint.sh for Docker deployment migration handling

* refactor(infra): embedded migrations, Docker simplification, and config consolidation

- Embed SQL migrations into Go binary, removing shell-based migration scripts
- Consolidate config files into conf/ directory (app.example.toml, app.docker.toml, app.dev.toml)
- Simplify Docker setup: remove initdb.d scripts, streamline nginx config and entrypoint
- Remove legacy CLI, feishu-echo commands, and obsolete incremental migration files
- Update install script and docs to require sudo for one-click install
- Add mise tasks for dev environment orchestration

* chore: recover migrations

---------

Co-authored-by: Acbox <acbox0328@gmail.com>
2026-02-19 00:20:27 +08:00
BBQ df7876a30c feat: add media asset system, channel lifecycle refactor, and chat attachments (#54) 2026-02-17 19:06:46 +08:00
Acbox 961ecbb96d feat: search provider 2026-02-16 01:22:15 +08:00
Ran 0fffed1d1e fix(web): vite url config error
- golang server support for arm64 arg
2026-02-15 03:45:09 +08:00
Ran e6580d7837 fix(deploy): docker health check failure 2026-02-14 22:44:23 +08:00
Ran 5c46d41ebf fix(deploy): vite config lazy load 2026-02-14 22:16:55 +08:00
Acbox f964cbca69 fix (docker): missing web keys in docker config 2026-02-14 20:45:33 +08:00
Acbox 1c85e0106f fix(docker): Dockerfile.web 2026-02-14 20:43:08 +08:00
Acbox 94d1cea488 fix(docker): Dockerfile.agent 2026-02-14 20:39:49 +08:00
Acbox 82e9f12b7f refactor(web): change old port (7003) and deploy port (80) to port 8082 2026-02-14 19:51:55 +08:00
BBQ f9be6baa4e fix(containerd): add pid:host for CNI netns access and runtime deps to MCP image
- Add pid: host to containerd service so server can access MCP container
  network namespaces via /proc/PID/ns/net for CNI setup
- Add Node.js, npm, Python 3, uv to embedded MCP image rootfs so users
  can run npx/uvx MCP servers inside containers
2026-02-13 01:55:11 +08:00
BBQ 944461f031 feat(mcp): add Node.js and Python runtime to MCP container image
- Install nodejs + npm (provides npx for JS/TS MCP servers)
- Install python3 + uv (provides uvx for Python MCP servers)
- Add bash and curl as base utilities
2026-02-13 00:41:00 +08:00
BBQ c53d35740e feat(deploy): self-contained containerd with embedded MCP image
- Add Dockerfile.containerd: multi-stage build that compiles MCP binary,
  assembles rootfs, creates Docker image tar, and bundles it with containerd
- Add containerd-entrypoint.sh: auto-imports MCP image on first start
- Fix MCP image reference: rename busybox_image to image in config,
  use fully-qualified docker.io/library/memoh-mcp:latest everywhere
- Make image ref configurable via config.toml instead of hardcoded
- Simplify deploy.sh: remove manual nerdctl/containerd-install steps
2026-02-12 23:50:48 +08:00
BBQ 1c15eb2146 refactor(core): restructure conversation/channel/message domains and modernize deployment
- Replace chat package with conversation flow architecture
- Add channel identity avatar support (migration 0002)
- Refactor channel adapters, identities, and message routing
- Update frontend: simplify composables, modernize UI components
- Improve Docker builds with cache mounts and version metadata
- Optimize healthchecks and simplify service dependencies
2026-02-12 20:55:03 +08:00
Ran 01cb6c85db fix(deploy): many docker compose bug 2026-02-12 08:23:25 +08:00
zenhouke 057e95cb9a refactor(deploy): consolidate configuration and reorganize docker files 2026-02-12 02:16:41 +08:00