mirror of
https://github.com/memohai/Memoh.git
synced 2026-04-27 07:16:19 +09:00
1680316c7f
* refactor(agent): replace TypeScript agent gateway with in-process Go agent using twilight-ai SDK
- Remove apps/agent (Bun/Elysia gateway), packages/agent (@memoh/agent),
internal/bun runtime manager, and all embedded agent/bun assets
- Add internal/agent package powered by twilight-ai SDK for LLM calls,
tool execution, streaming, sential logic, tag extraction, and prompts
- Integrate ToolGatewayService in-process for both built-in and user MCP
tools, eliminating HTTP round-trips to the old gateway
- Update resolver to convert between sdk.Message and ModelMessage at the
boundary (resolver_messages.go), keeping agent package free of
persistence concerns
- Prepend user message before storeRound since SDK only returns output
messages (assistant + tool)
- Clean up all Docker configs, TOML configs, nginx proxy, Dockerfile.agent,
and Go config structs related to the removed agent gateway
- Update cmd/agent and cmd/memoh entry points with setter-based
ToolGateway injection to avoid FX dependency cycles
* fix(web): move form declaration before computed properties that reference it
The `form` reactive object was declared after computed properties like
`selectedMemoryProvider` and `isSelectedMemoryProviderPersisted` that
reference it, causing a TDZ ReferenceError during setup.
* fix: prevent UTF-8 character corruption in streaming text output
StreamTagExtractor.Push() used byte-level string slicing to hold back
buffer tails for tag detection, which could split multi-byte UTF-8
characters. After json.Marshal replaced invalid bytes with U+FFFD,
the corruption became permanent — causing garbled CJK characters (�)
in agent responses.
Add safeUTF8SplitIndex() to back up split points to valid character
boundaries. Also fix byte-level truncation in command/formatter.go
and command/fs.go to use rune-aware slicing.
* fix: add agent error logging and fix Gemini tool schema validation
- Log agent stream errors in both SSE and WebSocket paths with bot/model context
- Fix send tool `attachments` parameter: empty `items` schema rejected by
Google Gemini API (INVALID_ARGUMENT), now specifies `{"type": "string"}`
- Upgrade twilight-ai to d898f0b (includes raw body in API error messages)
* chore(ci): remove agent gateway from Docker build and release pipelines
Agent gateway has been replaced by in-process Go agent; remove the
obsolete Docker image matrix entry, Bun/UPX CI steps, and agent-binary
build logic from the release script.
* fix: preserve attachment filename, metadata, and container path through persistence
- Add `name` column to `bot_history_message_assets` (migration 0034) to
persist original filenames across page refreshes.
- Add `metadata` JSONB column (migration 0035) to store source_path,
source_url, and other context alongside each asset.
- Update SQL queries, sqlc-generated code, and all Go types (MessageAsset,
AssetRef, OutboundAssetRef, FileAttachment) to carry name and metadata
through the full lifecycle.
- Extract filenames from path/URL in AttachmentsResolver before clearing
raw paths; enrich streaming event metadata with name, source_path, and
source_url in both the WebSocket and channel inbound ingestion paths.
- Implement `LinkAssets` on message service and `LinkOutboundAssets` on
flow resolver so WebSocket-streamed bot attachments are persisted to the
correct assistant message after streaming completes.
- Frontend: update MessageAsset type with metadata field, pass metadata
through to attachment items, and reorder attachment-block.vue template
so container files (identified by metadata.source_path) open in the
sidebar file manager instead of triggering a download.
* refactor(agent): decouple built-in tools from MCP, load via ToolProvider interface
Migrate all 13 built-in tool providers from internal/mcp/providers/ to
internal/agent/tools/ using the twilight-ai sdk.Tool structure. The agent
now loads tools through a ToolProvider interface instead of the MCP
ToolGatewayService, which is simplified to only manage external federation
sources. This enables selective tool loading and removes the coupling
between business tools and the MCP protocol layer.
* refactor(flow): split monolithic resolver.go into focused modules
Break the 1959-line resolver.go into 12 files organized by concern:
- resolver.go: core orchestration (Resolver struct, resolve, Chat, prepareRunConfig)
- resolver_stream.go: streaming (StreamChat, StreamChatWS, tryStoreStream)
- resolver_trigger.go: schedule/heartbeat triggers
- resolver_attachments.go: attachment routing, inlining, encoding
- resolver_history.go: message loading, deduplication, token trimming
- resolver_store.go: persistence (storeRound, storeMessages, asset linking)
- resolver_memory.go: memory provider integration
- resolver_model_selection.go: model selection and candidate matching
- resolver_identity.go: display name and channel identity resolution
- resolver_settings.go: bot settings, loop detection, inbox
- user_header.go: YAML front-matter formatting
- resolver_util.go: shared utilities (sanitize, normalize, dedup, UUID)
* fix(agent): enable Anthropic extended thinking by passing ReasoningConfig to provider
Anthropic's thinking requires WithThinking() at provider creation time,
unlike OpenAI which uses per-request ReasoningEffort. The config was
never wired through, so Claude models could not trigger thinking.
* refactor(agent): extract prompts into embedded markdown templates
Move inline prompt strings from prompt.go into separate .md files under
internal/agent/prompts/, using {{key}} placeholders and a simple render
engine. Remove obsolete SystemPromptParams fields (Language,
MaxContextLoadTime, Channels, CurrentChannel) and their call-site usage.
* fix: lint
285 lines
9.3 KiB
Go
285 lines
9.3 KiB
Go
package tools
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"log/slog"
|
|
"math"
|
|
"strings"
|
|
|
|
sdk "github.com/memohai/twilight-ai/sdk"
|
|
|
|
"github.com/memohai/memoh/internal/workspace/bridge"
|
|
)
|
|
|
|
const defaultContainerExecWorkDir = "/data"
|
|
|
|
type ContainerProvider struct {
|
|
clients bridge.Provider
|
|
execWorkDir string
|
|
logger *slog.Logger
|
|
}
|
|
|
|
func NewContainerProvider(log *slog.Logger, clients bridge.Provider, execWorkDir string) *ContainerProvider {
|
|
if log == nil {
|
|
log = slog.Default()
|
|
}
|
|
wd := strings.TrimSpace(execWorkDir)
|
|
if wd == "" {
|
|
wd = defaultContainerExecWorkDir
|
|
}
|
|
return &ContainerProvider{clients: clients, execWorkDir: wd, logger: log.With(slog.String("tool", "container"))}
|
|
}
|
|
|
|
func (p *ContainerProvider) Tools(_ context.Context, session SessionContext) ([]sdk.Tool, error) {
|
|
wd := p.execWorkDir
|
|
sess := session
|
|
return []sdk.Tool{
|
|
{
|
|
Name: "read",
|
|
Description: fmt.Sprintf("Read file content inside the bot container. Supports pagination for large files. Max %d lines / %d bytes per call.", readMaxLines, readMaxBytes),
|
|
Parameters: map[string]any{
|
|
"type": "object",
|
|
"properties": map[string]any{
|
|
"path": map[string]any{"type": "string", "description": fmt.Sprintf("File path (relative to %s or absolute inside container)", wd)},
|
|
"line_offset": map[string]any{"type": "integer", "description": "Line number to start reading from (1-indexed). Default: 1.", "minimum": 1, "default": 1},
|
|
"n_lines": map[string]any{"type": "integer", "description": fmt.Sprintf("Number of lines to read per call. Default: %d. Max: %d.", readMaxLines, readMaxLines), "minimum": 1, "maximum": readMaxLines, "default": readMaxLines},
|
|
},
|
|
"required": []string{"path"},
|
|
},
|
|
Execute: func(ctx *sdk.ToolExecContext, input any) (any, error) {
|
|
return p.execRead(ctx.Context, sess, inputAsMap(input))
|
|
},
|
|
},
|
|
{
|
|
Name: "write",
|
|
Description: "Write file content inside the bot container.",
|
|
Parameters: map[string]any{
|
|
"type": "object",
|
|
"properties": map[string]any{
|
|
"path": map[string]any{"type": "string", "description": fmt.Sprintf("File path (relative to %s or absolute inside container)", wd)},
|
|
"content": map[string]any{"type": "string", "description": "File content"},
|
|
},
|
|
"required": []string{"path", "content"},
|
|
},
|
|
Execute: func(ctx *sdk.ToolExecContext, input any) (any, error) {
|
|
return p.execWrite(ctx.Context, sess, inputAsMap(input))
|
|
},
|
|
},
|
|
{
|
|
Name: "list",
|
|
Description: "List directory entries inside the bot container.",
|
|
Parameters: map[string]any{
|
|
"type": "object",
|
|
"properties": map[string]any{
|
|
"path": map[string]any{"type": "string", "description": fmt.Sprintf("Directory path (relative to %s or absolute inside container)", wd)},
|
|
"recursive": map[string]any{"type": "boolean", "description": "List recursively"},
|
|
},
|
|
"required": []string{"path"},
|
|
},
|
|
Execute: func(ctx *sdk.ToolExecContext, input any) (any, error) {
|
|
return p.execList(ctx.Context, sess, inputAsMap(input))
|
|
},
|
|
},
|
|
{
|
|
Name: "edit",
|
|
Description: "Replace exact text in a file inside the bot container.",
|
|
Parameters: map[string]any{
|
|
"type": "object",
|
|
"properties": map[string]any{
|
|
"path": map[string]any{"type": "string", "description": fmt.Sprintf("File path (relative to %s or absolute inside container)", wd)},
|
|
"old_text": map[string]any{"type": "string", "description": "Exact text to find"},
|
|
"new_text": map[string]any{"type": "string", "description": "Replacement text"},
|
|
},
|
|
"required": []string{"path", "old_text", "new_text"},
|
|
},
|
|
Execute: func(ctx *sdk.ToolExecContext, input any) (any, error) {
|
|
return p.execEdit(ctx.Context, sess, inputAsMap(input))
|
|
},
|
|
},
|
|
{
|
|
Name: "exec",
|
|
Description: fmt.Sprintf("Execute a command in the bot container. Runs in the bot's data directory (%s) by default.", wd),
|
|
Parameters: map[string]any{
|
|
"type": "object",
|
|
"properties": map[string]any{
|
|
"command": map[string]any{"type": "string", "description": "Shell command to run (e.g. ls -la, cat file.txt)"},
|
|
"work_dir": map[string]any{"type": "string", "description": fmt.Sprintf("Working directory inside the container (default: %s)", wd)},
|
|
},
|
|
"required": []string{"command"},
|
|
},
|
|
Execute: func(ctx *sdk.ToolExecContext, input any) (any, error) {
|
|
return p.execExec(ctx.Context, sess, inputAsMap(input))
|
|
},
|
|
},
|
|
}, nil
|
|
}
|
|
|
|
func (p *ContainerProvider) normalizePath(path string) string {
|
|
path = strings.TrimSpace(path)
|
|
if path == "" {
|
|
return path
|
|
}
|
|
prefix := p.execWorkDir
|
|
if prefix == "" {
|
|
prefix = defaultContainerExecWorkDir
|
|
}
|
|
if path == prefix {
|
|
return "."
|
|
}
|
|
if strings.HasPrefix(path, prefix+"/") {
|
|
return strings.TrimLeft(strings.TrimPrefix(path, prefix+"/"), "/")
|
|
}
|
|
return path
|
|
}
|
|
|
|
func (p *ContainerProvider) getClient(ctx context.Context, botID string) (*bridge.Client, error) {
|
|
botID = strings.TrimSpace(botID)
|
|
if botID == "" {
|
|
return nil, errors.New("bot_id is required")
|
|
}
|
|
client, err := p.clients.MCPClient(ctx, botID)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("container not reachable: %w", err)
|
|
}
|
|
return client, nil
|
|
}
|
|
|
|
func (p *ContainerProvider) execRead(ctx context.Context, session SessionContext, args map[string]any) (any, error) {
|
|
client, err := p.getClient(ctx, session.BotID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
filePath := p.normalizePath(StringArg(args, "path"))
|
|
if filePath == "" {
|
|
return nil, errors.New("path is required")
|
|
}
|
|
lineOffset := int32(1)
|
|
if offset, ok, err := IntArg(args, "line_offset"); err != nil {
|
|
return nil, fmt.Errorf("invalid line_offset: %w", err)
|
|
} else if ok {
|
|
if offset < 1 {
|
|
return nil, errors.New("line_offset must be >= 1")
|
|
}
|
|
if offset > math.MaxInt32 {
|
|
return nil, errors.New("line_offset exceeds maximum")
|
|
}
|
|
lineOffset = int32(offset)
|
|
}
|
|
nLines := int32(readMaxLines)
|
|
if n, ok, err := IntArg(args, "n_lines"); err != nil {
|
|
return nil, fmt.Errorf("invalid n_lines: %w", err)
|
|
} else if ok {
|
|
if n < 1 {
|
|
return nil, errors.New("n_lines must be >= 1")
|
|
}
|
|
if n > readMaxLines {
|
|
n = readMaxLines
|
|
}
|
|
nLines = int32(n) //nolint:gosec // bounded by readMaxLines
|
|
}
|
|
resp, err := client.ReadFile(ctx, filePath, lineOffset, nLines)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if resp.GetBinary() {
|
|
return nil, errors.New("file appears to be binary. Read tool only supports text files")
|
|
}
|
|
return map[string]any{"content": resp.GetContent(), "total_lines": resp.GetTotalLines()}, nil
|
|
}
|
|
|
|
func (p *ContainerProvider) execWrite(ctx context.Context, session SessionContext, args map[string]any) (any, error) {
|
|
client, err := p.getClient(ctx, session.BotID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
filePath := p.normalizePath(StringArg(args, "path"))
|
|
content := StringArg(args, "content")
|
|
if filePath == "" {
|
|
return nil, errors.New("path is required")
|
|
}
|
|
if err := client.WriteFile(ctx, filePath, []byte(content)); err != nil {
|
|
return nil, err
|
|
}
|
|
return map[string]any{"ok": true}, nil
|
|
}
|
|
|
|
func (p *ContainerProvider) execList(ctx context.Context, session SessionContext, args map[string]any) (any, error) {
|
|
client, err := p.getClient(ctx, session.BotID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
dirPath := p.normalizePath(StringArg(args, "path"))
|
|
if dirPath == "" {
|
|
dirPath = "."
|
|
}
|
|
recursive, _, _ := BoolArg(args, "recursive")
|
|
entries, err := client.ListDir(ctx, dirPath, recursive)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
entriesMaps := make([]map[string]any, len(entries))
|
|
for i, e := range entries {
|
|
entriesMaps[i] = map[string]any{
|
|
"path": e.GetPath(), "is_dir": e.GetIsDir(), "size": e.GetSize(),
|
|
"mode": e.GetMode(), "mod_time": e.GetModTime(),
|
|
}
|
|
}
|
|
return map[string]any{"path": dirPath, "entries": entriesMaps}, nil
|
|
}
|
|
|
|
func (p *ContainerProvider) execEdit(ctx context.Context, session SessionContext, args map[string]any) (any, error) {
|
|
client, err := p.getClient(ctx, session.BotID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
filePath := p.normalizePath(StringArg(args, "path"))
|
|
oldText := StringArg(args, "old_text")
|
|
newText := StringArg(args, "new_text")
|
|
if filePath == "" || oldText == "" {
|
|
return nil, errors.New("path, old_text and new_text are required")
|
|
}
|
|
reader, err := client.ReadRaw(ctx, filePath)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
defer func() { _ = reader.Close() }()
|
|
raw, err := io.ReadAll(reader)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
updated, err := applyEdit(string(raw), filePath, oldText, newText)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if err := client.WriteFile(ctx, filePath, []byte(updated)); err != nil {
|
|
return nil, err
|
|
}
|
|
return map[string]any{"ok": true}, nil
|
|
}
|
|
|
|
func (p *ContainerProvider) execExec(ctx context.Context, session SessionContext, args map[string]any) (any, error) {
|
|
botID := strings.TrimSpace(session.BotID)
|
|
client, err := p.getClient(ctx, botID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
command := strings.TrimSpace(StringArg(args, "command"))
|
|
if command == "" {
|
|
return nil, errors.New("command is required")
|
|
}
|
|
workDir := strings.TrimSpace(StringArg(args, "work_dir"))
|
|
if workDir == "" {
|
|
workDir = p.execWorkDir
|
|
}
|
|
result, err := client.Exec(ctx, command, workDir, 30)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
stdout := pruneToolOutputText(result.Stdout, "tool result (exec stdout)")
|
|
stderr := pruneToolOutputText(result.Stderr, "tool result (exec stderr)")
|
|
return map[string]any{"stdout": stdout, "stderr": stderr, "exit_code": result.ExitCode}, nil
|
|
}
|