Files
Memoh/db/migrations/0061_unify_providers.down.sql
T
Acbox Liu 8d5c38f0e5 refactor: unify providers and models tables (#338)
* refactor: unify providers and models tables

- Rename `llm_providers` → `providers`, `llm_provider_oauth_tokens` → `provider_oauth_tokens`
- Remove `tts_providers` and `tts_models` tables; speech models now live in the unified `models` table with `type = 'speech'`
- Replace top-level `api_key`/`base_url` columns with a JSONB `config` field on `providers`
- Rename `llm_provider_id` → `provider_id` across all references
- Add `edge-speech` client type and `conf/providers/edge.yaml` default provider
- Create new read-only speech endpoints (`/speech-providers`, `/speech-models`) backed by filtered views of the unified tables
- Remove old TTS CRUD handlers; simplify speech page to read-only + test
- Update registry loader to skip malformed YAML files instead of failing entirely
- Fix YAML quoting for model names containing colons in openrouter.yaml
- Regenerate sqlc, swagger, and TypeScript SDK

* fix: exclude speech providers from providers list endpoint

ListProviders now filters out client_type matching '%-speech' so Edge
and future speech providers no longer appear on the Providers page.
ListSpeechProviders uses the same pattern match instead of hard-coding
'edge-speech'.

* fix: use explicit client_type list instead of LIKE pattern

Replace '%-speech' pattern with explicit IN ('edge-speech') for both
ListProviders (exclusion) and ListSpeechProviders (inclusion). New
speech client types must be added to both queries.

* fix: use EXECUTE for dynamic SQL in migrations referencing old schema

PL/pgSQL pre-validates column/table references in static SQL statements
inside DO blocks before evaluating IF/RETURN guards. This caused
migrations 0010-0061 to fail on fresh databases where the canonical
schema uses `providers`/`provider_id` instead of `llm_providers`/
`llm_provider_id`.

Wrap all SQL that references potentially non-existent old schema objects
(llm_providers, llm_provider_id, tts_providers, tts_models, etc.) in
EXECUTE strings so they are only parsed at runtime when actually reached.

* fix: revert canonical schema to use llm_providers for migration compatibility

The CI migrations workflow (up → down → up) failed because 0061 down
renames `providers` back to `llm_providers`, but 0001 down only dropped
`providers` — leaving `llm_providers` as a remnant. On the second
migrate up, 0010 found the stale `llm_providers` and tried to reference
`models.llm_provider_id` which no longer existed.

Revert 0001 canonical schema to use original names (llm_providers,
tts_providers, tts_models) so incremental migrations work naturally and
0061 handles the final rename. Remove EXECUTE wrappers and unnecessary
guards from migrations that now always operate on llm_providers.

* fix: icons

* fix: sync canonical schema with 0061 migration to fix sqlc column mismatch

0001_init.up.sql still used old names (llm_providers, llm_provider_id)
and included dropped tts_providers/tts_models tables. sqlc could not
parse the PL/pgSQL EXECUTE in migration 0061, so generated code retained
stale columns (input_modalities, supports_reasoning) causing runtime
"column does not exist" errors when adding models.

- Update 0001_init.up.sql to current schema (providers, provider_id,
  no tts tables, add provider_oauth_tokens)
- Use ALTER TABLE IF EXISTS in 0010/0041/0042 for backward compat
- Regenerate sqlc

* fix: guard all legacy migrations against fresh schema for CI compat

On fresh databases, 0001_init.up.sql creates providers/provider_id
(not llm_providers/llm_provider_id). Migrations 0013, 0041, 0046, 0047
referenced the old names without guards, causing CI migration failures.

- 0013: check llm_provider_id column exists before adding old constraint
- 0041: check llm_providers table exists before backfill/constraint DDL
- 0046: wrap CREATE TABLE in DO block with llm_providers existence check
- 0047: use ALTER TABLE IF EXISTS + DO block guard
2026-04-08 01:03:44 +08:00

80 lines
3.5 KiB
SQL

-- 0061_unify_providers (rollback)
-- Reverse the provider unification: restore llm_providers, tts_providers, tts_models.
-- Step 1: Rename provider_oauth_tokens back
ALTER INDEX IF EXISTS idx_provider_oauth_tokens_state RENAME TO idx_llm_provider_oauth_tokens_state;
ALTER TABLE provider_oauth_tokens RENAME COLUMN provider_id TO llm_provider_id;
ALTER TABLE provider_oauth_tokens RENAME TO llm_provider_oauth_tokens;
-- Step 2: Recreate tts_providers and tts_models
CREATE TABLE IF NOT EXISTS tts_providers (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
provider TEXT NOT NULL,
config JSONB NOT NULL DEFAULT '{}'::jsonb,
enable BOOLEAN NOT NULL DEFAULT false,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT tts_providers_name_unique UNIQUE (name)
);
CREATE TABLE IF NOT EXISTS tts_models (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
model_id TEXT NOT NULL,
name TEXT,
tts_provider_id UUID NOT NULL REFERENCES tts_providers(id) ON DELETE CASCADE,
config JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_tts_models_provider_id ON tts_models(tts_provider_id);
-- Step 3: Migrate speech providers back to tts_providers
INSERT INTO tts_providers (id, name, provider, config, enable, created_at, updated_at)
SELECT id, name,
CASE WHEN client_type = 'edge-speech' THEN 'edge' ELSE client_type END,
config, enable, created_at, updated_at
FROM providers
WHERE client_type = 'edge-speech';
-- Step 4: Migrate speech models back to tts_models
INSERT INTO tts_models (id, model_id, name, tts_provider_id, config, created_at, updated_at)
SELECT id, model_id, name, provider_id, config, created_at, updated_at
FROM models
WHERE type = 'speech';
-- Step 5: Update bots FK back to tts_models
ALTER TABLE bots DROP CONSTRAINT IF EXISTS bots_tts_model_id_fkey;
ALTER TABLE bots ADD CONSTRAINT bots_tts_model_id_fkey
FOREIGN KEY (tts_model_id) REFERENCES tts_models(id) ON DELETE SET NULL;
-- Step 6: Remove speech models and providers from unified tables
DELETE FROM models WHERE type = 'speech';
DELETE FROM providers WHERE client_type = 'edge-speech';
-- Step 7: Restore models type CHECK
ALTER TABLE models DROP CONSTRAINT models_type_check;
ALTER TABLE models ADD CONSTRAINT models_type_check CHECK (type IN ('chat', 'embedding'));
-- Step 8: Rename provider_id back to llm_provider_id
ALTER TABLE models RENAME CONSTRAINT models_provider_id_model_id_unique TO models_provider_model_id_unique;
ALTER TABLE models RENAME COLUMN provider_id TO llm_provider_id;
-- Step 9: Restore client_type CHECK
ALTER TABLE providers DROP CONSTRAINT providers_client_type_check;
ALTER TABLE providers ADD CONSTRAINT llm_providers_client_type_check CHECK (
client_type IN ('openai-responses', 'openai-completions', 'anthropic-messages', 'google-generative-ai', 'openai-codex')
);
-- Step 10: Restore api_key and base_url columns from config
ALTER TABLE providers ADD COLUMN base_url TEXT NOT NULL DEFAULT '';
ALTER TABLE providers ADD COLUMN api_key TEXT NOT NULL DEFAULT '';
UPDATE providers SET
base_url = COALESCE(config->>'base_url', ''),
api_key = COALESCE(config->>'api_key', '');
ALTER TABLE providers DROP COLUMN config;
-- Step 11: Rename providers back to llm_providers
ALTER TABLE providers RENAME CONSTRAINT providers_name_unique TO llm_providers_name_unique;
ALTER TABLE providers RENAME TO llm_providers;