refactor: unify providers and models tables (#338)

* refactor: unify providers and models tables

- Rename `llm_providers` → `providers`, `llm_provider_oauth_tokens` → `provider_oauth_tokens`
- Remove `tts_providers` and `tts_models` tables; speech models now live in the unified `models` table with `type = 'speech'`
- Replace top-level `api_key`/`base_url` columns with a JSONB `config` field on `providers`
- Rename `llm_provider_id` → `provider_id` across all references
- Add `edge-speech` client type and `conf/providers/edge.yaml` default provider
- Create new read-only speech endpoints (`/speech-providers`, `/speech-models`) backed by filtered views of the unified tables
- Remove old TTS CRUD handlers; simplify speech page to read-only + test
- Update registry loader to skip malformed YAML files instead of failing entirely
- Fix YAML quoting for model names containing colons in openrouter.yaml
- Regenerate sqlc, swagger, and TypeScript SDK

* fix: exclude speech providers from providers list endpoint

ListProviders now filters out client_type matching '%-speech' so Edge
and future speech providers no longer appear on the Providers page.
ListSpeechProviders uses the same pattern match instead of hard-coding
'edge-speech'.

* fix: use explicit client_type list instead of LIKE pattern

Replace '%-speech' pattern with explicit IN ('edge-speech') for both
ListProviders (exclusion) and ListSpeechProviders (inclusion). New
speech client types must be added to both queries.

* fix: use EXECUTE for dynamic SQL in migrations referencing old schema

PL/pgSQL pre-validates column/table references in static SQL statements
inside DO blocks before evaluating IF/RETURN guards. This caused
migrations 0010-0061 to fail on fresh databases where the canonical
schema uses `providers`/`provider_id` instead of `llm_providers`/
`llm_provider_id`.

Wrap all SQL that references potentially non-existent old schema objects
(llm_providers, llm_provider_id, tts_providers, tts_models, etc.) in
EXECUTE strings so they are only parsed at runtime when actually reached.

* fix: revert canonical schema to use llm_providers for migration compatibility

The CI migrations workflow (up → down → up) failed because 0061 down
renames `providers` back to `llm_providers`, but 0001 down only dropped
`providers` — leaving `llm_providers` as a remnant. On the second
migrate up, 0010 found the stale `llm_providers` and tried to reference
`models.llm_provider_id` which no longer existed.

Revert 0001 canonical schema to use original names (llm_providers,
tts_providers, tts_models) so incremental migrations work naturally and
0061 handles the final rename. Remove EXECUTE wrappers and unnecessary
guards from migrations that now always operate on llm_providers.

* fix: icons

* fix: sync canonical schema with 0061 migration to fix sqlc column mismatch

0001_init.up.sql still used old names (llm_providers, llm_provider_id)
and included dropped tts_providers/tts_models tables. sqlc could not
parse the PL/pgSQL EXECUTE in migration 0061, so generated code retained
stale columns (input_modalities, supports_reasoning) causing runtime
"column does not exist" errors when adding models.

- Update 0001_init.up.sql to current schema (providers, provider_id,
  no tts tables, add provider_oauth_tokens)
- Use ALTER TABLE IF EXISTS in 0010/0041/0042 for backward compat
- Regenerate sqlc

* fix: guard all legacy migrations against fresh schema for CI compat

On fresh databases, 0001_init.up.sql creates providers/provider_id
(not llm_providers/llm_provider_id). Migrations 0013, 0041, 0046, 0047
referenced the old names without guards, causing CI migration failures.

- 0013: check llm_provider_id column exists before adding old constraint
- 0041: check llm_providers table exists before backfill/constraint DDL
- 0046: wrap CREATE TABLE in DO block with llm_providers existence check
- 0047: use ALTER TABLE IF EXISTS + DO block guard
This commit is contained in:
Acbox Liu
2026-04-08 01:03:44 +08:00
committed by GitHub
parent 43c4153938
commit 8d5c38f0e5
78 changed files with 3163 additions and 5668 deletions
+184 -438
View File
@@ -889,6 +889,10 @@ definitions:
properties:
message_id:
type: string
preview:
type: string
sender:
type: string
target:
type: string
type: object
@@ -2073,6 +2077,10 @@ definitions:
type: array
created_at:
type: string
display_content:
type: string
event_id:
type: string
external_message_id:
type: string
id:
@@ -2125,12 +2133,12 @@ definitions:
properties:
config:
$ref: '#/definitions/models.ModelConfig'
llm_provider_id:
type: string
model_id:
type: string
name:
type: string
provider_id:
type: string
type:
$ref: '#/definitions/models.ModelType'
type: object
@@ -2152,12 +2160,12 @@ definitions:
$ref: '#/definitions/models.ModelConfig'
id:
type: string
llm_provider_id:
type: string
model_id:
type: string
name:
type: string
provider_id:
type: string
type:
$ref: '#/definitions/models.ModelType'
type: object
@@ -2180,10 +2188,12 @@ definitions:
enum:
- chat
- embedding
- speech
type: string
x-enum-varnames:
- ModelTypeChat
- ModelTypeEmbedding
- ModelTypeSpeech
models.TestResponse:
properties:
latency_ms:
@@ -2211,12 +2221,12 @@ definitions:
properties:
config:
$ref: '#/definitions/models.ModelConfig'
llm_provider_id:
type: string
model_id:
type: string
name:
type: string
provider_id:
type: string
type:
$ref: '#/definitions/models.ModelType'
type: object
@@ -2227,12 +2237,11 @@ definitions:
type: object
providers.CreateRequest:
properties:
api_key:
type: string
base_url:
type: string
client_type:
type: string
config:
additionalProperties: {}
type: object
icon:
type: string
metadata:
@@ -2241,18 +2250,16 @@ definitions:
name:
type: string
required:
- base_url
- client_type
- name
type: object
providers.GetResponse:
properties:
api_key:
type: string
base_url:
type: string
client_type:
type: string
config:
additionalProperties: {}
type: object
created_at:
type: string
enable:
@@ -2304,12 +2311,11 @@ definitions:
type: object
providers.UpdateRequest:
properties:
api_key:
type: string
base_url:
type: string
client_type:
type: string
config:
additionalProperties: {}
type: object
enable:
type: boolean
icon:
@@ -2565,6 +2571,8 @@ definitions:
type: integer
compaction_threshold:
type: integer
discuss_probe_model_id:
type: string
heartbeat_enabled:
type: boolean
heartbeat_interval:
@@ -2604,6 +2612,8 @@ definitions:
type: integer
compaction_threshold:
type: integer
discuss_probe_model_id:
type: string
heartbeat_enabled:
type: boolean
heartbeat_interval:
@@ -2627,25 +2637,6 @@ definitions:
tts_model_id:
type: string
type: object
tts.CreateModelRequest:
properties:
config:
additionalProperties: {}
type: object
model_id:
type: string
name:
type: string
tts_provider_id:
type: string
type: object
tts.CreateProviderRequest:
properties:
name:
type: string
provider:
type: string
type: object
tts.ModelCapabilities:
properties:
formats:
@@ -2672,26 +2663,6 @@ definitions:
name:
type: string
type: object
tts.ModelResponse:
properties:
config:
additionalProperties: {}
type: object
created_at:
type: string
id:
type: string
model_id:
type: string
name:
type: string
provider_type:
type: string
tts_provider_id:
type: string
updated_at:
type: string
type: object
tts.ParamConstraint:
properties:
default:
@@ -2720,8 +2691,30 @@ definitions:
provider:
type: string
type: object
tts.ProviderResponse:
tts.SpeechModelResponse:
properties:
config:
additionalProperties: {}
type: object
created_at:
type: string
id:
type: string
model_id:
type: string
name:
type: string
provider_id:
type: string
provider_type:
type: string
updated_at:
type: string
type: object
tts.SpeechProviderResponse:
properties:
client_type:
type: string
created_at:
type: string
enable:
@@ -2730,8 +2723,6 @@ definitions:
type: string
name:
type: string
provider:
type: string
updated_at:
type: string
type: object
@@ -2743,21 +2734,6 @@ definitions:
text:
type: string
type: object
tts.UpdateModelRequest:
properties:
config:
additionalProperties: {}
type: object
name:
type: string
type: object
tts.UpdateProviderRequest:
properties:
enable:
type: boolean
name:
type: string
type: object
tts.VoiceInfo:
properties:
id:
@@ -7924,6 +7900,138 @@ paths:
summary: List search provider metadata
tags:
- search-providers
/speech-models:
get:
description: List all models of type 'speech' (filtered view of unified models
table)
produces:
- application/json
responses:
"200":
description: OK
schema:
items:
$ref: '#/definitions/tts.SpeechModelResponse'
type: array
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: List all speech models
tags:
- speech-models
/speech-models/{id}:
get:
parameters:
- description: Model ID
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: OK
schema:
$ref: '#/definitions/tts.SpeechModelResponse'
"404":
description: Not Found
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Get a speech model
tags:
- speech-models
/speech-models/{id}/capabilities:
get:
parameters:
- description: Model ID
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: OK
schema:
$ref: '#/definitions/tts.ModelCapabilities'
"404":
description: Not Found
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Get speech model capabilities
tags:
- speech-models
/speech-models/{id}/test:
post:
consumes:
- application/json
description: Synthesize text using a specific model's config and return audio
parameters:
- description: Model ID
in: path
name: id
required: true
type: string
- description: Text to synthesize
in: body
name: request
required: true
schema:
$ref: '#/definitions/tts.TestSynthesizeRequest'
produces:
- application/octet-stream
responses:
"200":
description: Audio data
schema:
type: file
"400":
description: Bad Request
schema:
$ref: '#/definitions/handlers.ErrorResponse'
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Test speech model synthesis
tags:
- speech-models
/speech-providers:
get:
description: List providers that support speech (filtered view of unified providers
table)
produces:
- application/json
responses:
"200":
description: OK
schema:
items:
$ref: '#/definitions/tts.SpeechProviderResponse'
type: array
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: List speech providers
tags:
- speech-providers
/speech-providers/meta:
get:
description: List available speech provider types with their models and capabilities
responses:
"200":
description: OK
schema:
items:
$ref: '#/definitions/tts.ProviderMetaResponse'
type: array
summary: List speech provider metadata
tags:
- speech-providers
/supermarket/mcps:
get:
parameters:
@@ -8052,368 +8160,6 @@ paths:
summary: List all tags from supermarket
tags:
- supermarket
/tts-models:
get:
produces:
- application/json
responses:
"200":
description: OK
schema:
items:
$ref: '#/definitions/tts.ModelResponse'
type: array
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: List all TTS models
tags:
- tts-models
post:
consumes:
- application/json
description: Manually create a TTS model under a specific provider
parameters:
- description: TTS model configuration
in: body
name: request
required: true
schema:
$ref: '#/definitions/tts.CreateModelRequest'
produces:
- application/json
responses:
"201":
description: Created
schema:
$ref: '#/definitions/tts.ModelResponse'
"400":
description: Bad Request
schema:
$ref: '#/definitions/handlers.ErrorResponse'
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Create a TTS model
tags:
- tts-models
/tts-models/{id}:
delete:
parameters:
- description: Model ID
in: path
name: id
required: true
type: string
responses:
"204":
description: No Content
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Delete a TTS model
tags:
- tts-models
get:
parameters:
- description: Model ID
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: OK
schema:
$ref: '#/definitions/tts.ModelResponse'
"404":
description: Not Found
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Get a TTS model
tags:
- tts-models
put:
consumes:
- application/json
parameters:
- description: Model ID
in: path
name: id
required: true
type: string
- description: Updated configuration
in: body
name: request
required: true
schema:
$ref: '#/definitions/tts.UpdateModelRequest'
produces:
- application/json
responses:
"200":
description: OK
schema:
$ref: '#/definitions/tts.ModelResponse'
"400":
description: Bad Request
schema:
$ref: '#/definitions/handlers.ErrorResponse'
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Update a TTS model
tags:
- tts-models
/tts-models/{id}/capabilities:
get:
parameters:
- description: Model ID
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: OK
schema:
$ref: '#/definitions/tts.ModelCapabilities'
"404":
description: Not Found
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Get TTS model capabilities
tags:
- tts-models
/tts-models/{id}/test:
post:
consumes:
- application/json
description: Synthesize text using a specific model's config and return audio
parameters:
- description: Model ID
in: path
name: id
required: true
type: string
- description: Text to synthesize
in: body
name: request
required: true
schema:
$ref: '#/definitions/tts.TestSynthesizeRequest'
produces:
- application/octet-stream
responses:
"200":
description: Audio data
schema:
type: file
"400":
description: Bad Request
schema:
$ref: '#/definitions/handlers.ErrorResponse'
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Test TTS model synthesis
tags:
- tts-models
/tts-providers:
get:
parameters:
- description: Provider type filter
in: query
name: provider
type: string
produces:
- application/json
responses:
"200":
description: OK
schema:
items:
$ref: '#/definitions/tts.ProviderResponse'
type: array
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: List TTS providers
tags:
- tts-providers
post:
consumes:
- application/json
description: Create a TTS provider and auto-import its available models
parameters:
- description: TTS provider configuration
in: body
name: request
required: true
schema:
$ref: '#/definitions/tts.CreateProviderRequest'
produces:
- application/json
responses:
"201":
description: Created
schema:
$ref: '#/definitions/tts.ProviderResponse'
"400":
description: Bad Request
schema:
$ref: '#/definitions/handlers.ErrorResponse'
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Create a TTS provider
tags:
- tts-providers
/tts-providers/{id}:
delete:
parameters:
- description: Provider ID
in: path
name: id
required: true
type: string
responses:
"204":
description: No Content
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Delete a TTS provider
tags:
- tts-providers
get:
parameters:
- description: Provider ID
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: OK
schema:
$ref: '#/definitions/tts.ProviderResponse'
"404":
description: Not Found
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Get a TTS provider
tags:
- tts-providers
put:
consumes:
- application/json
parameters:
- description: Provider ID
in: path
name: id
required: true
type: string
- description: Updated configuration
in: body
name: request
required: true
schema:
$ref: '#/definitions/tts.UpdateProviderRequest'
produces:
- application/json
responses:
"200":
description: OK
schema:
$ref: '#/definitions/tts.ProviderResponse'
"400":
description: Bad Request
schema:
$ref: '#/definitions/handlers.ErrorResponse'
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Update a TTS provider
tags:
- tts-providers
/tts-providers/{id}/import-models:
post:
description: Discover and import available models from the TTS adapter
parameters:
- description: Provider ID
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: OK
schema:
items:
$ref: '#/definitions/tts.ModelResponse'
type: array
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: Import models for a TTS provider
tags:
- tts-providers
/tts-providers/{id}/models:
get:
parameters:
- description: Provider ID
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: OK
schema:
items:
$ref: '#/definitions/tts.ModelResponse'
type: array
"500":
description: Internal Server Error
schema:
$ref: '#/definitions/handlers.ErrorResponse'
summary: List models for a TTS provider
tags:
- tts-providers
/tts-providers/meta:
get:
description: List available TTS provider types with their models and capabilities
responses:
"200":
description: OK
schema:
items:
$ref: '#/definitions/tts.ProviderMetaResponse'
type: array
summary: List TTS provider metadata
tags:
- tts-providers
/users:
get:
description: List users