Static 多 Provider 生产部署

本场景部署一个 ScootGate 节点，使用本地 TOML 管理路由：

chat-fast 同时路由到一个 OpenAI 后端和一个 OpenAI-compatible 备用后端；
claude-fast 通过独立 Anthropic Messages 协议面路由；
所有 Provider 流量经 TLS HTTP CONNECT 代理；
API listener 原生终止 TLS；
用量事件经持久 outbox 投递 MQTT。

1. 拓扑

客户端
  │ https://gate.example.com:8443
  ▼
ScootGate prod-a
  ├── HTTPS ──► auth control plane
  ├── MQTTS ──► MQTT broker
  └── TLS CONNECT ──► egress proxy
                          ├──► api.openai.com
                          ├──► compatible.example.net
                          └──► api.anthropic.com

health listener 只绑定 127.0.0.1:9090，由同机探针或本机代理读取。

2. 前置条件

准备以下资源：

gate.example.com 的证书与私钥；
可访问三个 Provider 的 HTTP CONNECT 代理及专用账号；
授权快照 HTTPS 端点；
MQTTS broker；
三份 Provider 凭据；
/var/lib/scootgate 持久目录。

建议权限：

install -d -m 0700 -o scootgate -g scootgate /var/lib/scootgate
install -d -m 0700 -o scootgate -g scootgate /var/lib/scootgate/usage-outbox
install -d -m 0700 -o scootgate -g scootgate /run/secrets
install -d -m 0700 -o scootgate -g scootgate /etc/scootgate/tls

Provider key 应由 secret manager 直接挂载为：

/run/secrets/openai-primary-api-key
/run/secrets/compatible-backup-api-key
/run/secrets/anthropic-primary-api-key

文件权限设为 0400，不要用示例占位符启动生产节点。

3. 完整配置

下载：static-production.toml

instance_id = "scootgate-prod-a"

[server]
listen = "0.0.0.0:8443"
max_concurrent_requests = 2048
request_head_timeout_ms = 5000

[server.tls]
cert = "/etc/scootgate/tls/fullchain.pem"
key = "/etc/scootgate/tls/privkey.pem"

[health]
listen = "127.0.0.1:9090"
max_concurrent_requests = 64
request_head_timeout_ms = 2000

[auth]
snapshot_url = "https://control.example.com/v1/scootgate/auth-snapshot"
token = "REPLACE_WITH_AUTH_CONTROL_PLANE_TOKEN"
cache_path = "/var/lib/scootgate/auth-snapshot.json"
poll_interval_secs = 30

[egress]
kind = "http_connect"
addr = "egress-proxy.internal.example.com:8443"
username = "scootgate-prod-a"
password = "REPLACE_WITH_EGRESS_SERVICE_PASSWORD"
tls = true
connect_timeout_ms = 5000

[api]
max_request_body_bytes = 1048576
response_head_timeout_ms = 30000
stream_idle_timeout_ms = 300000

[routing]
max_pre_send_attempts = 2

[routing.load_balancing]
algorithm = "weighted_p2c"
ewma_alpha = 0.2
initial_first_byte_ms = 1000
slow_start_secs = 30

[routing.circuit_breaker]
failure_threshold = 3
failure_window_secs = 30
cooldown_secs = 30
half_open_max_requests = 1
throttle_cooldown_secs = 10

[credentials.openai-primary]
file = "/run/secrets/openai-primary-api-key"

[credentials.compatible-backup]
file = "/run/secrets/compatible-backup-api-key"

[credentials.anthropic-primary]
file = "/run/secrets/anthropic-primary-api-key"

[backends.openai-primary]
protocol = "openai_compatible"
upstream = "https://api.openai.com"
credential_ref = "openai-primary"
max_inflight_per_node = 64

[backends.compatible-backup]
protocol = "openai_compatible"
upstream = "https://compatible.example.net"
credential_ref = "compatible-backup"
max_inflight_per_node = 32

[backends.anthropic-primary]
protocol = "anthropic_messages"
upstream = "https://api.anthropic.com"
credential_ref = "anthropic-primary"
max_inflight_per_node = 64

[models.chat-fast]
protocol = "openai_compatible"

[[models.chat-fast.targets]]
backend = "openai-primary"
model = "gpt-4o-mini"
weight = 100
supports_stream = true
supports_tools = true
supports_json_schema = true

[[models.chat-fast.targets]]
backend = "compatible-backup"
model = "vendor-chat-fast"
weight = 40
supports_stream = true
supports_tools = false
supports_json_schema = false

[models.claude-fast]
protocol = "anthropic_messages"

[[models.claude-fast.targets]]
backend = "anthropic-primary"
model = "claude-3-5-haiku-latest"
weight = 100
supports_stream = true
supports_tools = true
supports_json_schema = true

[usage]
schema = 2
delivery = "mqtt_outbox"
authoritative = false
outbox_path = "/var/lib/scootgate/usage-outbox"
max_outbox_bytes = 1073741824
flush_interval_ms = 500
on_full = "drop_newest"

[mqtt]
broker = "mqtts://mqtt.internal.example.com:8883"
username = "scootgate-prod-a"
password = "REPLACE_WITH_MQTT_SERVICE_PASSWORD"
topic_prefix = "scootgate/prod"
client_id = "scootgate-prod-a"
queue_capacity = 8192

[shutdown]
grace_period_secs = 60

[log]
level = "info"

上线前至少替换：

auth control-plane URL 与 token；
egress proxy 地址、账号与密码；
compatible 后端域名和真实模型名；
MQTT 地址、账号与密码；
TLS、Provider secret 与状态路径。

chat-fast 的第二个 target 不支持 tools/json schema。带这些能力的请求会只在 openai-primary 候选中选择，而不是把不兼容请求发给备用后端。

4. 授权快照

示例客户端 key 是 demo-client-key，快照只保存它的 SHA-256：

{
  "version": 7,
  "keys": [
    {
      "key_id": "demo-active",
      "tenant_id": "tenant-demo",
      "key_sha256": "299ccbc5cc6d53eb7860d96cb94e58213a550cd3aaa12d18f6ec2a3b305808a4",
      "status": "active"
    },
    {
      "key_id": "demo-revoked",
      "tenant_id": "tenant-demo",
      "key_sha256": "7077d33f1bf541153b1040c55f8df93735c8b118e9ab74d39e0f3a2f26e49b4f",
      "status": "revoked"
    }
  ]
}

下载：auth-snapshot.json

控制面接口要求：

GET /v1/scootgate/auth-snapshot?since=0
Authorization: Bearer REPLACE_WITH_AUTH_CONTROL_PLANE_TOKEN

有更新：返回 200 与完整 JSON；
无更新：返回 304；
后续版本必须严格大于当前版本。

示例中的 revoked-client-key 已标记 revoked，用它请求必须返回 401。

5. 启动

将渲染后的配置保存为 /etc/scootgate/scootgate.toml 并设置 0600：

scootgate --config /etc/scootgate/scootgate.toml

启动日志出现 listener 地址不代表已经接流；必须等待：

curl --fail --silent http://127.0.0.1:9090/readyz

冷启动没有授权缓存时，节点会保持 unready，直到控制面返回有效快照。

6. 验证

以下命令假设证书由 /etc/scootgate/tls/ca.pem 信任：

curl --fail \
  --resolve gate.example.com:8443:127.0.0.1 \
  --cacert /etc/scootgate/tls/ca.pem \
  -H 'Authorization: Bearer demo-client-key' \
  https://gate.example.com:8443/v1/models

应只看到公共模型 chat-fast 与 claude-fast，不能出现 backend ID、 Provider origin 或真实凭据引用。

OpenAI-compatible 请求：

curl --fail \
  --resolve gate.example.com:8443:127.0.0.1 \
  --cacert /etc/scootgate/tls/ca.pem \
  -H 'Authorization: Bearer demo-client-key' \
  -H 'Content-Type: application/json' \
  -d '{"model":"chat-fast","messages":[{"role":"user","content":"ping"}]}' \
  https://gate.example.com:8443/v1/chat/completions

Anthropic 请求：

curl --fail \
  --resolve gate.example.com:8443:127.0.0.1 \
  --cacert /etc/scootgate/tls/ca.pem \
  -H 'x-api-key: demo-client-key' \
  -H 'anthropic-version: 2023-06-01' \
  -H 'Content-Type: application/json' \
  -d '{"model":"claude-fast","max_tokens":32,"messages":[{"role":"user","content":"ping"}]}' \
  https://gate.example.com:8443/v1/messages

验证代理日志中能看到三个 Provider 的 CONNECT，Provider 请求头中只有 ScootGate 持有的上游凭据，不能出现 demo-client-key。

7. 预期负载行为

无 tools/json schema 的 chat-fast 请求在两个 target 间按权重、在途数和首包 EWMA 选择；
tools/json schema 请求只选择 openai-primary；
claude-fast 永远不会进入 OpenAI-compatible 池；
备用 target 连续传输失败后熔断，冷却结束只允许一个 half-open 探测；
Provider 429 不触发当前请求重试。

权重是选择输入，不保证小样本严格按 100:40 分布。

8. 故障演练

操作	验收结果
使用 `revoked-client-key`	401，Provider 与代理均无请求
停止 egress proxy	502，三个 Provider 均无直连
让备用后端 TLS 证书不可信	仅在确认未发送字节时安全改选一次
让 Provider 返回 401	客户端 502，该 backend 后续不再收流
停止 MQTT	API 继续服务，`usage_outbox.pending_events` 增长
填满 outbox 配额	`dropped_events` 增长，API 继续服务
SIGTERM	`/readyz` 不再接流，进程在 60 秒 grace period 内退出

完成以上验证后，再把节点加入生产入口。

Keyboard shortcuts

ScootGate