优化前缀缓存机制和UI,并引入一些新工具.preview

This commit is contained in:
minorQAQ committed 2026-08-19 15:16:17 +08:00
1 parent 863abe7c76
commit 6131c2563e
102 files changed
+9087 -4351

No files matched your search

+35 -10
View File
@@ -85,42 +85,64 @@ sudo apt update && sudo apt install -y ffmpeg libavcodec-dev libavformat-dev lib
| `LOCAL_MODEL_DIR` / `MODEL_PATH` / `VOXCPM_MODEL_PATH` | 模型目录 | 全部 |
| `CUDA_VISIBLE_DEVICES` | GPU 设备号 | 全部 |
| `SPECIFIC_GPU` | 单 GPU 指定(写 `CUDA_VISIBLE_DEVICES`) | image_gen / tts |
| `SERVER_API_KEY` | API Key 鉴权(空=不校验,任意值放行) | 全部 |
| `SERVER_MODELNAME` | 服务模型名(请求 body.model 须匹配;空=不校验) | 全部 |
> **鉴权与模型名校验**:`SERVER_API_KEY` 为空时服务不校验 API Key(任意值放行);
> `SERVER_MODELNAME` 为空时服务不校验请求中的 `model` 字段。
> 部署时如需启用,示例:
> ```bash
> SERVER_API_KEY=sk-xxx SERVER_MODELNAME=Qwen3-ASR-1.7B bash start_asr.sh
> ```
### LLM · 8900
```bash
bash start_llm.sh
# 启用鉴权与自定义模型名
# SERVER_API_KEY=sk-xxx SERVER_MODELNAME=Qwen/Qwen3.6-35B-A3B-FP8 bash start_llm.sh
# 测试
python3 test_llm.py --url http://127.0.0.1:8900 # 非流式
python3 test_llm.py --url http://127.0.0.1:8900 --stream # 流式
```
> vLLM 加载 Qwen3.6-35B-A3B-FP8,OpenAI 兼容 API,使用 `chat_template.jinja` 格式化对话。
> `--served-model-name` 由 `SERVER_MODELNAME` 控制(默认 `Qwen/Qwen3.6-35B-A3B-FP8`);
> `SERVER_API_KEY` 非空时传给 vLLM `--api-key`,请求需携带 `Authorization: Bearer <key>`。
### ASR · 8901
```bash
bash start_asr.sh
# 启用鉴权与自定义模型名
# SERVER_API_KEY=sk-xxx SERVER_MODELNAME=Qwen3-ASR-1.7B bash start_asr.sh
# 测试
python3 test_asr.py --audio /path/to/audio.wav --url http://127.0.0.1:8901/v1/chat/completions
```
> vLLM 加载 Qwen3-ASR-1.7B,`--served-model-name` 由 `SERVER_MODELNAME` 控制(默认 `Qwen3-ASR-1.7B`)。
### TTS · 8902(流式)
```bash
bash start_streaming_tts.sh
# 启用鉴权与自定义模型名
# SERVER_API_KEY=sk-xxx SERVER_MODELNAME=VoxCPM1.5 bash start_streaming_tts.sh
# 测试(流式返回 wav 分块)
curl -X POST http://127.0.0.1:8902/stream_tts \
-H "Content-Type: application/json" \
-d '{"text": "你好,这是一个流式语音合成测试。", "request_id": "test-1"}'
-d '{"model": "VoxCPM1.5", "text": "你好,这是一个流式语音合成测试。", "request_id": "test-1"}'
```
> VoxCPM 流式合成,边生成边返回 wav chunk,配合 `reference_audio.wav` 控制音色。
> 配置了 `SERVER_MODELNAME` 时请求 body 须带 `model` 字段;配置了 `SERVER_API_KEY` 时须带 `Authorization: Bearer <key>` 头。
### RAG · 8903(Embedding + Reranker)
```bash
bash start_rag_server.sh
# 启用鉴权与自定义模型名
# SERVER_API_KEY=sk-xxx SERVER_MODELNAME=Qwen3-Embedding-0.6B bash start_rag_server.sh
python3 test_rag.py
```
@@ -128,10 +150,12 @@ python3 test_rag.py
```bash
LOCAL_MODEL_DIR=./models/Tongyi-MAI/Z-Image-Turbo SPECIFIC_GPU=3 bash start_image_gen.sh
# 启用鉴权与自定义模型名
# SERVER_API_KEY=sk-xxx SERVER_MODELNAME=Z-Image-Turbo bash start_image_gen.sh
# 测试
curl -X POST http://127.0.0.1:8904/generate \
-H "Content-Type: application/json" \
-d '{"prompt": "一只橘猫坐在窗台上,阳光洒落", "width": 1024, "height": 1024, "return_format": "base64"}' \
-d '{"model": "Z-Image-Turbo", "prompt": "一只橘猫坐在窗台上,阳光洒落", "width": 1024, "height": 1024, "return_format": "base64"}' \
| python3 -c "import sys,json; d=json.load(sys.stdin); print(f'成功:{d.get(\"success\")} 尺寸:{d.get(\"width\")}x{d.get(\"height\")} 耗时:{d.get(\"elapsed_seconds\",\"?\")}s')"
```
@@ -171,14 +195,15 @@ for p in 8900 8901 8902 8903 8904; do kill $(lsof -t -i:$p); done
## 五、接入 Agent
在 minor Agent 配置向导(或 `env_config.json`)中填入对应地址即可启用各能力:
四个本地服务(ASR / TTS / RAG / ImageGen)在 Agent 的 `env_config.json` **`models` 列表中注册**(设置 → 环境变量 → LLM 模型列表,或直接编辑 `src/agent/config/env_config.json`),`base_url` 指向本地映射端口,`api_key` 留空(对应服务端未启用鉴权)。Agent 运行时按 `model` 字段匹配取得服务地址:
| Agent 能力 | 配置字段 | 示例值 |
|-----------|----------|--------|
| LLM 对话 | `models[].base_url` | `http://localhost:8900/v1` |
| 语音识别 | `ASR_BASE_URL` | `http://localhost:8901/v1` |
| 语音合成 | `STREAMING_TTS_URL` | `http://localhost:8902` |
| 知识库检索 | `RAG_BASE_URL` | `http://localhost:8903` |
| 图像生成 | (由 `image_gen` 工具读取) | `http://localhost:8904` |
| Agent 能力 | 模型列表条目 | 字段示例 |
|-----------|-------------|----------|
| 语音识别 | `name: "ASR 语音识别"` | `model: "Qwen3-ASR-1.7B"`, `base_url: "http://localhost:8901"`, `api_key: ""` |
| 语音合成 | `name: "TTS 语音合成"` | `model: "VoxCPM1.5"`, `base_url: "http://localhost:8902"`, `api_key: ""` |
| 知识库检索 | `name: "RAG 检索"` | `model: "Qwen3-Embedding-0.6B"`, `base_url: "http://localhost:8903"`, `api_key: ""` |
| 图像生成 | `name: "图像生成"` | `model: "Z-Image-Turbo"`, `base_url: "http://localhost:8904"`, `api_key: ""` |
> 💡 若服务端部署时设置了 `SERVER_API_KEY` / `SERVER_MODELNAME`,请把模型条目中的 `api_key` 与 `model` 改成与之一致,Agent 发出的请求会携带 `Authorization: Bearer <api_key>` 与 `model` 字段。
>
> 💡 不必自建本服务,Agent 也支持任何 **OpenAI 兼容 API**(云端 Qwen / DeepSeek / OpenAI 等),填入 `base_url` + `api_key` 即可。
+45 -2
View File
@@ -26,6 +26,8 @@ import torch
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
DEFAULT_PORT = 8904
DEFAULT_MODEL_ID = "Tongyi-MAI/Z-Image-Turbo"
DEFAULT_MODEL_NAME = "Z-Image-Turbo" # 默认服务模型名(校验请求 body.model 用)
DEFAULT_API_KEY = "" # 默认空:不校验 API Key(任意值放行)
# 本地模型路径:优先读环境变量 LOCAL_MODEL_DIR,未设置则回退到远程 ID
LOCAL_MODEL_DIR = os.environ.get("LOCAL_MODEL_DIR", "")
@@ -45,6 +47,39 @@ warnings.filterwarnings("ignore", category=FutureWarning)
from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import JSONResponse, Response
# 服务参数(--api-key / --model-name):
# api_key 为空:不校验(任意值放行);model_name 为空:不校验请求 model 字段
_model_args: dict = {"api_key": "", "model_name": ""}
def _check_api_key(request: Request):
"""校验请求 API Key:服务未配置 key(空)时放行任意值;否则校验 Authorization / X-API-Key。
校验失败抛出 HTTP 401。
"""
api_key = _model_args.get("api_key", "")
if not api_key:
return
header = request.headers.get("Authorization", "")
if header.startswith("Bearer "):
provided = header[len("Bearer "):].strip()
else:
provided = request.headers.get("X-API-Key", "").strip()
if provided != api_key:
raise HTTPException(status_code=401, detail="API Key 无效或缺失")
def _check_model_name(body: dict):
"""校验请求 modelname:服务未配置模型名(空)时放行任意值;否则要求 body.model 匹配。
校验失败抛出 HTTP 400。
"""
model_name = _model_args.get("model_name", "")
if not model_name:
return
if str(body.get("model") or "").strip() != model_name:
raise HTTPException(status_code=400, detail=f"模型名不匹配: 期望 '{model_name}',收到 '{body.get('model') or ''}'")
# Lazy-loaded pipeline singleton
_pipe = None
_pipe_lock = threading.Lock()
@@ -129,6 +164,10 @@ async def generate_image(request: Request):
except Exception:
raise HTTPException(status_code=400, detail="请求体必须为 JSON")
# 鉴权与模型名校验
_check_api_key(request)
_check_model_name(body)
prompt = (body.get("prompt") or "").strip()
if not prompt:
raise HTTPException(status_code=400, detail="prompt 不能为空")
@@ -194,14 +233,18 @@ async def health():
# ======================== 启动入口 ========================
_model_args: dict = {}
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Z-Image-Turbo Image Generation Server")
parser.add_argument("--port", type=int, default=DEFAULT_PORT, help=f"服务端口(默认 {DEFAULT_PORT})")
parser.add_argument("--model-name", type=str, default=DEFAULT_MODEL_NAME,
help=f"服务模型名(校验请求 body.model;默认 {DEFAULT_MODEL_NAME},留空则不校验)")
parser.add_argument("--api-key", type=str, default=DEFAULT_API_KEY,
help="API Key(校验 Authorization: Bearer <key> 或 X-API-Key;默认空=任意值放行)")
args = parser.parse_args()
_model_args["port"] = args.port
_model_args["model_name"] = args.model_name
_model_args["api_key"] = args.api_key
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=args.port)
+59 -3
View File
@@ -1,7 +1,9 @@
import os
import argparse
import torch
import torch.nn.functional as F
from fastapi import FastAPI
from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from typing import List
from modelscope import AutoTokenizer, AutoModel, AutoModelForCausalLM
@@ -16,6 +18,37 @@ quantization_config = BitsAndBytesConfig(load_in_8bit=True)
EMBED_MODEL_PATH = os.path.join(BASE_DIR, "models", "Qwen", "Qwen3-Embedding-0___6B")
RERANK_MODEL_PATH = os.path.join(BASE_DIR, "models", "Qwen", "Qwen3-Reranker-4B")
# ===================== 服务参数(--api-key / --model-name)=====================
# api_key 为空:不校验(任意值放行);model_name 为空:不校验请求 model 字段
_model_args: dict = {"api_key": "", "model_name": ""}
def check_api_key(request: Request):
"""校验请求 API Key:服务未配置 key(空)时放行任意值;否则校验 Authorization / X-API-Key。
校验失败抛出 HTTP 401。
"""
api_key = _model_args.get("api_key", "")
if not api_key:
return
header = request.headers.get("Authorization", "")
if header.startswith("Bearer "):
provided = header[len("Bearer "):].strip()
else:
provided = request.headers.get("X-API-Key", "").strip()
if provided != api_key:
raise HTTPException(status_code=401, detail="API Key 无效或缺失")
def check_model_name(model: str | None):
"""校验请求 modelname:服务未配置模型名(空)时放行任意值;否则要求请求 model 匹配。
校验失败抛出 HTTP 400。
"""
model_name = _model_args.get("model_name", "")
if not model_name:
return
if str(model or "").strip() != model_name:
raise HTTPException(status_code=400, detail=f"模型名不匹配: 期望 '{model_name}',收到 '{model or ''}'")
# ===================== 初始化 FastAPI =====================
app = FastAPI(title="RAG Backend with 8-bit Qwen3 Models")
@@ -87,6 +120,7 @@ def load_models():
# ===================== API 请求体定义 =====================
class EmbedRequest(BaseModel):
task_description: str = "Given a web search query, retrieve relevant passages that answer the query"
model: str | None = None # 可选,服务配置了 --model-name 时须匹配
queries: List[str]
documents: List[str] # 可选,若只查询文本可只传 queries
@@ -96,6 +130,7 @@ class EmbedResponse(BaseModel):
class RerankRequest(BaseModel):
task_description: str = "Given a web search query, retrieve relevant passages that answer the query"
model: str | None = None # 可选,服务配置了 --model-name 时须匹配
query: str
documents: List[str]
@@ -104,7 +139,9 @@ class RerankResponse(BaseModel):
# ===================== Embedding 端点 =====================
@app.post("/embed", response_model=EmbedResponse)
async def get_embeddings(req: EmbedRequest):
async def get_embeddings(req: EmbedRequest, request: Request):
check_api_key(request)
check_model_name(req.model)
# 为 query 加上指令,document 不加指令
instructed_queries = [get_detailed_instruct(req.task_description, q) for q in req.queries]
# 所有文本拼成一个 batch:先是 queries,后是 documents(如果有)
@@ -132,7 +169,9 @@ async def get_embeddings(req: EmbedRequest):
# ===================== Reranker 端点 =====================
@app.post("/rerank", response_model=RerankResponse)
async def rerank_documents(req: RerankRequest):
async def rerank_documents(req: RerankRequest, request: Request):
check_api_key(request)
check_model_name(req.model)
# 构造带格式的输入对
pairs = [
format_rerank_instruction(req.task_description, req.query, doc)
@@ -166,3 +205,20 @@ async def rerank_documents(req: RerankRequest):
@app.get("/health")
def health():
return {"status": "ok"}
# ======================== 启动入口 ========================
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Qwen3 RAG Server (Embedding + Reranker)")
parser.add_argument("--port", type=int, default=8903, help="服务端口(默认 8903)")
parser.add_argument("--model-name", type=str, default="Qwen3-Embedding-0.6B",
help="服务模型名(校验请求 body.model;默认 Qwen3-Embedding-0.6B,留空则不校验)")
parser.add_argument("--api-key", type=str, default="",
help="API Key(校验 Authorization: Bearer <key> 或 X-API-Key;默认空=任意值放行)")
args = parser.parse_args()
_model_args["api_key"] = args.api_key
_model_args["model_name"] = args.model_name
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=args.port)
+24 -10
View File
@@ -6,6 +6,9 @@ SCRIPT_DIR="$(cd -- "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENV_DIR="${SCRIPT_DIR}/myenv"
MODEL_DIR="${SCRIPT_DIR}/models/Qwen/Qwen3-ASR-1.7B"
HOST_PORT="${HOST_PORT:-8901}"
# 鉴权与模型名校验:SERVER_API_KEY 为空时 vLLM 不校验 API Key(任意值放行)
SERVER_API_KEY="${SERVER_API_KEY:-}"
SERVER_MODELNAME="${SERVER_MODELNAME:-Qwen3-ASR-1.7B}"
# 激活虚拟环境
source "${VENV_DIR}/bin/activate"
@@ -18,6 +21,10 @@ fi
echo "Starting vLLM server on port ${HOST_PORT} ..."
echo "Model path: ${MODEL_DIR}"
echo "Served model name: ${SERVER_MODELNAME}"
if [ -n "${SERVER_API_KEY}" ]; then
echo "API Key 校验已启用"
fi
# 进入脚本目录
cd "${SCRIPT_DIR}"
@@ -34,14 +41,21 @@ export TRANSFORMERS_OFFLINE=1
export CUDA_VISIBLE_DEVICES=3
# 启动 vLLM 服务
vllm serve "${MODEL_DIR}" \
--port "${HOST_PORT}" \
--host 0.0.0.0 \
--quantization fp8 \
--dtype auto \
--kv-cache-dtype fp8 \
--max-model-len 16384 \
--allowed-local-media-path / \
--tensor-parallel-size 1 \
--gpu-memory-utilization 0.15 \
ASR_ARGS=(
--port "${HOST_PORT}"
--host 0.0.0.0
--quantization fp8
--dtype auto
--kv-cache-dtype fp8
--max-model-len 16384
--allowed-local-media-path /
--tensor-parallel-size 1
--gpu-memory-utilization 0.15
--max-num-batched-tokens 4096
--served-model-name "${SERVER_MODELNAME}"
)
# API Key 为空时不传 --api-key(vLLM 默认放行任意值)
if [ -n "${SERVER_API_KEY}" ]; then
ASR_ARGS+=(--api-key "${SERVER_API_KEY}")
fi
vllm serve "${MODEL_DIR}" "${ASR_ARGS[@]}"
+8 -1
View File
@@ -4,6 +4,9 @@ set -euo pipefail
# ---------- 配置 ----------
HOST_PORT="${HOST_PORT:-8904}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# 鉴权与模型名校验:SERVER_API_KEY / SERVER_MODELNAME 为空时服务端不校验(任意值放行)
SERVER_API_KEY="${SERVER_API_KEY:-}"
SERVER_MODELNAME="${SERVER_MODELNAME:-Z-Image-Turbo}"
# ---------- Python 虚拟环境 ----------
if [ -f "${SCRIPT_DIR}/../myenv/bin/activate" ]; then
@@ -20,5 +23,9 @@ if [ -n "${CUDA_DEV}" ]; then
fi
echo "[Z-Image] 启动图像生成服务 (端口=${HOST_PORT})"
echo "[Z-Image] 服务模型名: ${SERVER_MODELNAME}"
if [ -n "${SERVER_API_KEY}" ]; then
echo "[Z-Image] API Key 校验已启用"
fi
cd "${SCRIPT_DIR}"
exec python image_gen_server.py --port "${HOST_PORT}"
exec python image_gen_server.py --port "${HOST_PORT}" --model-name "${SERVER_MODELNAME}" --api-key "${SERVER_API_KEY}"
+29 -16
View File
@@ -5,6 +5,9 @@ SCRIPT_DIR="$(cd -- "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENV_DIR="${SCRIPT_DIR}/myenv"
HOST_PORT="${HOST_PORT:-8900}"
MODEL_DIR="${SCRIPT_DIR}/models/Qwen/Qwen3___6-35B-A3B-FP8"
# 鉴权与模型名校验:SERVER_API_KEY 为空时 vLLM 不校验 API Key(任意值放行)
SERVER_API_KEY="${SERVER_API_KEY:-}"
SERVER_MODELNAME="${SERVER_MODELNAME:-Qwen/Qwen3.6-35B-A3B-FP8}"
# 激活虚拟环境
source "${VENV_DIR}/bin/activate"
@@ -17,6 +20,10 @@ fi
echo "Starting vLLM server on port ${HOST_PORT} ..."
echo "Model path: ${MODEL_DIR}"
echo "Served model name: ${SERVER_MODELNAME}"
if [ -n "${SERVER_API_KEY}" ]; then
echo "API Key 校验已启用"
fi
# 设置环境变量
export VLLM_USE_MODELSCOPE=true
@@ -28,19 +35,25 @@ export TRANSFORMERS_OFFLINE=1
export CUDA_VISIBLE_DEVICES=3
# 启动 vLLM 服务
vllm serve "${MODEL_DIR}" \
--port "${HOST_PORT}" \
--host 0.0.0.0 \
--tensor-parallel-size 1 \
--max-model-len 262144 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}' \
--enable-prefix-caching \
--max_num_batched_tokens 4096 \
--max_num_seqs 8 \
--kv-cache-dtype fp8_e4m3 \
--chat-template "${SCRIPT_DIR}/chat_template.jinja" \
--safetensors-load-strategy eager \
--served-model-name Qwen/Qwen3.6-35B-A3B-FP8
VLLM_ARGS=(
--port "${HOST_PORT}"
--host 0.0.0.0
--tensor-parallel-size 1
--max-model-len 262144
--reasoning-parser qwen3
--enable-auto-tool-choice
--tool-call-parser qwen3_coder
--speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'
--enable-prefix-caching
--max_num_batched_tokens 4096
--max_num_seqs 8
--kv-cache-dtype fp8_e4m3
--chat-template "${SCRIPT_DIR}/chat_template.jinja"
--safetensors-load-strategy eager
--served-model-name "${SERVER_MODELNAME}"
)
# API Key 为空时不传 --api-key(vLLM 默认放行任意值)
if [ -n "${SERVER_API_KEY}" ]; then
VLLM_ARGS+=(--api-key "${SERVER_API_KEY}")
fi
vllm serve "${MODEL_DIR}" "${VLLM_ARGS[@]}"
+10 -3
View File
@@ -5,6 +5,9 @@ set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENV_DIR="${VENV_DIR:-${SCRIPT_DIR}/myenv}"
HOST_PORT="${HOST_PORT:-8903}"
# 鉴权与模型名校验:SERVER_API_KEY / SERVER_MODELNAME 为空时服务端不校验(任意值放行)
SERVER_API_KEY="${SERVER_API_KEY:-}"
SERVER_MODELNAME="${SERVER_MODELNAME:-Qwen3-Embedding-0.6B}"
# 激活虚拟环境
source "${VENV_DIR}/bin/activate"
@@ -16,6 +19,10 @@ if lsof -i "tcp:${HOST_PORT}" -t >/dev/null 2>&1; then
fi
echo "Starting Qwen3 RAG server on port ${HOST_PORT} ..."
echo "Served model name: ${SERVER_MODELNAME}"
if [ -n "${SERVER_API_KEY}" ]; then
echo "API Key 校验已启用"
fi
# 进入脚本目录
cd "${SCRIPT_DIR}"
@@ -34,7 +41,7 @@ export MODELSCOPE_CACHE="${MODELSCOPE_CACHE:-${SCRIPT_DIR}/models}"
export CUDA_VISIBLE_DEVICES=3
# 启动 FastAPI 服务
uvicorn rag_server:app \
--host 0.0.0.0 \
exec python rag_server.py \
--port "${HOST_PORT}" \
--workers 1
--model-name "${SERVER_MODELNAME}" \
--api-key "${SERVER_API_KEY}"
+10 -1
View File
@@ -7,6 +7,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# VoxCPM 模型路径(可覆盖)
VOXCPM_MODEL_PATH="${VOXCPM_MODEL_PATH:-${SCRIPT_DIR}/models/VoxCPM1__5}"
# 鉴权与模型名校验:SERVER_API_KEY / SERVER_MODELNAME 为空时服务端不校验(任意值放行)
SERVER_API_KEY="${SERVER_API_KEY:-}"
SERVER_MODELNAME="${SERVER_MODELNAME:-VoxCPM1.5}"
# ---------- Python 虚拟环境 ----------
if [ -f "${SCRIPT_DIR}/../myenv/bin/activate" ]; then
@@ -21,7 +24,13 @@ export CUDA_VISIBLE_DEVICES="${CUDA_DEV}"
echo "[VoxCPM TTS] 启动流式 TTS 服务 (端口=${HOST_PORT}, CUDA=${CUDA_DEV})"
echo "[VoxCPM TTS] 模型路径: ${VOXCPM_MODEL_PATH}"
echo "[VoxCPM TTS] 服务模型名: ${SERVER_MODELNAME}"
if [ -n "${SERVER_API_KEY}" ]; then
echo "[VoxCPM TTS] API Key 校验已启用"
fi
cd "${SCRIPT_DIR}"
exec python streaming_tts_server.py \
--port "${HOST_PORT}" \
--model-path "${VOXCPM_MODEL_PATH}"
--model-path "${VOXCPM_MODEL_PATH}" \
--model-name "${SERVER_MODELNAME}" \
--api-key "${SERVER_API_KEY}"
+50 -1
View File
@@ -39,6 +39,8 @@ except ImportError:
# 默认参数(可通过命令行覆盖)
DEFAULT_MODEL_PATH = os.path.join(BASE_DIR, "models", "VoxCPM1__5")
DEFAULT_PORT = 8902
DEFAULT_MODEL_NAME = "VoxCPM1.5" # 默认服务模型名(校验请求 body.model 用)
DEFAULT_API_KEY = "" # 默认空:不校验 API Key(任意值放行)
# 参考音频(用于统一音色,同一人物说的一段话 + 对应文本)
# 如果配置了这对参数,所有合成都会克隆该音色,各句音色保持一致
@@ -161,6 +163,37 @@ def _build_sse(data: dict) -> str:
return f"data: {json.dumps(data, ensure_ascii=False)}\n\n"
def _check_api_key(request: Request) -> str | None:
"""校验请求 API Key:服务未配置 key(空)时放行任意值;否则校验 Authorization / X-API-Key。
返回 None 表示通过,否则返回错误提示文本。
"""
api_key = _model_args.get("api_key", "")
if not api_key:
return None
header = request.headers.get("Authorization", "")
if header.startswith("Bearer "):
provided = header[len("Bearer "):].strip()
else:
provided = request.headers.get("X-API-Key", "").strip()
if provided == api_key:
return None
return "API Key 无效或缺失"
def _check_model_name(body: dict) -> str | None:
"""校验请求 modelname:服务未配置模型名(空)时放行任意值;否则要求 body.model 匹配。
返回 None 表示通过,否则返回错误提示文本。
"""
model_name = _model_args.get("model_name", "")
if not model_name:
return None
if str(body.get("model") or "").strip() == model_name:
return None
return f"模型名不匹配: 期望 '{model_name}',收到 '{body.get('model') or ''}'"
def _stream_audio_events(text: str, request_id: str, model_path: str):
"""生成 SSE 音频流事件(同步,在独立线程中运行)。"""
import numpy as np
@@ -237,7 +270,7 @@ def _stream_audio_events(text: str, request_id: str, model_path: str):
# ======================== FastAPI ========================
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
from fastapi.responses import JSONResponse, StreamingResponse
@asynccontextmanager
@@ -260,12 +293,22 @@ app = FastAPI(title="VoxCPM Streaming TTS", lifespan=lifespan)
@app.post("/stream_tts")
async def stream_tts(request: Request):
"""流式 TTS 端点:接收文本,SSE 流式返回 PCM f32le 音频块。"""
# 鉴权校验(服务配置了 API Key 时)
auth_error = _check_api_key(request)
if auth_error:
return JSONResponse(status_code=401, content={"detail": auth_error})
try:
body = await request.json()
except Exception:
form = await request.form()
body = dict(form)
# 模型名校验(服务配置了模型名时)
model_error = _check_model_name(body)
if model_error:
return JSONResponse(status_code=400, content={"detail": model_error})
text = (body.get("text") or "").strip()
request_id = body.get("request_id") or f"tts-{int(time.time() * 1000)}"
@@ -317,9 +360,15 @@ if __name__ == "__main__":
parser = argparse.ArgumentParser(description="VoxCPM Streaming TTS Server")
parser.add_argument("--port", type=int, default=DEFAULT_PORT, help=f"服务端口(默认 {DEFAULT_PORT})")
parser.add_argument("--model-path", type=str, default=DEFAULT_MODEL_PATH, help="VoxCPM 模型目录")
parser.add_argument("--model-name", type=str, default=DEFAULT_MODEL_NAME,
help=f"服务模型名(校验请求 body.model;默认 {DEFAULT_MODEL_NAME},留空则不校验)")
parser.add_argument("--api-key", type=str, default=DEFAULT_API_KEY,
help="API Key(校验 Authorization: Bearer <key> 或 X-API-Key;默认空=任意值放行)")
args = parser.parse_args()
_model_args["model_path"] = args.model_path
_model_args["model_name"] = args.model_name
_model_args["api_key"] = args.api_key
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=args.port)