auto_agent/llm_server/README.md

This bundle is self-contained for running Qwen/Qwen3.6-35B-A3B-FP8 with local/vllm-qwen36-nvfp4:v0.19.1-cu130 on another Spark machine.

Contents:

• image/: Docker image tarball
• modelscope/: bundled ModelScope cache for the model weights
• chat_template.jinja: chat template mounted into the container
• start_vllm.sh: loads the image if needed and starts the container
• stop_vllm.sh: stops the container

Usage after extraction:

cd vllm-qwen36-bundle
chmod +x start_vllm.sh stop_vllm.sh
./start_vllm.sh

Optional overrides:

• HOST_PORT=8001 ./start_vllm.sh
• CONTAINER_NAME=my-vllm ./start_vllm.sh

The startup script recreates the Qwen/Qwen3.6-35B-A3B-FP8 symlink inside the bundled ModelScope cache so the bundle remains portable across machines.

audio

前台运行，Ctrl+C 直接停止

./start_audio.sh

后台运行

nohup ./start_audio.sh > /dev/null 2>&1 &

后台运行时用 stop 停止

./stop_audio.sh