36 lines
982 B
Markdown
36 lines
982 B
Markdown
This bundle is self-contained for running `Qwen/Qwen3.6-35B-A3B-FP8` with
|
|
`local/vllm-qwen36-nvfp4:v0.19.1-cu130` on another Spark machine.
|
|
|
|
Contents:
|
|
- `image/`: Docker image tarball
|
|
- `modelscope/`: bundled ModelScope cache for the model weights
|
|
- `chat_template.jinja`: chat template mounted into the container
|
|
- `start_vllm.sh`: loads the image if needed and starts the container
|
|
- `stop_vllm.sh`: stops the container
|
|
|
|
Usage after extraction:
|
|
|
|
```bash
|
|
cd vllm-qwen36-bundle
|
|
chmod +x start_vllm.sh stop_vllm.sh
|
|
./start_vllm.sh
|
|
```
|
|
|
|
Optional overrides:
|
|
- `HOST_PORT=8001 ./start_vllm.sh`
|
|
- `CONTAINER_NAME=my-vllm ./start_vllm.sh`
|
|
|
|
The startup script recreates the `Qwen/Qwen3.6-35B-A3B-FP8` symlink inside the
|
|
bundled ModelScope cache so the bundle remains portable across machines.
|
|
|
|
|
|
*************audio*************
|
|
# 前台运行，Ctrl+C 直接停止
|
|
./start_audio.sh
|
|
|
|
# 后台运行
|
|
nohup ./start_audio.sh > /dev/null 2>&1 &
|
|
|
|
# 后台运行时用 stop 停止
|
|
./stop_audio.sh
|