Strata

https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF

export HF_HUB_OFFLINE=1
source /data/D/python312_venv/bin/activate
cd /data/E/Strata-0.1.38
IQ3_S:
./setup.sh --gguf-dir /data/H/LLM/Qwen3.8-Flash-Next-GSQ-RCO-GGUF_ISTA-DASLab \
  --model IQ3_S\
  --family qwen \
  --data-dir /data/E/Strata-data \
  --context 262144 \
  --kv int8 \
  --experimental-speed-projection off \
  --vision yes \
  --port 8099
UD-Q4_K_XL:
./setup.sh --gguf-dir /data/E/Qwen3.8-Flash-Next-GGUF_unsloth \
  --model UD-Q4_K_XL \
  --family unsloth \
  --data-dir /data/E/Strata-data \
  --context 262144 \
  --kv int8 \
  --experimental-speed-projection off \
  --vision yes \
  --port 8099
IQ3_S                120-160 token/s  235520/262144死循环
UD-Q4_K_XL  70-110 token/s    115520/131072medium稳定，194560/204800medium稳定，221184/245760medium稳定，245760/262144medium稳定，245760/262144xhigh思考过多直到超限导致卡住一段时间
第一次启动时需要calibrate，添加vision支持：
nano strata-unsloth-ud-q4_k_xl.json
 "args": [
  ...
  "--vision",
  "--resident-budget-gib",
  "80",
  ...
 ]
 "fit_max_tokens": true,
 "reasoning_budget_tokens": 16000,
 "vision": {
  "exe": "/data/E/Strata-0.1.38/engine/strata-vision",
  "mmproj": "/data/E/Strata-data/models/mmproj-Qwen3.8-Flash-Next-BF16-unsloth.gguf",
  "model": "/data/E/Qwen3.8-Flash-Next-GGUF_unsloth/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf",
  "gpu": true,
  "max_tokens": 1024
 }
上面的 "reasoning_budget_tokens": 16000 用于限制思考长度，防止超长思考，很重要
后续使用这条命令启动：
.venv/bin/python -m serve.server --engine strata --config strata-unsloth-ud-q4_k_xl.json  --port 8099

cc switch参数：
Body覆盖：
{
  "max_tokens": 32000,
  "output_config": {
    "effort": "medium"
  }
}
～/.claude/settings.json
  "model": "deepseek-v4-pro-2[1M]",
  "modelSettings": {
    "deepseek-v4-pro-2": {
      "effortLevel": "medium"
    }
  }

.venv/bin/python -m serve.server --engine strata --config strata-unsloth-ud-q4_k_xl.json  --port 8099 --api-monitor
打开网页：
http://127.0.0.1:8099/api-monitor
在agent中发消息检查log中的参数已否以更新,或者手动测试：
curl http://127.0.0.1:8099/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer xxx" \
-d '{
"model":"Qwen3.8-Flash-Next",
"messages":[
 {
  "role":"user",
  "content":"介绍一下中国"
 }
],
"max_tokens": 32000,
"output_config": {
  "effort": "medium"
}
}'

全局吞吐（总 token / 总时间）:
开启vision:
平均预填充 (prefill) 3.7K tok/s, 平均解码 (decode) 77 tok/s
关闭vision:
平均预填充 (prefill) 3.8K tok/s, 平均解码 (decode) 82.4 tok/s