Sixi-ASI Engine 1.0.0-rc1: download it, run Qwen3.8 on a Jetson Thor, serve 32 requests at once

Radek (Sixi) · 14 November 2026 · download, measured results and benchmark scripts

Until now, running sixi-asi meant building it. The first release candidate of 1.0 is one 50 MB archive for the NVIDIA Jetson AGX Thor: unpack it, download a model with one command, and talk to Qwen3.8-27B at 44 tokens per second, or serve thirty-two requests at once at 171 tokens per second in total. Every number below was measured on the files of that archive.

tar xzf sixi-asi-1.0.0-rc1-jetson-thor-aarch64.tar.gz
cd sixi-asi-1.0.0-rc1-jetson-thor-aarch64
./sixi setup      # checks that this machine can run the release
./sixi pull       # Qwen3.8-27B and its drafter, 18.9 GB
./sixi chat       # talk to it in the terminal

NVIDIA Jetson AGX Thor, Qwen3.8-27B	1.0.0-rc1
One stream, default weights (UD-Q4_K_XL v3.1)	44.1 tok/s, 3.7x plain decoding
4 / 16 / 32 requests at once, in total	98 / 152 / 171 tok/s
llama.cpp v1.0.0 on the same machine and weights, with its built-in MTP drafts	29.3 tok/s
16 agents with an 18k-token prompt each: wait for the next turn	under 3 s
JSON answers that parse, with a schema, 120 cases	120 of 120 (103 without the constraint)
Quality gate	GSM8K 28/30, HumanEval 19/20, tool calls and a needle test pass
Qwen3.8-Flash-Next, one stream	53 tok/s on code, 45 in conversation

Total tokens per second with 4, 8, 16 and 32 concurrent requests on NVIDIA Jetson AGX Thor, Qwen3.8-27B Q4_K_M with the DSpine drafter. Version 0.10: 94.2, 113.5, 128.8, 129.4. Version 1.0.0-rc1: 102.9, 129.0, 143.7, 172.0.

What is in the archive

The engine, one command to drive it, and the scripts the numbers come from.

./sixi run answers one prompt; --json returns exactly one JSON object and --schema file.json an object that matches your schema.
./sixi serve --parallel 4 is an OpenAI-compatible server on http://127.0.0.1:8089/v1: any OpenAI client works by changing the base URL.
./sixi service install keeps that server in the background as a systemd user service, the way a local model service does. It is ready 8 to 15 seconds after a start and ./sixi chat then uses it instead of loading the model again.
Two models by name: Qwen3.8-27B with the DSpine drafter (the measured configuration) and Qwen3.8-Flash-Next with its MTP draft head. Both are Apache-2.0 and are downloaded from their authors' Hugging Face repositories, resumable and checksummed.

Answers that always parse

New in this version: response_format with json_object or a JSON schema. A model asked for JSON usually writes it, and now and then closes an answer with ") or "> instead of "}: in the test below, 7 answers in 30. With the constraint the model can only write tokens that keep the document valid, so a finished answer always parses and nothing follows its closing brace. It holds with speculative drafts and with parallel streams, and reasoning stays free: the constraint starts after it.

Same prompt, schema included	With `response_format`	Without
10 agent-style schemas, prompts in five languages	30/30 valid	23/30
30 general schemas	90/90	80/90
General schemas, sampled instead of greedy	90/90
General schemas with reasoning on	30/30

The cost is about 2% of the speed on answers the model would have written correctly anyway. One practical note: put the schema in the prompt as well. The constraint guarantees the form; the prompt is what makes the model want those keys.

Next to llama.cpp v1.0.0

llama.cpp 1.0 came out this week, and the engine now runs on it. Its release notes recommend the model's built-in MTP head for speculative decoding, so that is the comparison: the same Thor, the same weights, the same five prompts.

Decode tokens per second, one stream, Qwen3.8-27B on NVIDIA Jetson AGX Thor: Sixi-ASI 1.0.0-rc1 with the DSpine drafter on Q4_K_M 46.0, TensorRT Edge-LLM 0.11.1 DFlash2 45.4, Sixi-ASI 1.0.0-rc1 DSpine on UD-Q4_K_XL (default) 44.1, llama.cpp v1.0.0 llama-server with built-in MTP 29.3, llama.cpp v1.0.0 llama-server plain decoding 12.1

Plain decoding runs at 12.1 tok/s, llama-server with built-in MTP at 29.3, this engine at 44.1 on the default weights and 46.0 on Q4_K_M. A quantized KV cache, also recommended there, changes nothing at short contexts on this machine (27.3 tok/s with three drafts against 27.2). A caution about this kind of table: it is five prompts, and two runs whose answers differ in a few tokens can differ by several percent, so I read it to the nearest ten percent, not to the decimal.

How the numbers were taken

On the released archive, not on a development build, with nothing else on the GPU.
Greedy decoding, identical prompts for every engine, 256 generated tokens; totals for several streams include prompt loading.
The figures for 8, 16 and 32 streams are means of three runs (112.9-122.8, 145.9-155.1, 169.3-172.6 tok/s). A single run at 8 to 16 streams spreads by several percent, and after three days without a restart the machine itself was 10% slower there, for every build including the previous release. It was restarted before the final measurements.
The scripts are in the archive (scripts/), so the tables can be repeated on another Thor.

1.0.0-rc1 performs like 0.11 on one stream and on many, and adds structured answers, a newer backend and a form you can install in a minute. From this version it is released in binary form, free to use for any purpose. Full tables: docs/RESULTS.md.

Download: github.com/radeksixi/sixi-asi, release v1.0.0-rc1. Previous write-up: sixi-asi 0.11.