Measuring the first token

A startup benchmark is useful only when the timer boundary and the success condition match the operator experience it is meant to describe. ColdSnap’s accepted August qualification measures time to the first non-empty streamed token—not process creation, a health endpoint, or a model loader milestone.

Timer boundary

For each run, time begins at Docker container State.StartedAt and ends when the serving workload emits its first non-empty streamed model token. Artifact preparation, verification, and capsule pulls completed before container start are outside this measurement.

That makes the result a focused activation-runtime comparison. It does not claim to measure every piece of an end-to-end scheduler placement or image-distribution workflow.

Cache discipline

Linux page cache was cleared on both hosts before every sample. Each reported value is the median of three runs. The matrix contains vanilla, recovery, and native paths across the declared engine and model combinations.

Every accepted restore sample had to return the exact validation response. The August matrix recorded 36 passing ColdSnap restore samples and six passing vanilla controls.

Reading the headline

The strongest accepted case compares a 229.914-second vanilla vLLM start with a 33.149-second n610 native restore for Qwen3.8 27B FP8 on two DGX Spark GB10 hosts. That is the source of the 6.9× headline.

It is not a universal multiplier. Different engines, models, snapshot drivers, weight providers, runtime images, and topologies produce different results and each combination needs its own qualification.

The benchmark hub lists the exact accepted headline cases and links to the underlying qualification material.