INT4
INT4 Quantization
Precision, memory, context and runtime fit are evaluated from release evidence before an order becomes available.
Quantization
BF16 / FP16Source
FP8 / INT8Estimated VRAM
INT4Validated per release
GPU targeting
ModelWeights and KV cache
Validation stateContext and concurrency
Runtime packagingvLLM, SGLang
Quality retention target
QuantizationEstimated until measured during engineering
Estimated VRAMValidated per release
Runtime packagingMeasured