Qwen3.5 0.8B

Architecture and calculation

Gated DeltaNet states use FP32; convolution states use the model dtype. Full-attention layers alone grow with text length. MTP speculation is excluded from cache.

• 6 full-attention layers × 2 KV heads × (256 key + 256 value dimensions).

18 recurrent layers maintain a fixed state per request, using FP32 state and 16-bit convolution buffers. They do not allocate a full attention cache for every token.

Precision is a storage estimate. Quantized files include scales and some higher-precision tensors; use the actual file size when known. A format also needs backend and hardware support. Full checkpoint count includes media/auxiliary weights when present; a text-only conversion can be smaller. Image/audio encoder working memory is not included. Full checkpoint count may include optional MTP or draft weights; cached speculative tokens are extra.

License: Apache 2.0. See the publisher’s terms.