GLM-5.3 Flash

Architecture and calculation

Cache layout: vLLM compressed MLA + KDA, BF16 cache.

FLASHINFER_MLA_SPARSE_SM90 supports BF16 and 512-element no-RoPE geometry; also supported by Blackwell FlashInfer sparse backend. Pooled index storage requires token block size multiple of128. ceil(T/4) reserves the open pool as a conservative extra index entry.

11 MLA layers retain 512-element latents; pooled DSA indices retain one 132-byte entry per four tokens plus four raw key/gate tail slots. 34 KDA layers keep FP32 64×128×128 recurrent matrices and BF16 convolution states. Cache tensor payload for vLLM with BF16 main cache, FP32 recurrent state, no speculative decoding, and no prefix-state checkpoint copies. Page padding, attention workspaces and allocator reserves are additional. Sparse top-k selection reduces computation, not retention of the main full-context cache.

11 compressed-attention layers store 512 latent + 0 position values per token.

• Pooled DSA index keys and scales: 11 × 132 values × 1 bytes per 4 tokens, per request.
• DSA raw key and gate tail: 11 × 256 values × 2 bytes × 4 fixed rows, per request.
• KDA recurrent matrices: 34 × 1048576 values × 4 bytes × 1 fixed rows, per request.
• KDA convolution states: 34 × 73728 values × 2 bytes × 1 fixed rows, per request.

Precision is a storage estimate. Quantized files include scales and some higher-precision tensors; use the actual file size when known. A format also needs backend and hardware support. Full checkpoint count includes media/auxiliary weights when present; a text-only conversion can be smaller. Image/audio encoder working memory is not included. Full checkpoint count may include optional MTP or draft weights; cached speculative tokens are extra.

License: MIT. See the publisher’s terms.