Granite 4.2 8B
Current dense Granite for reasoning, code and multilingual tools.
Stored parameters 8.8B
Native context 128Ktokens
ArchitectureDense
Extended context: 524,288 tokens. Publisher advertises 512K extension from a 128K native context; requires validated long-context runtime settings.
Memory by precision and context
GiB · one request · FP16 cache · default single-device buffers. Extended contexts are marked *. Text inference only.
| Weight precision | Weights | 8K total | 32K total | 128K total | 256K* total | 512K* total |
|---|---|---|---|---|---|---|
| Q4 · smaller | 4.9 | 7.2 | 10.9 | 25.9 | 45.9 | 85.9 |
| Q5 · medium | 5.8 | 8.1 | 11.8 | 26.8 | 46.8 | 86.8 |
| Q8 · larger | 8.7 | 10.9 | 14.7 | 29.7 | 49.7 | 89.7 |
| FP16 / BF16 | 16.4 | 18.9 | 22.7 | 37.7 | 57.7 | 97.7 |
Hardware fit at Q4 and 8K
One request, FP16 cache. Multi-GPU and compressed or sliding caches require compatible runtime support.
View 199 hardware configurations
| Hardware | Memory needed | Available | Estimate | Open plan |
|---|---|---|---|---|
| GeForce RTX 50508 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 50608 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 5060 Ti 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 5060 Ti 16 GB16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 507012 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 5070 Ti16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 508016 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 509032 GiB · GPU | 7.2 GiB | 31.0 GiB | Room to spare | |
| GeForce RTX 3060 12 GB12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 4060 Ti 16 GB16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 4070 Ti SUPER16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 309024 GiB · GPU | 7.2 GiB | 23.0 GiB | Room to spare | |
| GeForce RTX 3090 Ti24 GiB · GPU | 7.2 GiB | 23.0 GiB | Room to spare | |
| GeForce RTX 409024 GiB · GPU | 7.2 GiB | 23.0 GiB | Room to spare | |
| GeForce RTX 40608 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 4060 Ti 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 4070 GDDR6X12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 4070 SUPER12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 4070 Ti12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 408016 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 4080 SUPER16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 5090 Laptop GPU 24 GB24 GiB · GPU | 7.2 GiB | 23.0 GiB | Room to spare | |
| GeForce RTX 5080 Laptop GPU 16 GB16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 5070 Ti Laptop GPU 12 GB12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 5070 Laptop GPU 12 GB12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 5070 Laptop GPU 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 5060 Laptop GPU 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 5050 Laptop GPU 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 4090 Laptop GPU 16 GB16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| GeForce RTX 4080 Laptop GPU 12 GB12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 4070 Laptop GPU 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 4060 Laptop GPU 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 4050 Laptop GPU 6 GB6 GiB · GPU | 7.2 GiB | 5.0 GiB | Over capacity | |
| RTX PRO 2000 Blackwell16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| RTX PRO 4000 Blackwell SFF24 GiB · GPU | 7.2 GiB | 23.0 GiB | Room to spare | |
| RTX PRO 4000 Blackwell24 GiB · GPU | 7.2 GiB | 23.0 GiB | Room to spare | |
| RTX PRO 4500 Blackwell Workstation32 GiB · GPU | 7.2 GiB | 31.0 GiB | Room to spare | |
| RTX PRO 5000 Blackwell 48 GB48 GiB · GPU | 7.2 GiB | 47.0 GiB | Room to spare | |
| RTX PRO 5000 Blackwell 72 GB72 GiB · GPU | 7.2 GiB | 71.0 GiB | Room to spare | |
| RTX PRO 6000 Blackwell Workstation96 GiB · GPU | 7.2 GiB | 95.0 GiB | Room to spare | |
| RTX PRO 6000 Blackwell Max-Q96 GiB · GPU | 7.2 GiB | 95.0 GiB | Room to spare | |
| RTX A600048 GiB · GPU | 7.2 GiB | 47.0 GiB | Room to spare | |
| RTX 6000 Ada Generation48 GiB · GPU | 7.2 GiB | 47.0 GiB | Room to spare | |
| RTX 5000 Ada Generation32 GiB · GPU | 7.2 GiB | 31.0 GiB | Room to spare | |
| A100 80 GB PCIe80 GiB · GPU | 7.2 GiB | 78.0 GiB | Room to spare | |
| A100 80 GB SXM80 GiB · GPU | 7.2 GiB | 78.0 GiB | Room to spare | |
| H100 80 GB SXM80 GiB · GPU | 7.2 GiB | 78.0 GiB | Room to spare | |
| H100 NVL 94 GB PCIe94 GiB · GPU | 7.2 GiB | 92.0 GiB | Room to spare | |
| H200 141 GB SXM141 GiB · GPU | 7.2 GiB | 139.0 GiB | Room to spare | |
| H200 NVL 141 GB PCIe141 GiB · GPU | 7.2 GiB | 139.0 GiB | Room to spare | |
| L40S48 GiB · GPU | 7.2 GiB | 46.0 GiB | Room to spare | |
| B200 180 GB SXM180 GiB · GPU | 7.2 GiB | 178.0 GiB | Room to spare | |
| B300 288 GB SXM288 GiB · GPU | 7.2 GiB | 286.0 GiB | Room to spare | |
| A4048 GiB · GPU | 7.2 GiB | 46.0 GiB | Room to spare | |
| Radeon RX 680016 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon RX 6800 XT16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon RX 6950 XT16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon RX 7600 XT16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon RX 7700 XT12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| Radeon RX 7800 XT16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon RX 7900 GRE16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon RX 7900 XT20 GiB · GPU | 7.2 GiB | 19.0 GiB | Room to spare | |
| Radeon RX 7900 XTX24 GiB · GPU | 7.2 GiB | 23.0 GiB | Room to spare | |
| Radeon RX 9060 XT 16 GB16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon RX 9070 GRE12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| Radeon RX 907016 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon RX 9070 XT16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Radeon AI PRO R970032 GiB · GPU | 7.2 GiB | 31.0 GiB | Room to spare | |
| Radeon PRO W7800 32 GB32 GiB · GPU | 7.2 GiB | 31.0 GiB | Room to spare | |
| Radeon PRO W7800 48 GB48 GiB · GPU | 7.2 GiB | 47.0 GiB | Room to spare | |
| Radeon PRO W790048 GiB · GPU | 7.2 GiB | 47.0 GiB | Room to spare | |
| Instinct MI210 PCIe64 GiB · GPU | 7.2 GiB | 62.0 GiB | Room to spare | |
| Instinct MI300X OAM192 GiB · GPU | 7.2 GiB | 190.0 GiB | Room to spare | |
| Instinct MI325X OAM256 GiB · GPU | 7.2 GiB | 254.0 GiB | Room to spare | |
| Instinct MI350X OAM288 GiB · GPU | 7.2 GiB | 286.0 GiB | Room to spare | |
| Instinct MI355X OAM288 GiB · GPU | 7.2 GiB | 286.0 GiB | Room to spare | |
| Instinct MI350P PCIe144 GiB · GPU | 7.2 GiB | 142.0 GiB | Room to spare | |
| Arc Pro B5016 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Arc Pro B6024 GiB · GPU | 7.2 GiB | 23.0 GiB | Room to spare | |
| Arc Pro B6532 GiB · GPU | 7.2 GiB | 31.0 GiB | Room to spare | |
| Arc Pro B7032 GiB · GPU | 7.2 GiB | 31.0 GiB | Room to spare | |
| Arc A770 16 GB16 GiB · GPU | 7.2 GiB | 15.0 GiB | Room to spare | |
| Arc B57010 GiB · GPU | 7.2 GiB | 9.0 GiB | Tight fit | |
| Arc B58012 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| Mac mini M1 · 8 GB (8-core GPU)8 GiB · Unified memory | 7.2 GiB | 6.0 GiB | Over capacity | |
| Mac mini M1 · 16 GB (8-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| Mac mini M2 · 8 GB (10-core GPU)8 GiB · Unified memory | 7.2 GiB | 6.0 GiB | Over capacity | |
| Mac mini M2 · 16 GB (10-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| Mac mini M2 · 24 GB (10-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| Mac mini M2 Pro · 16 GB (16-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| Mac mini M2 Pro · 32 GB (16-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Mac mini M2 Pro · 16 GB (19-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| Mac mini M2 Pro · 32 GB (19-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| MacBook Air M3 · 8 GB (8-core GPU)8 GiB · Unified memory | 7.2 GiB | 6.0 GiB | Over capacity | |
| MacBook Air M3 · 16 GB (8-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| MacBook Air M3 · 24 GB (8-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| MacBook Air M3 · 8 GB (10-core GPU)8 GiB · Unified memory | 7.2 GiB | 6.0 GiB | Over capacity | |
| MacBook Air M3 · 16 GB (10-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| MacBook Air M3 · 24 GB (10-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| MacBook Pro M1 Pro · 16 GB (16-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| MacBook Pro M1 Pro · 32 GB (16-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Mac Studio M1 Max · 32 GB (24-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Mac Studio M1 Max · 64 GB (24-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M1 Max · 32 GB (32-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Mac Studio M1 Max · 64 GB (32-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M1 Ultra · 64 GB (48-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M1 Ultra · 128 GB (48-core GPU)128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Mac Studio M1 Ultra · 64 GB (64-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M1 Ultra · 128 GB (64-core GPU)128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Mac Studio M2 Max · 32 GB (30-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Mac Studio M2 Max · 64 GB (30-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M2 Max · 32 GB (38-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Mac Studio M2 Max · 64 GB (38-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M2 Max · 96 GB (38-core GPU)96 GiB · Unified memory | 7.2 GiB | 72.0 GiB | Room to spare | |
| Mac Studio M2 Ultra · 64 GB (60-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M2 Ultra · 128 GB (60-core GPU)128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Mac Studio M2 Ultra · 192 GB (60-core GPU)192 GiB · Unified memory | 7.2 GiB | 144.0 GiB | Room to spare | |
| Mac Studio M2 Ultra · 64 GB (76-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M2 Ultra · 128 GB (76-core GPU)128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Mac Studio M2 Ultra · 192 GB (76-core GPU)192 GiB · Unified memory | 7.2 GiB | 144.0 GiB | Room to spare | |
| MacBook Pro M3 Pro · 18 GB (18-core GPU)18 GiB · Unified memory | 7.2 GiB | 13.5 GiB | Room to spare | |
| MacBook Pro M3 Pro · 36 GB (18-core GPU)36 GiB · Unified memory | 7.2 GiB | 27.0 GiB | Room to spare | |
| MacBook Pro M3 Max · 36 GB (30-core GPU)36 GiB · Unified memory | 7.2 GiB | 27.0 GiB | Room to spare | |
| MacBook Pro M3 Max · 96 GB (30-core GPU)96 GiB · Unified memory | 7.2 GiB | 72.0 GiB | Room to spare | |
| MacBook Pro M3 Max · 48 GB (40-core GPU)48 GiB · Unified memory | 7.2 GiB | 36.0 GiB | Room to spare | |
| MacBook Pro M3 Max · 64 GB (40-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| MacBook Pro M3 Max · 128 GB (40-core GPU)128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Mac mini M4 · 16 GB (10-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| Mac mini M4 · 24 GB (10-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| Mac mini M4 · 32 GB (10-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Mac mini M4 Pro · 24 GB (16-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| Mac mini M4 Pro · 48 GB (16-core GPU)48 GiB · Unified memory | 7.2 GiB | 36.0 GiB | Room to spare | |
| Mac mini M4 Pro · 64 GB (16-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac mini M4 Pro · 24 GB (20-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| Mac mini M4 Pro · 48 GB (20-core GPU)48 GiB · Unified memory | 7.2 GiB | 36.0 GiB | Room to spare | |
| Mac mini M4 Pro · 64 GB (20-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M4 Max · 36 GB (32-core GPU)36 GiB · Unified memory | 7.2 GiB | 27.0 GiB | Room to spare | |
| Mac Studio M4 Max · 48 GB (40-core GPU)48 GiB · Unified memory | 7.2 GiB | 36.0 GiB | Room to spare | |
| Mac Studio M4 Max · 64 GB (40-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Mac Studio M4 Max · 128 GB (40-core GPU)128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Mac Studio M3 Ultra · 96 GB (60-core GPU)96 GiB · Unified memory | 7.2 GiB | 72.0 GiB | Room to spare | |
| Mac Studio M3 Ultra · 256 GB (60-core GPU)256 GiB · Unified memory | 7.2 GiB | 192.0 GiB | Room to spare | |
| Mac Studio M3 Ultra · 96 GB (80-core GPU)96 GiB · Unified memory | 7.2 GiB | 72.0 GiB | Room to spare | |
| Mac Studio M3 Ultra · 256 GB (80-core GPU)256 GiB · Unified memory | 7.2 GiB | 192.0 GiB | Room to spare | |
| Mac Studio M3 Ultra · 512 GB (80-core GPU)512 GiB · Unified memory | 7.2 GiB | 384.0 GiB | Room to spare | |
| MacBook Pro M5 · 16 GB (10-core GPU)16 GiB · Unified memory | 7.2 GiB | 12.0 GiB | Room to spare | |
| MacBook Pro M5 · 24 GB (10-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| MacBook Pro M5 · 32 GB (10-core GPU)32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| MacBook Pro M5 Pro · 24 GB (16-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| MacBook Pro M5 Pro · 48 GB (16-core GPU)48 GiB · Unified memory | 7.2 GiB | 36.0 GiB | Room to spare | |
| MacBook Pro M5 Pro · 24 GB (20-core GPU)24 GiB · Unified memory | 7.2 GiB | 18.0 GiB | Room to spare | |
| MacBook Pro M5 Pro · 48 GB (20-core GPU)48 GiB · Unified memory | 7.2 GiB | 36.0 GiB | Room to spare | |
| MacBook Pro M5 Pro · 64 GB (20-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| MacBook Pro M5 Max · 36 GB (32-core GPU)36 GiB · Unified memory | 7.2 GiB | 27.0 GiB | Room to spare | |
| MacBook Pro M5 Max · 48 GB (40-core GPU)48 GiB · Unified memory | 7.2 GiB | 36.0 GiB | Room to spare | |
| MacBook Pro M5 Max · 64 GB (40-core GPU)64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| MacBook Pro M5 Max · 128 GB (40-core GPU)128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Ryzen AI Max+ 395 · 32 GB32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Ryzen AI Max+ 395 · 64 GB64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Ryzen AI Max+ 395 · 128 GB128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Ryzen AI Max+ 392 · 32 GB32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Ryzen AI Max+ 392 · 64 GB64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Ryzen AI Max+ 392 · 128 GB128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| Ryzen AI Max+ 388 · 32 GB32 GiB · Unified memory | 7.2 GiB | 24.0 GiB | Room to spare | |
| Ryzen AI Max+ 388 · 64 GB64 GiB · Unified memory | 7.2 GiB | 48.0 GiB | Room to spare | |
| Ryzen AI Max+ 388 · 128 GB128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| NVIDIA DGX Spark · 128 GB128 GiB · Unified memory | 7.2 GiB | 96.0 GiB | Room to spare | |
| 2 × GeForce RTX 309048 GiB · Multi-GPU | 8.2 GiB | 46.0 GiB | Conditional · Room to spare | |
| 4 × GeForce RTX 309096 GiB · Multi-GPU | 10.2 GiB | 92.0 GiB | Conditional · Room to spare | |
| 2 × GeForce RTX 409048 GiB · Multi-GPU | 8.2 GiB | 46.0 GiB | Conditional · Room to spare | |
| 2 × GeForce RTX 509064 GiB · Multi-GPU | 8.2 GiB | 62.0 GiB | Conditional · Room to spare | |
| 4 × GeForce RTX 5090128 GiB · Multi-GPU | 10.2 GiB | 124.0 GiB | Conditional · Room to spare | |
| 2 × GeForce RTX 5060 Ti 16 GB32 GiB · Multi-GPU | 8.2 GiB | 30.0 GiB | Conditional · Room to spare | |
| 2 × RTX A600096 GiB · Multi-GPU | 8.2 GiB | 94.0 GiB | Conditional · Room to spare | |
| 2 × RTX PRO 5000 Blackwell 72 GB144 GiB · Multi-GPU | 8.2 GiB | 142.0 GiB | Conditional · Room to spare | |
| 2 × RTX PRO 6000 Blackwell Max-Q192 GiB · Multi-GPU | 8.2 GiB | 190.0 GiB | Conditional · Room to spare | |
| 4 × RTX PRO 6000 Blackwell Max-Q384 GiB · Multi-GPU | 10.2 GiB | 380.0 GiB | Conditional · Room to spare | |
| 2 × Radeon AI PRO R970064 GiB · Multi-GPU | 8.2 GiB | 62.0 GiB | Conditional · Room to spare | |
| 4 × Radeon AI PRO R9700128 GiB · Multi-GPU | 10.2 GiB | 124.0 GiB | Conditional · Room to spare | |
| 2 × Arc Pro B7064 GiB · Multi-GPU | 8.2 GiB | 62.0 GiB | Conditional · Room to spare | |
| 4 × Arc Pro B70128 GiB · Multi-GPU | 10.2 GiB | 124.0 GiB | Conditional · Room to spare | |
| 2 × Instinct MI210 PCIe128 GiB · Multi-GPU | 8.2 GiB | 124.0 GiB | Conditional · Room to spare | |
| 2 × H200 NVL 141 GB PCIe282 GiB · Multi-GPU | 8.2 GiB | 278.0 GiB | Conditional · Room to spare | |
| 4 × H200 NVL 141 GB PCIe564 GiB · Multi-GPU | 10.2 GiB | 556.0 GiB | Conditional · Room to spare | |
| GeForce RTX 3050 6 GB6 GiB · GPU | 7.2 GiB | 5.0 GiB | Over capacity | |
| GeForce RTX 3050 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 3060 Ti 8 GB GDDR68 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 30708 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 3070 Ti8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| GeForce RTX 3080 10 GB10 GiB · GPU | 7.2 GiB | 9.0 GiB | Tight fit | |
| GeForce RTX 3080 12 GB12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| GeForce RTX 3080 Ti12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| Radeon RX 66008 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| Radeon RX 6600 XT8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| Radeon RX 6650 XT8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| Radeon RX 6700 XT12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| Radeon RX 6750 XT12 GiB · GPU | 7.2 GiB | 11.0 GiB | Room to spare | |
| Arc A750 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity | |
| Arc A770 8 GB8 GiB · GPU | 7.2 GiB | 7.0 GiB | Over capacity |
Architecture and calculation
Text decoding cache; runtime workspaces and optional speculative caches are additional.
- 40 full-attention layers × 8 KV heads × (128 key + 128 value dimensions).
Precision estimates need quantization scales and some higher-precision tensors. Backend support and actual file sizes vary. Whole-checkpoint counts can include auxiliary weights.
License: Apache 2.0. See the publisher’s terms.