Models / Mixtral 8x7B Instruct

Mixtral 8x7B Instruct

MoE enthusiast
HuggingFace → HuggingBay → Mistral AI → Find for my hardware

46.7B total, 12.9B active per token (8x7B MoE). Requires ~24GB for Q4_K_M.

coding chat general
Parameters
46.7B
Context
32k
License
apache 2.0
Developer
Mistral AI
Origin
🇫🇷 France
Released
Dec 2023

Scores

Coding
62
Reasoning
65
General
68

Related models

Mistral Nemo 12B
12.2B consumer 7GB min RAM $0.02/M in Jul 18, 2024

Save your hardware and every model page answers the real question: will it run on your machine, and how fast?

Join free - save your rig →

Run it locally

Per-quant memory needs and a static "can you run it?" reference - no rig entry required

Q3_K_M
19.0GB 19.0GB min 21.0GB rec
Compact, moderate quality loss
Q4_K_M
26.5GB 26.5GB min 28.3GB rec
Balanced - the usual local sweet spot

The reference hardware

Schematic of the NVIDIA Jetson Orin NX 16GB reference rig - 16GB unified memory, 102 GB/s aggregate bandwidth
Schematic of the Single GTX 1080 Ti (11GB) reference rig - 11GB VRAM, 484 GB/s aggregate bandwidth
Schematic of the 4x H100 80GB (320GB) reference rig - 320GB VRAM, 13400 GB/s aggregate bandwidth
Schematic of the NVIDIA DGX Station 748GB reference rig - 748GB unified memory, 8000 GB/s aggregate bandwidth
Schematic of the 8x RTX 3090 rack (192GB) reference rig - 192GB VRAM, 7489 GB/s aggregate bandwidth
Schematic of the 4x RTX 5090 (128GB) reference rig - 128GB VRAM, 7168 GB/s aggregate bandwidth
Schematic of the AMD Instinct MI300X (192GB) reference rig - 192GB VRAM, 5324 GB/s aggregate bandwidth
Schematic of the 4x RTX 4090 (96GB) reference rig - 96GB VRAM, 4032 GB/s aggregate bandwidth
Schematic of the 2x RTX 5090 (64GB) reference rig - 64GB VRAM, 3584 GB/s aggregate bandwidth
Schematic of the 2x RTX 3090 (48GB) reference rig - 48GB VRAM, 1872 GB/s aggregate bandwidth
Schematic of the Single RTX 5090 (32GB) reference rig - 32GB VRAM, 1792 GB/s aggregate bandwidth
Schematic of the RTX PRO 6000 Blackwell (96GB) reference rig - 96GB VRAM, 1792 GB/s aggregate bandwidth
Schematic of the Mac Studio M4 Ultra 192GB reference rig - 192GB unified memory, 1092 GB/s aggregate bandwidth
Schematic of the Mac Studio M4 Ultra 512GB reference rig - 512GB unified memory, 1092 GB/s aggregate bandwidth
Schematic of the Single RTX 4090 (24GB) reference rig - 24GB VRAM, 1008 GB/s aggregate bandwidth
Schematic of the MacBook Pro M5 Max 128GB reference rig - 128GB unified memory, 614 GB/s aggregate bandwidth
Schematic of the Dual EPYC 9004 + 768GB DDR5-4800 reference rig - 768GB unified memory, 460 GB/s aggregate bandwidth
Schematic of the DGX Spark 128GB unified reference rig - 128GB unified memory, 273 GB/s aggregate bandwidth
Schematic of the Ryzen AI Max+ 395 128GB reference rig - 128GB unified memory, 256 GB/s aggregate bandwidth
Schematic of the Jetson AGX Orin 64GB reference rig - 64GB unified memory, 204 GB/s aggregate bandwidth
Schematic of the Epyc + 512GB DDR4-3200 + 2x RTX 3090 reference rig - 560GB unified memory, 204 GB/s aggregate bandwidth
Schematic of the Epyc + 512GB DDR4-2400 + 2x RTX 3090 reference rig - 560GB unified memory, 153 GB/s aggregate bandwidth

22 reference configs, drawn in-house. Scroll for more.

Can you run it? - reference rigs

Rig Q3_K_M Q4_K_M
NVIDIA Jetson Orin NX 16GB no -> cloud no -> cloud
Single GTX 1080 Ti (11GB) no -> cloud no -> cloud
4x H100 80GB (320GB) fast 1010.1t/s fast 786.7t/s
NVIDIA DGX Station 748GB fast 603.0t/s fast 469.7t/s
8x RTX 3090 rack (192GB) fast 564.6t/s fast 439.7t/s
4x RTX 5090 (128GB) fast 540.3t/s fast 420.8t/s
AMD Instinct MI300X (192GB) fast 401.4t/s fast 312.6t/s
4x RTX 4090 (96GB) fast 303.9t/s fast 236.7t/s
2x RTX 5090 (64GB) fast 270.2t/s fast 210.4t/s
2x RTX 3090 (48GB) fast 141.1t/s fast 109.9t/s
Single RTX 5090 (32GB) fast 135.1t/s fast 105.2t/s
RTX PRO 6000 Blackwell (96GB) fast 135.1t/s fast 105.2t/s
Mac Studio M4 Ultra 192GB fast 89.8t/s fast 69.9t/s
Mac Studio M4 Ultra 512GB fast 89.8t/s fast 69.9t/s
Single RTX 4090 (24GB) fast 76.0t/s offload
MacBook Pro M5 Max 128GB fast 50.5t/s fast 39.3t/s
Dual EPYC 9004 + 768GB DDR5-4800 fast 34.7t/s fast 27.1t/s
DGX Spark 128GB unified fast 20.6t/s ok 16.0t/s
Ryzen AI Max+ 395 128GB ok 19.3t/s ok 15.0t/s
Jetson AGX Orin 64GB ok 15.4t/s ok 12.0t/s
Epyc + 512GB DDR4-3200 + 2x RTX 3090 ok 15.4t/s ok 12.0t/s
Epyc + 512GB DDR4-2400 + 2x RTX 3090 ok 11.6t/s ok 9.0t/s

Fit tiers use the same will-it-run logic as the rig finder. For comfortable fits, the badge reflects decode speed: fast >=20 t/s, ok 8-20 t/s, slow <8 t/s. t/s is a bandwidth estimate, not a measured benchmark.

Download options

Q3_K_M official -10% vs fp16
19.0GB dl 19.0GB min 21.0GB rec
REC RAM vs largest quant
Q4_K_M official -5% vs fp16
26.5GB dl 26.5GB min 28.3GB rec
REC RAM vs largest quant

Or run it in the cloud

No per-token API provider pricing tracked for Mixtral 8x7B Instruct yet. For flagship list prices, see the calculator.

See who runs Mistral AI in production →

PRICE HISTORY

Inference cost over time

Data accumulates from the first daily sync - longer ranges populate over time. Prices come from OpenRouter snapshots, not a historical API.

Loading price history...