Llama 3.1 8B Instruct
consumerAI-generated content marks
The provider reports that this model does not add embedded watermarks or provenance metadata to generated output.
- 8.0B
- 128k
- llama 3
- 🇺🇸 USA
- Jul 2024
Scores
Score per dollar
3100 pts per $/M input
general_score (62) divided by cheapest input price ($0.02/M). Higher is better value. See live pricing.
Related models
Save your hardware and every model page answers the real question: will it run on your machine, and how fast?
Join free - save your rig →Run it locally
Per-quant memory needs and a static "can you run it?" reference - no rig entry required
The reference hardware
22 reference configs, drawn in-house. Scroll for more.
Can you run it? - reference rigs
| Rig | Q4_K_M | Q8_0 | FP16 |
|---|---|---|---|
| 4x H100 80GB (320GB) | fast 1125.5t/s | fast 733.5t/s | fast 408.4t/s |
| NVIDIA DGX Station 748GB | fast 672.0t/s | fast 437.9t/s | fast 243.8t/s |
| 8x RTX 3090 rack (192GB) | fast 629.1t/s | fast 410.0t/s | fast 228.2t/s |
| 4x RTX 5090 (128GB) | fast 602.1t/s | fast 392.4t/s | fast 218.4t/s |
| AMD Instinct MI300X (192GB) | fast 447.3t/s | fast 291.5t/s | fast 162.3t/s |
| 4x RTX 4090 (96GB) | fast 338.7t/s | fast 220.7t/s | fast 122.9t/s |
| 2x RTX 5090 (64GB) | fast 301.0t/s | fast 196.2t/s | fast 109.2t/s |
| 2x RTX 3090 (48GB) | fast 157.3t/s | fast 102.5t/s | fast 57.1t/s |
| Single RTX 5090 (32GB) | fast 150.5t/s | fast 98.1t/s | fast 54.6t/s |
| RTX PRO 6000 Blackwell (96GB) | fast 150.5t/s | fast 98.1t/s | fast 54.6t/s |
| Mac Studio M4 Ultra 192GB | fast 100.1t/s | fast 65.2t/s | fast 36.3t/s |
| Mac Studio M4 Ultra 512GB | fast 100.1t/s | fast 65.2t/s | fast 36.3t/s |
| Single RTX 4090 (24GB) | fast 84.7t/s | fast 55.2t/s | fast 30.7t/s |
| MacBook Pro M5 Max 128GB | fast 56.3t/s | fast 36.7t/s | fast 20.4t/s |
| Single GTX 1080 Ti (11GB) | fast 40.7t/s | fast 26.5t/s | offload |
| Dual EPYC 9004 + 768GB DDR5-4800 | fast 38.7t/s | fast 25.2t/s | ok 14.0t/s |
| DGX Spark 128GB unified | fast 22.9t/s | ok 14.9t/s | ok 8.3t/s |
| Ryzen AI Max+ 395 128GB | fast 21.5t/s | ok 14.0t/s | slow 7.8t/s |
| Jetson AGX Orin 64GB | ok 17.2t/s | ok 11.2t/s | slow 6.2t/s |
| Epyc + 512GB DDR4-3200 + 2x RTX 3090 | ok 17.2t/s | ok 11.2t/s | slow 6.2t/s |
| Epyc + 512GB DDR4-2400 + 2x RTX 3090 | ok 12.9t/s | ok 8.4t/s | slow 4.7t/s |
| NVIDIA Jetson Orin NX 16GB | ok 8.6t/s | slow 5.6t/s | tight |
Fit tiers use the same will-it-run logic as the rig finder. For comfortable fits, the badge reflects decode speed: fast >=20 t/s, ok 8-20 t/s, slow <8 t/s. t/s is a bandwidth estimate, not a measured benchmark.
Download options
Or run it in the cloud
Live per-provider pricing, throughput and uptime - refreshed about 11 hours ago via OpenRouter. Click a column to sort.
some pricing may be stale - last verified 2026-11-13
| Provider | Type | Input $/M | Output $/M | Cache $/M | Tok/s | Latency | Uptime | Value |
|---|---|---|---|---|---|---|---|---|
|
DeepInfra
|
API | 0.02 | 0.04 | - | - | - | 100.00% | best uptime |
|
Cloudflare
|
API | 0.15 | 0.29 | - | - | - | 100.00% | |
|
Together AI
stale
|
API | 0.18 | 0.18 | - | - | - | - | |
|
Fireworks AI
stale
|
API | 0.20 | 0.20 | - | - | - | - | |
|
CoreWeave
|
API | 0.22 | 0.22 | 0.220 | - | - | 100.00% | |
| API | 0.05 | 0.08 | 0.025 | - | - | 96.67% |
Default order: throughput among 95%+ uptime providers, then latency; subscriptions last. Sort by any column. Subscription rows show $/mo in the Value column - per-token columns are "-". Affiliate links are marked sponsored / nofollow. Confirm current pricing on the provider's site before committing.
Detailed API pricing page + JSON endpoint →
See who runs Meta in production →
Inference cost over time
Data accumulates from the first daily sync - longer ranges populate over time. Prices come from OpenRouter snapshots, not a historical API.
Tokenstead community
Need a second opinion on this model?
Ask the people running it on real hardware, then post your tok/s and fit notes so the next reader has a data point.
Something wrong or stale? Send a correction.