NVIDIA Jetson AGX Orin 64GB

NVIDIA’s embedded module family, three generations on this site: Orin Nano Super (8GB, 68 GB/s, $249, 2024 - the “Super” refresh of the 2023 board), Orin NX 16GB (102 GB/s, $499, 2023), and AGX Orin 64GB (205 GB/s, $899, 2022). All run JetPack Linux on ARM with CUDA.

What it does well:

• Power envelope: 10-60W total, passively or lightly cooled - the only CUDA platform here that runs on a battery or a solar rig.
• CUDA on a budget: $249 buys a real NVIDIA stack for 1-8B models, with the full llama.cpp and MNN toolchain.

Where it falls short: 68 GB/s on the Nano Super caps single-stream decode near single-digit tok/s on 7B-class models; 8-64GB means only small dense models or heavily quantized MoE fit.

Run it locally: Jetson AI Lab containers, Ollama, or llama.cpp on JetPack. The modeldex results rank checkpoints by fit and estimated tok/s.