Audio version — Estimated duration: 5 min 50 sec
Backend Comparison — Which Installation to Choose?
Not sure which backend to pick? This guide helps you choose based on your hardware and priorities.
Quick Decision Flowchart
Do you have a GPU?
├── No → [[llama-cpp/1-installation/cpu-installation|CPU Installation]]
└── Yes → Which vendor?
├── NVIDIA → [[llama-cpp/1-installation/cuda-installation|CUDA Installation]] (best perf)
├── AMD → [[llama-cpp/1-installation/amd-installation|HIP Installation]] (best perf)
├── Intel → [[llama-cpp/1-installation/intel-installation|SYCL Installation]] (best perf)
└── Other / want one build for all → [[llama-cpp/1-installation/vulkan-installation|Vulkan Installation]]
Backend Comparison Table
| Feature | CUDA | HIP | SYCL | Vulkan | CPU |
|---|---|---|---|---|---|
| GPU Vendor | NVIDIA | AMD | Intel | Any | None |
| Relative Speed | ★★★★★ | ★★★★☆ | ★★★★☆ | ★★★☆☆ | ★★☆☆☆ |
| Setup Complexity | Medium | Medium | Medium | Low | None |
| SDK Size | ~3 GB | ~2 GB | ~4 GB | ~500 MB | N/A |
| Multi-GPU | Yes | Yes | Yes | Yes | N/A |
| VRAM Sharing | Yes (CUDA) | Yes (HIP) | Yes (SYCL) | Yes (Vulkan) | System RAM |
| Portable Build | No | No | No | Yes | Yes |
Detailed Recommendations
NVIDIA GPU (Any)
Pick: CUDA. It’s the fastest and most mature backend. Vulkan also works but is 10-20% slower.
AMD GPU (Radeon RX 6000+)
Pick: HIP for best performance. If the HIP SDK gives you trouble or you need a single build for multiple AMD cards, Vulkan is a good fallback.
AMD GPU (Older / RX 5000 and below)
Pick: Vulkan. Older AMD GPUs have limited HIP support. Vulkan provides broad compatibility.
Intel Arc A-Series (A770, A750, etc.)
Pick: SYCL — it’s the native backend and gives the best performance. Vulkan works too but is slower.
Intel Integrated (Iris Xe, UHD)
Pick: SYCL if you need GPU acceleration. Vulkan is also an option. Note that iGPUs share system RAM, so performance is limited regardless.
Multiple Different GPUs (e.g., NVIDIA + Intel iGPU)
Pick: Vulkan — it’s the only backend that can target both GPUs with a single build. You can select which GPU at runtime via GGML_VULKAN_DEVICE.
Just Want to Try It Out / Minimal Hassle
Pick: CPU — no SDKs, no drivers to install. Just build and run. Then upgrade to a GPU backend when you want more speed.
Performance Expectations
Real-world speed depends on your GPU, model size, quantization, and context length. These are rough guidelines for a 7B-8B model at Q4_K_M:
| Hardware | Backend | Tokens/sec |
|---|---|---|
| CPU only (8 cores) | CPU | ~5-15 t/s |
| NVIDIA RTX 4090 | CUDA | ~80-120 t/s |
| NVIDIA RTX 3060 | CUDA | ~30-50 t/s |
| AMD RX 7900 XTX | HIP | ~60-90 t/s |
| Intel Arc A770 | SYCL | ~40-60 t/s |
| Intel Iris Xe | SYCL | ~10-20 t/s |
| Any mid-range GPU | Vulkan | ~80-90% of native |
Can I Build Multiple Backends?
Yes. You can pass multiple -DGGML_* flags to CMake to enable several backends in one build:
cmake .. -DGGML_CUDA=ON -DGGML_VULKAN=ONllama.cpp will select the best available backend at runtime. However, this increases build time and binary size. For most users, pick one backend.
Last Updated: 2026-07-19