AMD Installation (AMD GPUs)
If you have an AMD graphics card, you can accelerate model inference by utilizing HIP (Heterogeneous-interface for Portability). On Windows, this is achieved using the AMD HIP SDK, which allows llama.cpp to leverage your AMD GPU.
🧠 Key Concepts for Beginners
What is a GPU?
A GPU (Graphics Processing Unit) is a specialized processor designed to handle many small, repetitive tasks simultaneously. While a CPU is like a brilliant mathematician who can solve one complex problem at a time, a GPU is like a thousand students who can all solve simple arithmetic problems at the exact same moment. Large Language Models involve massive amounts of simple math, making them perfect for GPUs.
What is AMD HIP?
HIP (Heterogeneous-interface for Portability) is a software layer that acts like a universal translator. It allows code written for one type of hardware (like NVIDIA) to be easily “translated” and run on another type of hardware (like AMD). For you, it means you can use the power of your AMD card to run AI models efficiently.
🛠️ Prerequisites
In addition to the core tools mentioned in the Prerequisites guide, you must have:
- AMD GPU: A compatible AMD graphics card.
- AMD HIP SDK: This provides the necessary libraries to run HIP code on AMD hardware.
- Download: AMD HIP SDK Downloads.
- Installation: Follow the standard Windows installer.
- Verification: Open a terminal and type:
hipconfig --version
🚀 Step-by-Step Installation
The process is similar to the CPU and CUDA builds, but with a specific CMake flag for HIP support.
1. Clone the Repository
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp2. Create a Build Directory
mkdir build
cd build3. Configure the Build with HIP Support
We tell CMake to enable the HIP/ROCm backend using the -DGGML_HIPBLAS=ON flag.
cmake .. -DGGML_HIPBLAS=ON4. Compile the Project
cmake --build . --config Release✅ Verification
Once the build is complete, verify that the executable was built with HIP support.
To verify the build, run:
./bin/Release/llama-cli.exe --helpTo confirm HIP is actually working: When you run a model (see CLI Usage), check the initial log output in your terminal. You should see entries indicating that HIP or hipBLAS is being used and that your AMD GPU has been detected.
💡 Troubleshooting
- “HIP not found” error during CMake: This is usually due to the HIP SDK not being correctly added to your system’s PATH. Re-install the SDK or manually add the appropriate
bindirectory to your PATH, then restart your terminal. - Compatibility: Ensure your AMD graphics card and drivers are up to date to work with the latest version of the HIP SDK.
Last Updated: 2026-05-03