Audio version — Estimated duration: 7 min 44 sec

Common Issues & Solutions

Build Errors

”cmake is not recognized”

Cause: CMake isn’t in your PATH.

Fix:

1. Re-run the CMake installer and check “Add CMake to the system PATH”.
2. Restart your terminal.
3. See Environment Variables for manual PATH setup.

”git is not recognized”

Cause: Git isn’t in your PATH.

Fix: Same as CMake above — reinstall Git with PATH option, or add manually.

”CUDA not found” / “CMAKE_CUDA_COMPILER not found”

Cause: You used -DGGML_CUDA=ON but CUDA Toolkit isn’t installed or not in PATH.

Fix:

1. Install the NVIDIA CUDA Toolkit.
2. Ensure nvcc --version works in a new terminal.
3. Restart your terminal and re-run CMake.

Alternative: Build without CUDA (cmake .. without -DGGML_CUDA=ON) and use CPU or another backend.

”HIP not found” during AMD build

Cause: HIP SDK isn’t installed or environment isn’t set up.

Fix:

1. Install AMD HIP SDK.
2. Verify with hipconfig --version.

”icpx not found” during Intel build

Cause: oneAPI environment not activated.

Fix: Run the oneAPI setvars.bat or open “Intel oneAPI Command Prompt” from the Start Menu.

Build fails with syntax errors / weird C++ errors

Cause: Your Visual Studio installation is missing the C++ workload, or you’re not using the Developer Command Prompt.

Fix:

1. Open Visual Studio Installer → Modify → Check “Desktop development with C++“.
2. Use “Developer Command Prompt for VS” from Start Menu.

Runtime Errors

”llama_model_load: error loading model” / “magic number mismatch”

Cause: The file isn’t a valid GGUF model, or it’s corrupted.

Fix:

• Ensure you downloaded a .gguf file, not raw safetensors.
• Re-download the file.
• Try a different model/provider (e.g., Bartowski on Hugging Face).

“failed to allocate” / “CUDA out of memory”

Cause: Your GPU doesn’t have enough VRAM for the model + context.

Fix:

• Use a smaller quantization (Q4_K_M → Q3_K_M → Q2_K).
• Reduce -ngl to offload fewer layers.
• Reduce -c (context size) to use less VRAM.
• Use a smaller model (e.g., 3B instead of 8B).

“out of memory” on CPU

Cause: The model + context exceeds your system RAM.

Fix:

• Use a smaller quantization.
• Reduce context size (-c 2048 instead of 8192).
• Close other applications.

Model loads but gives gibberish / repeats the prompt

Cause: Wrong prompt template.

Fix:

• Use the correct prompt template for your model.
• Try -cnv (conversation mode) with the --template flag if your build supports it.
• Use llama-server — it handles templates automatically.

Very slow generation (< 5 tokens/second)

Cause: No GPU acceleration, or GPU not properly configured.

Fix:

• Ensure you’re using -ngl to offload layers to GPU.
• Check the startup logs for “CUDA” / “HIP” / “SYCL” / “Vulkan” — if you don’t see them, your GPU backend isn’t working.
• See Backend Comparison to pick the right backend.
• Set -t to your physical core count.

”No language model is loaded” in llama-server

Cause: The server started without a model (you need to provide -m).

Fix: Stop the server and restart with the -m flag pointing to your GGUF file.

Terminal shows Chinese/Unicode characters instead of English

Cause: Windows terminal encoding issue.

Fix: In PowerShell, run:

$OutputEncoding = [System.Text.Encoding]::UTF8

Performance Issues

Model uses CPU even though I have a GPU

Cause: You built without GPU support, or the GPU backend failed.

Fix:

1. Check the startup logs — do you see “CUDA” / “HIP” / “SYCL” / “Vulkan”?
2. If not, rebuild with the correct -DGGML_*=ON flag.
3. Verify your GPU is detected: run nvidia-smi (NVIDIA), hipconfig (AMD), or check Device Manager (Intel).

GPU utilization is low (< 50%)

Cause: Small models don’t fully utilize the GPU; or you’re bottlenecked by CPU-GPU transfer.

Fix:

• Use a larger model or larger batch size (-b 2048).
• Ensure you offloaded enough layers (-ngl 99).
• Small models (≤3B) may actually run faster on CPU with enough threads.

”out of memory” during cmake --build

Cause: Compilation requires significant RAM, especially with CUDA.

Fix:

• Reduce parallel build jobs: cmake --build . --config Release -- -j 2
• Close other memory-intensive applications.

---

Last Updated: 2026-07-19