LIVE
Mistral Makes Large 4 Available as Open Weights29/10/26 · Mistral AI|Gemini 4 Carbon tops the Artificial Analysis Intelligence Index v529/10/26 · Google|Mistral Large 4 now available on Hugging Face29/10/26 · Mistral AI|Google Introduces "Gemini 4 Carbon" as Next Step in Its Model Lineup28/10/26 · Google|DeepMind unveils Genie 4, a new generation of world model27/10/26 · Google DeepMind|Anthropic files IPO prospectus, with risk disclosures taking center stage26/10/26 · Anthropic|Robot policy trained only in simulation matches top real-world systems26/10/26|OpenAI unveils GPT-6.1 Terra, a model built for computer use26/10/26 · OpenAI|New benchmark checks whether LLMs actually test their temporal medical claims29/10/26|An ODE-based sampling method speeds up discrete diffusion models29/10/26|OpenAI revises its Model Spec with five guiding values for AI behavior29/10/26 · OpenAI|Inside Proton's internal clash over its AI pivot29/10/26 · Proton|Mistral Makes Large 4 Available as Open Weights29/10/26 · Mistral AI|Gemini 4 Carbon tops the Artificial Analysis Intelligence Index v529/10/26 · Google|Mistral Large 4 now available on Hugging Face29/10/26 · Mistral AI|Google Introduces "Gemini 4 Carbon" as Next Step in Its Model Lineup28/10/26 · Google|DeepMind unveils Genie 4, a new generation of world model27/10/26 · Google DeepMind|Anthropic files IPO prospectus, with risk disclosures taking center stage26/10/26 · Anthropic|Robot policy trained only in simulation matches top real-world systems26/10/26|OpenAI unveils GPT-6.1 Terra, a model built for computer use26/10/26 · OpenAI|New benchmark checks whether LLMs actually test their temporal medical claims29/10/26|An ODE-based sampling method speeds up discrete diffusion models29/10/26|OpenAI revises its Model Spec with five guiding values for AI behavior29/10/26 · OpenAI|Inside Proton's internal clash over its AI pivot29/10/26 · Proton|
AppRisingofficial publisher

llama.cpp

App : A product you use directly

The C/C++ inference engine behind Ollama, LM Studio, and most local AI tools.

Indispensable behind the scenes, worth using directly only if you are comfortable with the command line.

GitHub · 132,260 ★
Overall score
8.9 /10
Security: nothing to flag
Utility
8.5
Reliability
10.0
Security
No caveat on record
Ease
Technical · Technical

Security reflects where data goes, requested permissions and incidents on record at the verification date. It is not a guarantee of zero risk: check the deployment mode and the connections you enable.

What it does, concretely

Running a large language model on your own computer usually requires significant resources and complex setup. llama.cpp solves this by providing an inference engine written in pure C/C++, with no external dependencies, capable of running LLM and VLM models on ordinary hardware, both locally and in the cloud.

In practice, llama.cpp loads models in GGUF format and runs them by leveraging the CPU, GPU, or both simultaneously. It supports quantization from 1.5 to 8 bits to reduce memory requirements, and relies on various backends: Metal for Apple Silicon, CUDA for Nvidia GPUs, HIP for AMD, Vulkan, OpenCL, and more. A command-line interface and an OpenAI-compatible API server are included out of the box.

llama.cpp is aimed at developers, researchers, and advanced users who want to run models locally with full control over configuration. It is also the engine running behind the scenes in tools like Ollama or LM Studio, without end users needing to install it directly.

It is not suited for people looking for a simple graphical interface or who are not comfortable with the command line. Installation and configuration require a minimum level of technical knowledge.

What it offers

  • ·Local inference of LLM and VLM models in pure C/C++, with no external dependencies.
  • ·Model quantization from 1.5 to 8 bits to reduce memory usage and speed up execution.
  • ·Support for many hardware backends: Apple Silicon (Metal), Nvidia GPU (CUDA), AMD GPU (HIP), Vulkan, OpenCL, RISC-V, and more.
  • ·Hybrid CPU+GPU inference to run models larger than the available VRAM.
  • ·Built-in server compatible with the OpenAI API, accessible via the llama serve command.
  • ·Command-line interface (llama cli) allowing you to download and run a model directly from Hugging Face.
  • ·Built-in web UI available when using the llama serve server.
  • ·Cross-platform support: macOS, Windows, Linux, Docker, and even Android.

In practice

Price
Free
Commercial use
Commercial use: yes
Available on
macos, windows, linux, cli
Licence
MIT

Security and data

Rating
Security: nothing to flag
Your data
Data stays local
Maintenance
Active
Last release
15 October 2026

Links and repository checked on 25 October 2026 · added on 1 March 2023

How to install llama.cpp

1. Go to the GitHub releases page (https://github.com/ggml-org/llama.cpp/releases) and download the binary for your system (Windows, macOS, Linux). 2. Extract the archive to a folder of your choice. 3. Open a terminal and navigate to that folder. 4. Run a model with the command: llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF (replace the model name as needed). 5. To start the OpenAI-compatible server, use: llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF. 6. Security note: only download binaries from the official GitHub page to avoid risks from unverified third-party sources.
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

In the same category

Whole category Run locally →