LIVE
Google Introduces "Gemini 4 Carbon" as Next Step in Its Model Lineup28/10/26 · Google|DeepMind unveils Genie 4, a new generation of world model27/10/26 · Google DeepMind|Anthropic files IPO prospectus, with risk disclosures taking center stage26/10/26 · Anthropic|Robot policy trained only in simulation matches top real-world systems26/10/26|OpenAI unveils GPT-6.1 Terra, a model built for computer use26/10/26 · OpenAI|Google's AI gives evasive answers about who won the 2020 US election28/10/26 · Google|NVIDIA introduces Nemotron 3.5 Transcribe, a promptable multilingual transcription model28/10/26 · NVIDIA|OpenAI and ServiceNow deepen partnership on enterprise workflows28/10/26 · OpenAI|Failed startups' data finds a second life training AI models28/10/26|Anthropic expands AI for Science program to 20 foundations and research institutions28/10/26 · Anthropic|Anthropic shares early results on using Claude to accelerate drug development27/10/26 · Anthropic|GEPA-Curator: Filtering Failures to Better Train LLM Agents27/10/26|Google Introduces "Gemini 4 Carbon" as Next Step in Its Model Lineup28/10/26 · Google|DeepMind unveils Genie 4, a new generation of world model27/10/26 · Google DeepMind|Anthropic files IPO prospectus, with risk disclosures taking center stage26/10/26 · Anthropic|Robot policy trained only in simulation matches top real-world systems26/10/26|OpenAI unveils GPT-6.1 Terra, a model built for computer use26/10/26 · OpenAI|Google's AI gives evasive answers about who won the 2020 US election28/10/26 · Google|NVIDIA introduces Nemotron 3.5 Transcribe, a promptable multilingual transcription model28/10/26 · NVIDIA|OpenAI and ServiceNow deepen partnership on enterprise workflows28/10/26 · OpenAI|Failed startups' data finds a second life training AI models28/10/26|Anthropic expands AI for Science program to 20 foundations and research institutions28/10/26 · Anthropic|Anthropic shares early results on using Claude to accelerate drug development27/10/26 · Anthropic|GEPA-Curator: Filtering Failures to Better Train LLM Agents27/10/26|
AppRisingofficial publisher

llama.cpp

App : A product you use directly

The C/C++ inference engine behind Ollama, LM Studio, and most local AI tools.

Indispensable behind the scenes, worth using directly only if you are comfortable with the command line.

GitHub · 132,260 ★
Overall score
8.9 /10
Security: nothing to flag
Utility
8.5
Reliability
10.0
Security
No caveat on record
Ease
Technical · Technical

Security reflects where data goes, requested permissions and incidents on record at the verification date. It is not a guarantee of zero risk: check the deployment mode and the connections you enable.

What it does, concretely

Running a large language model on your own computer usually requires significant resources and complex setup. llama.cpp solves this by providing an inference engine written in pure C/C++, with no external dependencies, capable of running LLM and VLM models on ordinary hardware, both locally and in the cloud.

In practice, llama.cpp loads models in GGUF format and runs them by leveraging the CPU, GPU, or both simultaneously. It supports quantization from 1.5 to 8 bits to reduce memory requirements, and relies on various backends: Metal for Apple Silicon, CUDA for Nvidia GPUs, HIP for AMD, Vulkan, OpenCL, and more. A command-line interface and an OpenAI-compatible API server are included out of the box.

llama.cpp is aimed at developers, researchers, and advanced users who want to run models locally with full control over configuration. It is also the engine running behind the scenes in tools like Ollama or LM Studio, without end users needing to install it directly.

It is not suited for people looking for a simple graphical interface or who are not comfortable with the command line. Installation and configuration require a minimum level of technical knowledge.

What it offers

  • ·Local inference of LLM and VLM models in pure C/C++, with no external dependencies.
  • ·Model quantization from 1.5 to 8 bits to reduce memory usage and speed up execution.
  • ·Support for many hardware backends: Apple Silicon (Metal), Nvidia GPU (CUDA), AMD GPU (HIP), Vulkan, OpenCL, RISC-V, and more.
  • ·Hybrid CPU+GPU inference to run models larger than the available VRAM.
  • ·Built-in server compatible with the OpenAI API, accessible via the llama serve command.
  • ·Command-line interface (llama cli) allowing you to download and run a model directly from Hugging Face.
  • ·Built-in web UI available when using the llama serve server.
  • ·Cross-platform support: macOS, Windows, Linux, Docker, and even Android.

In practice

Price
Free
Commercial use
Commercial use: yes
Available on
macos, windows, linux, cli
Licence
MIT

Security and data

Rating
Security: nothing to flag
Your data
Data stays local
Maintenance
Active
Last release
15 October 2026

Links and repository checked on 25 October 2026 · added on 1 March 2023

How to install llama.cpp

1. Go to the GitHub releases page (https://github.com/ggml-org/llama.cpp/releases) and download the binary for your system (Windows, macOS, Linux). 2. Extract the archive to a folder of your choice. 3. Open a terminal and navigate to that folder. 4. Run a model with the command: llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF (replace the model name as needed). 5. To start the OpenAI-compatible server, use: llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF. 6. Security note: only download binaries from the official GitHub page to avoid risks from unverified third-party sources.
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

In the same category

Whole category Run locally →