llama.cpp

Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.

View on GitHub Website Active

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

Ollama

ollama/ollama

Dead-simple local LLM runner with a one-line install, model registry, and OpenAI-compatible API.

185k 19kGopushed 2 hours ago

Foundry Local

microsoft/Foundry-Local

Open-source on-device AI platform covering discovery, model running, sandboxed execution, and evaluation of open models.

unknown unknownactivity unknown

Potato OS

slomin/potato-os

Linux distribution for fully local AI inference on Raspberry Pi 5 and 4, optimized for running open models at the edge.

unknown unknownactivity unknown

MLC-LLM

mlc-ai/mlc-llm

Deployment engine that compiles and runs LLMs across browsers, mobile devices, and local hardware.

unknown unknownactivity unknown

WebLLM

mlc-ai/web-llm

High-performance in-browser LLM inference engine. Runs models directly in the browser with WebGPU acceleration.

unknown unknownactivity unknown

llama-cpp-python

abetlen/llama-cpp-python

Official Python bindings for llama.cpp.

unknown unknownactivity unknown