llmash

Ollama-compatible LLM server on a llama.cpp fork, tuned for speculative decoding on NVIDIA GPUs. Keeps Ollama's commands, API and model store, drafts with the model's own MTP head, and can uncensor a model with a control vector applied at load. MIT licensed.

View on GitHub Unknown

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

Ollama

ollama/ollama

Dead-simple local LLM runner with a one-line install, model registry, and OpenAI-compatible API.

185k 19kGopushed 2 hours ago

llama.cpp

ggml-org/llama.cpp

Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.

133k 25kC++pushed 3 hours ago

Foundry Local

microsoft/Foundry-Local

Open-source on-device AI platform covering discovery, model running, sandboxed execution, and evaluation of open models.

unknown unknownactivity unknown

Potato OS

slomin/potato-os

Linux distribution for fully local AI inference on Raspberry Pi 5 and 4, optimized for running open models at the edge.

unknown unknownactivity unknown

MLC-LLM

mlc-ai/mlc-llm

Deployment engine that compiles and runs LLMs across browsers, mobile devices, and local hardware.

unknown unknownactivity unknown

WebLLM

mlc-ai/web-llm

High-performance in-browser LLM inference engine. Runs models directly in the browser with WebGPU acceleration.

unknown unknownactivity unknown