LiteRT-LM

Production-ready runtime for deploying LLMs on edge devices with low-latency inference and optimized hardware paths for mobile and embedded platforms.

View on GitHub Unknown

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

Ollama

ollama/ollama

Dead-simple local LLM runner with a one-line install, model registry, and OpenAI-compatible API.

185k 19kGopushed 3 hours ago

llama.cpp

ggml-org/llama.cpp

Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.

133k 25kC++pushed 3 hours ago

Foundry Local

microsoft/Foundry-Local

Open-source on-device AI platform covering discovery, model running, sandboxed execution, and evaluation of open models.

unknown unknownactivity unknown

Potato OS

slomin/potato-os

Linux distribution for fully local AI inference on Raspberry Pi 5 and 4, optimized for running open models at the edge.

unknown unknownactivity unknown

MLC-LLM

mlc-ai/mlc-llm

Deployment engine that compiles and runs LLMs across browsers, mobile devices, and local hardware.

unknown unknownactivity unknown

WebLLM

mlc-ai/web-llm

High-performance in-browser LLM inference engine. Runs models directly in the browser with WebGPU acceleration.

unknown unknownactivity unknown