Results for "openai-compatible"

Terminal-native AI chat client supporting OpenClaw, Hermes, Ollama, and any OpenAI-compatible backend — with routines, multi-agent switching, and local agent skills, all in the TUI. Apache-2.0 licensed.

Manages unified access to generative AI services built on Envoy Gateway. Kubernetes-native AI gateway for routing, load balancing, and managing LLM traffic with enterprise-grade reliability. Apache 2.0 licensed.

OpenAI-compatible proxy gateway that stacks the free tiers of multiple LLM providers behind a single endpoint with automatic failover and rate tracking. MIT licensed.

Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.

FastAPI-based API server for ExLlamaV2/V3 backends. OpenAI-compatible API with support for model loading/unloading, embeddings, speculative decoding, multi-LoRA, and streaming.

Production-grade platform for running any open-source LLMs as OpenAI-compatible API endpoints. Supports 50+ models with built-in streaming, batching, and auto-acceleration. Apache 2.0 licensed.

Python and TypeScript client library, with an optional local OpenAI-compatible server, that pools the free tiers of Gemini, Groq, OpenRouter, Cloudflare Workers AI and other providers with key rotation and failover. MIT licensed.

Open platform for training, serving, and evaluating large language model chatbots. Powers Chatbot Arena (lmarena.ai) serving 10M+ requests for 70+ LLMs. Includes training code for Vicuna, MT-Bench evaluation, and distributed multi-model serving with OpenAI-compatible APIs. Apache 2.0 licensed.

Official open-source codebase for HuggingChat.

MIT-licensed, standard-library-only Go CLI that validates `/models` and opt-in `/chat/completions` behavior for OpenAI-compatible APIs and emits bounded diagnostics without credentials or response bodies.

Native C implementation of Qwen3.8-Flash-Next for local CPU inference, with terminal chat, a resident OpenAI-compatible API, and reproducible benchmarks.

LLM gateway with a unified OpenAI-compatible API across providers, routing, load balancing, fallbacks, guardrails, and observability. Apache 2.0 licensed.

Unified, production-ready inference API for LLMs, speech, and multimodal models. Drop-in GPT replacement with single-line code changes. Supports thousands of models with auto-batching and distributed inference.

Official Python bindings for llama.cpp.

Apple-centric inference server for local-first AI workflows with model management, GPU orchestration, and OpenAI-compatible APIs for self-hosted deployment. Apache 2.0 licensed.

Self-hosted Apache-2.0-licensed AI gateway written in Rust, with OpenAI-compatible endpoints, an Anthropic-compatible Messages endpoint, model routing, traffic policies, Prometheus metrics, and OTLP trace export.

Intelligent model swapping proxy for llama.cpp. Enables seamless hot-swapping between different GGUF models without restarting the server, with automatic model loading/unloading and OpenAI-compatible API. MIT licensed.

Deployment engine that compiles and runs LLMs across browsers, mobile devices, and local hardware.

Open-source intelligent model router with CMA-ES evolved routing weights, federated learning for privacy-preserving collaborative routing, and human-interpretable explanations for every decision. OpenAI-compatible API with web dashboard. MIT licensed.

Most popular self-hosted ChatGPT-style interface.

Multi-agent harness whose host agent plans complex tasks as DAGs and delegates them to built-in research, coding, design, and on-call agents or to third-party agents over ACP, CLI, or OpenAI-compatible APIs, with cross-session memory and an experimental self-evolution loop that installs only verified changes. Apache 2.0 licensed.

Local-first AI app framework.

Beautiful local-first coding agent running in your terminal. Built for privacy and control with support for multiple AI providers via OpenRouter.

Multi-modal inference stack extending vLLM for image, audio, and video generation workloads with a unified serving interface. Apache 2.0 licensed.