InternLM/lmdeploy
Toolkit for compressing, deploying, and serving LLMs from OpenMMLab. 4-bit inference with 2.4x higher performance than FP16, distributed multi-model serving across machines.
lm-sys/FastChat
Open platform for training, serving, and evaluating large language model chatbots. Powers Chatbot Arena (lmarena.ai) serving 10M+ requests for 70+ LLMs. Includes training code for Vicuna, MT-Bench evaluation, and distributed multi-model serving with OpenAI-compatible APIs. Apache 2.0 licensed.
looptroop-ai/LoopTroop
Local-first AI coding workspace that orchestrates multi-model planning councils, git worktrees, and task loops. MIT licensed.
ChatGPTNextWeb/NextChat
Light and fast AI assistant supporting Web, iOS, macOS, Android, Linux, and Windows. One-click deploy with multi-model support. MIT licensed.
enricoros/big-AGI
AI suite for power users with multi-model "Beam" chats, AI personas, voice, text-to-image, code execution, and PDF import. MIT licensed.
arc53/DocsGPT
Private AI platform for building intelligent agents and assistants with enterprise search. Features Agent Builder, deep research tools, multi-format document analysis, and multi-model support. MIT licensed.
bentoml/BentoML
Unified framework to build, ship, and scale AI apps.
neuml/txtai
All-in-one AI framework for semantic search, LLM orchestration and language model workflows. Embeddings database with customizable pipelines.
Intelligent-Internet/ii-agent
New open-source framework to build and deploy intelligent agents with support for Claude, Gemini, and OpenAI models. Apache 2.0 licensed.
ianarawjo/ChainForge
Visual programming environment for battle-testing prompts and evaluating LLM outputs. Features node-based prompt chains, multi-model comparison, and hypothesis testing. MIT licensed.
linshenkx/prompt-optimizer
AI prompt optimization tool with multi-round iterative improvements, dual-mode optimization for system and user prompts, and multi-model support. Available as web app, desktop app, Chrome extension, and Docker deployment. AGPL-3.0 licensed.
jundot/omlx
Apple-centric inference server for local-first AI workflows with model management, GPU orchestration, and OpenAI-compatible APIs for self-hosted deployment. Apache 2.0 licensed.