Results for "open-weights"

Edge-native Mixture-of-Experts serving engine for running frontier-scale open-weight models across consumer GPUs, CPUs, and host memory, with OpenAI- and Anthropic-compatible APIs. Apache-2.0 licensed.

Open-weights live music model for streaming generation and real-time interaction. Apache 2.0 licensed.

OpenAI open-weight model repository with inference examples, recipes, and deployment guidance.

Native inference engine optimized for DeepSeek V4 and GLM models with Metal, CUDA, and ROCm support.

Open-source mixture-of-experts language model family optimized for long-horizon planning, agentic tasks, and coding. Apache 2.0 licensed.

Zero-dependency Python CLI that plans LLM VRAM before you serve: real per-token KV-cache math, FITS/TIGHT/OOM verdicts, max safe context, and paste-ready vLLM/llama.cpp flags. MIT licensed.

Toolkit for compressing, deploying, and serving LLMs from OpenMMLab. 4-bit inference with 2.4x higher performance than FP16, distributed multi-model serving across machines.

Batteries-included LangChain agent harness for building and running structured multi-agent workflows with reusable runtime patterns.

Reproducible training recipe and open weights for a Mistral-Nemo-12B creative-writing fine-tune built with QLoRA and anti-slop DPO, with GGUF quants and the lowest slop score among the same-size 12B models tested on EQ-Bench Creative Writing v3. Apache-2.0 licensed.

Production-grade platform for running any open-source LLMs as OpenAI-compatible API endpoints. Supports 50+ models with built-in streaming, batching, and auto-acceleration. Apache 2.0 licensed.

Local-first macOS AI app that runs LLM chat, image generation, voice transcription, and personal memory/RAG fully on-device via llama.cpp - nothing leaves the machine. AGPL-3.0 licensed.

State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.

Fast, flexible Rust-native LLM inference engine built on Candle. Supports text, vision, audio, image generation, and embeddings with hardware-aware auto-tuning.

Self-hosted AI pull request reviewer for GitHub, Forgejo and Gitea that scores PRs, enforces merge gates as status checks and posts inline fixes, using local GGUF models via llama.cpp or a hosted model API. AGPL-3.0 licensed.

Single-GPU 70B inference stack with strong memory/performance optimizations for local deployment on commodity hardware.

GPU cluster manager that orchestrates inference engines like vLLM and SGLang. Automated engine selection, parameter optimization, and distributed multi-GPU deployment for high-performance AI workloads.

Linux distribution for fully local AI inference on Raspberry Pi 5 and 4, optimized for running open models at the edge.

Hardware-aware local inference engine that profiles host hardware, recommends suitable open models, and tunes execution across Apple Silicon, NVIDIA, AMD, and CPU. Apache 2.0 licensed.

NVIDIA's official high-performance inference backend.