Inference Kernels & Runtime Primitives

Communication library for efficient expert-parallel training/inference pipelines, reducing MoE cross-device communication overhead. Apache 2.0 licensed.

High-performance CUTLASS-based Kimi Delta Attention CUDA kernels for linear attention mechanisms. MIT licensed.

Clean, high-performance FP8 GEMM kernels with fine-grained scaling for modern inference workloads. Apache 2.0 licensed.

CUDA-accelerated algorithms and ANN building blocks for high-performance similarity search, clustering, and matrix learning workloads.

CUDA tile primitives and kernel templates for accelerating transformer attention blocks. MIT licensed.

TT-Metalium + TT-NN operator stack for building and optimizing kernels on Tenstorrent AI accelerators. Apache 2.0 licensed.

Compact implementation of SGLang designed to demystify modern LLM serving systems. Educational yet production-quality with RadixAttention, continuous batching, and speculative decoding. MIT licensed.