Model Training & Optimization Utilities

Simple API to make training scripts run on any hardware (multi-GPU, TPU, mixed precision) with minimal code changes.

Microsoft's deep learning optimization library for extreme-scale training (ZeRO, offloading, MoE).

Fast exact attention kernels that reduce memory usage and accelerate transformer training and inference.

Optimized transformer building blocks and attention operators for PyTorch.

High-level wrapper for PyTorch that removes boilerplate and adds best practices.

Deep learning library providing practitioners with high-level components for state-of-the-art results. Built on PyTorch with a focus on usability and transfer learning. Apache 2.0 licensed.

High-level library for training and evaluating neural networks in PyTorch with an engine, events & handlers system for maximum flexibility. BSD-3-Clause licensed.

High-performance inference and training for ONNX models across hardware.

Flexible, powerful tensor operations for readable and reliable code. Supports PyTorch, JAX, TensorFlow, NumPy, MLX.

Simple, safe way to store and distribute tensors. Fast, secure alternative to pickle for model serialization.

Machine learning metrics for distributed, scalable PyTorch applications. 80+ metrics with built-in distributed synchronization.

PyTorch native quantization and sparsity for training and inference. Drop-in optimizations for production deployment.

Game theoretic approach to explain the output of any machine learning model. Industry standard for model interpretability.

scikit-learn compatible neural network library that wraps PyTorch. Seamlessly integrate PyTorch models with scikit-learn pipelines, grid search, and cross-validation.

Supercharge your model training. MosaicML's PyTorch training library with built-in algorithms for efficient training (FSDP, gradient compression, progressive resizing) and seamless distributed training on large-scale clusters. Apache 2.0 licensed.

PyTorch extension for mixed precision training and distributed training optimizations. Powers many production deep learning workloads with tools for automatic mixed precision (AMP), distributed data parallel, and fused optimizers. BSD-3-Clause licensed.