Hugging Face Accelerate

Microsoft's deep learning optimization library for extreme-scale training (ZeRO, offloading, MoE).

Fast exact attention kernels that reduce memory usage and accelerate transformer training and inference.

Optimized transformer building blocks and attention operators for PyTorch.

High-level wrapper for PyTorch that removes boilerplate and adds best practices.

Deep learning library providing practitioners with high-level components for state-of-the-art results. Built on PyTorch with a focus on usability and transfer learning. Apache 2.0 licensed.

High-level library for training and evaluating neural networks in PyTorch with an engine, events & handlers system for maximum flexibility. BSD-3-Clause licensed.