NVIDIA Apex

PyTorch extension for mixed precision training and distributed training optimizations. Powers many production deep learning workloads with tools for automatic mixed precision (AMP), distributed data parallel, and fused optimizers. BSD-3-Clause licensed.

View on GitHub Unknown

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

Hugging Face Accelerate

huggingface/accelerate

Simple API to make training scripts run on any hardware (multi-GPU, TPU, mixed precision) with minimal code changes.

unknown unknownactivity unknown

DeepSpeed

microsoft/DeepSpeed

Microsoft's deep learning optimization library for extreme-scale training (ZeRO, offloading, MoE).

unknown unknownactivity unknown

FlashAttention

Dao-AILab/flash-attention

Fast exact attention kernels that reduce memory usage and accelerate transformer training and inference.

unknown unknownactivity unknown

xFormers

facebookresearch/xformers

Optimized transformer building blocks and attention operators for PyTorch.

unknown unknownactivity unknown

PyTorch Lightning

Lightning-AI/lightning

High-level wrapper for PyTorch that removes boilerplate and adds best practices.

unknown unknownactivity unknown

fastai

fastai/fastai

Deep learning library providing practitioners with high-level components for state-of-the-art results. Built on PyTorch with a focus on usability and transfer learning. Apache 2.0 licensed.

unknown unknownactivity unknown