FlashAttention

Fast exact attention kernels that reduce memory usage and accelerate transformer training and inference.

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

Hugging Face Accelerate

huggingface/accelerate

Simple API to make training scripts run on any hardware (multi-GPU, TPU, mixed precision) with minimal code changes.

unknown unknownactivity unknown

DeepSpeed

microsoft/DeepSpeed

Microsoft's deep learning optimization library for extreme-scale training (ZeRO, offloading, MoE).

unknown unknownactivity unknown

xFormers

facebookresearch/xformers

Optimized transformer building blocks and attention operators for PyTorch.

unknown unknownactivity unknown

PyTorch Lightning

Lightning-AI/lightning

High-level wrapper for PyTorch that removes boilerplate and adds best practices.

unknown unknownactivity unknown

fastai

fastai/fastai

Deep learning library providing practitioners with high-level components for state-of-the-art results. Built on PyTorch with a focus on usability and transfer learning. Apache 2.0 licensed.

unknown unknownactivity unknown

PyTorch Ignite

pytorch/ignite

High-level library for training and evaluating neural networks in PyTorch with an engine, events & handlers system for maximum flexibility. BSD-3-Clause licensed.

unknown unknownactivity unknown