Megatron-LM

Distributed training framework and reference codebase for large transformer models at scale.

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

DeepSpeed

deepspeedai/DeepSpeed

Extreme-scale training optimizations.

unknown unknownactivity unknown

Colossal-AI

hpcaitech/ColossalAI

Unified system for 100B+ models.

unknown unknownactivity unknown

Ray Train

ray-project/ray

Scalable distributed training.

unknown unknownactivity unknown

Nanotron (Hugging Face)

huggingface/nanotron

Minimalistic 3D-parallelism LLM pretraining with tensor, pipeline, and data parallelism. Designed for simplicity and speed.

unknown unknownactivity unknown

RLinf

RLinf/RLinf

Scalable open-source RL infrastructure for post-training foundation models via reinforcement learning. Features M2Flow paradigm for embodied AI and agentic workflows with real-world robotics integrations. Apache 2.0 licensed.

unknown unknownactivity unknown

dstack

dstackai/dstack

Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal. MPL-2.0 licensed.

unknown unknownactivity unknown