Streaming (MosaicML)

High-performance data streaming library for efficient neural network training. Streams training data from cloud storage (S3, GCS, Azure) with local caching and deterministic shuffling. Apache 2.0 licensed.

View on GitHub Unknown

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

DeepSpeed

deepspeedai/DeepSpeed

Extreme-scale training optimizations.

unknown unknownactivity unknown

Colossal-AI

hpcaitech/ColossalAI

Unified system for 100B+ models.

unknown unknownactivity unknown

Megatron-LM

NVIDIA/Megatron-LM

Distributed training framework and reference codebase for large transformer models at scale.

unknown unknownactivity unknown

Ray Train

ray-project/ray

Scalable distributed training.

unknown unknownactivity unknown

Nanotron (Hugging Face)

huggingface/nanotron

Minimalistic 3D-parallelism LLM pretraining with tensor, pipeline, and data parallelism. Designed for simplicity and speed.

unknown unknownactivity unknown

RLinf

RLinf/RLinf

Scalable open-source RL infrastructure for post-training foundation models via reinforcement learning. Features M2Flow paradigm for embodied AI and agentic workflows with real-world robotics integrations. Apache 2.0 licensed.

unknown unknownactivity unknown