NLP & Transformers

Industrial-strength natural language processing with 75+ languages, transformer pipelines, and production-grade NER, parsing, and text classification.

The de facto standard library for pretrained NLP models. 1M+ models, 250,000+ downloads/day. BERT, GPT, Llama, Qwen, and hundreds more.

Classic library for sentence and image embeddings.

Fast state-of-the-art tokenizers for training and inference.

FAIR Sequence Modeling Toolkit 2. Complete rewrite of fairseq with modern PyTorch APIs, native support for LLM training (70B+ models), vLLM integration, and first-party recipes for instruction finetuning and preference optimization. MIT licensed.

Self-hosted machine translation API powered by the Argos Translate engine. AGPL-3.0 licensed.