SWE-rebench (Nebius)

Continuously updated benchmark with 21,000+ real-world SWE tasks for evaluating agentic LLMs. Decontaminated, mined from GitHub.

View project Unknown

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

LiveBench

LiveBench/LiveBench

Contamination-free LLM benchmark with objective ground-truth scoring. ICLR 2025 spotlight paper featuring frequently-updated questions from recent sources. Tests math, coding, reasoning, language, instruction following, and data analysis.

unknown unknownactivity unknown

lm-evaluation-harness (EleutherAI)

EleutherAI/lm-evaluation-harness

De-facto standard for generative model evaluation.

unknown unknownactivity unknown

HELM (Stanford)

stanford-crfm/helm

Holistic Evaluation of Language Models.

unknown unknownactivity unknown

SWE-bench

SWE-bench/SWE-bench

Evaluates LLMs on real-world GitHub issues from 15+ Python repositories.

unknown unknownactivity unknown

GAIA

Real-world multi-step agentic benchmark.

unknown unknownactivity unknown

OpenCompass

open-compass/opencompass

Evaluation platform for benchmarking language and multimodal models across large benchmark suites.

unknown unknownactivity unknown