VLMEvalKit

Open-source evaluation toolkit for large multi-modality models (LMMs). Supports 220+ LMMs and 80+ benchmarks including MMMU, MathVista, and ChartQA. Powers the OpenVLM Leaderboard. Apache 2.0 licensed.

View on GitHub Unknown

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

LiveBench

LiveBench/LiveBench

Contamination-free LLM benchmark with objective ground-truth scoring. ICLR 2025 spotlight paper featuring frequently-updated questions from recent sources. Tests math, coding, reasoning, language, instruction following, and data analysis.

unknown unknownactivity unknown

lm-evaluation-harness (EleutherAI)

EleutherAI/lm-evaluation-harness

De-facto standard for generative model evaluation.

unknown unknownactivity unknown

HELM (Stanford)

stanford-crfm/helm

Holistic Evaluation of Language Models.

unknown unknownactivity unknown

SWE-bench

SWE-bench/SWE-bench

Evaluates LLMs on real-world GitHub issues from 15+ Python repositories.

unknown unknownactivity unknown

GAIA

Real-world multi-step agentic benchmark.

unknown unknownactivity unknown

OpenCompass

open-compass/opencompass

Evaluation platform for benchmarking language and multimodal models across large benchmark suites.

unknown unknownactivity unknown