Deequ

Library built on top of Apache Spark for defining "unit tests for data". Measures data quality in large datasets with constraint verification, anomaly detection, and incremental validation. Used at Amazon for production data quality. Apache 2.0 licensed.

View on GitHub Unknown

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

Great Expectations

great-expectations/great_expectations

Always know what to expect from your data. Data validation, profiling, and documentation for data pipelines. Apache 2.0 licensed.

unknown unknownactivity unknown

ydata-profiling

ydataai/ydata-profiling

One line of code for comprehensive data quality profiling and exploratory data analysis. Generates detailed reports for Pandas and Spark DataFrames including statistics, correlations, missing values, and data quality alerts. MIT licensed.

unknown unknownactivity unknown

Soda Core

sodadata/soda-core

Data contracts engine for the modern data stack. Define data quality checks in YAML and automatically validate schema and data across your pipelines. Supports 20+ data sources including Snowflake, BigQuery, and PostgreSQL. Apache 2.0 licensed.

unknown unknownactivity unknown

TFX (TensorFlow Extended)

tensorflow/tfx

End-to-end platform for deploying production ML pipelines. Data validation, transformation, model training, and serving with TensorFlow. Powers Google's production ML infrastructure. Apache 2.0 licensed.

unknown unknownactivity unknown