Deepchecks
Open Source
comprehensively validating your machine learning models
and data with minimal effort, in both the research and the
production phases.
Key Capabilities of Deepchecks Open Source
Data Integrity
When you have a fresh dataset, and
want to validate your data’s correctness
and uncover inconsistencies such as
conflicting labels or data duplicates.
Model Evaluation
When you have a trained model, and
want to examine performance metrics,
compare it to various benchmarks, and
create a clear and granular picture for
validating the model’s behavior (e.g. are
segments where it under-performs).
Train-Test Validation
When you have separate datasets
(such as train and test, or training data
collected at different times), and want
to validate that they are representative
of each other and don’t have issues
such as drift or leakage.
ML Validation Continuity from Research to Production
You can use the exact set (or a subset)
of the checks that were used during
research for CI/CD and Production
monitoring. That ensures that the deep
knowledge that you data science team
has will be used by the ML Engineers in
later model/data lifecycle phase.
Code-Level Root Cause Analysis
You can segment the data to get to the
area where the model/data seem to fail
and then handle that to the data
science team for code level analysis.
This means quicker root cause analysis
cycles (up to 70% of the time is usually
spent on the initial analysis, which is
saved here).
Integrations
Deepchecks Open Source: For ML Practitioners From Research to Production
and data with minimal effort, in both the reseach and the production phases. This includes checks related
to various types of issues, such as model performance, data integrity, distribution mismatches, and more. Model
and data validation is one of the most important processes that Data Scientists and ML Engineers are dealing with
while scaling up from the “laboratory phase” to ML Systems that are providing continuous value. We would
typically recommend “kicking the tires” with the Deepchecks Testing module as a first step, and continue to
deploying the Deepchecks Monitoring module when the timiing is right.
How Does It Work?
Suites are composed of checks. Each check contains outputs displayed in a notebook and/or conditions with a pass/fail output.
Conditions can be added or removed from a
check;
Checks can be edited or added/removed to a
suite;
Suites can be created from scratch or forked
from an existing suite.
Testing: Key Features & Checks
Data Integrity
suite = data_integrity()
suite_result = suite.run(train_dataset)
check = StringMismatch()
result = check.run(dataset)
Train Test Validation
suite = train_test_validation()
suite_result = suite.run(train_dataset, test_dataset)
check = PredictionDrift()
result = check.run(train_dataset, test_dataset)
Model Evaluation
suite = model_evaluation()
suite_result = suite.run(train_dataset, test_dataset, model)
check = WeakSegmentsPerformance()
result = check.run(test_dataset, model)
Checks for Unstructured Data
pip install -U “deepchecks[nlp]”
pip install -U “deepchecks[nlp-properties]”
pip install -U “deepchecks[vision]”
Monitoring: Key Features & Capabilities
Metrics & Alerts
from the testing library to calculate metrics and
display their progress over time. Anomalies in
these metrics trigger alerts that can be
consumed in the UI or using 3rd party apps like
Slack, mail or PagerDuty.
Code-Level Root Cause
Analysis
Deepchecks is designed to help reach the root
cause quickly. Deepchecks’ product philosophy is
to enhance an analysis journey that starts from
our UI and continues in a notebook or IDE, using
the production data and the relevant test suites.
Deployment That Keeps All
Your Data on Your Premise
Deepchecks Open Source is deployed in a
setting that keeps all data on your cloud. The
default 'press-enter' deployment enables you to
connect Deepchecks Monitoring to your data
without dealing with data processing protocols or
anonymization.
deployment methods please check out Deepchecks Hub.
Open Source & Community
Join our Slack Community
Recent Blog Posts
Brain John Aboze