Deepchecks For LLM Evaluation
How Does It Work?
Deepchecks Open Source
Install Deepchecks Open Source to Take Care of Both Testing and Monitoring of Your ML
In the past, Deepchecks Open Source was focused only on testing ML during the research phase. As of June 2023, Deepchecks expanded this to a combination of testing and monitoring, so that data scientists and ML engineers can take multiple steps in their validation journey while remaining in the open source realm.
users iteratively running test
suites on their data and models,
from within a notebook or an IDE.
the test suites you know (and
love) as part of your CI/CD, using
tools like GitHub Actions or
about tracking data and models in
production to make sure your ML
system is behaving as expected.
pip install -U deepchecks from deepchecks.tabular.suites import train_test_validation suite_result = train_test_validation().run(train_dataset, test_dataset)on: push: ['main'] job: run_suites: train_dataset: 'load.py:get_train_dataset'
suites: ['custom_suites.py:my_model_evaluation_suite']dc_client = deepchecks_client.DeepchecksClient(host, API_TOKEN) model_version = dc_client.create_tabular_model_version(model_name, version_name, schema_file, reference_dataset) model_version.log_batch(sample_ids, data, timestamps, predictions)
Combining the Continuous Validation of Multiple Models, in a Managed and Secure Setting, from Research to Production
Deepchecks Supported Data Types
data lifecycle from training to production
pip install -U deepchecks from deepchecks.tabular.suites import train_test_validation
Run Test Suites for CI/CD
integrating just a few lines of code to your CI/CD
scripts. This will help you to ensure that your re-trained
model will not cause issues when deployed to
on: push: ['main'] job: run_suites: train_dataset: 'load.py:get_train_dataset'