๐ŸŽ‰ Deepchecksโ€™ New Major Release: Evaluation for LLM-Based Apps!  Click here to find out more ๐Ÿš€

What needs to be versioned in ML development?

Randall Hendricks
Randall HendricksAnswered

The answer depends on where in the model development process you currently find yourself in. We need to examine each stage of development to determine version control for machine learning models.

  • Selection. Choosing an appropriate model is the first step in selecting an appropriate algorithm. You may need to test out many algorithms before settling on one. Each chosen algorithm needs its machine-learning development environment so that modifications can be tracked independently and the most effective model may be picked.Your model’s performance may fluctuate as you write it or as you implement various performance tweaks, so it’s important to keep track of them as you go.
  • Training. Take note of the training hyperparameters. When tuning a model’s hyperparameters, you may split out each parameter into its branch and monitor how the model responds to its various tweaks.
    Along with the model’s code and hyperparameters, trained parameters must also be versioned to guarantee that the same trained weights can be reproduced and that reverting to this version does not need additional training time.
  • Evaluation. To evaluate a model, its creator will run it on data it has never seen before.
    Every step’s performance matrix should be recorded with the hold-out and the outcomes on the model versioning machine learning. When you’ve settled on the optimal model training settings, you may evaluate the integration branch as indicated in the model training section.
  • Validation. Be mindful of the model’s performance over time and document each validation result. When contrasting models, keep tabs on the validation matrices each one was put through.
  • Deployment. Keep track of which version was used and what modifications were made between releases. In this way, you may have a staged deployment by releasing a new version to production first while continuing to refine and enhance your model.

When integrating many features, deployment time and complexity can be affected by knowing what was done and how.

Testing. CI/CD. Monitoring.

Because ML systems are more fragile than you think. All based on our open-source core.

Our GithubInstall Open SourceBook a Demo

Subscribe to Our Newsletter

Do you want to stay informed? Keep up-to-date with industry news, the latest trends in MLOps, and observability of ML systems.