MLOps: DevOps for Machine Learning
MLOps: DevOps for Machine Learning
Collaboration and teamwork are two of the keys to the success of large companies.
Software development projects are executed following an application-centric approach. Specifically, they focus on the platform, instrumentation, and processes required to support applications. They answer questions such as: What infrastructure is needed to support the application? What tools can be used to automate it? What is the release process for quality control and production?
AI/ML projects, on the other hand, follow processes with many small iterations to refine data quality, model configuration, and result analysis according to the business questions posed. They typically do not consider a delivery process or interaction with other teams (such as IT). Today, software development teams are still unfamiliar with the tools, languages, and artifacts used in data science projects.
Software development teams and data scientists work with two independent methodologies but share a common goal: deploying an AI application into production. Bridging the gap between these two ways of working requires effort. AI/ML projects need to incorporate some of the operational and deployment practices that make software development effective, while software development projects need to accommodate the AI/ML development process to automate deployment and release of AI/ML models.
MLOps helps us understand how to build a CI/CD pipeline for ML/AI projects.
The traditional process for bringing an ML model into production—and the one that needs to be automated—is:
(a) Train the model
(b) Package the model
(c) Validate its performance
(d) Decide to deploy it
(e) Monitor it
(f) Decide to update (retrain)