The MLOps Way: Utilizing the 3 Pipeline Design and Production REST API
For MLOps, we map the CI pipeline and the CD pipeline to the 3 pipeline design and production REST API.
The 3 pipeline design, first coined by Pau Labarta Bajo, is great for laying the foundations of a complete AI system, where the required pipelines include the feature, training, and inference pipelines, sitting on top of a feature store and model registry. For organizations who are just beginning to deploy their AI prototypes in production, incorporating MLOps (machine learning operations) is key. MLOps incorporates the concepts of continuous integration (CI), continuous deployment (CD), continuous monitoring (CM), and continuous training (CT) for seamless development, testing, deployment, and monitoring of ML models in production. MLOps helps to automate and orchestrate ML workflows seamlessly, from ML development (CI), to ML testing (CI), to ML deployment (CD), to ML monitoring (CM), to ML retraining (CT).
The MLOps framework utilizing the 3 pipeline design helps to modularize the ML workflows that can scale. So we start with the 3 pipeline design, where everything is done manually between the 3 pipelines and dev/test/prod environments. Once these pipelines and environments are designed and built, then MLOps can be instituted to add automation and orchestration between these pipelines and environments, which is important for scaling ML models into enterprise applications and keeping up with multiple ML models in production. In addition, the use of REST APIs and Docker helps to create production-ready AI systems.
The MLOps way utilizing the 3 pipeline design, REST API, and Docker (based on Pau Lambarta Bajo’s article below):
This article explores an MLOps approach using the 3 pipeline design: Feature Pipeline, Training Pipeline, and Continuous Deployment (CD) Pipeline, along with the use of REST APIs and Docker to create production-ready systems.
Feature Pipeline: Data Preparation
The Feature Pipeline is where raw data is transformed into a suitable format for training models. This stage involves cleaning the data, engineering features, and selecting the most relevant attributes. The output is a clean, feature-rich dataset ready to be fed into the model training process.
Training Pipeline: Model Development
Once the data is prepared, the Training Pipeline takes over. This pipeline is responsible for training the machine learning model using the prepared features. The model undergoes a rigorous training process, where it learns patterns from the data and optimizes its parameters to make accurate predictions. The trained model is then evaluated for its performance, and if it meets the required standards, it is serialized (commonly pickled) and pushed to a model registry.
After data preparation in the Feature pipeline, and training the model in the Training pipeline, we now look to automate and orchestrate the pipelines with Git and CI/CD via MLOps framework:
Step 1: Code Versioning with Git
With the trained model ready, the next step is to Push code to a new branch and open a Pull Request (PR). This is a crucial practice in software engineering that also applies to MLOps. By pushing the code to a new branch, you ensure that your main branch remains stable. The PR allows for peer review and integration of the new code with the existing codebase.
Step 2: Continuous Integration (CI)
After the PR is opened, the Continuous Integration (CI) process is triggered. The CI pipeline ensures that the trained model is ready for integration and deployment into production, and is designed to:
1. Run Unit Tests: These tests ensure that the code changes have not introduced any new bugs or broken existing functionality.
2. Run Training Script: The training script is executed to validate the new model’s performance.
3. Validate ML Model: The model’s metrics are assessed to ensure it meets the required benchmarks. If the model passes all tests, it is serialized and pushed to the model registry.
Step 3: Continuous Deployment (CD)
Once the CI pipeline successfully completes, the Continuous Deployment (CD) process takes over. The CD pipeline is responsible for deploying the trained model into a production environment. This involves:
1. Pulling the Pickled Model: The CD pipeline retrieves the serialized model from the model registry.
2. Incorporating the Model into Inference Script: The model is integrated into an inference.py script, where a function predict(features) -> target is defined to make predictions based on the input features.
3. Building a REST API: A REST API is built around the inference script, allowing external applications to interact with the model.
4. Packaging with Docker: The entire application, including the model, inference script, and dependencies, is packaged into a Docker image. This ensures that the model can be deployed consistently across various compute environments.
Automated Deployment with GitHub Actions
The beauty of MLOps lies in automation. In this workflow, the CD Pipeline is automatically triggered via GitHub Actions once the CI pipeline completes successfully. GitHub Actions orchestrates the following steps:
1. Push Docker Image to Registry: The Docker image is pushed to an application-specific Docker registry, ensuring that it is versioned and stored securely.
2. Trigger Deployment: The deployment process is initiated, deploying the Docker container to a compute platform such as Kubernetes, AWS, or Google Cloud. This step ensures that the model is available as a production-ready API endpoint, ready to serve predictions to users.
(For more details, please refer to Pau Lambarta Bajo’s article link above)
To reiterate- it’s important to first build the pipelines, following the 3 pipeline design. The 3 pipeline design is great for laying the foundation, but everything is done manually between pipelines/environments. Once you have the foundational pipelines/environments/infrastructure, then you can add automation and orchestration by overlaying MLOps on top of it.
So for deploying ML models at enterprise scale, we need to ramp up our vision to include the MLOps way, where the CI pipeline sits on top of the feature and training pipelines, with its main purpose to version the code and determine if the model is ready for production. Once the CI pipeline is successfully completed, then it automatically triggers the CD pipeline, to deploy the Docker image built in the CI pipeline to a production ready API endpoint.
But solutions architects and developers need to understand the 3 pipeline design first- I see people jumping onto CI/CD for deploying ML models, without really knowing what feature and training pipelines are. Without the feature and training pipelines, what are you going to deploy in the inference pipeline?
CI/CD for ML is really the automation and orchestration that occurs when the ML model is developed and productionized via the feature, training, and inference pipelines.
In the next article, we will discuss CM/CT for monitoring and retraining ML models in production. CM/CT is a requirement for ML models in production, as the model degrades over time, due to either data or concept drift.
Conclusion
By adopting an MLOps approach that integrates pipelines for data preparation, model training, and continuous deployment, organizations can streamline their machine learning workflows. The use of REST APIs and Docker ensures that models are not only trained effectively but are also deployed and maintained with minimal friction. This automated and scalable approach is essential for organizations looking to leverage machine learning in a production environment.
If you're looking for support, here is how to contact me:
Coaching and Mentorship: I offer coaching and mentorship; book a coaching session here