Containerized Model Serving on Red Hat OpenShift

Question

Pulumi · Accepted Answer

To containerize a model for serving on Red Hat OpenShift, you need to take several steps, which include containerizing your application, creating a deployment configuration for OpenShift, and exposing the application via a route so it can be accessed from outside the OpenShift cluster. Below you’ll find a Pulumi program written in Python that accomplishes the setup of a containerized model serving on Azure Red Hat OpenShift.

### Prerequisites:
1. An Azure account with permissions to create resources.
2. Pulumi CLI installed and configured with an appropriate Azure subscription.
3. `pulumi-azure-native` and `pulumi-docker` packages installed for Pulumi with Python.
4. Docker installed for building and pushing your container image.
5. Your machine learning model ready to be containerized.
6. Your Red Hat OpenShift cluster already up and running on Azure.

### Containerizing the Model
Before we dive into the Pulumi code, your model needs to be containerized. This typically involves:

- Writing a `Dockerfile` that specifies how to build the container image, including installing dependencies, copying your model to the image, and defining the command or process that runs when the container starts.
- Building the docker image and pushing it to a container registry, such as Azure Container Registry (ACR) or Docker Hub.

Here's an example of what a `Dockerfile` might look like:

```Dockerfile
# Use an official Python runtime as a base image
FROM python:3.8-slim

# Set the working directory in the container
WORKDIR /usr/src/app

# Copy the current directory contents into the container at /usr/src/app
COPY . .

# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Make port 80 available to the world outside this container
EXPOSE 80

# Define environment variable
ENV NAME World

# Run app.py when the container launches
CMD ["python", "./app.py"]
```

### The Pulumi Program
Next, here's a simplified Pulumi program that sets up the necessary Azure Red Hat OpenShift (ARO) resources. The code does not include the actual model serving application but assumes you have a container image ready for deployment.

```python
import pulumi
import pulumi_azure_native as azure_native

# Configurations for the OpenShift deployment
image = "your-container-registry/your-image"  # Replace with your container image
app_name = "model-serving-app"  # Name of the application
resource_group_name = "myResourceGroup"  # Replace with your resource group name

# Fetch the OpenShift cluster resource from Azure
open_shift_cluster = azure_native.redhatopenshift.get_open_shift_cluster(
    resource_group_name=resource_group_name,
    resource_name="myCluster"  # Replace with your cluster name
)

# Define an Azure Container Registry to store the image if not already present
acr = azure_native.containerregistry.Registry("myContainerRegistry",
    resource_group_name=resource_group_name,
    sku=azure_native.containerregistry.SkuArgs(
        name="Basic"
    ),
)

# Build and Push to Azure Container Registry (in reality, you would do this outside of Pulumi)
# Here, we are simply demonstrating how you might define a custom resource for building and
# pushing a container image using the `pulumi-docker` package.
import pulumi_docker as docker

docker.Image("model-serving-image",
    image_name=acr.login_server.apply(lambda server: f"{server}/{app_name}:v1"),
    build=docker.DockerBuild(context="path/to/Dockerfile/folder"),
    registry=docker.ImageRegistry(
        server=acr.login_server,
        username=acr.admin_username,
        password=acr.admin_password,
    ),
)

# Export the container registry URL
pulumi.export('container_registry_url', acr.login_server)
```

### Steps in the Program:

1. **Define Container Image**: We have a placeholder for the image that needs to be deployed. You would replace `'your-container-registry/your-image'` with the path to your image in a container registry.

2. **Configure OpenShift Cluster & ACR**: We fetch an existing OpenShift cluster (`azure_native.redhatopenshift.get_open_shift_cluster`) from Azure and create an Azure Container Registry where our container image will reside.

3. **Build & Push Container Image**: The Docker image is defined using `pulumi_docker.Image`. This part assumes you have a folder containing your `Dockerfile` and all necessary code. The code will build the image and push it to the newly created ACR. In practice, this usually happens in a CI/CD pipeline outside of your infrastructure code. Here we illustrate the concept.

### Deploying the Containerized Model to OpenShift

Deploying your application on OpenShift typically involves creating an `Application` resource that references your container image. This could be done using the `kubectl` tool, OpenShift CLI (`oc`), or through clients that talk to the OpenShift API.

As of my knowledge cutoff in 2023, direct management of OpenShift resources through Pulumi is limited to fetching existing clusters and using custom resources for specific OpenShift-related tasks within Azure. For deeper OpenShift management such as defining DeploymentConfigs, Services, or Routes, you would need to use `kubectl` or `oc`.

### Conclusion
The provided Pulumi program helps you to set up an Azure Container Registry and build & push your containerized model serving application image. You would then complete the setup by deploying the application on an OpenShift cluster using the OpenShift tools and referring to the image stored in ACR.