Instrukcja do tego ćwiczenia Pobierz notatnik (.ipynb)

Creating a Real-Time Inferencing Service¶

You've spent a lot of time in this course training and registering machine learning models. Now it's time to deploy a model as a real-time service that clients can use to get predictions from new data.

Connect to Your Workspace¶

The first thing you need to do is to connect to your workspace using the Azure ML SDK.

Note: If the authenticated session with your Azure subscription has expired since you completed the previous exercise, you'll be prompted to reauthenticate.

In [ ]:

from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()
ml_client = MLClient.from_config(credential=credential)
print(f"Ready to use Azure ML to work with {ml_client.workspace_name}")

Train and Register a Model¶

The scoring script used later in this lab expects the registered diabetes_model to be a plain scikit-learn model saved with joblib (a CUSTOM_MODEL asset containing a diabetes_model.pkl file) - not the MLflow-format model registered in some of the earlier labs (Lab 3B, Lab 6A). Run the cell below to train and register a model in that expected format, so the deployment steps that follow work regardless of what other labs you've already completed.

In [ ]:

import pandas as pd
import numpy as np
import joblib
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import roc_auc_score
from azure.ai.ml.entities import Model
from azure.ai.ml.constants import AssetTypes

# load the diabetes dataset
print("Loading Data...")
diabetes = pd.read_csv('data/diabetes.csv')

# Separate features and labels
X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values

# Split data into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)

# Train a decision tree model
print('Training a decision tree model')
model = DecisionTreeClassifier().fit(X_train, y_train)

# calculate accuracy
y_hat = model.predict(X_test)
acc = np.average(y_hat == y_test)
print('Accuracy:', acc)

# calculate AUC
y_scores = model.predict_proba(X_test)
auc = roc_auc_score(y_test, y_scores[:,1])
print('AUC: ' + str(auc))

# Save the trained model
model_file = 'diabetes_model.pkl'
joblib.dump(value=model, filename=model_file)

# Register the model
registered_model = Model(
    path=model_file,
    type=AssetTypes.CUSTOM_MODEL,
    name="diabetes_model",
    description="A decision tree model that classifies patients by their likelihood of being diabetic.",
    tags={"Training context": "Inline Training"},
    properties={"AUC": str(auc), "Accuracy": str(acc)},
)
ml_client.models.create_or_update(registered_model)

print('Model trained and registered.')

Deploy a Model to a Managed Online Endpoint¶

You have trained and registered a machine learning model that classifies patients based on the likelihood of them having diabetes. This model could be used in a production environment such as a doctor's surgery where only patients deemed to be at risk need to be subjected to a clinical test for diabetes. To support this scenario, you will deploy the model to a managed online endpoint for real-time inferencing.

First, let's determine what models you have registered in the workspace.

In [ ]:

print("Registered models:")
for model in ml_client.models.list():
    print(f"\t{model.name}")

# Just as with data assets: the version and type belong to a specific version
latest = ml_client.models.get(name="diabetes_model", label="latest")
print(f"\n{latest.name}: latest version {latest.version}, type {latest.type}")

Right, now let's get the model that we want to deploy. By default, if we specify a model name, the latest version will be returned.

In [ ]:

model = ml_client.models.get(name="diabetes_model", label="latest")
print(model.name, 'version', model.version)

We're going to deploy this model to a managed online endpoint, and this will require some code and configuration files; so let's create a folder for those.

In [ ]:

import os

folder_name = 'diabetes_service'

# Create a folder for the deployment files
experiment_folder = './' + folder_name
os.makedirs(folder_name, exist_ok=True)

print(folder_name, 'folder created.')

The endpoint deployment will need some Python code to load the input data, get the model, and generate and return predictions. We'll save this code in an entry script (also known as a scoring script) that will be deployed to the endpoint:

In [ ]:

%%writefile $folder_name/score_diabetes.py
import json
import joblib
import numpy as np
import os

# Called when the deployment is initialized
def init():
    global model
    # Get the path to the deployed model file and load it
    # (AZUREML_MODEL_DIR is set by the online endpoint and points to the
    # folder containing the registered model files)
    model_path = os.path.join(os.getenv("AZUREML_MODEL_DIR"), "diabetes_model.pkl")
    model = joblib.load(model_path)

# Called when a request is received
def run(raw_data):
    # Get the input data as a numpy array
    data = np.array(json.loads(raw_data)['data'])
    # Get a prediction from the model
    predictions = model.predict(data)
    # Get the corresponding classname for each prediction (0 or 1)
    classnames = ['not-diabetic', 'diabetic']
    predicted_classes = []
    for prediction in predictions:
        predicted_classes.append(classnames[prediction])
    # Return the list of predictions - the inference server turns it into JSON
    return predicted_classes

The endpoint will run in a container, and the container will need to install any required Python dependencies when it gets initialized. In this case, our scoring code requires scikit-learn, so we'll create a conda environment file that lists the required packages.

In [ ]:

%%writefile $folder_name/diabetes_env.yml
name: diabetes-env
dependencies:
  - python=3.8
  - numpy
  - scikit-learn
  - pip
  - pip:
      - azureml-inference-server-http

In [ ]:

# Print the .yml file
with open(folder_name + "/diabetes_env.yml", "r") as f:
    print(f.read())

Now you're ready to deploy. We'll create an endpoint named diabetes-endpoint with a single deployment named blue. The deployment process includes the following steps:

Create a managed online endpoint.
Create a managed online deployment that specifies the model, the scoring script and environment, and the compute resources to use. In SDK v2 there's no ACI or AKS to choose between - all managed online endpoints run on Azure-managed compute.
Route all of the endpoint's traffic to the new deployment.
Verify the status of the deployment.

More Information: For more details about model deployment, see the documentation.

Deployment will take some time as it first builds a container image for the environment, and then provisions compute and deploys the model to it. When deployment has completed successfully, the deployment's provisioning state will be Succeeded.

In [ ]:

from azure.ai.ml.entities import (
    ManagedOnlineEndpoint,
    ManagedOnlineDeployment,
    Environment,
    CodeConfiguration,
)

endpoint_name = "diabetes-endpoint"

# Create a managed online endpoint
endpoint = ManagedOnlineEndpoint(
    name=endpoint_name,
    description="Real-time diabetes classification service",
    auth_mode="key",
)
ml_client.online_endpoints.begin_create_or_update(endpoint).result()

# Define the environment used to run the scoring script
env = Environment(
    conda_file=f"{folder_name}/diabetes_env.yml",
    image="mcr.microsoft.com/azureml/openmpi5.0-ubuntu24.04",
)

# Create a managed online deployment
blue_deployment = ManagedOnlineDeployment(
    name="blue",
    endpoint_name=endpoint_name,
    model=model,
    environment=env,
    code_configuration=CodeConfiguration(code=folder_name, scoring_script="score_diabetes.py"),
    instance_type="Standard_DS2_v2",
    instance_count=1,
)
ml_client.online_deployments.begin_create_or_update(blue_deployment).result()

# Route all traffic for the endpoint to the blue deployment
endpoint.traffic = {"blue": 100}
ml_client.online_endpoints.begin_create_or_update(endpoint).result()

print("Deployment complete.")

Hopefully, the deployment has been successful and you can see a provisioning state of Succeeded. If not, you can use the following code to check the status and get the deployment logs to help you troubleshoot.

In [ ]:

endpoint = ml_client.online_endpoints.get(name=endpoint_name)
print(endpoint.provisioning_state)

logs = ml_client.online_deployments.get_logs(
    name="blue", endpoint_name=endpoint_name, lines=50
)
print(logs)

# If you need to make a change and redeploy, you may need to delete the deployment first using the following code:
# ml_client.online_deployments.begin_delete(name="blue", endpoint_name=endpoint_name)

Take a look at your workspace in the Azure web interface and view the Endpoints page, which shows the deployed endpoints in your workspace.

You can also retrieve the names of online endpoints in your workspace by running the following code:

In [ ]:

for online_endpoint in ml_client.online_endpoints.list():
    print(online_endpoint.name)

Use the Endpoint¶

With the endpoint deployed, now you can consume it from a client application.

In [ ]:

import json

x_new = [[2,180,74,24,21,23.9091702,1.488172308,22]]
print ('Patient: {}'.format(x_new[0]))

# Save the sample data as a JSON request file
request_file_name = "sample-data.json"
with open(request_file_name, "w") as f:
    json.dump({"data": x_new}, f)

# Call the endpoint, passing the request file (the endpoint will also accept the data in binary format)
predictions = ml_client.online_endpoints.invoke(
    endpoint_name=endpoint_name,
    deployment_name="blue",
    request_file=request_file_name,
)

# Get the predicted class - it'll be the first (and only) one.
predicted_classes = json.loads(predictions)
print(predicted_classes[0])

You can also send multiple patient observations to the endpoint, and get back a prediction for each one.

In [ ]:

import json

# This time our input is an array of two feature arrays
x_new = [[2,180,74,24,21,23.9091702,1.488172308,22],
         [0,148,58,11,179,39.19207553,0.160829008,45]]

# Save the sample data as a JSON request file
request_file_name = "sample-data.json"
with open(request_file_name, "w") as f:
    json.dump({"data": x_new}, f)

# Call the endpoint, passing the request file
predictions = ml_client.online_endpoints.invoke(
    endpoint_name=endpoint_name,
    deployment_name="blue",
    request_file=request_file_name,
)

# Get the predicted classes.
predicted_classes = json.loads(predictions)

for i in range(len(x_new)):
    print ("Patient {}".format(x_new[i]), predicted_classes[i] )

The code above uses the Azure ML SDK to connect to the managed online endpoint and use it to generate predictions from your diabetes classification model. In production, a model is likely to be consumed by business applications that do not use the Azure ML SDK, but simply make HTTP requests to the endpoint.

Let's determine the URL to which these applications must submit their requests:

In [ ]:

endpoint = ml_client.online_endpoints.get(name=endpoint_name)
scoring_uri = endpoint.scoring_uri
print(scoring_uri)

Now that you know the endpoint URI, an application can make an HTTP request, sending the patient data in JSON format, and receive back the predicted class(es). Since the endpoint was created with auth_mode="key", the request must include an Authorization header with a valid key.

In [ ]:

import requests
import json

# Retrieve an authentication key for the endpoint
keys = ml_client.online_endpoints.get_keys(name=endpoint_name)
primary_key = keys.primary_key

x_new = [[2,180,74,24,21,23.9091702,1.488172308,22],
         [0,148,58,11,179,39.19207553,0.160829008,45]]

# Convert the array to a serializable list in a JSON document
input_json = json.dumps({"data": x_new})

# Set the content type and authorization headers
headers = {
    'Content-Type':'application/json',
    'Authorization': f'Bearer {primary_key}',
}

response = requests.post(scoring_uri, input_json, headers = headers)
predicted_classes = json.loads(response.json())

for i in range(len(x_new)):
    print ("Patient {}".format(x_new[i]), predicted_classes[i] )

You've deployed your model to a managed online endpoint using key-based authentication. Managed online endpoints handle load balancing, scaling, and monitoring for you, without requiring you to provision or manage any Azure Container Instances or Kubernetes clusters yourself. For production scenarios where tokens should expire, you can create the endpoint with auth_mode="aml_token" or auth_mode="aad_token" instead of auth_mode="key".

More Information For more information about deploying a model to an online endpoint, see the documentation

Clean Up¶

Managed online endpoints keep their underlying compute running (and incurring cost) until you delete them. If you don't need the endpoint anymore, run the following cell to delete it:

In [ ]:

# ml_client.online_endpoints.begin_delete(name=endpoint_name)