Working with Compute Targets¶
You've used the Azure Machine Learning SDK v2 to run several jobs, all of them on your local compute (in this case, the Azure Machine Learning compute instance). Now it's time to see how you can leverage cloud compute to increase the scalability of your compute contexts.
Connect to Your Workspace¶
The first thing you need to do is to connect to your workspace using the Azure ML SDK v2.
Note: If the authenticated session with your Azure subscription has expired since you completed the previous exercise, you'll be prompted to reauthenticate.
from importlib.metadata import version
from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential
credential = DefaultAzureCredential()
ml_client = MLClient.from_config(credential=credential)
print(f"Ready to use Azure ML {version('azure-ai-ml')} to work with {ml_client.workspace_name}")
Prepare Data for an Experiment¶
In this lab, you'll use a dataset containing details of diabetes patients. This lab is about compute targets, not data assets, so rather than depending on the exact type/version of the shared diabetes_mltable data asset from earlier labs (which is registered as an mltable), you'll just pass the local data/diabetes.csv file directly as a uri_file input to the job - the SDK uploads it automatically when the job is submitted.
Create a Compute Target¶
In many cases, your local compute resources may not be sufficient to process a complex or long-running experiment that needs to process a large volume of data; and you may want to take advantage of the ability to dynamically create and use compute resources in the cloud.
Azure Machine Learning supports a range of compute targets, which you can define in your workspace and use to run jobs; paying for the resources only when using them. When you set up the workspace in the first lab, you created a compute cluster called aml-cluster, so let's verify that it exists (and if not, create it) so we can use it to run training jobs.
from azure.ai.ml.entities import AmlCompute
from azure.core.exceptions import ResourceNotFoundError
cluster_name = "aml-cluster"
# Verify that cluster exists
try:
training_cluster = ml_client.compute.get(cluster_name)
print('Found existing cluster, use it.')
except ResourceNotFoundError:
# If not, create it
training_cluster = AmlCompute(
name=cluster_name,
type="amlcompute",
size="Standard_D2as_v4",
min_instances=0,
max_instances=4,
idle_time_before_scale_down=300,
)
training_cluster = ml_client.begin_create_or_update(training_cluster).result()
print(training_cluster.name, "is", training_cluster.provisioning_state)
Run an Experiment on Remote Compute¶
Now that you have created your compute, you can use it to run jobs. The following code creates a folder for job files (which may already exist from the previous lab, but run it anyway!)
import os
# Create a folder for the experiment files
experiment_folder = 'diabetes_training_tree'
os.makedirs(experiment_folder, exist_ok=True)
print(experiment_folder, 'folder created')
Next, create the Python script file for the job. This will overwrite the script you used in the previous lab.
%%writefile $experiment_folder/diabetes_training.py
# Import libraries
import argparse
import os
import pandas as pd
import numpy as np
import joblib
import mlflow
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import roc_auc_score
from sklearn.metrics import roc_curve
import matplotlib.pyplot as plt
parser = argparse.ArgumentParser()
parser.add_argument('--training-data', type=str, dest='training_data', help='path to the training data')
args = parser.parse_args()
# load the diabetes data (passed as an input)
print("Loading Data...")
diabetes = pd.read_csv(args.training_data)
# Separate features and labels
X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values
# Split data into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)
# Train a decision tree model
print('Training a decision tree model')
model = DecisionTreeClassifier().fit(X_train, y_train)
# calculate accuracy
y_hat = model.predict(X_test)
acc = np.average(y_hat == y_test)
print('Accuracy:', acc)
mlflow.log_metric('Accuracy', float(acc))
# calculate AUC
y_scores = model.predict_proba(X_test)
auc = roc_auc_score(y_test,y_scores[:,1])
print('AUC: ' + str(auc))
mlflow.log_metric('AUC', float(auc))
# plot ROC curve
fpr, tpr, thresholds = roc_curve(y_test, y_scores[:,1])
fig = plt.figure(figsize=(6, 4))
# Plot the diagonal 50% line
plt.plot([0, 1], [0, 1], 'k--')
# Plot the FPR and TPR achieved by our model
plt.plot(fpr, tpr)
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curve')
mlflow.log_figure(fig, "ROC.png")
plt.show()
os.makedirs('outputs', exist_ok=True)
# note file saved in the outputs folder is automatically uploaded into the job record
joblib.dump(value=model, filename='outputs/diabetes_model.pkl')
Now you're ready to run the job on the compute you created.
Note: The job will take quite a lot longer because a container image must be built with the conda environment, and then the cluster nodes must be started and the image deployed before the script can be run. For a simple job like the diabetes training script, this may seem inefficient; but imagine you needed to run a more complex job that takes several hours - dynamically creating more scalable compute may reduce the overall time significantly.
from azure.ai.ml import command, Input
from azure.ai.ml.entities import Environment
from azure.ai.ml.constants import AssetTypes
# Define the conda dependencies for the job
conda_spec = {
"name": "diabetes-experiment-env",
"channels": ["conda-forge"],
"dependencies": [
"python=3.10",
"scikit-learn",
"ipykernel",
"matplotlib",
"pandas",
"pip",
{
"pip": [
# No mlflow package - the plugin pulls a compatible version itself.
# Adding it here breaks artifact logging.
"azureml-mlflow",
"pyarrow",
]
},
],
}
# Create (or re-create) the environment, in case the previous lab wasn't completed
diabetes_env = Environment(
name="diabetes-experiment-env",
description="A custom environment for training the diabetes model",
conda_file=conda_spec,
image="mcr.microsoft.com/azureml/openmpi5.0-ubuntu24.04",
)
registered_env = ml_client.environments.create_or_update(diabetes_env)
job = command(
code=experiment_folder,
command="python diabetes_training.py --training-data ${{inputs.diabetes}}",
inputs={"diabetes": Input(type=AssetTypes.URI_FILE, path="data/diabetes.csv")},
environment=registered_env,
compute=cluster_name, # Use the compute target created previously
display_name="diabetes-train-tree-remote",
experiment_name="diabetes-training",
)
returned_job = ml_client.jobs.create_or_update(job)
ml_client.jobs.stream(returned_job.name)
While you're waiting for the job to run, you can check on the status of the compute in the streamed log above or in Azure Machine Learning studio. You can also check the status of the compute using the code below. Note that it will take a while before the state changes from steady to resizing (now might be a good time to take a coffee break!)
Note: The SDK v2 compute object returned here doesn't expose the live node count the way the v1 SDK did - watch the Compute page in Studio to see the nodes scale up in real time.
cluster_status = ml_client.compute.get(cluster_name)
print(f"State: {cluster_status.provisioning_state}")
print(f"Min instances: {cluster_status.min_instances}, Max instances: {cluster_status.max_instances}")
After the job has finished, you can get the metrics and outputs generated by the job. This time, the outputs will include logs for building the environment image and managing the compute.
import mlflow
mlflow.set_tracking_uri(ml_client.workspaces.get(ml_client.workspace_name).mlflow_tracking_uri)
mlflow_run = mlflow.get_run(returned_job.name)
print("Metrics:")
for key, value in mlflow_run.data.metrics.items():
print(f" {key}: {value}")
print(f"\nView the job in Azure Machine Learning studio: {returned_job.studio_url}")
Now you can register the model that was trained by the job.
from azure.ai.ml.entities import Model
from azure.ai.ml.constants import AssetTypes
# Register the model
model = Model(
path=f"azureml://jobs/{returned_job.name}/outputs/artifacts/paths/outputs/diabetes_model.pkl",
type=AssetTypes.CUSTOM_MODEL,
name="diabetes_model",
description="A diabetes classification model",
tags={"Training context": "Command job on aml-cluster (Decision Tree)"},
properties={
"AUC": str(mlflow_run.data.metrics.get("AUC")),
"Accuracy": str(mlflow_run.data.metrics.get("Accuracy")),
},
)
ml_client.models.create_or_update(model)
# List registered models
for m in ml_client.models.list(name="diabetes_model"):
print(m.name, 'version:', m.version)
for tag_name in m.tags:
print('\t', tag_name, ':', m.tags[tag_name])
for prop_name in m.properties:
print('\t', prop_name, ':', m.properties[prop_name])
print('\n')
More Information: For more information about compute targets in Azure Machine Learning, see the documentation.