Training Models¶
The central goal of machine learning is to train predictive models that can be used by applications. In Azure Machine Learning, you can use scripts to train models leveraging common machine learning frameworks like Scikit-Learn, TensorFlow, PyTorch, and others. You can run these training scripts as jobs in order to track metrics and outputs - in particular, the trained models - and then register the resulting models in your workspace.
Connect to Your Workspace¶
The first thing you need to do is to connect to your workspace using the Azure ML SDK v2.
Note: If the authenticated session with your Azure subscription has expired since you completed the previous exercise, you'll be prompted to reauthenticate.
from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential
credential = DefaultAzureCredential()
ml_client = MLClient.from_config(credential=credential)
print(f"Ready to use Azure ML to work with {ml_client.workspace_name}")
Create a Training Script¶
You're going to use a Python script to train a machine learning model based on the diabetes data, so let's start by creating a folder for the script and data files.
import os, shutil
# Create a folder for the experiment files
training_folder = 'diabetes-training'
os.makedirs(training_folder, exist_ok=True)
# Copy the data file into a data subfolder, so the script can load it with the
# same relative path it uses in this repo
os.makedirs(os.path.join(training_folder, 'data'), exist_ok=True)
shutil.copy('data/diabetes.csv', os.path.join(training_folder, 'data', 'diabetes.csv'))
Now you're ready to create the training script and save it in the folder.
The script calls mlflow.sklearn.autolog() so that parameters and metrics are logged for you, without listing each one by hand. The model itself is saved explicitly, to a named job output: autolog can do that too, but that route depends on the MLflow version matching the Azure ML plugin, whereas an explicit save always works.
%%writefile $training_folder/diabetes_training.py
# Import libraries
import argparse
import pandas as pd
import numpy as np
import mlflow
import mlflow.sklearn
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
# Azure ML passes in the folder where the finished model is expected
parser = argparse.ArgumentParser()
parser.add_argument('--model_output', type=str, dest='model_output',
required=True,
help="folder of the job's named output for the finished model")
args = parser.parse_args()
# Log parameters and metrics automatically. We handle the model ourselves -
# see the comment at the end of this script.
mlflow.sklearn.autolog(log_models=False, log_datasets=False)
# Set regularization hyperparameter
reg = 0.01
mlflow.log_param('regularization_rate', reg)
# load the diabetes dataset
print("Loading Data...")
diabetes = pd.read_csv('data/diabetes.csv')
# Separate features and labels
X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values
# Split data into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)
# Train a logistic regression model
print('Training a logistic regression model with regularization rate of', reg)
model = LogisticRegression(C=1/reg, solver="liblinear").fit(X_train, y_train)
# calculate accuracy
y_hat = model.predict(X_test)
acc = np.average(y_hat == y_test)
print('Accuracy:', acc)
mlflow.log_metric('Accuracy', acc)
# calculate AUC
y_scores = model.predict_proba(X_test)
auc = roc_auc_score(y_test,y_scores[:,1])
print('AUC: ' + str(auc))
mlflow.log_metric('AUC', auc)
# Save the model to the job's named output. log_model() will not work here:
# azureml-mlflow supports MLflow 2.16 at the latest.
mlflow.sklearn.save_model(sk_model=model, path=args.model_output)
print('Model saved to:', args.model_output)
Run the Script as a Job¶
Previously, you ran scripts inline in a notebook. Now you'll use a command job to run the training script on compute, so that it's tracked as an Azure Machine Learning job and you can view its metrics, outputs, and registered model in Azure Machine Learning studio.
In this case, you'll use the aml-cluster compute cluster you created earlier, and a curated environment that already includes scikit-learn and MLflow.
from azure.ai.ml import command, Output
from azure.ai.ml.constants import AssetTypes
# configure the job
job = command(
code=training_folder,
command="python diabetes_training.py --model_output ${{outputs.model_output}}",
outputs={
# Nazwane wyjscie: Azure ML przygotuje katalog i zapamieta, ze lezy
# w nim model w formacie MLflow.
"model_output": Output(type=AssetTypes.MLFLOW_MODEL),
},
environment="azureml://registries/azureml/environments/sklearn-1.5/labels/latest",
compute="aml-cluster",
display_name="diabetes-training",
experiment_name="diabetes-training",
)
# submit the job
returned_job = ml_client.jobs.create_or_update(job)
# stream the job logs while it runs
ml_client.jobs.stream(returned_job.name)
While the job runs, you can open it in Azure Machine Learning studio using the link below.
print(returned_job.services["Studio"].endpoint)
You can also retrieve the metrics and parameters logged by the job using the MLflow client, once the job has finished.
from mlflow.tracking import MlflowClient
client = MlflowClient()
job_run = client.get_run(returned_job.name)
print("Metrics:")
for key, value in job_run.data.metrics.items():
print(key, value)
print("\nParameters:")
for key, value in job_run.data.params.items():
print(key, value)
Register the Trained Model¶
The script saved the model to the job output named model_output. Azure ML knows an MLflow model lives there because that is how the output was declared, so registering it is a matter of pointing at that output.
Registering gives the model a name and the next version number. That makes it possible to retrieve it later, and means you won't need a separate scoring script when you deploy it in a later lab.
from azure.ai.ml.entities import Model
from azure.ai.ml.constants import AssetTypes
# Register the model from the job's named output (not from MLflow artifacts)
model = Model(
path=f"azureml://jobs/{returned_job.name}/outputs/model_output",
name="diabetes_model",
type=AssetTypes.MLFLOW_MODEL,
description="Diabetes classification model trained with scikit-learn.",
tags={"training_context": "command job"},
)
registered_model = ml_client.models.create_or_update(model)
print(f"Registered model: {registered_model.name}, version: {registered_model.version}")
# List all versions of the registered model
for m in ml_client.models.list(name="diabetes_model"):
print(m.name, 'version:', m.version)
Create a Parameterized Training Script¶
You can increase the flexibility of your training job by adding parameters to your script, enabling you to repeat the same training job with different settings. In this case, you'll add a parameter for the regularization rate used by the logistic regression algorithm when training the model.
Again, let's start by creating a folder for the parameterized script and the training data.
import os, shutil
# Create a folder for the experiment files
training_folder = 'diabetes-training-params'
os.makedirs(training_folder, exist_ok=True)
# Copy the data file into a data subfolder, so the script can load it with the
# same relative path it uses in this repo
os.makedirs(os.path.join(training_folder, 'data'), exist_ok=True)
shutil.copy('data/diabetes.csv', os.path.join(training_folder, 'data', 'diabetes.csv'))
Now let's create a script containing a parameter for the regularization rate hyperparameter.
%%writefile $training_folder/diabetes_training.py
# Import libraries
import argparse
import pandas as pd
import numpy as np
import mlflow
import mlflow.sklearn
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
# Azure ML passes in the folder where the finished model is expected
parser = argparse.ArgumentParser()
parser.add_argument('--model_output', type=str, dest='model_output',
required=True,
help="folder of the job's named output for the finished model")
parser.add_argument('--reg_rate', type=float, dest='reg', default=0.01)
args = parser.parse_args()
# Log parameters and metrics automatically. We handle the model ourselves -
# see the comment at the end of this script.
mlflow.sklearn.autolog(log_models=False, log_datasets=False)
reg = args.reg
mlflow.log_param('regularization_rate', reg)
# load the diabetes dataset
print("Loading Data...")
diabetes = pd.read_csv('data/diabetes.csv')
# Separate features and labels
X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values
# Split data into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)
# Train a logistic regression model
print('Training a logistic regression model with regularization rate of', reg)
model = LogisticRegression(C=1/reg, solver="liblinear").fit(X_train, y_train)
# calculate accuracy
y_hat = model.predict(X_test)
acc = np.average(y_hat == y_test)
print('Accuracy:', acc)
mlflow.log_metric('Accuracy', acc)
# calculate AUC
y_scores = model.predict_proba(X_test)
auc = roc_auc_score(y_test,y_scores[:,1])
print('AUC: ' + str(auc))
mlflow.log_metric('AUC', auc)
# Save the model to the job's named output. log_model() will not work here:
# azureml-mlflow supports MLflow 2.16 at the latest.
mlflow.sklearn.save_model(sk_model=model, path=args.model_output)
print('Model saved to:', args.model_output)
Pass Parameters to the Job¶
Previously, you ran the training script with its default regularization rate. Now you'll run the parameterized script and pass the --reg_rate argument as part of the job's command, so you can repeat the same training job with different regularization rates without editing the script.
from azure.ai.ml import command, Output
from azure.ai.ml.constants import AssetTypes
# configure the job, passing the regularization rate as a command-line argument
job = command(
code=training_folder,
command="python diabetes_training.py --reg_rate 0.1 --model_output ${{outputs.model_output}}",
outputs={
# Nazwane wyjscie: Azure ML przygotuje katalog i zapamieta, ze lezy
# w nim model w formacie MLflow.
"model_output": Output(type=AssetTypes.MLFLOW_MODEL),
},
environment="azureml://registries/azureml/environments/sklearn-1.5/labels/latest",
compute="aml-cluster",
display_name="diabetes-training",
experiment_name="diabetes-training",
)
# submit the job
returned_job = ml_client.jobs.create_or_update(job)
# stream the job logs while it runs
ml_client.jobs.stream(returned_job.name)
Once again, you can retrieve the metrics and parameters logged by the job.
from mlflow.tracking import MlflowClient
client = MlflowClient()
job_run = client.get_run(returned_job.name)
print("Metrics:")
for key, value in job_run.data.metrics.items():
print(key, value)
print("\nParameters:")
for key, value in job_run.data.params.items():
print(key, value)
Register a New Version of the Model¶
Now that you've trained a new model, you can register it as a new version of diabetes_model in the workspace.
from azure.ai.ml.entities import Model
from azure.ai.ml.constants import AssetTypes
# Register the model from the job's named output (not from MLflow artifacts)
model = Model(
path=f"azureml://jobs/{returned_job.name}/outputs/model_output",
name="diabetes_model",
type=AssetTypes.MLFLOW_MODEL,
description="Diabetes classification model trained with scikit-learn.",
tags={"training_context": "command job"},
)
registered_model = ml_client.models.create_or_update(model)
print(f"Registered model: {registered_model.name}, version: {registered_model.version}")
# List all versions of the registered model
for m in ml_client.models.list(name="diabetes_model"):
print(m.name, 'version:', m.version)
On the File menu, select Close and Halt to close this notebook. Then return to the lab instructions.