Instrukcja do tego ćwiczenia Pobierz notatnik (.ipynb)

Running Experiments¶

You can use the Azure Machine Learning SDK to submit and track jobs that run code, log metrics, and generate outputs. This is at the core of most machine learning operations in Azure Machine Learning.

Connect to Your Workspace¶

The first thing you need to do is to connect to your workspace using the Azure ML SDK v2.

Note: If the authenticated session with your Azure subscription has expired since you completed the previous exercise, you'll be prompted to reauthenticate.

In [ ]:

from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()
ml_client = MLClient.from_config(credential=credential)

print(f"Ready to use Azure ML to work with {ml_client.workspace_name}")

Track a Run with MLflow¶

One of the most fundamental tasks that data scientists need to perform is to create and run experiments that process and analyze data. In this exercise, you'll learn how to use MLflow tracking - which is built into Azure Machine Learning - to run Python code in this notebook and record values extracted from data. In this case, you'll use a simple dataset that contains details of patients that have been tested for diabetes. You'll run some code to explore the data, extracting statistics, visualizations, and data samples. Most of the code you'll use is fairly generic Python, such as you might run in any data exploration process. However, with the addition of a few lines, the code uses MLflow to log details of the run to your Azure Machine Learning workspace.

In [ ]:

import mlflow
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

# Start an MLflow run to track this exploration
run = mlflow.start_run(run_name="diabetes-exploration")
print("Starting run:", run.info.run_id)

# load the data from a local file
data = pd.read_csv('data/diabetes.csv')

# Count the rows and log the result
row_count = (len(data))
mlflow.log_metric('observations', row_count)
print('Analyzing {} rows of data'.format(row_count))

# Plot and log the count of diabetic vs non-diabetic patients
diabetic_counts = data['Diabetic'].value_counts()
fig = plt.figure(figsize=(6,6))
ax = fig.gca()    
diabetic_counts.plot.bar(ax = ax) 
ax.set_title('Patients with Diabetes') 
ax.set_xlabel('Diagnosis') 
ax.set_ylabel('Patients')
plt.show()
fig.savefig('label-distribution.png')
mlflow.log_artifact('label-distribution.png')

# log distinct pregnancy counts
pregnancies = sorted(data.Pregnancies.unique().tolist())
mlflow.log_param('pregnancy_categories', pregnancies)

# Log summary statistics for numeric columns
med_columns = ['PlasmaGlucose', 'DiastolicBloodPressure', 'TricepsThickness', 'SerumInsulin', 'BMI']
summary_stats = data[med_columns].describe().to_dict()
for col in summary_stats:
    for stat, value in summary_stats[col].items():
        mlflow.log_metric(f"{col}_{stat}", value)

# Save a sample of the data and log it as an artifact
data.sample(100).to_csv('sample.csv', index=False, header=True)
mlflow.log_artifact('sample.csv')

# End the run
mlflow.end_run()

View Run Results¶

After the run has finished, you can use the MLflow client to get information about the run and its outputs:

In [ ]:

import json
from mlflow.tracking import MlflowClient

client = MlflowClient()
run_data = client.get_run(run.info.run_id)

# Get logged metrics
print("Metrics:")
print(json.dumps(run_data.data.metrics, indent=2))

# Get logged parameters
print("\nParameters:")
print(json.dumps(run_data.data.params, indent=2))

# Get output artifacts
print("\nArtifacts:")
for artifact in client.list_artifacts(run.info.run_id):
    print(artifact.path)

You can follow a job's progress and review its logged metrics directly in Azure Machine Learning studio.

Open Azure Machine Learning studio, select Jobs, and find the run under the diabetes-experiment experiment. When viewing the run in Azure Machine Learning studio, note the following:

The Overview tab shows the general properties of the run.
The Metrics tab enables you to select logged metrics and view them as tables or charts.
The Images tab enables you to select and view any images or plots that were logged in the run (in this case, the label-distribution.png plot).
The Outputs + logs tab shows the output files and artifacts generated by the run.
The Code tab shows a snapshot of the files used to produce the run.

Run a Script as a Job¶

In the previous example, you ran code inline in this notebook. A more flexible solution is to create a separate script for the code, store it in a folder along with any other files it needs, and then use Azure ML to run it as a command job on compute you choose. This makes it easy to move from ad hoc exploration on a compute instance to reproducible, scalable runs on a compute cluster.

First, let's create a folder for the script files, and copy the data into it:

In [ ]:

import os, shutil

# Create a folder for the experiment files
folder_name = 'diabetes-experiment-files'
experiment_folder = './' + folder_name
os.makedirs(folder_name, exist_ok=True)

# Copy the data file into a data subfolder, so the script can load it with the
# same relative path it uses in this repo
os.makedirs(os.path.join(folder_name, 'data'), exist_ok=True)
shutil.copy('data/diabetes.csv', os.path.join(folder_name, 'data', 'diabetes.csv'))

Now we'll create a Python script containing the code for our job, and save it in the folder.

Note: running the following cell just creates the script file - it doesn't run it!

In [ ]:

%%writefile $folder_name/diabetes_experiment.py
import pandas as pd
import os
import mlflow

# load the diabetes dataset
data = pd.read_csv('data/diabetes.csv')

# Count the rows and log the result
row_count = (len(data))
mlflow.log_metric('observations', row_count)
print('Analyzing {} rows of data'.format(row_count))

# Count and log the label counts
diabetic_counts = data['Diabetic'].value_counts()
print(diabetic_counts)
for k, v in diabetic_counts.items():
    mlflow.log_metric('Label:' + str(k), v)

# Save a sample of the data in the outputs folder (job outputs are captured automatically)
os.makedirs('outputs', exist_ok=True)
data.sample(100).to_csv("outputs/sample.csv", index=False, header=True)

Note the following about this script:

It uses mlflow.log_metric() to log metrics. Azure Machine Learning attaches an MLflow run to every job automatically, so anything you log this way ends up in the job's history.
It loads the diabetes data from the data subfolder next to the script. Everything in the job's code folder is uploaded with the job, so the script finds the file at the same relative path when it runs in the cloud.
It creates a folder named outputs and writes the sample file to it - files written to outputs are automatically captured as part of the job.

Now you're almost ready to run the script as a job. There are just a few things you need to define:

The environment in which to run the script - in this case, we'll use a curated Azure Machine Learning environment that already includes common Python packages.
The compute on which to run the job - in this case, the aml-cluster compute cluster you created in an earlier lab.
The command that specifies how to run the script.

Note: Don't worry too much about the environment configuration for now - we'll explore it in more depth later.

The following cell configures a command job, and then submits it.

In [ ]:

from azure.ai.ml import command

# configure the job
job = command(
    code=experiment_folder,
    command="python diabetes_experiment.py",
    environment="azureml://registries/azureml/environments/sklearn-1.5/labels/latest",
    compute="aml-cluster",
    display_name="diabetes-experiment",
    experiment_name="diabetes-experiment",
)

# submit the job
returned_job = ml_client.jobs.create_or_update(job)

# stream the job logs while it runs
ml_client.jobs.stream(returned_job.name)

As before, you can use Azure Machine Learning studio to view the outputs generated by the job, and you can also write code to retrieve the metrics and files it generated:

In [ ]:

from mlflow.tracking import MlflowClient

client = MlflowClient()
job_run = client.get_run(returned_job.name)

# Get logged metrics
print("Metrics:")
for key, value in job_run.data.metrics.items():
    print(key, value)

print("\nOutput files:")
for artifact in client.list_artifacts(returned_job.name):
    print(artifact.path)

View Job History¶

Now that you've run the same script multiple times, you can view the history in Azure Machine Learning studio and explore each job. Or you can list jobs and filter by experiment name using the SDK:

In [ ]:

for logged_job in ml_client.jobs.list():
    if logged_job.experiment_name == "diabetes-experiment":
        print('Job name:', logged_job.name)
        print('Status:', logged_job.status)
        print('Display name:', logged_job.display_name)

More Information: To find out more about running jobs, see Train models with Azure Machine Learning CLI, SDK, and REST API in the Azure ML documentation. For details of how to log metrics with MLflow, see Log metrics, parameters, and files with MLflow.