sagemaker.core.analytics

Placeholder docstring

Classes

AnalyticsMetricsBase()	Base class for tuning job or training job analytics classes.
ArtifactAnalytics([sort_by, sort_order, ...])	Fetch artifact data and make them accessible for analytics.
ExperimentAnalytics([experiment_name, ...])	Fetch trial component data and make them accessible for analytics.
HyperparameterTuningJobAnalytics(...[, ...])	Fetch results about a hyperparameter tuning job and make them accessible for analytics.
TrainingJobAnalytics(training_job_name[, ...])	Fetch training curve data from CloudWatch Metrics for a specific training job.

class sagemaker.core.analytics.AnalyticsMetricsBase

Bases: object

Base class for tuning job or training job analytics classes.

Understands common functionality like persistence and caching.

clear_cache()

Clear the object of all local caches of API methods.

So that the next time any properties are accessed they will be refreshed from the service.

dataframe(force_refresh=False)

A pandas dataframe with lots of interesting results about this object.

Created by calling SageMaker List and Describe APIs and converting them into a convenient tabular summary.

Parameters:

force_refresh (bool) – Set to True to fetch the latest data from SageMaker API.

export_csv(filename)

Persists the analytics dataframe to a file.

Parameters:

filename (str) – The name of the file to save to.

class sagemaker.core.analytics.ArtifactAnalytics(sort_by=None, sort_order=None, source_uri=None, artifact_type=None, sagemaker_session=None)

Bases: AnalyticsMetricsBase

Fetch artifact data and make them accessible for analytics.

class sagemaker.core.analytics.ExperimentAnalytics(experiment_name=None, search_expression=None, sort_by=None, sort_order=None, metric_names=None, parameter_names=None, sagemaker_session=None, input_artifact_names=None, output_artifact_names=None)

Bases: AnalyticsMetricsBase

Fetch trial component data and make them accessible for analytics.

MAX_TRIAL_COMPONENTS = 10000

clear_cache()

Clear the object of all local caches of API methods.

property name

Name of the Experiment being analyzed.

class sagemaker.core.analytics.HyperparameterTuningJobAnalytics(hyperparameter_tuning_job_name, sagemaker_session=None)

Bases: AnalyticsMetricsBase

Fetch results about a hyperparameter tuning job and make them accessible for analytics.

clear_cache()

Clear the object of all local caches of API methods.

description(force_refresh=False)

Call DescribeHyperParameterTuningJob for the hyperparameter tuning job.

Parameters:

force_refresh (bool) – Set to True to fetch the latest data from SageMaker API.

Returns:

The Amazon SageMaker response for DescribeHyperParameterTuningJob.

Return type:

dict

property name

Name of the HyperparameterTuningJob being analyzed

training_job_summaries(force_refresh=False)

A (paginated) list of everything from ListTrainingJobsForTuningJob.

Parameters:

force_refresh (bool) – Set to True to fetch the latest data from SageMaker API.

Returns:

The Amazon SageMaker response for ListTrainingJobsForTuningJob.

Return type:

dict

property tuning_ranges

A dictionary describing the ranges of all tuned hyperparameters.

The keys are the names of the hyperparameter, and the values are the ranges.

The output can take one of two forms:

• If the ‘TrainingJobDefinition’ field is present in the job description, the output

  is a dictionary constructed from ‘ParameterRanges’ in ‘HyperParameterTuningJobConfig’ of the job description. The keys are the parameter names, while the values are the parameter ranges. Example: >>> { >>> “eta”: {“MaxValue”: “1”, “MinValue”: “0”, “Name”: “eta”}, >>> “gamma”: {“MaxValue”: “10”, “MinValue”: “0”, “Name”: “gamma”}, >>> “iterations”: {“MaxValue”: “100”, “MinValue”: “50”, “Name”: “iterations”}, >>> “num_layers”: {“MaxValue”: “30”, “MinValue”: “5”, “Name”: “num_layers”}, >>> }

• If the ‘TrainingJobDefinitions’ field (list) is present in the job description,

  the output is a dictionary with keys as the ‘DefinitionName’ values from all items in ‘TrainingJobDefinitions’, and each value would be a dictionary constructed from ‘HyperParameterRanges’ in each item in ‘TrainingJobDefinitions’ in the same format as above Example: >>> { >>> “estimator_1”: { >>> “eta”: {“MaxValue”: “1”, “MinValue”: “0”, “Name”: “eta”}, >>> “gamma”: {“MaxValue”: “10”, “MinValue”: “0”, “Name”: “gamma”}, >>> }, >>> “estimator_2”: { >>> “framework”: {“Values”: [“TF”, “MXNet”], “Name”: “framework”}, >>> “gamma”: {“MaxValue”: “1.0”, “MinValue”: “0.2”, “Name”: “gamma”} >>> } >>> }

For more details about the ‘TrainingJobDefinition’ and ‘TrainingJobDefinitions’ fields in job description, see https://botocore.readthedocs.io/en/latest/reference/services/sagemaker.html#SageMaker.Client.create_hyper_parameter_tuning_job

class sagemaker.core.analytics.TrainingJobAnalytics(training_job_name, metric_names=None, sagemaker_session=None, start_time=None, end_time=None, period=None)

Bases: AnalyticsMetricsBase

Fetch training curve data from CloudWatch Metrics for a specific training job.

CLOUDWATCH_NAMESPACE = '/aws/sagemaker/TrainingJobs'

clear_cache()

Clear the object of all local caches of API methods.

This is so that the next time any properties are accessed they will be refreshed from the service.

property name

Name of the TrainingJob being analyzed