sagemaker.core.common_utils

sagemaker.core.common_utils#

Placeholder docstring

Functions

`aws_partition`(region)	Given a region name (ex: "cn-north-1"), return the corresponding aws partition ("aws-cn").
`base_from_name`(name)	Extract the base name of the resource name (for use with future resource name generation).
`base_name_from_image`(image[, default_base_name])	Extract the base name of the image to use as the 'algorithm name' for the job.
`build_dict`(key, value)	Return a dict of key and value pair if value is not None, otherwise return an empty dict.
`camel_case_to_pascal_case`(data)	Iteratively updates a dictionary to convert all keys from snake_case to PascalCase.
`camel_to_snake`(camel_case_string)	Converts camelCase to snake_case_string using a regex.
`can_model_package_source_uri_autopopulate`(...)	Checks if the source_uri can lead to auto-population of information in the Model registry.
`check_and_get_run_experiment_config`([...])	Check user input experiment_config or get it from the current Run object if exists.
`construct_container_object`(obj, ...)	Function to construct container object.
`create_paginator_config`([max_items, page_size])	Placeholder docstring
`create_tar_file`(source_files[, target])	Create a tar file containing all the source_files
`custom_extractall_tarfile`(tar, extract_path)	Extract a tarfile, optionally using data_filter if available.
`deep_override_dict`(dict1, dict2[, skip_keys])	Overrides any overlapping contents of dict1 with the contents of dict2.
`download_file`(bucket_name, path, target, ...)	Download a Single File from S3 into a local path
`download_file_from_url`(url, dst, ...)	Placeholder docstring
`download_folder`(bucket_name, prefix, target, ...)	Download a folder from S3 to a local path
`extract_instance_rate_per_hour`(price_data)	Extract instance rate per hour for the given Price JSON data.
`flatten_dict`(d[, max_flatten_depth])	Flatten a dictionary object.
`format_tags`(tags)	Process tags to turn them into the expected format for Sagemaker.
`get_config_value`(key_path, config)	Placeholder Docstring
`get_domain_for_region`(region)	Returns the domain for the given region.
`get_instance_rate_per_hour`(instance_type, region)	Gets instance rate per hour for the given instance type.
`get_instance_type_family`(instance_type)	Return the family of the instance type.
`get_module`(module_name)	Import a module.
`get_nested_value`(dictionary, nested_keys)	Returns a nested value from the given dictionary, and None if none present.
`get_resource_name_from_arn`(arn)	Extract the resource name from an ARN string.
`get_sagemaker_config_value`(...[, ...])	Returns the value that corresponds to the provided key from the configuration file.
`get_short_version`(framework_version)	Return short version in the format of x.x
`instance_supports_kms`(instance_type)	Returns True if SageMaker allows KMS keys to be attached to the instance.
`list_tags`(sagemaker_session, resource_arn[, ...])	List the tags given an Amazon Resource Name.
`name_from_base`(base[, max_length, short])	Append a timestamp to the provided string.
`name_from_image`(image[, max_length])	Create a training job name based on the image name and a timestamp.
`nested_set_dict`(d, keys, value)	Set a value to a sequence of nested keys.
`pop_out_unused_kwarg`(arg_name, kwargs[, ...])	Pop out the unused key-word argument and give a warning.
`remove_tag_with_key`(key, tags)	Remove a tag with the given key from the list of tags.
`repack_model`(inference_script, ...[, kms_key])	Unpack model tarball and creates a new model tarball with the provided code script.
`resolve_class_attribute_from_config`(clazz, ...)	Utility method that merges config values to data classes.
`resolve_nested_dict_value_from_config`(...[, ...])	Utility method that sets the value of a key path in a nested dictionary .
`resolve_value_from_config`([direct_input, ...])	Decides which value for the caller to use.
`retries`(max_retry_count, ...[, seconds_to_sleep])	Retries until max retry count is reached.
`retry_with_backoff`(callable_func[, ...])	Retry with backoff until maximum attempts are reached
`sagemaker_short_timestamp`()	Return a timestamp that is relatively short in length
`sagemaker_timestamp`()	Return a timestamp with millisecond precision.
`secondary_training_status_changed`(...)	Returns true if training job's secondary status message has changed.
`secondary_training_status_message`(...)	Returns a string contains last modified time and the secondary training job status message.
`set_nested_value`(dictionary, nested_keys, ...)	Sets a nested value in a dictionary.
`stringify_object`(obj)	Returns string representation of object, returning only non-None fields.
`sts_regional_endpoint`(region)	Get the AWS STS endpoint specific for the given region.
`tag_exists`(tag, curr_tags)	Returns True if `tag` already exists.
`to_string`(obj)	Convert an object to string
`unflatten_dict`(d)	Unflatten dict-like object.
`unique_name_from_base`(base[, max_length])	Placeholder Docstring
`unique_name_from_base_uuid4`(base[, max_length])	Append a UUID to the provided string.
`update_container_with_inference_params`([...])	Function to check if inference recommender parameters exist and update container.
`update_list_of_dicts_with_values_from_config`(...)	Updates a list of dictionaries with missing values that are present in Config.
`update_nested_dictionary_with_values_from_config`(...)	Updates a nested dictionary with missing values that are present in Config.
`volume_size_supported`(instance_type)	Returns True if SageMaker allows volume_size to be used for the instance type.
`walk_and_apply_json`(json_obj, apply[, stop_keys])	Recursively walks a json object and applies a given function to the keys.

Classes

`DataConfig`()	Abstract base class for accessing data config hosted in AWS resources.
`DeferredError`(exception)	Stores an exception and raises it at a later time if this object is accessed in any way.
`LogState`()	Placeholder docstring
`ModelApprovalStatusEnum`(value)	Model package approval status enumerator
`S3DataConfig`(sagemaker_session, bucket_name, ...)	This class extends the DataConfig class to fetch a data config file hosted on S3

class sagemaker.core.common_utils.DataConfig[source]#

Bases: ABC

Abstract base class for accessing data config hosted in AWS resources.

Provides a skeleton for customization by overriding of method fetch_data_config.

abstract fetch_data_config()[source]#

Abstract method implementing retrieval of data config from a pre-configured data source.

Returns:: The data configuration object.
Return type:: object

class sagemaker.core.common_utils.DeferredError(exception)[source]#

Bases: object

Stores an exception and raises it at a later time if this object is accessed in any way.

Useful to allow soft-dependencies on imports, so that the ImportError can be raised again later if code actually relies on the missing library.

Example:

try:
    import obscurelib
except ImportError as e:
    logger.warning("Failed to import obscurelib. Obscure features will not work.")
    obscurelib = DeferredError(e)

class sagemaker.core.common_utils.LogState[source]#

Bases: object

Placeholder docstring

COMPLETE = 5#

JOB_COMPLETE = 4#

STARTING = 1#

TAILING = 3#

WAIT_IN_PROGRESS = 2#

class sagemaker.core.common_utils.ModelApprovalStatusEnum(value)[source]#

Bases: str, Enum

Model package approval status enumerator

APPROVED = 'Approved'#

PENDING_MANUAL_APPROVAL = 'PendingManualApproval'#

REJECTED = 'Rejected'#

class sagemaker.core.common_utils.S3DataConfig(sagemaker_session, bucket_name, prefix)[source]#

Bases: DataConfig

This class extends the DataConfig class to fetch a data config file hosted on S3

fetch_data_config()[source]#

Fetches data configuration from a S3 bucket.

Returns:: The JSON object containing data configuration.
Return type:: object

get_data_bucket(region_requested=None)[source]#

Provides the bucket containing the data for specified region.

Parameters:: region_requested (str) – The region for which the data is beig requested.
Returns:: Name of the S3 bucket containing datasets in the requested region.
Return type:: str

sagemaker.core.common_utils.aws_partition(region)[source]#

Given a region name (ex: “cn-north-1”), return the corresponding aws partition (“aws-cn”).

Parameters:

region (str) – The region name for which to return the corresponding partition.
Ex – “cn-north-1”

Returns:

partition corresponding to the region name passed in. Ex: “aws-cn”

Return type:

str

sagemaker.core.common_utils.base_from_name(name)[source]#

Extract the base name of the resource name (for use with future resource name generation).

This function looks for timestamps that match the ones produced by name_from_base().

Parameters:: name (str) – The resource name.
Returns:: The base name, as extracted from the resource name.
Return type:: str

sagemaker.core.common_utils.base_name_from_image(image, default_base_name=None)[source]#

Extract the base name of the image to use as the ‘algorithm name’ for the job.

Parameters:

image (str) – Image name.
default_base_name (str) – The default base name

Returns:

Algorithm name, as extracted from the image name.

Return type:

str

sagemaker.core.common_utils.build_dict(key, value)[source]#

Return a dict of key and value pair if value is not None, otherwise return an empty dict.

Parameters:

key (str) – input key
value (str) – input value

Returns:

dict of key and value or an empty dict.

Return type:

dict

sagemaker.core.common_utils.camel_case_to_pascal_case(data: Dict[str, Any]) → Dict[str, Any][source]#

Iteratively updates a dictionary to convert all keys from snake_case to PascalCase.

Parameters:: data (dict) – The dictionary to be updated.
Returns:: The updated dictionary with keys in PascalCase.
Return type:: dict

sagemaker.core.common_utils.camel_to_snake(camel_case_string: str) → str[source]#

Converts camelCase to snake_case_string using a regex.

This regex cannot handle whitespace (“camelString TwoWords”)

sagemaker.core.common_utils.can_model_package_source_uri_autopopulate(source_uri: str)[source]#

Checks if the source_uri can lead to auto-population of information in the Model registry.

Parameters:: source_uri (str) – The source uri.
Returns:: True if the source_uri can lead to auto-population, False otherwise.
Return type:: bool

sagemaker.core.common_utils.check_and_get_run_experiment_config(experiment_config: dict | None = None) → dict[source]#

Check user input experiment_config or get it from the current Run object if exists.

Parameters:

experiment_config (dict) – The experiment_config supplied by the user.

Returns:

Return the user supplied experiment_config if it is not None.: Otherwise fetch the experiment_config from the current Run object if exists.

Return type:

dict

sagemaker.core.common_utils.construct_container_object(obj, data_input_configuration, framework, framework_version, nearest_model_name)[source]#

Function to construct container object.

Parameters:

framework (str) – Machine learning framework of the model package container image (default: None).
framework_version (str) – Framework version of the Model Package Container Image (default: None).
nearest_model_name (str) – Name of a pre-trained machine learning benchmarked by Amazon SageMaker Inference Recommender (default: None).
data_input_configuration (str) – Input object for the model (default: None).
obj (dict) – object to be updated.

Returns:

container object

Return type:

dict

sagemaker.core.common_utils.create_paginator_config(max_items: int | None = None, page_size: int | None = None) → Dict[str, int][source]#: Placeholder docstring

sagemaker.core.common_utils.create_tar_file(source_files, target=None)[source]#

Create a tar file containing all the source_files

Parameters:

source_files – (List[str]): List of file paths that will be contained in the tar file
target

Returns:

path to created tar file

Return type:

(str)

sagemaker.core.common_utils.custom_extractall_tarfile(tar, extract_path)[source]#

Extract a tarfile, optionally using data_filter if available.

# TODO: The function and it’s usages can be deprecated once SageMaker Python SDK is upgraded to use Python 3.12+

If the tarfile has a data_filter attribute, it will be used to extract the contents of the file. Otherwise, the _get_safe_members function will be used to filter bad paths and bad links.

Parameters:

tar (tarfile.TarFile) – The opened tarfile object.
extract_path (str) – The path to extract the contents of the tarfile.

Returns:

None

sagemaker.core.common_utils.deep_override_dict(dict1: Dict[str, Any], dict2: Dict[str, Any], skip_keys: List[str] | None = None) → Dict[str, Any][source]#: Overrides any overlapping contents of dict1 with the contents of dict2.

sagemaker.core.common_utils.download_file(bucket_name, path, target, sagemaker_session)[source]#

Download a Single File from S3 into a local path

Parameters:

bucket_name (str) – S3 bucket name
path (str) – file path within the bucket
target (str) – destination directory for the downloaded file.
sagemaker_session (sagemaker.core.helper.session.Session) – a sagemaker session to interact with S3.

sagemaker.core.common_utils.download_file_from_url(url, dst, sagemaker_session)[source]#: Placeholder docstring

sagemaker.core.common_utils.download_folder(bucket_name, prefix, target, sagemaker_session)[source]#

Download a folder from S3 to a local path

Parameters:

bucket_name (str) – S3 bucket name
prefix (str) – S3 prefix within the bucket that will be downloaded. Can be a single file.
target (str) – destination path where the downloaded items will be placed
sagemaker_session (sagemaker.core.helper.session.Session) – a sagemaker session to interact with S3.

sagemaker.core.common_utils.extract_instance_rate_per_hour(price_data: Dict[str, Any]) → Dict[str, str] | None[source]#

Extract instance rate per hour for the given Price JSON data.

Parameters:: price_data (Dict[str, Any]) – The Price JSON data.
Returns:: Instance rate per hour.
Return type:: Optional[Dict[str, str], None]

sagemaker.core.common_utils.flatten_dict(d: Dict[str, Any], max_flatten_depth=None) → Dict[str, Any][source]#

Flatten a dictionary object.

d (Dict[str, Any]):: The dict that will be flattened.
max_flatten_depth (Optional[int]):: Maximum depth to merge.

sagemaker.core.common_utils.format_tags(tags: List[Dict[str, str | PipelineVariable]] | Dict[str, str | PipelineVariable]) → List[Dict[str, str | PipelineVariable]][source]#: Process tags to turn them into the expected format for Sagemaker.

sagemaker.core.common_utils.get_config_value(key_path, config)[source]#: Placeholder Docstring

sagemaker.core.common_utils.get_domain_for_region(region: str) → str[source]#

Returns the domain for the given region.

Parameters:: region (str) – AWS region name.

sagemaker.core.common_utils.get_instance_rate_per_hour(instance_type: str, region: str) → Dict[str, str] | None[source]#

Gets instance rate per hour for the given instance type.

Parameters:

instance_type (str) – The instance type.
region (str) – The region.

Returns:

Instance rate per hour. Example: {‘name’: ‘Instance Rate’, ‘unit’: ‘USD/Hrs’, ‘value’: ‘1.125’}.

Return type:

Optional[Dict[str, str]]

Raises:

Exception – An exception is raised if the IAM role is not authorized to perform pricing:GetProducts. or unexpected event happened.

sagemaker.core.common_utils.get_instance_type_family(instance_type: str) → str[source]#

Return the family of the instance type.

Regex matches either “ml.<family>.<size>” or “ml_<family>. If input is None or there is no match, return an empty string.

sagemaker.core.common_utils.get_module(module_name)[source]#

Import a module.

Parameters:: module_name (str) – name of the module to import.
Returns:: The imported module.
Return type:: object
Raises:: Exception – when the module name is not found

sagemaker.core.common_utils.get_nested_value(dictionary: dict, nested_keys: List[str])[source]#

Returns a nested value from the given dictionary, and None if none present.

Raises: ValueError if the dictionary structure does not match the nested_keys

sagemaker.core.common_utils.get_resource_name_from_arn(arn)[source]#

Extract the resource name from an ARN string.

Parameters:: arn (str) – An ARN.
Returns:: The resource name.
Return type:: str

sagemaker.core.common_utils.get_sagemaker_config_value(sagemaker_session, key, sagemaker_config: dict | None = None)[source]#

Returns the value that corresponds to the provided key from the configuration file.

Parameters:

key – Key Path of the config file entry.
sagemaker_session (sagemaker.core.helper.session.Session) – A SageMaker Session object, used for SageMaker interactions.
sagemaker_config (dict) – The sdk defaults config that is normally accessed through a Session object by doing session.sagemaker_config. (default: None) This parameter will be checked for the config value if (and only if) sagemaker_session is None. This parameter exists for the rare cases where no Session provided but a default Session cannot be initialized before config injection is needed. In that case, the config dictionary may be loaded and passed here before a default Session object is created.

Returns:

The corresponding default value in the configuration file.

Return type:

object

sagemaker.core.common_utils.get_short_version(framework_version)[source]#

Return short version in the format of x.x

Parameters:: framework_version – The version string to be shortened.
Returns:: The short version string
Return type:: str

sagemaker.core.common_utils.instance_supports_kms(instance_type: str) → bool[source]#

Returns True if SageMaker allows KMS keys to be attached to the instance.

Raises:: ValueError – If the instance type is improperly formatted.

sagemaker.core.common_utils.list_tags(sagemaker_session, resource_arn, max_results=50)[source]#

List the tags given an Amazon Resource Name.

Parameters:

resource_arn (str) – The Amazon Resource Name (ARN) for which to get the tags list.
max_results (int) – The maximum number of results to include in a single page. This method takes care of that abstraction and returns a full list.

sagemaker.core.common_utils.name_from_base(base, max_length=63, short=False)[source]#

Append a timestamp to the provided string.

This function assures that the total length of the resulting string is not longer than the specified max length, trimming the input parameter if necessary.

Parameters:

base (str) – String used as prefix to generate the unique name.
max_length (int) – Maximum length for the resulting string (default: 63).
short (bool) – Whether or not to use a truncated timestamp (default: False).

Returns:

Input parameter with appended timestamp.

Return type:

str

sagemaker.core.common_utils.name_from_image(image, max_length=63)[source]#

Create a training job name based on the image name and a timestamp.

Parameters:

image (str) – Image name.

Returns:

Training job name using the algorithm from the image name and a: timestamp.

max_length (int): Maximum length for the resulting string (default: 63).

Return type:

str

sagemaker.core.common_utils.nested_set_dict(d: Dict[str, Any], keys: List[str], value: Any) → None[source]#: Set a value to a sequence of nested keys.

sagemaker.core.common_utils.pop_out_unused_kwarg(arg_name: str, kwargs: dict, override_val: str | None = None)[source]#

Pop out the unused key-word argument and give a warning.

Parameters:

arg_name (str) – The name of the argument to be checked if it is unused.
kwargs (dict) – The key-word argument dict.
override_val (str) – The value used to override the unused argument (default: None).

Remove a tag with the given key from the list of tags.

Parameters:

key (str) – The key of the tag to remove.
tags (Optional[Tags]) – The current list of tags.

Returns:

The updated list of tags with the tag removed.

Return type:

Optional[Tags]

sagemaker.core.common_utils.repack_model(inference_script, source_directory, dependencies, model_uri, repacked_model_uri, sagemaker_session, kms_key=None)[source]#

Unpack model tarball and creates a new model tarball with the provided code script.

This function does the following: - uncompresses model tarball from S3 or local system into a temp folder - replaces the inference code from the model with the new code provided - compresses the new model tarball and saves it in S3 or local file system

Parameters:

inference_script (str) – path or basename of the inference script that will be packed into the model
source_directory (str) – path including all the files that will be packed into the model
dependencies (list[str]) –
A list of paths to directories (absolute or relative) with any additional libraries that will be exported to the container (default: []). The library folders will be copied to SageMaker in the same folder where the entrypoint is copied. Example
The following call >>> Estimator(entry_point=’train.py’, dependencies=[‘my/libs/common’, ‘virtual-env’]) results in the following inside the container:
>>> $ ls
>>> opt/ml/code >>> |------ train.py >>> |------ common >>> |------ virtual-env
model_uri (str) – S3 or file system location of the original model tar
repacked_model_uri (str) – path or file system location where the new model will be saved
sagemaker_session (sagemaker.core.helper.session.Session) – a sagemaker session to interact with S3.
kms_key (str) – KMS key ARN for encrypting the repacked model file

Returns:

path to the new packed model

Return type:

str

sagemaker.core.common_utils.resolve_class_attribute_from_config(clazz: type | None, instance: object | None, attribute: str, config_path: str, default_value=None, sagemaker_session=None)[source]#

Utility method that merges config values to data classes.

Takes an instance of a class and, if not already set, sets the instance’s attribute to a value fetched from the sagemaker_config or the default_value.

Uses this order of prioritization to determine what the value of the attribute should be: 1. current value of attribute 2. config value 3. default_value 4. does not set it

Parameters:

clazz (Optional[type]) – Class of ‘instance’. Used to generate a new instance if the instance is None. If None is provided here, no new object will be created if ‘instance’ doesnt exist. Note: if provided, the constructor should set default values to None; Otherwise, the constructor’s non-None default will be left as-is even if a config value was defined.
instance (Optional[object]) – instance of the Class ‘clazz’ that has an attribute of ‘attribute’ to set
attribute (str) – attribute of the instance to set if not already set
config_path (str) – a string denoting the path to use to lookup the config value in the sagemaker config
default_value – the value to use if not present elsewhere
sagemaker_session (sagemaker.core.helper.session.Sessionn) – A SageMaker Session object, used for SageMaker interactions (default: None).

Returns:

The updated class instance that should be used by the caller instead of the ‘instance’ parameter that was passed in.

sagemaker.core.common_utils.resolve_nested_dict_value_from_config(dictionary: dict, nested_keys: List[str], config_path: str, default_value: object | None = None, sagemaker_session=None)[source]#

Utility method that sets the value of a key path in a nested dictionary .

This method takes a dictionary and, if not already set, sets the value for the provided list of nested keys to the value fetched from the sagemaker_config or the default_value.

Uses this order of prioritization to determine what the value of the attribute should be: (1) current value of nested key, (2) config value, (3) default_value, (4) does not set it

Parameters:

dictionary – The dict to update.
nested_keys – The paths of keys where the value should be checked and set if needed.
config_path (str) – A string denoting the path used to find the config value in the
config. (sagemaker)
default_value – The value to use if not present elsewhere.
sagemaker_session (sagemaker.core.helper.session.Session) – A SageMaker Session object, used for SageMaker interactions (default: None).

Returns:

The updated dictionary that should be used by the caller instead of the ‘dictionary’ parameter that was passed in.

sagemaker.core.common_utils.resolve_value_from_config(direct_input=None, config_path: str | None = None, default_value=None, sagemaker_session=None, sagemaker_config: dict | None = None)[source]#

Decides which value for the caller to use.

Note: This method incorporates information from the sagemaker config.

Uses this order of prioritization: 1. direct_input 2. config value 3. default_value 4. None

Parameters:

direct_input – The value that the caller of this method starts with. Usually this is an input to the caller’s class or method.
config_path (str) – A string denoting the path used to lookup the value in the sagemaker config.
default_value – The value used if not present elsewhere.
sagemaker_session (sagemaker.core.helper.session.Session) – A SageMaker Session object, used for SageMaker interactions (default: None).
sagemaker_config (dict) – The sdk defaults config that is normally accessed through a Session object by doing session.sagemaker_config. (default: None) This parameter will be checked for the config value if (and only if) sagemaker_session is None. This parameter exists for the rare cases where the user provided no Session but a default Session cannot be initialized before config injection is needed. In that case, the config dictionary may be loaded and passed here before a default Session object is created.

Returns:

The value that should be used by the caller

sagemaker.core.common_utils.retries(max_retry_count, exception_message_prefix, seconds_to_sleep=10)[source]#

Retries until max retry count is reached.

Parameters:

max_retry_count (int) – The retry count.
exception_message_prefix (str) – The message to include in the exception on failure.
seconds_to_sleep (int) – The number of seconds to sleep between executions.

sagemaker.core.common_utils.retry_with_backoff(callable_func, num_attempts=8, botocore_client_error_code=None)[source]#

Retry with backoff until maximum attempts are reached

Parameters:

callable_func (Callable) – The callable function to retry.
num_attempts (int) – The maximum number of attempts to retry.(Default: 8)
botocore_client_error_code (str) – The specific Botocore ClientError exception error code on which to retry on. If provided other exceptions will be raised directly w/o retry. If not provided, retry on any exception. (Default: None)

sagemaker.core.common_utils.sagemaker_short_timestamp()[source]#: Return a timestamp that is relatively short in length

sagemaker.core.common_utils.sagemaker_timestamp()[source]#: Return a timestamp with millisecond precision.

sagemaker.core.common_utils.secondary_training_status_changed(current_job_description, prev_job_description)[source]#

Returns true if training job’s secondary status message has changed.

Parameters:

current_job_description – Current job description, returned from DescribeTrainingJob call.
prev_job_description – Previous job description, returned from DescribeTrainingJob call.

Returns:

Whether the secondary status message of a training job changed or not.

Return type:

boolean

sagemaker.core.common_utils.secondary_training_status_message(job_description, prev_description)[source]#

Returns a string contains last modified time and the secondary training job status message.

Parameters:

job_description – Returned response from DescribeTrainingJob call
prev_description – Previous job description from DescribeTrainingJob call

Returns:

Job status string to be printed.

Return type:

str

sagemaker.core.common_utils.set_nested_value(dictionary: dict, nested_keys: List[str], value_to_set: object)[source]#

Sets a nested value in a dictionary.

This sets a nested value inside the given dictionary and returns the new dictionary. Note: if provided an unintended list of nested keys, this can overwrite an unexpected part of the dict. Recommended to use after a check with get_nested_value first

sagemaker.core.common_utils.stringify_object(obj: Any) → str[source]#: Returns string representation of object, returning only non-None fields.

sagemaker.core.common_utils.sts_regional_endpoint(region)[source]#

Get the AWS STS endpoint specific for the given region.

We need this function because the AWS SDK does not yet honor the region_name parameter when creating an AWS STS client.

For the list of regional endpoints, see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html#id_credentials_region-endpoints.

Parameters:: region (str) – AWS region name
Returns:: AWS STS regional endpoint
Return type:: str

Returns True if tag already exists.

Parameters:

tag (TagsDict) – The tag dictionary.
curr_tags (Optional[Tags]) – The current tags.

Returns:

True if the tag exists.

Return type:

bool

sagemaker.core.common_utils.to_string(obj: object)[source]#

Convert an object to string

This helper function handles converting PipelineVariable object to string as well

Parameters:: obj (object) – The object to be converted

sagemaker.core.common_utils.unflatten_dict(d: Dict[str, Any]) → Dict[str, Any][source]#

Unflatten dict-like object.

d (Dict[str, Any]) :: The dict that will be unflattened.

sagemaker.core.common_utils.unique_name_from_base(base, max_length=63)[source]#: Placeholder Docstring

sagemaker.core.common_utils.unique_name_from_base_uuid4(base, max_length=63)[source]#

Append a UUID to the provided string.

This function is used to generate a name using UUID instead of timestamps for uniqueness.

Parameters:

base (str) – String used as prefix to generate the unique name.
max_length (int) – Maximum length for the resulting string (default: 63).

Returns:

Input parameter with appended timestamp.

Return type:

str

sagemaker.core.common_utils.update_container_with_inference_params(framework=None, framework_version=None, nearest_model_name=None, data_input_configuration=None, container_def=None, container_list=None)[source]#

Function to check if inference recommender parameters exist and update container.

Parameters:

framework (str) – Machine learning framework of the model package container image (default: None).
framework_version (str) – Framework version of the Model Package Container Image (default: None).
nearest_model_name (str) – Name of a pre-trained machine learning benchmarked by Amazon SageMaker Inference Recommender (default: None).
data_input_configuration (str) – Input object for the model (default: None).
container_def (dict) – object to be updated.
container_list (list) – list to be updated.

Returns:

dict with inference recommender params

Return type:

dict

sagemaker.core.common_utils.update_list_of_dicts_with_values_from_config(input_list, config_key_path, required_key_paths: List[str] | None = None, union_key_paths: List[List[str]] | None = None, sagemaker_session=None)[source]#

Updates a list of dictionaries with missing values that are present in Config.

In some cases, config file might introduce new parameters which requires certain other parameters to be provided as part of the input list. Without those parameters, the underlying service will throw an exception. This method provides the capability to specify required key paths.

In some other cases, config file might introduce new parameters but the service API requires either an existing parameter or the new parameter that was supplied by config but not both

Parameters:

input_list – The input list that was provided as a method parameter.
config_key_path – The Key Path in the Config file that corresponds to the input_list
parameter.
required_key_paths (List[str]) – List of required key paths that should be verified in the
missing (merged output. If a required key path is)
that (we will not perform the merge for)
item.
union_key_paths (List[List[str]]) – List of List of Key paths for which we need to verify
exist. (whether exactly zero/one of the parameters)
example (For) – If the resultant dictionary can have either ‘X1’ or ‘X2’ as parameter or
both (neither but not)
[['X1' (then pass)
'X2']]
sagemaker_session (sagemaker.core.helper.session.Session) – A SageMaker Session object, used for SageMaker interactions (default: None).

Returns:

No output. In place merge happens.

sagemaker.core.common_utils.update_nested_dictionary_with_values_from_config(source_dict, config_key_path, sagemaker_session=None) → dict[source]#

Updates a nested dictionary with missing values that are present in Config.

Parameters:

source_dict – The input nested dictionary that was provided as method parameter.
config_key_path – The Key Path in the Config file which corresponds to this
parameter. (source_dict)
sagemaker_session (sagemaker.core.helper.session.Session) – A SageMaker Session object, used for SageMaker interactions (default: None).

Returns:

The merged nested dictionary that is updated with missing values that are present in the Config file.

Return type:

dict

sagemaker.core.common_utils.volume_size_supported(instance_type: str) → bool[source]#

Returns True if SageMaker allows volume_size to be used for the instance type.

Raises:: ValueError – If the instance type is improperly formatted.

sagemaker.core.common_utils.walk_and_apply_json(json_obj: Dict[Any, Any], apply, stop_keys: List[str] | None = ['metrics']) → Dict[Any, Any][source]#

Recursively walks a json object and applies a given function to the keys.

stop_keys (Optional[list[str]]): List of field keys that should stop the application function.: Any children of these keys will not have the application function applied to them.

sagemaker.core.common_utils

Contents

sagemaker.core.common_utils#