sagemaker.core.local.utils

sagemaker.core.local.utils#

Placeholder docstring

Functions

check_for_studio()

Helper function to determine if the run environment is studio.

copy_directory_structure(...)

Creates intermediate directory structure for relative_path.

get_child_process_ids(pid)

Retrieve all child pids for a certain pid

get_docker_host()

Discover remote docker host address (if applicable) or use "localhost"

get_using_dot_notation(dictionary, keys)

Extract keys from dictionary where keys is a string in dot notation.

kill_child_processes(pid)

Kill child processes

move_to_destination(source, destination, ...)

Move source to destination.

recursive_copy(source, destination)

A wrapper around shutil.copy_tree.
sagemaker.core.local.utils.check_for_studio()[source]#

Helper function to determine if the run environment is studio.

Returns (bool): Returns True if valid Studio request.

Raises:

NotImplementedError – if run environment = Studio and AppType not in STUDIO_APP_TYPES

sagemaker.core.local.utils.copy_directory_structure(destination_directory, relative_path)[source]#

Creates intermediate directory structure for relative_path.

Create all the intermediate directories required for relative_path to exist within destination_directory. This assumes that relative_path is a directory located within root_dir.

Examples

destination_directory: /tmp/destination relative_path: test/unit/ will create: /tmp/destination/test/unit

Parameters:

  • destination_directory (str) – root of the destination directory where the directory structure will be created.

  • relative_path (str) – relative path that will be created within destination_directory

sagemaker.core.local.utils.get_child_process_ids(pid)[source]#

Retrieve all child pids for a certain pid

Recursively scan each childs process tree and add it to the output

Parameters:

pid (int) – process id

Returns:

Child process ids

Return type:

(List[int])

sagemaker.core.local.utils.get_docker_host()[source]#

Discover remote docker host address (if applicable) or use “localhost”

Use “docker context inspect” to read current docker host endpoint url, url must start with “tcp://”

Args:

Returns:

Docker host DNS or IP address

Return type:

docker_host (str)

sagemaker.core.local.utils.get_using_dot_notation(dictionary, keys)[source]#

Extract keys from dictionary where keys is a string in dot notation.

Parameters:

  • dictionary (Dict)

  • keys (str)

Returns:

Nested object within dictionary as defined by “keys”

Raises:

ValueError if the provided key does not exist in input dictionary –

sagemaker.core.local.utils.kill_child_processes(pid)[source]#

Kill child processes

Kills all nested child process ids for a specific pid

Parameters:

pid (int) – process id

sagemaker.core.local.utils.move_to_destination(source, destination, job_name, sagemaker_session, prefix='')[source]#

Move source to destination.

Can handle uploading to S3.

Parameters:

  • source (str) – root directory to move

  • destination (str) – file:// or s3:// URI that source will be moved to.

  • job_name (str) – SageMaker job name.

  • sagemaker_session (sagemaker.Session) – a sagemaker_session to interact with S3 if needed

  • prefix (str, optional) – the directory on S3 used to save files, default to the root of destination

Returns:

destination URI

Return type:

(str)

sagemaker.core.local.utils.recursive_copy(source, destination)[source]#

A wrapper around shutil.copy_tree.

This won’t throw any exception when the source directory does not exist.

Parameters:

  • source (str) – source path

  • destination (str) – destination path

Contents