sagemaker.core.debugger.framework_profile

Contents

sagemaker.core.debugger.framework_profile#

Configuration for collecting framework metrics in SageMaker training jobs.

Classes

FrameworkProfile([local_path, ...])

Sets up the profiling configuration for framework metrics.
class sagemaker.core.debugger.framework_profile.FrameworkProfile(local_path='/opt/ml/output/profiler', file_max_size=10485760, file_close_interval=60, file_open_fail_threshold=50, detailed_profiling_config=None, dataloader_profiling_config=None, python_profiling_config=None, horovod_profiling_config=None, smdataparallel_profiling_config=None, start_step=None, num_steps=None, start_unix_time=None, duration=None)[source]#

Bases: object

Sets up the profiling configuration for framework metrics.

Validates user inputs and fills in default values if no input is provided. There are three main profiling options to choose from: DetailedProfilingConfig, DataloaderProfilingConfig, and PythonProfilingConfig.

The following list shows available scenarios of configuring the profiling options.

1. None of the profiling configuration, step range, or time range is specified. SageMaker Debugger activates framework profiling based on the default settings of each profiling option.

from sagemaker.debugger import ProfilerConfig, FrameworkProfile

profiler_config=ProfilerConfig(
    framework_profile_params=FrameworkProfile()
)

2. Target step or time range is specified to this FrameworkProfile class. The requested target step or time range setting propagates to all of the framework profiling options. For example, if you configure this class as following, all of the profiling options profiles the 6th step:

from sagemaker.debugger import ProfilerConfig, FrameworkProfile

profiler_config=ProfilerConfig(
    framework_profile_params=FrameworkProfile(start_step=6, num_steps=1)
)

3. Individual profiling configurations are specified through the *_profiling_config parameters. SageMaker Debugger profiles framework metrics only for the specified profiling configurations. For example, if the DetailedProfilingConfig class is configured but not the other profiling options, Debugger only profiles based on the settings specified to the DetailedProfilingConfig class. For example, the following example shows a profiling configuration to perform detailed profiling at step 10, data loader profiling at step 9 and 10, and Python profiling at step 12.

from sagemaker.debugger import ProfilerConfig, FrameworkProfile

profiler_config=ProfilerConfig(
    framework_profile_params=FrameworkProfile(
        detailed_profiling_config=DetailedProfilingConfig(start_step=10, num_steps=1),
        dataloader_profiling_config=DataloaderProfilingConfig(start_step=9, num_steps=2),
        python_profiling_config=PythonProfilingConfig(start_step=12, num_steps=1),
    )
)

If the individual profiling configurations are specified in addition to the step or time range, SageMaker Debugger prioritizes the individual profiling configurations and ignores the step or time range. For example, in the following code, the start_step=1 and num_steps=10 will be ignored.

from sagemaker.debugger import ProfilerConfig, FrameworkProfile

profiler_config=ProfilerConfig(
    framework_profile_params=FrameworkProfile(
        start_step=1,
        num_steps=10,
        detailed_profiling_config=DetailedProfilingConfig(start_step=10, num_steps=1),
        dataloader_profiling_config=DataloaderProfilingConfig(start_step=9, num_steps=2),
        python_profiling_config=PythonProfilingConfig(start_step=12, num_steps=1)
    )
)
Contents