sagemaker.core.workflow

sagemaker.core.workflow#

Workflow primitives for SageMaker pipelines.

This module contains the foundation types and primitives used by workflow orchestration. These primitives can be used by Train, Serve, and MLOps packages without creating circular dependencies.

For pipeline and step orchestration classes (Pipeline, TrainingStep, etc.), import from sagemaker.mlops.workflow instead.

Functions

`is_pipeline_parameter_string`(var)	Check if the variable is a pipeline parameter string
`is_pipeline_variable`(var)	Check if the variable is a pipeline variable

class sagemaker.core.workflow.Condition(condition_type: ConditionTypeEnum = NOTHING)[source]#

Bases: Entity

Abstract Condition entity.

condition_type#

The type of condition.

Type:: ConditionTypeEnum

condition_type: ConditionTypeEnum#

Bases: Condition

Generic comparison condition that can be used to derive specific condition comparisons.

left#

The execution variable, parameter, property, step output or Python primitive value to use in the comparison.

Type:: Union[ConditionValueType, PrimitiveType]

right#

The execution variable, parameter, property, step output or Python primitive value to compare to.

Type:: Union[ConditionValueType, PrimitiveType]

left: ExecutionVariable | Parameter | Properties | StepOutput | str | int | bool | float | None#

right: ExecutionVariable | Parameter | Properties | StepOutput | str | int | bool | float | None#

to_request() → Dict[str, Any] | List[Dict[str, Any]][source]#: Get the request structure for workflow service calls.

Bases: ConditionComparison

A condition for equality comparisons.

Bases: ConditionComparison

A condition for greater than comparisons.

Bases: ConditionComparison

A condition for greater than or equal to comparisons.

Bases: Condition

A condition to check membership.

to_request() → Dict[str, Any] | List[Dict[str, Any]][source]#: Get the request structure for workflow service calls.

Bases: ConditionComparison

A condition for less than comparisons.

Bases: ConditionComparison

A condition for less than or equal to comparisons.

class sagemaker.core.workflow.ConditionNot(expression: Condition)[source]#

Bases: Condition

A condition for negating another Condition.

to_request() → Dict[str, Any] | List[Dict[str, Any]][source]#: Get the request structure for workflow service calls.

class sagemaker.core.workflow.ConditionOr(conditions: List[Condition] | None = None)[source]#

Bases: Condition

A condition for taking the logical OR of a list of Condition instances.

to_request() → Dict[str, Any] | List[Dict[str, Any]][source]#: Get the request structure for workflow service calls.

class sagemaker.core.workflow.ConditionTypeEnum(*args, value=<object object>, **kwargs)[source]#

Bases: Enum

Condition type enum.

EQ = 'Equals'#

GT = 'GreaterThan'#

GTE = 'GreaterThanOrEqualTo'#

IN = 'In'#

LT = 'LessThan'#

LTE = 'LessThanOrEqualTo'#

NOT = 'Not'#

OR = 'Or'#

class sagemaker.core.workflow.DefaultEnumMeta(cls, bases, classdict, **kwds)[source]#

Bases: EnumMeta

An EnumMeta which defaults to the first value in the Enum list.

default = <object object>#

factory(*args, value=<object object>, **kwargs)#: Defaults to the first value in the Enum list.

class sagemaker.core.workflow.Entity[source]#

Bases: ABC

Base object for workflow entities.

Entities must implement the to_request method.

abstract to_request() → Dict[str, Any] | List[Dict[str, Any]][source]#: Get the request structure for workflow service calls.

class sagemaker.core.workflow.ExecutionVariable(name: str)[source]#

Bases: PipelineVariable

Pipeline execution variables for workflow.

property expr: Dict[str, Any] | List[Dict[str, Any]]#: The ‘Get’ expression dict for an ExecutionVariable.

to_string() → PipelineVariable[source]#

Prompt the pipeline to convert the pipeline variable to String in runtime

As ExecutionVariable is treated as String in runtime, no extra actions are needed.

class sagemaker.core.workflow.ExecutionVariables[source]#

Bases: object

Provide access to all available execution variables:

ExecutionVariables.START_DATETIME
ExecutionVariables.CURRENT_DATETIME
ExecutionVariables.PIPELINE_NAME
ExecutionVariables.PIPELINE_ARN
ExecutionVariables.PIPELINE_EXECUTION_ID
ExecutionVariables.PIPELINE_EXECUTION_ARN
ExecutionVariables.TRAINING_JOB_NAME
ExecutionVariables.PROCESSING_JOB_NAME

CURRENT_DATETIME = <sagemaker.core.workflow.execution_variables.ExecutionVariable object>#

PIPELINE_ARN = <sagemaker.core.workflow.execution_variables.ExecutionVariable object>#

PIPELINE_EXECUTION_ARN = <sagemaker.core.workflow.execution_variables.ExecutionVariable object>#

PIPELINE_EXECUTION_ID = <sagemaker.core.workflow.execution_variables.ExecutionVariable object>#

PIPELINE_NAME = <sagemaker.core.workflow.execution_variables.ExecutionVariable object>#

PROCESSING_JOB_NAME = <sagemaker.core.workflow.execution_variables.ExecutionVariable object>#

START_DATETIME = <sagemaker.core.workflow.execution_variables.ExecutionVariable object>#

TRAINING_JOB_NAME = <sagemaker.core.workflow.execution_variables.ExecutionVariable object>#

class sagemaker.core.workflow.Join(on: str = NOTHING, values: List = NOTHING)[source]#

Bases: PipelineVariable

Join together properties.

Examples: Build a Amazon S3 Uri with bucket name parameter and pipeline execution Id and use it as training input:

bucket = ParameterString('bucket', default_value='my-bucket')

TrainingInput(
    s3_data=Join(
        on='/',
        values=['s3:/', bucket, ExecutionVariables.PIPELINE_EXECUTION_ID]
    ),
    content_type="text/csv")

values#

The primitive type values, parameters, step properties, expressions to join.

Type:: List[Union[PrimitiveType, Parameter, PipelineVariable]]

on#

The string to join the values on (Defaults to “”).

Type:: str

property expr#: The expression dict for a Join function.

on: str#

to_string() → PipelineVariable[source]#

Prompt the pipeline to convert the pipeline variable to String in runtime

As Join is treated as String in runtime, no extra actions are needed.

values: List#

class sagemaker.core.workflow.JsonGet(step_name: str = None, property_file: PropertyFile | str | None = None, json_path: str = None, s3_uri: Join | None = None, step: Step = None)[source]#

Bases: PipelineVariable

Get JSON properties from PropertyFiles or S3 location.

step_name#

The step name from which to get the property file.

Type:: str

property_file#

Either a PropertyFile instance or the name of a property file.

Type:: Optional[Union[PropertyFile, str]]

json_path#

The JSON path expression to the requested value.

Type:: str

s3_uri#

The S3 location from which to fetch a Json file. The Json file is the output of a step defined with @step decorator.

Type:: Optional[sagemaker.workflow.functions.Join]

step#

The upstream step object which the s3_uri is associated to.

Type:: Step

property expr#: The expression dict for a JsonGet function.

json_path: str#

property_file: PropertyFile | str | None#

s3_uri: Join | None#

step: Step#

step_name: str#

class sagemaker.core.workflow.Parameter(name: str = NOTHING, parameter_type: ParameterTypeEnum = NOTHING, default_value: str | int | bool | float | None = None)[source]#

Bases: PipelineVariable, Entity

Pipeline parameter for workflow.

name#

The name of the parameter.

Type:: str

parameter_type#

The type of the parameter.

Type:: ParameterTypeEnum

default_value#

The default value of the parameter.

Type:: PrimitiveType

default_value: str | int | bool | float | None#

property expr: Dict[str, str]#: The ‘Get’ expression dict for a Parameter.

name: str#

parameter_type: ParameterTypeEnum#

to_request() → Dict[str, Any] | List[Dict[str, Any]][source]#: Get the request structure for workflow service calls.

class sagemaker.core.workflow.ParameterFloat(name: str, default_value: float | None = None)[source]#

Bases: Parameter

Float parameter for pipelines.

class sagemaker.core.workflow.ParameterInteger(name: str, default_value: int | None = None)[source]#

Bases: Parameter

Integer parameter for pipelines.

class sagemaker.core.workflow.ParameterString(name: str, default_value: str | None = None, enum_values: List[str] | None = None)[source]#

Bases: Parameter

String parameter for pipelines.

to_request() → Dict[str, Any] | List[Dict[str, Any]][source]#: Get the request structure for workflow service calls.

to_string() → PipelineVariable[source]#

Prompt the pipeline to convert the pipeline variable to String in runtime

As ParameterString is treated as String in runtime, no extra actions are needed.

class sagemaker.core.workflow.ParameterTypeEnum(*args, value=<object object>, **kwargs)[source]#

Bases: Enum

Parameter type enum.

BOOLEAN = 'Boolean'#

FLOAT = 'Float'#

INTEGER = 'Integer'#

STRING = 'String'#

property python_type: Type#: Provide the Python type of the enum value.

class sagemaker.core.workflow.Properties(step_name: str, path: str = None, shape_name: str = None, shape_names: List[str] = None, service_name: str = 'sagemaker', step: Step = None)[source]#

Bases: StepOutput

Properties for use in workflow expressions.

property expr#: The ‘Get’ expression dict for a Properties.

class sagemaker.core.workflow.PropertiesList(step_name: str, path: str, shape_name: str = None, service_name: str = 'sagemaker', step: Step = None)[source]#

Bases: Properties

PropertiesList for use in workflow expressions.

class sagemaker.core.workflow.PropertiesMap(step_name: str, path: str, shape_name: str = None, service_name: str = 'sagemaker', step: Step = None)[source]#

Bases: Properties

PropertiesMap for use in workflow expressions.

class sagemaker.core.workflow.PropertyFile(name: str, output_name: str, path: str)[source]#

Bases: PipelineVariable

Provides a property file struct.

name#

The name of the property file for reference with JsonGet functions.

Type:: str

output_name#

The name of the processing job output channel.

Type:: str

path#

The path to the file at the output channel location.

Type:: str

property expr: Dict[str, str]#: The expression dict for a PropertyFile.

name: str#

output_name: str#

path: str#

class sagemaker.core.workflow.StepOutput(step: Step = None)[source]#

Bases: PipelineVariable

Base class representing @step decorator outputs.

abstract property expr: Dict[str, Any] | List[Dict[str, Any]]#: Get the expression structure for workflow service calls.

sagemaker.core.workflow.get_step(step_output: StepOutput)[source]#

Get the step associated with this output.

Parameters:: step_output – A sagemaker.workflow.steps.StepOutput instance.
Returns:: A sagemaker.workflow.steps.Step instance.

sagemaker.core.workflow.is_pipeline_parameter_string(var: object) → bool[source]#

Check if the variable is a pipeline parameter string

Parameters:: var (object) – The variable to be verified.
Returns:: True if it is, False otherwise.
Return type:: bool

sagemaker.core.workflow.is_pipeline_variable(var: object) → bool[source]#

Check if the variable is a pipeline variable

Parameters:: var (object) – The variable to be verified.
Returns:: True if it is, False otherwise.
Return type:: bool

Modules

`conditions`	Conditions for condition steps.
`entities`	Defines the base entities used in workflow.
`execution_variables`	Pipeline parameters and conditions for workflow.
`functions`	The step definitions for workflow.
`parameters`	Pipeline parameters and conditions for workflow.
`pipeline_context`	The pipeline context for workflow
`pipeline_definition_config`	Pipeline experiment config for SageMaker pipeline.
`properties`	The properties definitions for workflow.
`step_outputs`	Base class representing step decorator outputs
`utilities`	Utilities to support workflow.

sagemaker.core.workflow

Contents

sagemaker.core.workflow#