sagemaker.core.remote_function.core.serialization

SageMaker remote function data serializer/deserializer.

Functions

deserialize_exception_from_s3(...)	Downloads from S3 and then deserializes exception.
deserialize_func_from_s3(sagemaker_session, ...)	Downloads from S3 and then deserializes data objects.
deserialize_obj_from_s3(sagemaker_session, ...)	Downloads from S3 and then deserializes data objects.
json_serialize_obj_to_s3(obj, json_key, ...)	Json serializes data object and uploads it to S3.
serialize_exception_to_s3(exc, ...[, s3_kms_key])	Serializes exception with traceback and uploads it to S3.
serialize_func_to_s3(func, ...[, s3_kms_key])	Serializes function and uploads it to S3.
serialize_obj_to_s3(obj, sagemaker_session, ...)	Serializes data object and uploads it to S3.

Classes

CloudpickleSerializer()

Serializer using cloudpickle.

class sagemaker.core.remote_function.core.serialization.CloudpickleSerializer

Bases: object

Serializer using cloudpickle.

static deserialize(s3_uri: str, bytes_to_deserialize: bytes) → Any

Downloads from S3 and then deserializes data objects.

Parameters:

• s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

• bytes_to_deserialize – bytes to be deserialized.

Returns :

List of deserialized python objects.

Raises:

DeserializationError – when fail to serialize object to bytes.

static serialize(obj: Any) → bytes

Serializes data object and uploads it to S3.

Parameters:

obj – object to be serialized and persisted

Raises:

SerializationError – when fail to serialize object to bytes.

sagemaker.core.remote_function.core.serialization.deserialize_exception_from_s3(sagemaker_session: Session, s3_uri: str) → Any

Downloads from S3 and then deserializes exception.

Parameters:

• sagemaker_session (sagemaker.core.helper.session.Session) – The underlying sagemaker session which AWS service calls are delegated to.

• s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

Returns :

Deserialized exception with traceback.

Raises:

DeserializationError – when fail to serialize object to bytes.

sagemaker.core.remote_function.core.serialization.deserialize_func_from_s3(sagemaker_session: Session, s3_uri: str) → Callable

Downloads from S3 and then deserializes data objects.

This method downloads the serialized training job outputs to a temporary directory and then deserializes them using dask.

Parameters:

• sagemaker_session (sagemaker.core.helper.session.Session) – The underlying sagemaker session which AWS service calls are delegated to.

• s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

Returns :

The deserialized function.

Raises:

DeserializationError – when fail to serialize function to bytes.

sagemaker.core.remote_function.core.serialization.deserialize_obj_from_s3(sagemaker_session: Session, s3_uri: str) → Any

Downloads from S3 and then deserializes data objects.

Parameters:

• sagemaker_session (sagemaker.core.helper.session.Session) – The underlying sagemaker session which AWS service calls are delegated to.

• s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

Returns :

Deserialized python objects.

Raises:

DeserializationError – when fail to serialize object to bytes.

sagemaker.core.remote_function.core.serialization.json_serialize_obj_to_s3(obj: Any, json_key: str, sagemaker_session: Session, s3_uri: str, s3_kms_key: str | None = None)

Json serializes data object and uploads it to S3.

If a function step’s output is data referenced by other steps via JsonGet, its output should be json serialized and uploaded to S3.

Parameters:

• obj – (Any) object to be serialized and persisted.

• json_key – (str) the json key pointing to function step output.

• sagemaker_session (sagemaker.core.helper.session.Session) – The underlying Boto3 session which AWS service calls are delegated to.

• s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

• s3_kms_key (str) – KMS key used to encrypt artifacts uploaded to S3.

sagemaker.core.remote_function.core.serialization.serialize_exception_to_s3(exc: Exception, sagemaker_session: Session, s3_uri: str, s3_kms_key: str | None = None)

Serializes exception with traceback and uploads it to S3.

Parameters:

• sagemaker_session (sagemaker.core.helper.session.Session) – The underlying Boto3 session which AWS service calls are delegated to.

• s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

• s3_kms_key (str) – KMS key used to encrypt artifacts uploaded to S3.

• exc – Exception to be serialized and persisted

Raises:

SerializationError – when fail to serialize object to bytes.

sagemaker.core.remote_function.core.serialization.serialize_func_to_s3(func: Callable, sagemaker_session: Session, s3_uri: str, s3_kms_key: str | None = None)

Serializes function and uploads it to S3.

Parameters:

• sagemaker_session (sagemaker.core.helper.session.Session) – The underlying Boto3 session which AWS service calls are delegated to.

• s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

• s3_kms_key (str) – KMS key used to encrypt artifacts uploaded to S3.

• func – function to be serialized and persisted

Raises:

SerializationError – when fail to serialize function to bytes.

sagemaker.core.remote_function.core.serialization.serialize_obj_to_s3(obj: Any, sagemaker_session: Session, s3_uri: str, s3_kms_key: str | None = None)

Serializes data object and uploads it to S3.

Parameters:

• sagemaker_session (sagemaker.core.helper.session.Session) – The underlying Boto3 session which AWS service calls are delegated to.

• s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

• s3_kms_key (str) – KMS key used to encrypt artifacts uploaded to S3.

• obj – object to be serialized and persisted

Raises:

SerializationError – when fail to serialize object to bytes.