sagemaker.core.iterators

sagemaker.core.iterators#

Implements iterators for deserializing data returned from an inference streaming endpoint.

Functions

handle_stream_errors(chunk)

Handle API Response errors within invoke_endpoint_with_response_stream API if any.

Classes

BaseIterator(event_stream)

Abstract base class for Inference Streaming iterators.

ByteIterator(event_stream)

A helper class for parsing the byte Event Stream input to provide Byte iteration.

LineIterator(event_stream)

A helper class for parsing the byte Event Stream input to provide Line iteration.
class sagemaker.core.iterators.BaseIterator(event_stream)[source]#

Bases: ABC

Abstract base class for Inference Streaming iterators.

Provides a skeleton for customization requiring the overriding of iterator methods __iter__ and __next__.

Tenets of iterator class for Streaming Inference API Response (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ sagemaker-runtime/client/invoke_endpoint_with_response_stream.html): 1. Needs to accept an botocore.eventstream.EventStream response. 2. Needs to implement logic in __next__ to:

2.1. Concatenate and provide next chunk of response from botocore.eventstream.EventStream.

While doing so parse the response_chunk[“PayloadPart”][“Bytes”].

2.2. If PayloadPart not in EventStream response, handle Errors

[Recommended to use iterators.handle_stream_errors method].

class sagemaker.core.iterators.ByteIterator(event_stream)[source]#

Bases: BaseIterator

A helper class for parsing the byte Event Stream input to provide Byte iteration.

class sagemaker.core.iterators.LineIterator(event_stream)[source]#

Bases: BaseIterator

A helper class for parsing the byte Event Stream input to provide Line iteration.

sagemaker.core.iterators.handle_stream_errors(chunk)[source]#

Handle API Response errors within invoke_endpoint_with_response_stream API if any.

Parameters:

chunk (dict) – A chunk of response received as part of botocore.eventstream.EventStream response object.

Raises:

  • ModelStreamError – If ModelStreamError error is detected in a chunk of botocore.eventstream.EventStream response object.

  • InternalStreamFailure – If InternalStreamFailure error is detected in a chunk of botocore.eventstream.EventStream response object.

Contents