sagemaker.serve.spec.inference_base

Holds templated classes to enable users to provide custom inference scripting capabilities

Classes

AsyncCustomOrchestrator()	Templated class to standardize async entrypoint-based inference scripts
CustomOrchestrator()	Templated class to standardize sync entrypoint-based inference scripts

class sagemaker.serve.spec.inference_base.AsyncCustomOrchestrator

Bases: ABC

Templated class to standardize async entrypoint-based inference scripts

abstract async handle(data, context=None)

Abstract class for defining an aynchronous entrypoint for the model server

class sagemaker.serve.spec.inference_base.CustomOrchestrator

Bases: ABC

Templated class to standardize sync entrypoint-based inference scripts

property client

Boto3 SageMaker runtime client to use with custom orchestrator

abstract handle(data, context=None)

Abstract class for defining an entrypoint for the model server