sagemaker.serve.spec.inference_spec

Implements class that holds custom load and invoke function of a model

Classes

InferenceSpec()

Abstract base class for holding custom load, invoke and prepare functions.

class sagemaker.serve.spec.inference_spec.InferenceSpec

Bases: ABC

Abstract base class for holding custom load, invoke and prepare functions.

Provides a skeleton for customization to override the methods load, invoke and prepare.

get_model()

Return HuggingFace model name for inference spec

abstract invoke(input_object: object, model: object)

Given model object and input, make inference and return the result.

Parameters:

• input_object (object) – The input to model

• model (object) – The model object

abstract load(model_dir: str)

Loads the model stored in model_dir and return the model object.

Parameters:

model_dir (str) – Path to the directory where the model is stored.

postprocess(predictions: object)

Custom post-processing function

prepare(*args, **kwargs)

Custom prepare function

preprocess(input_data: object)

Custom pre-processing function