sagemaker.core.lineage.query

This module contains code to query SageMaker lineage.

Classes

Edge(source_arn, destination_arn, ...)	A connecting edge for a lineage graph.
LineageEntityEnum(value)	Enum of lineage entities for use in a query filter.
LineageFilter([entities, sources, ...])	A filter used in a lineage query.
LineageQuery(sagemaker_session)	Creates an object used for performing lineage queries.
LineageQueryDirectionEnum(value)	Enum of query filter directions.
LineageQueryResult([edges, vertices, startarn])	A wrapper around the results of a lineage query.
LineageSourceEnum(value)	Enum of lineage types for use in a query filter.
PyvisVisualizer(graph_styles[, pyvis_options])	Create object used for visualizing graph using Pyvis library.
Vertex(arn, lineage_entity, lineage_source, ...)	A vertex for a lineage graph.

class sagemaker.core.lineage.query.Edge(source_arn: str, destination_arn: str, association_type: str)

Bases: object

A connecting edge for a lineage graph.

class sagemaker.core.lineage.query.LineageEntityEnum(value)

Bases: Enum

Enum of lineage entities for use in a query filter.

ACTION = 'Action'

ARTIFACT = 'Artifact'

CONTEXT = 'Context'

TRIAL = 'Trial'

TRIAL_COMPONENT = 'TrialComponent'

class sagemaker.core.lineage.query.LineageFilter(entities: List[LineageEntityEnum | str] | None = None, sources: List[LineageSourceEnum | str] | None = None, created_before: datetime | None = None, created_after: datetime | None = None, modified_before: datetime | None = None, modified_after: datetime | None = None, properties: Dict[str, str] | None = None)

Bases: object

A filter used in a lineage query.

class sagemaker.core.lineage.query.LineageQuery(sagemaker_session)

Bases: object

Creates an object used for performing lineage queries.

query(start_arns: List[str], direction: LineageQueryDirectionEnum = LineageQueryDirectionEnum.BOTH, include_edges: bool = True, query_filter: LineageFilter | None = None, max_depth: int = 10) → LineageQueryResult

Perform a lineage query.

Parameters:

• start_arns (List[str]) – A list of ARNs that will be used as the starting point for the query.

• direction (LineageQueryDirectionEnum, optional) – The direction of the query.

• include_edges (bool, optional) – If true, return edges in addition to vertices.

• query_filter (LineageQueryFilter, optional) – The query filter.

Returns:

The lineage query result.

Return type:

LineageQueryResult

class sagemaker.core.lineage.query.LineageQueryDirectionEnum(value)

Bases: Enum

Enum of query filter directions.

ASCENDANTS = 'Ascendants'

BOTH = 'Both'

DESCENDANTS = 'Descendants'

class sagemaker.core.lineage.query.LineageQueryResult(edges: List[Edge] | None = None, vertices: List[Vertex] | None = None, startarn: List[str] | None = None)

Bases: object

A wrapper around the results of a lineage query.

visualize(path: str | None = 'lineage_graph_pyvis.html', pyvis_options: Dict[str, Any] | None = None)

Visualize lineage query result.

Creates a PyvisVisualizer object to render network graph with Pyvis library. Pyvis library should be installed before using this method (run “pip install pyvis”) The elements(nodes & edges) are preprocessed in this method and sent to PyvisVisualizer for rendering graph.

Parameters:

• path (optional) – The path/filename of the rendered graph html file. (default path: “lineage_graph_pyvis.html”)

• pyvis_options (optional) – A dict containing PyVis options to customize visualization. (see https://visjs.github.io/vis-network/docs/network/#options for supported fields)

Returns:

The interactive visualization is presented as a static HTML file.

Return type:

display graph

class sagemaker.core.lineage.query.LineageSourceEnum(value)

Bases: Enum

Enum of lineage types for use in a query filter.

APPROVAL = 'Approval'

CHECKPOINT = 'Checkpoint'

DATASET = 'DataSet'

ENDPOINT = 'Endpoint'

IMAGE = 'Image'

MODEL = 'Model'

MODEL_DATA = 'ModelData'

MODEL_DEPLOYMENT = 'ModelDeployment'

MODEL_GROUP = 'ModelGroup'

MODEL_REPLACE = 'ModelReplaced'

PROCESSING_JOB = 'ProcessingJob'

TENSORBOARD = 'TensorBoard'

TRAINING_JOB = 'TrainingJob'

TRANSFORM_JOB = 'TransformJob'

class sagemaker.core.lineage.query.PyvisVisualizer(graph_styles, pyvis_options: Dict[str, Any] | None = None)

Bases: object

Create object used for visualizing graph using Pyvis library.

render(elements, path='lineage_graph_pyvis.html')

Render graph for lineage query result.

Parameters:

• elements –

  A dictionary that contains the node and the edges of the graph. .. rubric:: Example

  elements[“nodes”] contains list of tuples, each tuple represents a node

  format: (node arn, node lineage source, node lineage entity,

  node is start arn)

  elements[“edges”] contains list of tuples, each tuple represents an edge

  format: (edge source arn, edge destination arn, edge association type)

• path (optional) – The path/filename of the rendered graph html file. (default path: “lineage_graph_pyvis.html”)

Returns:

The interactive visualization is presented as a static HTML file.

Return type:

display graph

class sagemaker.core.lineage.query.Vertex(arn: str, lineage_entity: str, lineage_source: str, sagemaker_session)

Bases: object

A vertex for a lineage graph.

to_lineage_object()

Convert the Vertex object to its corresponding lineage object.

Returns:

A Vertex object to its corresponding Artifact,``Action``, Context or TrialComponent object.