Deployment

Functions for Inference and model deployment.

class anomalib.deploy.ExportType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

Model export type.

Examples

>>> from anomalib.deploy import ExportType
>>> ExportType.ONNX
'onnx'
>>> ExportType.OPENVINO
'openvino'
>>> ExportType.TORCH
'torch'

class anomalib.deploy.OpenVINOInferencer(path, metadata=None, device='AUTO', task=None, config=None)

Bases: Inferencer

OpenVINO implementation for the inference.

Parameters:

• path (str | Path) – Path to the openvino onnx, xml or bin file.

• metadata (str | Path | dict, optional) – Path to metadata file or a dict object defining the metadata. Defaults to None.

• device (str | None, optional) – Device to run the inference on (AUTO, CPU, GPU, NPU). Defaults to AUTO.

• task (TaskType | None, optional) – Task type. Defaults to None.

• config (dict | None, optional) – Configuration parameters for the inference Defaults to None.

Examples

Assume that we have an OpenVINO IR model and metadata files in the following structure:

$ tree weights
./weights
├── model.bin
├── model.xml
└── metadata.json

We could then create OpenVINOInferencer as follows:

>>> from anomalib.deploy.inferencers import OpenVINOInferencer
>>> inferencer = OpenVINOInferencer(
...     path="weights/model.xml",
...     metadata="weights/metadata.json",
...     device="CPU",
... )

This will ensure that the model is loaded on the CPU device and the metadata is loaded from the metadata.json file. To make a prediction, we can simply call the predict method:

>>> import cv2
>>> image = cv2.imread("path/to/image.jpg")
>>> image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
>>> result = inferencer.predict(image)

result will be an ImageResult object containing the prediction results. For example, to visualize the heatmap, we can do the following:

>>> from matplotlib import pyplot as plt
>>> plt.imshow(result.heatmap)

It is also possible to visualize the true and predicted masks if the task is TaskType.SEGMENTATION:

>>> plt.imshow(result.gt_mask)
>>> plt.imshow(result.pred_mask)

forward(image)

Forward-Pass input tensor to the model.

Parameters:

image (np.ndarray) – Input tensor.

Returns:

Output predictions.

Return type:

np.ndarray

load_model(path)

Load the OpenVINO model.

Parameters:

path (str | Path | tuple[bytes, bytes]) – Path to the onnx or xml and bin files or tuple of .xml and .bin data as bytes.

Returns:

Input and Output blob names

together with the Executable network.

Return type:

[tuple[str, str, ExecutableNetwork]]

post_process(predictions, metadata=None)

Post process the output predictions.

Parameters:

• predictions (np.ndarray) – Raw output predicted by the model.

• metadata (Dict, optional) – Metadata. Post-processing step sometimes requires additional metadata such as image shape. This variable comprises such info. Defaults to None.

Returns:

Post processed prediction results.

Return type:

dict[str, Any]

pre_process(image)

Pre-process the input image by applying transformations.

Parameters:

image (np.ndarray) – Input image.

Returns:

pre-processed image.

Return type:

np.ndarray

class anomalib.deploy.TorchInferencer(path, device='auto')

Bases: Inferencer

PyTorch implementation for the inference.

Parameters:

• path (str | Path) – Path to Torch model weights.

• device (str) – Device to use for inference. Options are auto, cpu, cuda. Defaults to auto.

Examples

Assume that we have a Torch pt model and metadata files in the following structure:

>>> from anomalib.deploy.inferencers import TorchInferencer
>>> inferencer = TorchInferencer(path="path/to/torch/model.pt", device="cpu")

This will ensure that the model is loaded on the CPU device. To make a prediction, we can simply call the predict method:

>>> import cv2
>>> image = cv2.imread("path/to/image.jpg")
>>> image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
>>> result = inferencer.predict(image)

result will be an ImageResult object containing the prediction results. For example, to visualize the heatmap, we can do the following:

>>> from matplotlib import pyplot as plt
>>> plt.imshow(result.heatmap)

It is also possible to visualize the true and predicted masks if the task is TaskType.SEGMENTATION:

>>> plt.imshow(result.gt_mask)
>>> plt.imshow(result.pred_mask)

forward(image)

Forward-Pass input tensor to the model.

Parameters:

image (torch.Tensor) – Input tensor.

Returns:

Output predictions.

Return type:

Tensor

load_model(path)

Load the PyTorch model.

Parameters:

path (str | Path) – Path to the Torch model.

Returns:

Torch model.

Return type:

(nn.Module)

post_process(predictions, metadata=None)

Post process the output predictions.

Parameters:

• predictions (Tensor | list[torch.Tensor] | dict[str, torch.Tensor]) – Raw output predicted by the model.

• metadata (dict, optional) – Meta data. Post-processing step sometimes requires additional meta data such as image shape. This variable comprises such info. Defaults to None.

Returns:

Post processed prediction results.

Return type:

dict[str, str | float | np.ndarray]

pre_process(image)

Pre process the input image by applying transformations.

Parameters:

image (np.ndarray) – Input image

Returns:

pre-processed image.

Return type:

Tensor

anomalib.deploy.export_to_onnx(model, input_size, export_root, transform, task=None, export_type=ExportType.ONNX)

Export model to onnx.

Parameters:

• model (AnomalyModule) – Model to export.

• input_size (list[int] | tuple[int, int]) – Image size used as the input for onnx converter.

• export_root (Path) – Path to the root folder of the exported model.

• transform (dict[str, Any] | AnomalibDataset | AnomalibDataModule | A.Compose) – Data transforms (augmentations) used for the model. When using dict, ensure that the transform dict is in the format required by Albumentations.

• task (TaskType | None) – Task type should be provided if transforms is of type dict or A.Compose object. Defaults to None.

• export_type (ExportType) – Mode to export the model. Since this method is used by OpenVINO export as well, we need to pass the export type so that the right export path is created. Defaults to ExportType.ONNX.

Returns:

Path to the exported onnx model.

Return type:

Path

Examples

Export the Lightning Model to ONNX:

>>> from anomalib.models import Patchcore
>>> from anomalib.data import Visa
>>> from anomalib.deploy import export_to_onnx
...
>>> datamodule = Visa()
>>> model = Patchcore()
...
>>> export_to_onnx(
...     model=model,
...     input_size=(224, 224),
...     export_root="path/to/export",
...     transform=datamodule.test_data.transform,
...     task=datamodule.test_data.task
... )

Using Custom Transforms: This example shows how to use a custom Compose object for the transform argument.

>>> import albumentations as A
>>> transform = A.Compose([A.Resize(224, 224), A.pytorch.ToTensorV2()])
...
>>> export_to_onnx(
...     model=model,
...     input_size=(224, 224),
...     export_root="path/to/export",
...     transform=transform,
...     task="segmentation",
... )

anomalib.deploy.export_to_openvino(export_root, model, input_size, transform, ov_args=None, task=None)

Convert onnx model to OpenVINO IR.

Parameters:

• export_root (Path) – Path to the export folder.

• model (AnomalyModule) – AnomalyModule to export.

• input_size (tuple[int, int]) – Input size of the model. Used for adding metadata to the IR.

• transform (dict[str, Any] | AnomalibDataset | AnomalibDataModule | A.Compose) – Data transforms (augmentations) used for the model. When using dict, ensure that the transform dict is in the format required by Albumentations.

• ov_args (Optional[dict[str, Any]]) – Model optimizer arguments for OpenVINO model conversion. Defaults to None.

• task (TaskType | None) – Task type should be provided if transforms is of type dict or A.Compose object. Defaults to None.

Returns:

Path to the exported onnx model.

Return type:

Path

Raises:

ModuleNotFoundError – If OpenVINO is not installed.

Returns:

Path to the exported OpenVINO IR.

Return type:

Path

Examples

Export the Lightning Model to OpenVINO IR: This example demonstrates how to export the Lightning Model to OpenVINO IR.

>>> from anomalib.models import Patchcore
>>> from anomalib.data import Visa
>>> from anomalib.deploy import export_to_openvino
...
>>> datamodule = Visa()
>>> model = Patchcore()
...
>>> export_to_openvino(
...     export_root="path/to/export",
...     model=model,
...     input_size=(224, 224),
...     transform=datamodule.test_data.transform,
...     task=datamodule.test_data.task
... )

Using Custom Transforms: This example shows how to use a custom Compose object for the transform argument.

>>> import albumentations as A
>>> transform = A.Compose([A.Resize(224, 224), A.pytorch.ToTensorV2()])
...
>>> export_to_openvino(
...     export_root="path/to/export",
...     model=model,
...     input_size=(224, 224),
...     transform=transform,
...     task="segmentation",
... )

anomalib.deploy.export_to_torch(model, export_root, transform, task=None)

Export AnomalibModel to torch.

Parameters:

• model (AnomalyModule) – Model to export.

• export_root (Path) – Path to the output folder.

• transform (dict[str, Any] | AnomalibDataset | AnomalibDataModule | A.Compose) – Data transforms (augmentations) used for the model. When using dict, ensure that the transform dict is in the format required by Albumentations.

• task (TaskType | None) – Task type should be provided if transforms is of type dict or A.Compose object. Defaults to None.

Returns:

Path to the exported pytorch model.

Return type:

Path

Examples

Assume that we have a model to train and we want to export it to torch format.

>>> from anomalib.data import Visa
>>> from anomalib.models import Patchcore
>>> from anomalib.engine import Engine
...
>>> datamodule = Visa()
>>> model = Patchcore()
>>> engine = Engine()
...
>>> engine.fit(model, datamodule)

Now that we have a model trained, we can export it to torch format.

>>> from anomalib.deploy import export_to_torch
...
>>> export_to_torch(
...     model=model,
...     export_root="path/to/export",
...     transform=datamodule.test_data.transform,
...     task=datamodule.test_data.task,
... )