Inference#

Functions for model inference and deployment.

This module provides functionality for deploying trained anomaly detection models and performing inference. It includes:

  • Model export utilities for converting models to different formats

  • Inference classes for making predictions:

Example

>>> from anomalib.deploy import TorchInferencer
>>> model = TorchInferencer(path="path/to/model.pt")
>>> predictions = model.predict(image="path/to/image.jpg")

The prediction contains anomaly maps and scores:

>>> predictions.anomaly_map  
tensor([[0.1, 0.2, ...]])
>>> predictions.pred_score  
tensor(0.86)
class anomalib.deploy.CompressionType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Model compression type when exporting to OpenVINO.

FP16#

Weight compression to FP16 precision. All weights are converted to FP16.

INT8#

Weight compression to INT8 precision. All weights are quantized to INT8, but are dequantized to floating point before inference.

INT8_PTQ#

Full integer post-training quantization to INT8 precision. All weights and operations are quantized to INT8. Inference is performed in INT8 precision.

INT8_ACQ#

Accuracy-control quantization to INT8 precision. Weights and operations are quantized to INT8, except those that would degrade model quality beyond an acceptable threshold. Inference uses mixed precision.

Example

>>> from anomalib.deploy import CompressionType
>>> compression = CompressionType.INT8_PTQ
>>> compression
'int8_ptq'
class anomalib.deploy.ExportType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Model export type.

Supported export formats for anomaly detection models.

ONNX#

Export model to ONNX format

OPENVINO#

Export model to OpenVINO IR format

TORCH#

Export model to PyTorch format

Example

>>> from anomalib.deploy import ExportType
>>> export_type = ExportType.ONNX
>>> export_type
'onnx'
class anomalib.deploy.OpenVINOInferencer(path, device='AUTO', config=None)#

Bases: object

OpenVINO inferencer for optimized model inference.

Parameters:
  • path (str | Path | tuple[bytes, bytes]) – Path to OpenVINO IR files (.xml and .bin) or ONNX model, or tuple of xml/bin data as bytes.

  • device (str | None, optional) – Inference device. Options: "AUTO", "CPU", "GPU", "NPU". Defaults to "AUTO".

  • config (dict | None, optional) – OpenVINO configuration parameters. Defaults to None.

Example

>>> from anomalib.deploy import OpenVINOInferencer
>>> model = OpenVINOInferencer(
...     path="model.xml",
...     device="CPU"
... )
>>> prediction = model.predict("test.jpg")
load_model(path)#

Load OpenVINO model from file or bytes.

Parameters:

path (str | Path | tuple[bytes, bytes]) – Path to model files or model data as bytes tuple.

Returns:

Tuple containing:
  • Input blob

  • Output blob

  • Compiled model

Return type:

tuple[Any, Any, Any]

Raises:

ValueError – If model path has invalid extension.

static post_process(predictions)#

Convert OpenVINO predictions to dictionary.

Parameters:

predictions (OVDict) – Raw predictions from OpenVINO model.

Returns:

Dictionary of prediction tensors.

Return type:

dict

static pre_process(image)#

Pre-process input image.

Parameters:

image (np.ndarray) – Input image.

Returns:

Pre-processed image with shape (N,C,H,W).

Return type:

np.ndarray

predict(image)#

Run inference on an input image.

Parameters:

image (str | Path | np.ndarray) – Input image as file path or array.

Returns:

Batch containing the predictions.

Return type:

NumpyImageBatch

Raises:

TypeError – If image input is invalid type.

class anomalib.deploy.TorchInferencer(path, device='auto')#

Bases: object

PyTorch inferencer for anomaly detection models.

Parameters:
  • path (str | Path) – Path to the PyTorch model weights file.

  • device (str, optional) – Device to use for inference. Options are "auto", "cpu", "cuda", "gpu". Defaults to "auto".

Example

>>> from anomalib.deploy import TorchInferencer
>>> model = TorchInferencer(path="path/to/model.pt")
>>> predictions = model.predict(image="path/to/image.jpg")
Raises:
  • ValueError – If an invalid device is specified.

  • ValueError – If the model file has an unknown extension.

  • KeyError – If the checkpoint file does not contain a model.

load_model(path)#

Load the PyTorch model.

Parameters:

path (str | Path) – Path to the PyTorch model file.

Returns:

Loaded PyTorch model in evaluation mode.

Return type:

nn.Module

Raises:

KeyError – If the checkpoint file does not contain a model.

Example

>>> model = TorchInferencer(path="path/to/model.pt")
>>> isinstance(model.model, nn.Module)
True
pre_process(image)#

Pre-process the input image.

Parameters:

image (torch.Tensor) – Input image tensor.

Returns:

Pre-processed image tensor.

Return type:

torch.Tensor

Example

>>> model = TorchInferencer(path="path/to/model.pt")
>>> image = torch.rand(3, 224, 224)
>>> processed = model.pre_process(image)
>>> processed.shape
torch.Size([1, 3, 224, 224])
predict(image)#

Predict anomalies for an input image.

Parameters:

image (str | Path | torch.Tensor) – Input image to predict. Can be a file path or PyTorch tensor.

Returns:

Prediction results containing anomaly maps and scores.

Return type:

ImageBatch

Example

>>> model = TorchInferencer(path="path/to/model.pt")
>>> predictions = model.predict("path/to/image.jpg")
>>> predictions.anomaly_map.shape  
torch.Size([1, 256, 256])