Inference#
Functions for model inference and deployment.
This module provides functionality for deploying trained anomaly detection models and performing inference. It includes:
Model export utilities for converting models to different formats
- Inference classes for making predictions:
Inferencer
: Base inferencer interfaceTorchInferencer
: For PyTorch modelsOpenVINOInferencer
: For OpenVINO IR models
Example
>>> from anomalib.deploy import TorchInferencer
>>> model = TorchInferencer(path="path/to/model.pt")
>>> predictions = model.predict(image="path/to/image.jpg")
The prediction contains anomaly maps and scores:
>>> predictions.anomaly_map
tensor([[0.1, 0.2, ...]])
>>> predictions.pred_score
tensor(0.86)
- class anomalib.deploy.CompressionType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#
-
Model compression type when exporting to OpenVINO.
- FP16#
Weight compression to FP16 precision. All weights are converted to FP16.
- INT8#
Weight compression to INT8 precision. All weights are quantized to INT8, but are dequantized to floating point before inference.
- INT8_PTQ#
Full integer post-training quantization to INT8 precision. All weights and operations are quantized to INT8. Inference is performed in INT8 precision.
- INT8_ACQ#
Accuracy-control quantization to INT8 precision. Weights and operations are quantized to INT8, except those that would degrade model quality beyond an acceptable threshold. Inference uses mixed precision.
Example
>>> from anomalib.deploy import CompressionType >>> compression = CompressionType.INT8_PTQ >>> compression 'int8_ptq'
- class anomalib.deploy.ExportType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#
-
Model export type.
Supported export formats for anomaly detection models.
- ONNX#
Export model to ONNX format
- OPENVINO#
Export model to OpenVINO IR format
- TORCH#
Export model to PyTorch format
Example
>>> from anomalib.deploy import ExportType >>> export_type = ExportType.ONNX >>> export_type 'onnx'
- class anomalib.deploy.OpenVINOInferencer(path, device='AUTO', config=None)#
Bases:
object
OpenVINO inferencer for optimized model inference.
- Parameters:
path (str | Path | tuple[bytes, bytes]) – Path to OpenVINO IR files (
.xml
and.bin
) or ONNX model, or tuple of xml/bin data as bytes.device (str | None, optional) – Inference device. Options:
"AUTO"
,"CPU"
,"GPU"
,"NPU"
. Defaults to"AUTO"
.config (dict | None, optional) – OpenVINO configuration parameters. Defaults to
None
.
Example
>>> from anomalib.deploy import OpenVINOInferencer >>> model = OpenVINOInferencer( ... path="model.xml", ... device="CPU" ... ) >>> prediction = model.predict("test.jpg")
- load_model(path)#
Load OpenVINO model from file or bytes.
- static post_process(predictions)#
Convert OpenVINO predictions to dictionary.
- Parameters:
predictions (OVDict) – Raw predictions from OpenVINO model.
- Returns:
Dictionary of prediction tensors.
- Return type:
- static pre_process(image)#
Pre-process input image.
- Parameters:
image (np.ndarray) – Input image.
- Returns:
Pre-processed image with shape (N,C,H,W).
- Return type:
np.ndarray
- class anomalib.deploy.TorchInferencer(path, device='auto')#
Bases:
object
PyTorch inferencer for anomaly detection models.
- Parameters:
Example
>>> from anomalib.deploy import TorchInferencer >>> model = TorchInferencer(path="path/to/model.pt") >>> predictions = model.predict(image="path/to/image.jpg")
- Raises:
ValueError – If an invalid device is specified.
ValueError – If the model file has an unknown extension.
KeyError – If the checkpoint file does not contain a model.
- load_model(path)#
Load the PyTorch model.
- Parameters:
path (str | Path) – Path to the PyTorch model file.
- Returns:
Loaded PyTorch model in evaluation mode.
- Return type:
nn.Module
- Raises:
KeyError – If the checkpoint file does not contain a model.
Example
>>> model = TorchInferencer(path="path/to/model.pt") >>> isinstance(model.model, nn.Module) True
- pre_process(image)#
Pre-process the input image.
- Parameters:
image (torch.Tensor) – Input image tensor.
- Returns:
Pre-processed image tensor.
- Return type:
Example
>>> model = TorchInferencer(path="path/to/model.pt") >>> image = torch.rand(3, 224, 224) >>> processed = model.pre_process(image) >>> processed.shape torch.Size([1, 3, 224, 224])
- predict(image)#
Predict anomalies for an input image.
- Parameters:
image (str | Path | torch.Tensor) – Input image to predict. Can be a file path or PyTorch tensor.
- Returns:
Prediction results containing anomaly maps and scores.
- Return type:
Example
>>> model = TorchInferencer(path="path/to/model.pt") >>> predictions = model.predict("path/to/image.jpg") >>> predictions.anomaly_map.shape torch.Size([1, 256, 256])