anomalib.utils.sweep.helpers

Helpers for benchmarking and hyperparameter optimization.

Submodules

Package Contents

Functions

get_sweep_callbacks(→ List[pytorch_lightning.Callback])

Gets callbacks relevant to sweep.

get_meta_data(→ Dict)

Get meta data for inference.

get_openvino_throughput(→ float)

Runs the generated OpenVINO model on a dummy dataset to get throughput.

get_torch_throughput(→ float)

Tests the model on dummy data. Images are passed sequentially to make the comparision with OpenVINO model fair.
anomalib.utils.sweep.helpers.get_sweep_callbacks(config: Union[omegaconf.ListConfig, omegaconf.DictConfig]) → List[pytorch_lightning.Callback][source]

Gets callbacks relevant to sweep.

Parameters

config (Union[DictConfig, ListConfig]) – Model config loaded from anomalib

Returns

List of callbacks

Return type

List[Callback]

anomalib.utils.sweep.helpers.get_meta_data(model: anomalib.models.components.AnomalyModule, input_size: Tuple[int, int]) → Dict[source]

Get meta data for inference.

Parameters

  • model (AnomalyModule) – Trained model from which the metadata is extracted.

  • input_size (Tuple[int, int]) – Input size used to resize the pixel level mean and std.

Returns

Metadata as dictionary.

Return type

(Dict)

anomalib.utils.sweep.helpers.get_openvino_throughput(config: Union[omegaconf.DictConfig, omegaconf.ListConfig], model_path: pathlib.Path, test_dataset: torch.utils.data.DataLoader, meta_data: Dict) → float[source]

Runs the generated OpenVINO model on a dummy dataset to get throughput.

Parameters

  • config (Union[DictConfig, ListConfig]) – Model config.

  • model_path (Path) – Path to folder containing the OpenVINO models. It then searches model.xml in the folder.

  • test_dataset (DataLoader) – The test dataset used as a reference for the mock dataset.

  • meta_data (Dict) – Metadata used for normalization.

Returns

Inference throughput

Return type

float

anomalib.utils.sweep.helpers.get_torch_throughput(config: Union[omegaconf.DictConfig, omegaconf.ListConfig], model: anomalib.models.components.AnomalyModule, test_dataset: torch.utils.data.DataLoader, meta_data: Dict) → float[source]

Tests the model on dummy data. Images are passed sequentially to make the comparision with OpenVINO model fair.

Parameters

  • config (Union[DictConfig, ListConfig]) – Model config.

  • model (Path) – Model on which inference is called.

  • test_dataset (DataLoader) – The test dataset used as a reference for the mock dataset.

  • meta_data (Dict) – Metadata used for normalization.

Returns

Inference throughput

Return type

float