Evaluator#
- class anomalib.metrics.Evaluator(val_metrics=None, test_metrics=None, compute_on_cpu=True)#
Bases:
Module,CallbackEvaluator module for LightningModule.
The Evaluator module is a PyTorch module that computes and logs metrics during validation and test steps. Each AnomalibModule should have an Evaluator module as a submodule to compute and log metrics during validation and test steps. An Evaluation module can be passed to the AnomalibModule as a parameter during initialization. When no Evaluator module is provided, the AnomalibModule will use a default Evaluator module that logs a default set of metrics.
- Parameters:
val_metrics (
AnomalibMetric|Sequence[AnomalibMetric] |None) – Validation metrics. Defaults to[].test_metrics (
AnomalibMetric|Sequence[AnomalibMetric] |None) – Test metrics. Defaults to[].compute_on_cpu (
bool) – Whether to compute metrics on CPU. Defaults toTrue.
Examples
>>> from anomalib.metrics import F1Score, AUROC >>> from anomalib.data import ImageBatch >>> import torch >>> >>> f1_score = F1Score(fields=["pred_label", "gt_label"]) >>> auroc = AUROC(fields=["pred_score", "gt_label"]) >>> >>> evaluator = Evaluator(test_metrics=[f1_score])
- metrics_to_cpu(metrics)#
Set the compute_on_cpu attribute of the metrics to True.
- Return type:
None
- on_test_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx=0)#
Update test metrics with the batch output.
- Return type:
None
- on_test_epoch_end(trainer, pl_module)#
Compute and log test metrics.
- Return type:
None
- on_validation_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx=0)#
Update validation metrics with the batch output.
- Return type:
None
- on_validation_epoch_end(trainer, pl_module)#
Compute and log validation metrics.
- Return type:
None
- setup(trainer, pl_module, stage)#
Move metrics to cpu if
num_devices == 1andcompute_on_cpuis set toTrue.- Return type:
None
- static validate_metrics(metrics)#
Validate metrics.
- Return type:
Sequence[AnomalibMetric]