:py:mod:`anomalib.utils.metrics` ================================ .. py:module:: anomalib.utils.metrics .. autoapi-nested-parse:: Custom anomaly evaluation metrics. Submodules ---------- .. toctree:: :titlesonly: :maxdepth: 1 adaptive_threshold/index.rst anomaly_score_distribution/index.rst aupr/index.rst aupro/index.rst auroc/index.rst collection/index.rst min_max/index.rst optimal_f1/index.rst plotting_utils/index.rst Package Contents ---------------- Classes ~~~~~~~ .. autoapisummary:: anomalib.utils.metrics.AdaptiveThreshold anomalib.utils.metrics.AnomalyScoreDistribution anomalib.utils.metrics.AUPR anomalib.utils.metrics.AUPRO anomalib.utils.metrics.AUROC anomalib.utils.metrics.MinMax anomalib.utils.metrics.OptimalF1 .. py:class:: AdaptiveThreshold(default_value: float = 0.5, **kwargs) Bases: :py:obj:`torchmetrics.Metric` Optimal F1 Metric. Compute the optimal F1 score at the adaptive threshold, based on the F1 metric of the true labels and the predicted anomaly scores. .. py:method:: update(preds: torch.Tensor, target: torch.Tensor) -> None Update the precision-recall curve metric. .. py:method:: compute() -> torch.Tensor Compute the threshold that yields the optimal F1 score. Compute the F1 scores while varying the threshold. Store the optimal threshold as attribute and return the maximum value of the F1 score. :returns: Value of the F1 score at the optimal threshold. .. py:class:: AnomalyScoreDistribution(**kwargs) Bases: :py:obj:`torchmetrics.Metric` Mean and standard deviation of the anomaly scores of normal training data. .. py:method:: update(anomaly_scores: Optional[torch.Tensor] = None, anomaly_maps: Optional[torch.Tensor] = None) -> None Update the precision-recall curve metric. .. py:method:: compute() -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor] Compute stats. .. py:class:: AUPR Bases: :py:obj:`torchmetrics.PrecisionRecallCurve` Area under the PR curve. .. py:method:: compute() -> torch.Tensor First compute PR curve, then compute area under the curve. :returns: Value of the AUPR metric .. py:method:: update(preds: torch.Tensor, target: torch.Tensor) -> None Update state with new values. Need to flatten new values as PrecicionRecallCurve expects them in this format for binary classification. :param preds: predictions of the model :type preds: Tensor :param target: ground truth targets :type target: Tensor .. py:method:: _compute() -> Tuple[torch.Tensor, torch.Tensor] Compute prec/rec value pairs. :returns: Tuple containing Tensors for rec and prec .. py:method:: generate_figure() -> Tuple[matplotlib.figure.Figure, str] Generate a figure containing the PR curve as well as the random baseline and the AUC. :returns: Tuple containing both the PR curve and the figure title to be used for logging :rtype: Tuple[Figure, str] .. py:class:: AUPRO(compute_on_step: bool = True, dist_sync_on_step: bool = False, process_group: Optional[Any] = None, dist_sync_fn: Callable = None, fpr_limit: float = 0.3) Bases: :py:obj:`torchmetrics.Metric` Area under per region overlap (AUPRO) Metric. .. py:attribute:: is_differentiable :annotation: :bool = False .. py:attribute:: higher_is_better :annotation: :Optional[bool] .. py:attribute:: full_state_update :annotation: :bool = False .. py:attribute:: preds :annotation: :List[torch.Tensor] .. py:attribute:: target :annotation: :List[torch.Tensor] .. py:method:: update(preds: torch.Tensor, target: torch.Tensor) -> None Update state with new values. :param preds: predictions of the model :type preds: Tensor :param target: ground truth targets :type target: Tensor .. py:method:: _compute() -> Tuple[torch.Tensor, torch.Tensor] Compute the pro/fpr value-pairs until the fpr specified by self.fpr_limit. It leverages the fact that the overlap corresponds to the tpr, and thus computes the overall PRO curve by aggregating per-region tpr/fpr values produced by ROC-construction. :raises ValueError: ValueError is raised if self.target doesn't conform with requirements imposed by kornia for connected component analysis. :returns: tuple containing final fpr and tpr values. :rtype: Tuple[Tensor, Tensor] .. py:method:: compute() -> torch.Tensor Fist compute PRO curve, then compute and scale area under the curve. :returns: Value of the AUPRO metric :rtype: Tensor .. py:method:: generate_figure() -> Tuple[matplotlib.figure.Figure, str] Generate a figure containing the PRO curve and the AUPRO. :returns: Tuple containing both the figure and the figure title to be used for logging :rtype: Tuple[Figure, str] .. py:method:: interp1d(old_x: torch.Tensor, old_y: torch.Tensor, new_x: torch.Tensor) -> torch.Tensor :staticmethod: Function to interpolate a 1D signal linearly to new sampling points. :param old_x: original 1-D x values (same size as y) :type old_x: Tensor :param old_y: original 1-D y values (same size as x) :type old_y: Tensor :param new_x: x-values where y should be interpolated at :type new_x: Tensor :returns: y-values at corresponding new_x values. :rtype: Tensor .. py:class:: AUROC Bases: :py:obj:`torchmetrics.ROC` Area under the ROC curve. .. py:method:: compute() -> torch.Tensor First compute ROC curve, then compute area under the curve. :returns: Value of the AUROC metric :rtype: Tensor .. py:method:: update(preds: torch.Tensor, target: torch.Tensor) -> None Update state with new values. Need to flatten new values as ROC expects them in this format for binary classification. :param preds: predictions of the model :type preds: Tensor :param target: ground truth targets :type target: Tensor .. py:method:: _compute() -> Tuple[torch.Tensor, torch.Tensor] Compute fpr/tpr value pairs. :returns: Tuple containing Tensors for fpr and tpr .. py:method:: generate_figure() -> Tuple[matplotlib.figure.Figure, str] Generate a figure containing the ROC curve, the baseline and the AUROC. :returns: Tuple containing both the figure and the figure title to be used for logging :rtype: Tuple[Figure, str] .. py:class:: MinMax(**kwargs) Bases: :py:obj:`torchmetrics.Metric` Track the min and max values of the observations in each batch. .. py:method:: update(predictions: torch.Tensor) -> None Update the min and max values. .. py:method:: compute() -> Tuple[torch.Tensor, torch.Tensor] Return min and max values. .. py:class:: OptimalF1(num_classes: int, **kwargs) Bases: :py:obj:`torchmetrics.Metric` Optimal F1 Metric. Compute the optimal F1 score at the adaptive threshold, based on the F1 metric of the true labels and the predicted anomaly scores. .. py:method:: update(preds: torch.Tensor, target: torch.Tensor) -> None Update the precision-recall curve metric. .. py:method:: compute() -> torch.Tensor Compute the value of the optimal F1 score. Compute the F1 scores while varying the threshold. Store the optimal threshold as attribute and return the maximum value of the F1 score. :returns: Value of the F1 score at the optimal threshold.