DFM#
Deep Feature Modeling (DFM) for anomaly detection.
This module provides a PyTorch Lightning implementation of the DFM model for anomaly detection. The model extracts deep features from images using a pre-trained CNN backbone and fits a Gaussian model on these features to detect anomalies.
Paper: https://arxiv.org/abs/1909.11786
Example
>>> from anomalib.models.image import Dfm
>>> model = Dfm(
... backbone="resnet50",
... layer="layer3",
... pre_trained=True
... )
Notes
The model uses a pre-trained backbone to extract features and fits a PCA transformation followed by a Gaussian model during training. No gradient updates are performed on the backbone.
See also
anomalib.models.image.dfm.torch_model.DFMModel
:PyTorch implementation of the DFM model.
- class anomalib.models.image.dfm.lightning_model.Dfm(backbone='resnet50', layer='layer3', pre_trained=True, pooling_kernel_size=4, pca_level=0.97, score_type='fre', pre_processor=True, post_processor=True, evaluator=True, visualizer=True)#
Bases:
MemoryBankMixin
,AnomalibModule
DFM Lightning Module.
- Parameters:
backbone (str) – Name of the backbone CNN network. Defaults to
"resnet50"
.layer (str) – Name of the layer to extract features from the backbone. Defaults to
"layer3"
.pre_trained (bool, optional) – Whether to use a pre-trained backbone. Defaults to
True
.pooling_kernel_size (int, optional) – Kernel size for pooling features. Defaults to
4
.pca_level (float, optional) – Ratio of variance to preserve in PCA. Must be between 0 and 1. Defaults to
0.97
.score_type (str, optional) – Type of anomaly score to compute. Options are
"fre"
(feature reconstruction error) or"nll"
(negative log-likelihood). Defaults to"fre"
.pre_processor (PreProcessor | bool, optional) – Pre-processor to use. If
True
, uses the default pre-processor. IfFalse
, no pre-processing is performed. Defaults toTrue
.post_processor (PostProcessor | bool, optional) – Post-processor to use. If
True
, uses the default post-processor. IfFalse
, no post-processing is performed. Defaults toTrue
.evaluator (Evaluator | bool, optional) – Evaluator to use. If
True
, uses the default evaluator. IfFalse
, no evaluation is performed. Defaults toTrue
.visualizer (Visualizer | bool, optional) – Visualizer to use. If
True
, uses the default visualizer. IfFalse
, no visualization is performed. Defaults toTrue
.
- static configure_optimizers()#
Configure optimizers for training.
- Returns:
DFM doesn’t require optimization.
- Return type:
None
- fit()#
Fit the PCA transformation and Gaussian model to the embeddings.
The method aggregates embeddings collected during training and fits both the PCA transformation and Gaussian model used for scoring.
- Return type:
- property learning_type: LearningType#
Get the learning type of the model.
- Returns:
The model uses one-class learning.
- Return type:
LearningType
- training_step(batch, *args, **kwargs)#
Extract features from the input batch during training.
- Parameters:
batch (Batch) – Input batch containing images.
*args – Additional positional arguments (unused).
**kwargs – Additional keyword arguments (unused).
- Returns:
Dummy loss tensor for compatibility.
- Return type:
- validation_step(batch, *args, **kwargs)#
Compute predictions for the input batch during validation.
- Parameters:
batch (Batch) – Input batch containing images.
*args – Additional positional arguments (unused).
**kwargs – Additional keyword arguments (unused).
- Returns:
Dictionary containing anomaly scores and maps.
- Return type:
STEP_OUTPUT
PyTorch model for Deep Feature Modeling (DFM).
This module provides a PyTorch implementation of the DFM model for anomaly detection. The model extracts deep features from images using a pre-trained CNN backbone and fits a Gaussian model on these features to detect anomalies.
Example
>>> import torch
>>> from anomalib.models.image.dfm.torch_model import DFMModel
>>> model = DFMModel(
... backbone="resnet18",
... layer="layer4",
... pre_trained=True
... )
>>> batch = torch.randn(32, 3, 224, 224)
>>> features = model(batch) # Returns features during training
>>> predictions = model(batch) # Returns scores during inference
Notes
The model uses a pre-trained backbone to extract features and fits a PCA transformation followed by a Gaussian model during training. No gradient updates are performed on the backbone.
- class anomalib.models.image.dfm.torch_model.DFMModel(backbone, layer, pre_trained=True, pooling_kernel_size=4, n_comps=0.97, score_type='fre')#
Bases:
Module
Deep Feature Modeling (DFM) model for anomaly detection.
The model extracts deep features from images using a pre-trained CNN backbone and fits a Gaussian model on these features to detect anomalies.
- Parameters:
backbone (str) – Pre-trained model backbone from timm.
layer (str) – Layer from which to extract features.
pre_trained (bool, optional) – Whether to use pre-trained backbone. Defaults to
True
.pooling_kernel_size (int, optional) – Kernel size to pool features. Defaults to
4
.n_comps (float, optional) – Ratio for PCA components calculation. Defaults to
0.97
.score_type (str, optional) – Scoring type -
fre
ornll
. Defaults tofre
. Segmentation supported withfre
only. Fornll
, set task to classification.
Example
>>> model = DFMModel( ... backbone="resnet18", ... layer="layer4", ... pre_trained=True ... ) >>> input_tensor = torch.randn(32, 3, 224, 224) >>> output = model(input_tensor)
- fit(dataset)#
Fit PCA and Gaussian model to dataset.
- Parameters:
dataset (torch.Tensor) – Input dataset with shape
(n_samples, n_features)
.- Return type:
- forward(batch)#
Compute anomaly predictions from input images.
- Parameters:
batch (torch.Tensor) – Input images with shape
(batch_size, channels, height, width)
.- Returns:
- Model predictions. During
training returns features tensor. During inference returns
InferenceBatch
with prediction scores and anomaly maps.
- Return type:
Union[torch.Tensor, InferenceBatch]
- get_features(batch)#
Extract features from the pretrained network.
- Parameters:
batch (torch.Tensor) – Input images with shape
(batch_size, channels, height, width)
.- Returns:
- Features during
training, or tuple of (features, feature_shapes) during inference.
- Return type:
Union[torch.Tensor, Tuple[torch.Tensor, torch.Size]]
- score(features, feature_shapes)#
Compute anomaly scores.
Scores are either PCA-based feature reconstruction error (FRE) scores or Gaussian density-based NLL scores.
- Parameters:
features (torch.Tensor) – Features for scoring with shape
(n_samples, n_features)
.feature_shapes (tuple) – Shape of features tensor for anomaly map.
- Returns:
- Tuple containing
(scores, anomaly_maps). Anomaly maps are None for NLL scoring.
- Return type:
tuple[torch.Tensor, Optional[torch.Tensor]]
- class anomalib.models.image.dfm.torch_model.SingleClassGaussian#
Bases:
DynamicBufferMixin
Model Gaussian distribution over a set of points.
This class fits a single Gaussian distribution to a set of feature vectors and computes likelihood scores for new samples.
Example
>>> gaussian = SingleClassGaussian() >>> features = torch.randn(128, 100) # 100 samples of 128 dimensions >>> gaussian.fit(features) >>> scores = gaussian.score_samples(features)
- fit(dataset)#
Fit a Gaussian model to dataset X.
Covariance matrix is not calculated directly using
C = X.X^T
. Instead, it is represented using SVD of X:X = U.S.V^T
. Hence,C = U.S^2.U^T
. This simplifies the calculation of the log-likelihood without requiring full matrix inversion.- Parameters:
dataset (torch.Tensor) – Input dataset to fit the model with shape
(n_features, n_samples)
.- Return type:
- forward(dataset)#
Fit the model to the input dataset.
Transforms the input dataset based on singular values calculated earlier.
- Parameters:
dataset (torch.Tensor) – Input dataset with shape
(n_features, n_samples)
.- Return type:
- score_samples(features)#
Compute the negative log likelihood (NLL) scores.
- Parameters:
features (torch.Tensor) – Semantic features on which density modeling is performed with shape
(n_samples, n_features)
.- Returns:
NLL scores for each sample.
- Return type: