MH-PatchCore#
MH-PatchCore is a one-class image anomaly detector based on the Mahalanobis PatchCore arXiv preprint by Niccolò Ferrari, Oligert Osmani, and Evelina Lamma. It retains PatchCore’s patch-level nearest-neighbor scoring and adds PCA projection, shrinkage covariance whitening, and a bounded merge-reduce memory bank. The reference implementation is MIT-licensed.
Architecture#
Fitting makes three ordered passes over the normal training images:
PCA estimation.
Covariance estimation and Cholesky factorization in PCA space.
Whitened memory-bank construction.
At inference time, the same feature path projects and whitens each patch before nearest-neighbor scoring. The patch grid produces the anomaly map, while the most anomalous patch and its local memory-bank support produce the image score.
Usage#
from anomalib.data import MVTecAD
from anomalib.engine import Engine
from anomalib.models import MHPatchcore
datamodule = MVTecAD(root="./datasets/MVTecAD", category="bottle")
model = MHPatchcore()
engine = Engine()
engine.fit(model=model, datamodule=datamodule)
engine.test(model=model, datamodule=datamodule)
The model is also available through the CLI:
anomalib train --model MHPatchcore --data MVTecAD --data.category bottle
To use the repository configuration:
anomalib train \
--config examples/configs/model/mh_patchcore.yaml \
--data MVTecAD \
--data.category bottle
Default configuration#
Argument |
Default |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The default backbone uses the IMAGENET1K_V2 weights stored as
wide_resnet50_2-9ba9bcbe.pth (SHA-256
9ba9bcbebc349d733a72eb7608143fd754e4689ac8d8ce2916ca0bdff6443950).
Canonical preprocessing resizes images to 256×256, takes a 224×224 center crop,
and applies ImageNet normalization. The raw model returns pred_score and
anomaly_map; anomalib post-processing adds thresholded labels and masks.
Lifecycle and limitations#
Fitting requires exactly three epochs and one device. Validation begins only after the third pass finalizes the memory bank.
Epoch-boundary and fully fitted checkpoints can be restored. Checkpoints captured partway through a fitting pass are rejected because transient accumulators are not serialized.
Torch, ONNX, and OpenVINO export are supported after fitting.
The 224×224 crop with the default backbone and layers is the validated configuration. Alternative
timmbackbones need compatible layer names and are not covered by the reported benchmarks.
Detailed MVTec AD results and their separate published, native anomalib, and same-machine ordered protocols are available in the model package README.
Sample results#
API reference#
MH-PatchCore Lightning model.
- class anomalib.models.image.mh_patchcore.lightning_model.MHPatchcore(backbone='wide_resnet50_2.tv2_in1k', layers=('layer2', 'layer3'), pre_trained=True, pca_variance_ratio=0.99, covariance_shrinkage=0.07, memory_bank_size=1000, local_coreset_size=256, num_neighbors=9, pre_processor=True, post_processor=True, evaluator=True, visualizer=True)#
Bases:
MemoryBankMixin,AnomalibModuleMH-PatchCore Lightning module for anomaly detection.
MH-PatchCore fits its statistical state in three ordered passes over the training data: PCA, covariance whitening, and memory-bank construction.
- Parameters:
backbone (
str) – Name of the timm backbone. Defaults to"wide_resnet50_2.tv2_in1k".layers (
Sequence[str]) – Ordered backbone layers used for feature extraction. Defaults to("layer2", "layer3").pre_trained (
bool) – Whether to load pretrained backbone weights. Defaults toTrue.pca_variance_ratio (
float) – Fraction of explained variance retained by incremental PCA. Defaults to0.99.covariance_shrinkage (
float) – Fixed covariance shrinkage coefficient. Defaults to0.07.memory_bank_size (
int) – Maximum number of vectors in the finalized memory bank. Defaults to1000.local_coreset_size (
int) – Maximum number of vectors retained in each merge-reduce block. Defaults to256.num_neighbors (
int) – Number of memory-bank neighbors used for image score reweighting. Defaults to9.pre_processor (
PreProcessor|bool) – Preprocessor instance or flag to use the default. Defaults toTrue.post_processor (
PostProcessor|bool) – Postprocessor instance or flag to use the default. Defaults toTrue.evaluator (
Evaluator|bool) – Evaluator instance or flag to use the default. Defaults toTrue.visualizer (
Visualizer|bool) – Visualizer instance or flag to use the default. Defaults toTrue.
- static configure_optimizers()#
Return no optimizer because MH-PatchCore performs statistical fitting.
- Return type:
None
- static configure_pre_processor(image_size=None)#
Configure the canonical MH-PatchCore preprocessor.
- Parameters:
image_size (
tuple[int,int] |None) – Resize dimensions before the fixed224 x 224center crop. Defaults to(256, 256).- Returns:
- Resize, center-crop, and ImageNet normalization
transforms used by MH-PatchCore.
- Return type:
- Raises:
ValueError – If either resize dimension is smaller than the fixed center-crop size.
- fit()#
Finalize the active fitting pass.
- Raises:
RuntimeError – If the active pass received no training batches or the persisted fitting stage is invalid.
- Return type:
None
- property learning_type: LearningType#
Return the one-class learning type.
- Returns:
LearningType.ONE_CLASS.- Return type:
LearningType
- on_load_checkpoint(checkpoint)#
Reject mid-pass checkpoints, then restore shared Anomalib state.
Transient PCA, covariance, and merge-reduce accumulators are not serialized. Epoch-boundary and fully fitted checkpoints have a zero stage batch count and can be restored safely. Calling
superkeeps component enable/disable flags and safe post-processor config restore fromAnomalibModule.- Parameters:
checkpoint (
dict[str,Any]) – Lightning checkpoint to validate.- Raises:
RuntimeError – If the checkpoint was saved after processing one or more batches of an unfinished fitting pass.
- Return type:
None
- on_train_epoch_end()#
Finalize the active pass after its training epoch.
- Return type:
None
- on_validation_start()#
Finalize the memory bank immediately before final-epoch validation.
- Raises:
RuntimeError – If validation starts before the memory-bank pass.
- Return type:
None
- property trainer_arguments: dict[str, int]#
Return trainer arguments required by the three fitting passes.
- Returns:
Single-device, three-epoch trainer configuration.
- Return type:
dict[str, int]
- training_step(batch, *args, **kwargs)#
Update the statistical component for the active fitting pass.
- Parameters:
batch (
Batch) – Batch containing training images.*args – Additional positional arguments, which are unused.
**kwargs – Additional keyword arguments, which are unused.
- Returns:
Dummy scalar loss required by Lightning.
- Return type:
Tensor- Raises:
RuntimeError – If training continues after all fitting passes.
TypeError – If the Torch model does not return training embeddings.
- validation_step(batch, *args, **kwargs)#
Generate anomaly predictions for a validation batch.
- Parameters:
batch (
Batch) – Batch containing validation images and metadata.*args – Additional positional arguments, which are unused.
**kwargs – Additional keyword arguments, which are unused.
- Returns:
Batch updated with image scores and anomaly maps.
- Return type:
Union[Tensor,Mapping[str,Any],None]
PyTorch implementation of MH-PatchCore.
- class anomalib.models.image.mh_patchcore.torch_model.MHPatchcoreModel(backbone='wide_resnet50_2.tv2_in1k', layers=('layer2', 'layer3'), pre_trained=True, pca_variance_ratio=0.99, covariance_shrinkage=0.07, memory_bank_size=1000, local_coreset_size=256, num_neighbors=9)#
Bases:
ModuleExtract ordered patch embeddings for MH-PatchCore.
- Parameters:
backbone (
str) – Name of the timm backbone. Defaults to"wide_resnet50_2.tv2_in1k".layers (
Sequence[str]) – Ordered backbone layers used for feature extraction. Defaults to("layer2", "layer3").pre_trained (
bool) – Whether to load pretrained backbone weights. Defaults toTrue.pca_variance_ratio (
float) – Fraction of explained variance retained by incremental PCA. Defaults to0.99.covariance_shrinkage (
float) – Fixed covariance shrinkage coefficient. Defaults to0.07.memory_bank_size (
int) – Maximum number of vectors in the finalized memory bank. Defaults to1000.local_coreset_size (
int) – Maximum number of vectors retained in each merge-reduce block. Defaults to256.num_neighbors (
int) – Number of memory-bank neighbors used for image score reweighting. Defaults to9.
- Raises:
ValueError – If
layersis empty,pca_variance_ratiois outside(0, 1],covariance_shrinkageis outside[0, 1], or a memory-bank size or neighbor count is not a positive integer.
- forward(input_tensor)#
Extract embeddings or return anomaly predictions.
Training mode returns raw patch embeddings. Evaluation mode applies the fitted projection, whitening, memory-bank scoring, and anomaly-map path.
- Parameters:
input_tensor (
Tensor) – Image batch with shape[B, C, H, W].- Returns:
- Raw embeddings during training, or
anomaly scores and maps during evaluation.
- Return type:
Tensor|InferenceBatch- Raises:
RuntimeError – If evaluation is requested before all fitted state is available.
- generate_embedding(features)#
Construct ordered embeddings from hierarchical feature maps.
- Parameters:
features (
dict[str|int,Tensor]) – Feature maps keyed by backbone layer name.- Returns:
- Batch-major, row-major patch embeddings with 1024
features per patch.
- Return type:
Tensor