Dataclasses#
This guide explains how to use the dataclasses in Anomalib, from basic usage to advanced use cases across different modalities.
Basic Concepts#
Anomalib uses dataclasses to represent and validate data throughout the pipeline. Anomalib’s dataclasses are based on python’s
native dataclasses, but are extended with several useful features to facilitate input validation and easy conversion.
Dataclasses are used by the AnomalibDataset
and AnomalibDatamodule
to represent input data and ground truth annotations,
and by the AnomalibModule
to store the model predictions. For basic users, knowing how to access and update the fields
of Anomalib’s dataclasses is sufficient to cover most use-cases.
The dataclasses are designed to be:
Type-safe: All fields are validated to ensure correct types and shapes
Modality-specific: Specialized classes for images, videos, and depth data
Framework-specific: Support for both PyTorch and NumPy backends
Batch-aware: Handle both single items and batches of data
The dataclass system is built around two main concepts:
Item: Single data instance (e.g., one image)
Batch: Collection of items with batch processing capabilities
The Item and Batch classes are defined separately for the different data modalities in the libary. For example, when
working with image data, the relevant classes are ImageItem
and ImageBatch
.
Input- and Output fields#
All dataclasses are equipped with the following standard data fields:
Input Fields: Base fields for anomaly detection data
image
: Input image/videogt_label
: Ground truth labelgt_mask
: Ground truth segmentation maskmask_path
: Path to mask file
Output Fields: Fields for model predictions
anomaly_map
: Predicted anomaly heatmappred_score
: Predicted anomaly scorepred_mask
: Predicted segmentation maskpred_label
: Predicted labelexplanation
: Path to explanation visualization
Out of these standard fields, only image
is mandatory. All other fields are optional. In addition to the standard fields,
Anomalib’s dataclasses may contain additional modality-specific input and output fields, depending on the modality of the
data (Image, Video, Depth).
Basic Usage#
Creating a dataclass instance#
To create a new dataclass instance, simply pass the data to the constructor using the keyword arguments. For example, we could use the following code to create a new instance of an ImageItem
from a randomly generated image and an all-negative (no anomalous pixels) ground truth mask.
import torch
from anomalib.data import ImageItem
item = ImageItem(
image=torch.rand((3, 224, 224)),
image_path="path/to/my/image.png"
gt_label=0,
gt_mask=torch.zeros((224, 224)),
)
After creating the instance, you can directly access any of the provided fields of the dataclass
print(item.image_path) # "path/to/my/image.png"
print(item.image.shape) # torch.Size([3, 224, 224])
Similarly, we could create a batch of images, by directly defining an image tensor with a leading batch dimension. Let’s create a random batch consisting of 8 images.
import torch
from anomalib.data import ImageItem
batch = ImageBatch(
image=torch.rand((8, 3, 224, 224)),
gt_label=[0, ] * 8,
gt_mask=torch.zeros((8, 224, 224)),
)
Again, we can inspect the fields of the batch instance by accessing them directly. In addition, the Batch
class provides
a useful batch_size
property to quickly retrieve the number of items in the batch.
print(batch.image.shape) # torch.Size([8, 3, 224, 224])
print(batch.batch_size) # 8
Note: The above examples are for illustrative purposes. In general, most use-cases don’t require instantiating dataclasses explicitly, as Anomalib’s modules create and return the dataclass instances.
Validation and formatting#
The dataclass performs some validation checks to assert that the provided values have the expected shape and format, and automatically converts the values to the correct datatype where possible. This ensures that all instances of the dataclass will always use the same shapes and data types to represent the input- and output fields!
item = ImageItem(
image=torch.rand((8, 3, 224, 224))
)
# raises ValueError because provided value has one dimension too many (batch cannot be converted to single item).
batch = ImageBatch(
image=torch.rand((3, 224, 224)),
gt_label = [1],
)
print(batch.image.shape) # torch.Size([1, 3, 224, 224]) <-- leading batch dimension added automatically
print(batch.gt_label) # tensor([True]) <-- positive label converted to boolean tensor
Updating a dataclass instance#
To update a field of a dataclass instance, simply overwrite its value. The dataclass will automatically run the validation checks before assigning the updated value to the instance!
item = ImageItem(
image=torch.rand((3, 224, 224)),
gt_label=tensor(False),
)
# overwrite an existing field
item.gt_label = tensor(True)
print(item.gt_label) # tensor(True)
# assign a previously unassigned field
item.image_path = "path/to/my/image.png"
print(item.image_path) # "path/to/my/image.png"
# input validation and auto formatting still works
item.pred_score = 0.45
print(item.pred_score) # tensor(0.4500)
As an alternative method of updating dataclass fields, Anomalib’s dataclasses are equipped with the update
method. By default,
the update
method updates the dataclass instance inplace, meaning that the original instance will be modified.
item = ImageItem(
image=torch.rand((3, 224, 224)),
pred_score=0.33,
)
item.update(pred_score=0.87) # this is equivalent to item.pred_score=0.87
print(item.pred_score) # 0.87
If you want to keep the original item, you can pass inplace=False
, and use the new instance returned by update
.
item = ImageItem(
image=torch.rand((3, 224, 224)),
pred_score=0.33,
)
new_item = item.update(pred_score=0.87, inplace=False) # the original item will remain unmodified
print(item.pred_score) # 0.33
print(new_item.pred_score) # 0.87
The update
method can be useful in situations where you want to update multiple fields at once, for example from a dictionary
of predictions returned by your model. This can be achieved by specifying each field as a keyword argument, or by passing an
entire dictionary using the **
notation:
item.update(
pred_score=0.87,
pred_label=True,
)
# the following would have the same effect as the statement above
predictions = {
"pred_score": 0.87,
"pred_label": True,
}
item.update(**predictions)
Converting between items and batch#
It is very easy to switch between Item
and Batch
instances. To separate a Batch
instance into a list of Item
s, simply
use the items
property:
batch = ImageBatch(
image=torch.rand((4, 3, 360, 240))
)
items = batch.items # list of Item instances
Conversely, Batch
has a collate
method that can be invoked to create a new Batch
instance from a list of Item
s.
new_batch = ImageBatch.collate(items) # construct a new batch from a list of Items
It is also possible to directly iterate over the Item
s in a batch, without explicitly calling the items
property:
for item in batch:
# use item
Converting Between Frameworks#
All dataclasses support conversion between PyTorch and NumPy:
# Items
numpy_item = torch_item.to_numpy()
# Batches
numpy_batch = torch_batch.to_numpy()
Supported Modalities#
1. Image Data#
The most basic form, supporting RGB images:
from anomalib.data.dataclasses.torch import ImageItem, ImageBatch
# Single image
item = ImageItem(
image=torch.rand(3, 224, 224),
gt_label=torch.tensor(0),
image_path="image.jpg"
)
# Batch of images
batch = ImageBatch(
image=torch.rand(32, 3, 224, 224),
gt_label=torch.randint(0, 2, (32,)),
image_path=[f"image_{i}.jpg" for i in range(32)]
)
2. Video Data#
For video processing with temporal information:
from anomalib.data.dataclasses.torch import VideoItem, VideoBatch
# Single video item
item = VideoItem(
image=torch.rand(10, 3, 224, 224), # 10 frames
gt_label=torch.tensor(0),
video_path="path/to/video.mp4",
)
# Batch of video items
batch = VideoBatch(
image=torch.rand(32, 10, 3, 224, 224), # 32 videos, 10 frames
gt_label=torch.randint(0, 2, (32,)),
video_path=["video_{}.mp4".format(i) for i in range(32)],
)
3. Depth Data#
For RGB-D or depth-only processing:
from anomalib.data.dataclasses.torch import DepthItem, DepthBatch
# Single depth item
item = DepthItem(
image=torch.rand(3, 224, 224), # RGB image
depth_map=torch.rand(224, 224), # Depth map
image_path="rgb.jpg",
depth_path="depth.png",
)
# Batch of depth items
batch = DepthBatch(
image=torch.rand(32, 3, 224, 224), # RGB images
depth_map=torch.rand(32, 224, 224), # Depth maps
image_path=[f"rgb_{i}.jpg" for i in range(32)],
depth_path=[f"depth_{i}.png" for i in range(32)],
)
Best Practices#
Type Hints: Always use appropriate type hints when subclassing
Validation: Implement custom validators for special requirements
Batch Size: Keep batch dimensions consistent across all fields
Paths: Use relative paths when possible for portability
Batch Processing: Use batch operations when possible for better performance
Device Management: Keep tensors on the same device within a batch
Common Pitfalls#
Inconsistent Shapes: Ensure all batch dimensions match
Missing Fields: Required fields must be provided
Type Mismatches: Use correct tensor types (torch vs numpy)
Memory Leaks: Clear large batches when no longer needed
Path Issues: Use proper path separators for cross-platform compatibility
Device Mismatches: Ensure all tensors in a batch are on the same device
Batch Size Inconsistency: Maintain consistent batch sizes across all fields