Benchmark Job Generator#

class anomalib.pipelines.benchmark.generator.BenchmarkJobGenerator(accelerator)#

Bases: JobGenerator

Generate benchmark jobs for evaluating model performance.

This class generates benchmark jobs based on provided configurations for models, datasets and other parameters. Each job evaluates a specific model-dataset combination.

Parameters:

accelerator (str) – Type of accelerator to use for running the jobs (e.g. "cpu", "gpu").

Example

>>> from anomalib.pipelines.benchmark.generator import BenchmarkJobGenerator
>>> generator = BenchmarkJobGenerator(accelerator="gpu")
>>> args = {
...     "seed": 42,
...     "model": {"class_path": "Padim"},
...     "data": {"class_path": "MVTec", "init_args": {"category": "bottle"}}
... }
>>> jobs = list(generator.generate_jobs(args, None))
generate_jobs(args, previous_stage_result)#

Generate benchmark jobs from the provided arguments.

Parameters:
  • args (dict) – Dictionary containing job configuration including model, dataset and other parameters.

  • previous_stage_result (PREV_STAGE_RESULT) – Results from previous pipeline stage (unused).

Yields:

Generator[BenchmarkJob, None, None]

Generator yielding benchmark job

instances.

Return type:

Generator[BenchmarkJob, None, None]

Example

>>> generator = BenchmarkJobGenerator(accelerator="cpu")
>>> args = {
...     "seed": 42,
...     "model": {"class_path": "Padim"},
...     "data": {"class_path": "MVTec"}
... }
>>> jobs = list(generator.generate_jobs(args, None))
property job_class: type#

Get the job class used by this generator.

Returns:

The BenchmarkJob class.

Return type:

type