Parallel Runner

The parallel runners creates a pool of runners with the pool size equal to the number defined when creating the runner.

Each process in the pool has a process id assigned to it that is between 0-n_jobs. When a job is run using the parallel runner, the process id is passed to the job. The job can use this id to determine process specific logic. For example, if the pool size is equal to the number of GPUs, the job can use the process id to assign a specific GPU to the process.

Process pool executor.

exception anomalib.pipelines.components.runners.parallel.ParallelExecutionError

Bases: Exception

Pool execution error should be raised when one or more jobs fail in the pool.

class anomalib.pipelines.components.runners.parallel.ParallelRunner(generator, n_jobs)

Bases: Runner

Run the job in parallel using a process pool.

It creates a pool of processes and submits the jobs to the pool. This is useful when you have fixed resources that you want to re-use. Once a process is done, it is replaced with a new job.

Parameters:

• generator (JobGenerator) – The generator that generates the jobs.

• n_jobs (int) – The number of jobs to run in parallel.

Example

Creating a pool with the size of the number of available GPUs and submitting jobs to the pool. >>> ParallelRunner(generator, n_jobs=torch.cuda.device_count()) Each time a job is submitted to the pool, an additional parameter task_id will be passed to job.run method. The job can then use this task_id to assign a particular device to train on. >>> def run(self, arg1: int, arg2: nn.Module, task_id: int) -> None: >>> device = torch.device(f”cuda:{task_id}”) >>> model = arg2.to(device) >>> …

run(args, prev_stage_results=None)

Run the job in parallel.

Return type:

Any