:py:mod:`anomalib.data.utils` ============================= .. py:module:: anomalib.data.utils .. autoapi-nested-parse:: Helper utilities for data. Subpackages ----------- .. toctree:: :titlesonly: :maxdepth: 3 generators/index.rst Submodules ---------- .. toctree:: :titlesonly: :maxdepth: 1 download/index.rst image/index.rst split/index.rst Package Contents ---------------- Classes ~~~~~~~ .. autoapisummary:: anomalib.data.utils.DownloadProgressBar Functions ~~~~~~~~~ .. autoapisummary:: anomalib.data.utils.hash_check anomalib.data.utils.random_2d_perlin anomalib.data.utils.generate_output_image_filename anomalib.data.utils.get_image_filenames anomalib.data.utils.read_image .. py:class:: DownloadProgressBar(iterable: Optional[Iterable] = None, desc: Optional[str] = None, total: Optional[Union[int, float]] = None, leave: Optional[bool] = True, file: Optional[Union[io.TextIOWrapper, io.StringIO]] = None, ncols: Optional[int] = None, mininterval: Optional[float] = 0.1, maxinterval: Optional[float] = 10.0, miniters: Optional[Union[int, float]] = None, use_ascii: Optional[Union[bool, str]] = None, disable: Optional[bool] = False, unit: Optional[str] = 'it', unit_scale: Optional[Union[bool, int, float]] = False, dynamic_ncols: Optional[bool] = False, smoothing: Optional[float] = 0.3, bar_format: Optional[str] = None, initial: Optional[Union[int, float]] = 0, position: Optional[int] = None, postfix: Optional[Dict] = None, unit_divisor: Optional[float] = 1000, write_bytes: Optional[bool] = None, lock_args: Optional[tuple] = None, nrows: Optional[int] = None, colour: Optional[str] = None, delay: Optional[float] = 0, gui: Optional[bool] = False, **kwargs) Bases: :py:obj:`tqdm.tqdm` Create progress bar for urlretrieve. Subclasses `tqdm`. For information about the parameters in constructor, refer to `tqdm`'s documentation. :param iterable: Iterable to decorate with a progressbar. Leave blank to manually manage the updates. :type iterable: Optional[Iterable] :param desc: Prefix for the progressbar. :type desc: Optional[str] :param total: The number of expected iterations. If unspecified, len(iterable) is used if possible. If float("inf") or as a last resort, only basic progress statistics are displayed (no ETA, no progressbar). If `gui` is True and this parameter needs subsequent updating, specify an initial arbitrary large positive number, e.g. 9e9. :type total: Optional[Union[int, float]] :param leave: upon termination of iteration. If `None`, will leave only if `position` is `0`. :type leave: Optional[bool] :param file: Specifies where to output the progress messages (default: sys.stderr). Uses `file.write(str)` and `file.flush()` methods. For encoding, see `write_bytes`. :type file: Optional[Union[io.TextIOWrapper, io.StringIO]] :param ncols: The width of the entire output message. If specified, dynamically resizes the progressbar to stay within this bound. If unspecified, attempts to use environment width. The fallback is a meter width of 10 and no limit for the counter and statistics. If 0, will not print any meter (only stats). :type ncols: Optional[int] :param mininterval: Minimum progress display update interval [default: 0.1] seconds. :type mininterval: Optional[float] :param maxinterval: Maximum progress display update interval [default: 10] seconds. Automatically adjusts `miniters` to correspond to `mininterval` after long display update lag. Only works if `dynamic_miniters` or monitor thread is enabled. :type maxinterval: Optional[float] :param miniters: Minimum progress display update interval, in iterations. If 0 and `dynamic_miniters`, will automatically adjust to equal `mininterval` (more CPU efficient, good for tight loops). If > 0, will skip display of specified number of iterations. Tweak this and `mininterval` to get very efficient loops. If your progress is erratic with both fast and slow iterations (network, skipping items, etc) you should set miniters=1. :type miniters: Optional[Union[int, float]] :param use_ascii: If unspecified or False, use unicode (smooth blocks) to fill the meter. The fallback is to use ASCII characters " 123456789#". :type use_ascii: Optional[Union[bool, str]] :param disable: Whether to disable the entire progressbar wrapper [default: False]. If set to None, disable on non-TTY. :type disable: Optional[bool] :param unit: String that will be used to define the unit of each iteration [default: it]. :type unit: Optional[str] :param unit_scale: If 1 or True, the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc.) [default: False]. If any other non-zero number, will scale `total` and `n`. :type unit_scale: Union[bool, int, float] :param dynamic_ncols: If set, constantly alters `ncols` and `nrows` to the environment (allowing for window resizes) [default: False]. :type dynamic_ncols: Optional[bool] :param smoothing: Exponential moving average smoothing factor for speed estimates (ignored in GUI mode). Ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3]. :type smoothing: Optional[float] :param bar_format: Specify a custom bar string formatting. May impact performance. [default: '{l_bar}{bar}{r_bar}'], where l_bar='{desc}: {percentage:3.0f}%|' and r_bar='| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, ' '{rate_fmt}{postfix}]' Possible vars: l_bar, bar, r_bar, n, n_fmt, total, total_fmt, percentage, elapsed, elapsed_s, ncols, nrows, desc, unit, rate, rate_fmt, rate_noinv, rate_noinv_fmt, rate_inv, rate_inv_fmt, postfix, unit_divisor, remaining, remaining_s, eta. Note that a trailing ": " is automatically removed after {desc} if the latter is empty. :type bar_format: Optional[str] :param initial: The initial counter value. Useful when restarting a progress bar [default: 0]. If using float, consider specifying `{n:.3f}` or similar in `bar_format`, or specifying `unit_scale`. :type initial: Optional[Union[int, float]] :param position: Specify the line offset to print this bar (starting from 0) Automatic if unspecified. Useful to manage multiple bars at once (eg, from threads). :type position: Optional[int] :param postfix: Specify additional stats to display at the end of the bar. Calls `set_postfix(**postfix)` if possible (dict). :type postfix: Optional[Dict] :param unit_divisor: [default: 1000], ignored unless `unit_scale` is True. :type unit_divisor: Optional[float] :param write_bytes: If (default: None) and `file` is unspecified, bytes will be written in Python 2. If `True` will also write bytes. In all other cases will default to unicode. :type write_bytes: Optional[bool] :param lock_args: Passed to `refresh` for intermediate output (initialisation, iterating, and updating). nrows (Optional[int]): The screen height. If specified, hides nested bars outside this bound. If unspecified, attempts to use environment height. The fallback is 20. :type lock_args: Optional[tuple] :param colour: Bar colour (e.g. 'green', '#00ff00'). :type colour: Optional[str] :param delay: Don't display until [default: 0] seconds have elapsed. :type delay: Optional[float] :param gui: WARNING: internal parameter - do not use. Use tqdm.gui.tqdm(...) instead. If set, will attempt to use matplotlib animations for a graphical output [default: False]. :type gui: Optional[bool] .. rubric:: Example >>> with DownloadProgressBar(unit='B', unit_scale=True, miniters=1, desc=url.split('/')[-1]) as p_bar: >>> urllib.request.urlretrieve(url, filename=output_path, reporthook=p_bar.update_to) .. py:method:: update_to(chunk_number: int = 1, max_chunk_size: int = 1, total_size=None) Progress bar hook for tqdm. Based on https://stackoverflow.com/a/53877507 The implementor does not have to bother about passing parameters to this as it gets them from urlretrieve. However the context needs a few parameters. Refer to the example. :param chunk_number: The current chunk being processed. Defaults to 1. :type chunk_number: int, optional :param max_chunk_size: Maximum size of each chunk. Defaults to 1. :type max_chunk_size: int, optional :param total_size: Total download size. Defaults to None. :type total_size: [type], optional .. py:function:: hash_check(file_path: pathlib.Path, expected_hash: str) Raise assert error if hash does not match the calculated hash of the file. :param file_path: Path to file. :type file_path: Path :param expected_hash: Expected hash of the file. :type expected_hash: str .. py:function:: random_2d_perlin(shape: Tuple, res: Tuple[Union[int, torch.Tensor], Union[int, torch.Tensor]], fade=lambda t: 6 * t**5 - 15 * t**4 + 10 * t**3) -> Union[numpy.ndarray, torch.Tensor] Returns a random 2d perlin noise array. :param shape: Shape of the 2d map. :type shape: Tuple :param res: Tuple of scales for perlin noise for height and width dimension. :type res: Tuple[Union[int, Tensor]] :param fade: Function used for fading the resulting 2d map. Defaults to equation 6*t**5-15*t**4+10*t**3. :type fade: _type_, optional :returns: Random 2d-array/tensor generated using perlin noise. :rtype: Union[np.ndarray, Tensor] .. py:function:: generate_output_image_filename(input_path: Union[str, pathlib.Path], output_path: Union[str, pathlib.Path]) -> pathlib.Path Generate an output filename to save the inference image. This function generates an output filaname by checking the input and output filenames. Input path is the input to infer, and output path is the path to save the output predictions specified by the user. The function expects ``input_path`` to always be a file, not a directory. ``output_path`` could be a filename or directory. If it is a filename, the function checks if the specified filename exists on the file system. If yes, the function calls ``duplicate_filename`` to duplicate the filename to avoid overwriting the existing file. If ``output_path`` is a directory, this function adds the parent and filenames of ``input_path`` to ``output_path``. :param input_path: Path to the input image to infer. :type input_path: Union[str, Path] :param output_path: Path to output to save the predictions. Could be a filename or a directory. :type output_path: Union[str, Path] .. rubric:: Examples >>> input_path = Path("datasets/MVTec/bottle/test/broken_large/000.png") >>> output_path = Path("datasets/MVTec/bottle/test/broken_large/000.png") >>> generate_output_image_filename(input_path, output_path) PosixPath('datasets/MVTec/bottle/test/broken_large/000_1.png') >>> input_path = Path("datasets/MVTec/bottle/test/broken_large/000.png") >>> output_path = Path("results/images") >>> generate_output_image_filename(input_path, output_path) PosixPath('results/images/broken_large/000.png') :raises ValueError: When the ``input_path`` is not a file. :returns: The output filename to save the output predictions from the inferencer. :rtype: Path .. py:function:: get_image_filenames(path: Union[str, pathlib.Path]) -> List[pathlib.Path] Get image filenames. :param path: Path to image or image-folder. :type path: Union[str, Path] :returns: List of image filenames :rtype: List[Path] .. py:function:: read_image(path: Union[str, pathlib.Path]) -> numpy.ndarray Read image from disk in RGB format. :param path: path to the image file :type path: str, Path .. rubric:: Example >>> image = read_image("test_image.jpg") :returns: image as numpy array