avalanche.benchmarks.generators.filelist_benchmark

avalanche.benchmarks.generators.filelist_benchmark(root: Optional[Union[str, pathlib.Path]], train_file_lists: Sequence[Union[str, pathlib.Path]], test_file_lists: Sequence[Union[str, pathlib.Path]], *, other_streams_file_lists: Optional[Dict[str, Sequence[Union[str, pathlib.Path]]]] = None, task_labels: Sequence[int], complete_test_set_only: bool = False, train_transform=None, train_target_transform=None, eval_transform=None, eval_target_transform=None, other_streams_transforms: Optional[Dict[str, Tuple[Any, Any]]] = None) avalanche.benchmarks.scenarios.classification_scenario.GenericCLScenario

Creates a benchmark instance given a list of filelists and the respective task labels. A separate dataset will be created for each filelist and each of those datasets will be considered a separate experience.

This helper functions is the best shot when loading Caffe-style dataset based on filelists.

Beware that this helper function is limited is the following two aspects:

  • The resulting benchmark instance and the intermediate datasets used to populate it will be of type CLASSIFICATION. There is no way to change this.

  • Task labels can only be defined by choosing a single task label for each experience (the same task label is applied to all patterns of experiences sharing the same position in different streams).

Despite those constraints, this helper function is usually sufficiently powerful to cover most continual learning benchmarks based on file lists.

When in need to create a similar benchmark instance starting from an in-memory list of paths, then the similar helper function create_generic_benchmark_from_paths() can be used.

When in need to create a benchmark instance in which task labels are defined in a more fine-grained way, then consider using create_multi_dataset_generic_benchmark() by passing properly initialized AvalancheDataset instances.

Parameters
  • root – The root path of the dataset. Can be None.

  • train_file_lists – A list of filelists describing the paths of the training patterns for each experience.

  • test_file_lists – A list of filelists describing the paths of the test patterns for each experience.

  • other_streams_file_lists – A dictionary describing the content of custom streams. Keys must be valid stream names (letters and numbers, not starting with a number) while the value must be a list of filelists (same as train_file_lists and test_file_lists parameters). If this dictionary contains the definition for “train” or “test” streams then those definition will override the train_file_lists and test_file_lists parameters.

  • task_labels – A list of task labels. Must contain at least a value for each experience. Each value describes the task label that will be applied to all patterns of a certain experience. For more info on that, see the function description.

  • complete_test_set_only – If True, only the complete test set will be returned by the benchmark. This means that the test_file_lists parameter must be list with a single element (the complete test set). Alternatively, can be a plain string or Path object. Defaults to False.

  • train_transform – The transformation to apply to the training data, e.g. a random crop, a normalization or a concatenation of different transformations (see torchvision.transform documentation for a comprehensive list of possible transformations). Defaults to None.

  • train_target_transform – The transformation to apply to training patterns targets. Defaults to None.

  • eval_transform – The transformation to apply to the test data, e.g. a random crop, a normalization or a concatenation of different transformations (see torchvision.transform documentation for a comprehensive list of possible transformations). Defaults to None.

  • eval_target_transform – The transformation to apply to test patterns targets. Defaults to None.

  • other_streams_transforms – Transformations to apply to custom streams. If no transformations are defined for a custom stream, then “train” transformations will be used. This parameter must be a dictionary mapping stream names to transformations. The transformations must be a two elements tuple where the first element defines the X transformation while the second element is the Y transformation. Those elements can be None. If this dictionary contains the transformations for “train” or “test” streams then those transformations will override the train_transform, train_target_transform, eval_transform and eval_target_transform parameters.

Returns

A GenericCLScenario instance.