avalanche.benchmarks.classic.CLEAR

avalanche.benchmarks.classic.CLEAR(*, data_name: str = 'clear10', evaluation_protocol: str = 'streaming', feature_type: str | None = None, seed: int | None = None, train_transform: Any | None = None, eval_transform: Any | None = None, dataset_root: str | Path | None = None)[source]

Creates a Domain-Incremental benchmark for CLEAR 10 & 100 with 10 & 100 illustrative classes and an n+1 th background class.

If the dataset is not present in the computer, this method will be able to automatically download and store it.

This generator supports benchmark construction of both ‘iid’ and ‘streaming’ evaluation_protocol. The main difference is:

‘iid’: Always sample testset from current task, which requires

splitting the data into 7:3 train:test with a given random seed.

‘streaming’: Use all data of next task as the testset for current task,

which does not split the data and does not require random seed.

The generator supports both Image and Feature (Tensor) datasets. If feature_type == None, then images will be used. If feature_type is specified, then feature tensors will be used.

The benchmark instance returned by this method will have two fields, train_stream and test_stream, which can be iterated to obtain training and test Experience. Each Experience contains the dataset and the associated task label.

Note that the train/test streams will still be data of current task, regardless of whether evaluation protocol is ‘iid’ or ‘streaming’. For ‘iid’ protocol, train stream is 70% of current task data, and test stream is 30% of current task data. For ‘streaming’ protocol, train stream is 100% of current task data, and test stream is just a duplicate of train stream.

The task label 0 will be assigned to each experience.

Parameters:
  • evaluation_protocol – Choose from [‘iid’, ‘streaming’] if chosen ‘iid’, then must specify a seed between [0,1,2,3,4]; if chosen ‘streaming’, then the seed will be ignored.

  • feature_type – Whether to return raw RGB images or feature tensors extracted by pre-trained models. Can choose between [None, ‘moco_b0’, ‘moco_imagenet’, ‘byol_imagenet’, ‘imagenet’]. If feature_type is None, then images will be returned. Otherwise feature tensors will be returned.

  • seed – If evaluation_protocol is iid, then must specify a seed value for train:test split. Choose between [0,1,2,3,4].

  • train_transform – The transformation to apply to the training data, e.g. a random crop, a normalization or a concatenation of different transformations (see torchvision.transform documentation for a comprehensive list of possible transformations). Defaults to None.

  • eval_transform – The transformation to apply to the test data, e.g. a random crop, a normalization or a concatenation of different transformations (see torchvision.transform documentation for a comprehensive list of possible transformations). Defaults to None.

  • dataset_root – The root path of the dataset. Defaults to None, which means that the default location for str(data_name) will be used.

Returns:

a properly initialized GenericCLScenario instance.