avalanche.training.SCR

class avalanche.training.SCR(model: ~avalanche.models.scr_model.SCRModel, optimizer: ~torch.optim.optimizer.Optimizer, augmentations=Compose(     Lambda() ), mem_size: int = 100, temperature: int = 0.1, train_mb_size: int = 1, batch_size_mem: int = 100, train_epochs: int = 1, eval_mb_size: int | None = 1, device='cpu', plugins: ~typing.Sequence[~avalanche.core.BaseSGDPlugin] | None = None, evaluator=<function default_evaluator>, eval_every=-1, peval_mode='epoch')

Supervised Contrastive Replay from https://arxiv.org/pdf/2103.13885.pdf. This strategy trains an encoder network in a self-supervised manner to cluster together examples of the same class while pushing away examples of different classes. It uses the Nearest Class Mean classifier on the embeddings produced by the encoder.

Accuracy cannot be monitored during training (no NCM classifier). During training, NCRLoss is monitored, while during eval CrossEntropyLoss is monitored.

The original paper uses an additional fine-tuning phase on the buffer at the end of each experience (called review trick, but not mentioned in the paper). This implementation does not implement the review trick.

__init__(model: ~avalanche.models.scr_model.SCRModel, optimizer: ~torch.optim.optimizer.Optimizer, augmentations=Compose(     Lambda() ), mem_size: int = 100, temperature: int = 0.1, train_mb_size: int = 1, batch_size_mem: int = 100, train_epochs: int = 1, eval_mb_size: int | None = 1, device='cpu', plugins: ~typing.Sequence[~avalanche.core.BaseSGDPlugin] | None = None, evaluator=<function default_evaluator>, eval_every=-1, peval_mode='epoch')

Parameters:

• model – an Avalanche model like the avalanche.models.SCRModel, where the train classifier uses a projection network (e.g., MLP) while the test classifier uses a NCM Classifier. Normalization should be applied between feature extractor and classifier.

• optimizer – PyTorch optimizer.

• augmentations – TorchVision Compose Transformations to augment the input minibatch. The augmented mini-batch will be concatenated to the original one (which includes the memory buffer). Note: only augmentations that can be applied to Tensors are supported.

• mem_size – replay memory size, used also at test time to compute class means.

• temperature – SCR Loss temperature.

• train_mb_size – mini-batch size for training. The default dataloader is a task-balanced dataloader that divides each mini-batch evenly between samples from all existing tasks in the dataset.

• batch_size_mem – number of examples drawn from the buffer.

• train_epochs – number of training epochs.

• eval_mb_size – mini-batch size for eval.

• device – PyTorch device where the model will be allocated.

• plugins – (optional) list of StrategyPlugins.

• evaluator – (optional) instance of EvaluationPlugin for logging and metric computations. None to remove logging.

• eval_every – the frequency of the calls to eval inside the training loop. -1 disables the evaluation. 0 means eval is called only at the end of the learning experience. Values >0 mean that eval is called every eval_every epochs and at the end of the learning experience.

• peval_mode – one of {‘epoch’, ‘iteration’}. Decides whether the periodic evaluation during training should execute every eval_every epochs or iterations (Default=’epoch’).

Methods

__init__(model, optimizer[, augmentations, ...])	param model: an Avalanche model like the avalanche.models.SCRModel,
backward()	Run the backward pass.
check_model_and_optimizer([...])
compute_class_means()
criterion()	Loss function for supervised problems.
eval(exp_list, **kwargs)	Evaluate the current model on a series of experiences and returns the last recorded value for each metric.
eval_dataset_adaptation(**kwargs)	Initialize self.adapted_dataset.
eval_epoch(**kwargs)	Evaluation loop over the current self.dataloader.
forward()	Compute the model's output given the current mini-batch.
make_eval_dataloader([num_workers, shuffle, ...])	Initializes the eval data loader. :param num_workers: How many subprocesses to use for data loading. 0 means that the data will be loaded in the main process. (default: 0). :param pin_memory: If True, the data loader will copy Tensors into CUDA pinned memory before returning them. Defaults to True. :param kwargs: :return:.
make_optimizer([reset_optimizer_state])	Optimizer initialization.
make_train_dataloader([num_workers, ...])	Data loader initialization.
model_adaptation([model])	Adapts the model to the current data.
optimizer_step()	Execute the optimizer step (weights update).
stop_training()	Signals to stop training at the next iteration.
train(experiences[, eval_streams])	Training loop.
train_dataset_adaptation(**kwargs)	Initialize self.adapted_dataset.
training_epoch(**kwargs)	Training epoch.

Attributes

is_eval	True if the strategy is in evaluation mode.
mb_task_id	Current mini-batch task labels.
mb_x	Current mini-batch input.
mb_y	Current mini-batch target.
mbatch	Current mini-batch.
mb_output	Model's output computed on the current mini-batch.
dataloader	Dataloader.
optimizer	PyTorch optimizer.
loss	Loss of the current mini-batch.
train_epochs	Number of training epochs.
train_mb_size	Training mini-batch size.
eval_mb_size	Eval mini-batch size.
retain_graph	Retain graph when calling loss.backward().
evaluator	EvaluationPlugin used for logging and metric computations.
clock	Incremental counters for strategy events.
adapted_dataset	Data used to train.
model	PyTorch model.
device	PyTorch device where the model will be allocated.
plugins	List of `SupervisedPlugin`s. .
experience	Current experience.
is_training	True if the strategy is in training mode.
current_eval_stream	Current evaluation stream.