avalanche.training.plugins.MASPlugin

class avalanche.training.plugins.MASPlugin(lambda_reg: float = 1.0, alpha: float = 0.5, verbose=False)[source]

Memory Aware Synapses (MAS) plugin.

Similarly to EWC, the MAS plugin computes the importance of each parameter at the end of each experience. The approach computes importance via a second pass on the dataset. MAS does not require supervision and estimates importance using the gradients of the L2 norm of the output. Importance is then used to add a penalty term to the loss function.

Technique introduced in: “Memory Aware Synapses: Learning what (not) to forget” by Aljundi et. al (2018).

Implementation based on FACIL, as in: https://github.com/mmasana/FACIL/blob/master/src/approach/mas.py

__init__(lambda_reg: float = 1.0, alpha: float = 0.5, verbose=False)[source]
Parameters:

  • lambda_reg – hyperparameter weighting the penalty term in the loss.

  • alpha – hyperparameter used to update the importance by also considering the influence in the previous experience.

  • verbose – when True, the computation of the influence shows a progress bar using tqdm.

Methods

__init__([lambda_reg, alpha, verbose])

param lambda_reg:

hyperparameter weighting the penalty term

after_backward(strategy, *args, **kwargs)

Called after criterion.backward() by the BaseTemplate.

after_eval(strategy, *args, **kwargs)

Called after eval by the BaseTemplate.

after_eval_dataset_adaptation(strategy, ...)

Called after eval_dataset_adaptation by the BaseTemplate.

after_eval_exp(strategy, *args, **kwargs)

Called after eval_exp by the BaseTemplate.

after_eval_forward(strategy, *args, **kwargs)

Called after model.forward() by the BaseTemplate.

after_eval_iteration(strategy, *args, **kwargs)

Called after the end of an iteration by the BaseTemplate.

after_forward(strategy, *args, **kwargs)

Called after model.forward() by the BaseTemplate.

after_train_dataset_adaptation(strategy, ...)

Called after train_dataset_adapatation by the BaseTemplate.

after_training(strategy, *args, **kwargs)

Called after train by the BaseTemplate.

after_training_epoch(strategy, *args, **kwargs)

Called after train_epoch by the BaseTemplate.

after_training_exp(strategy, **kwargs)

Called after train_exp by the BaseTemplate.

after_training_iteration(strategy, *args, ...)

Called after the end of a training iteration by the BaseTemplate.

after_update(strategy, *args, **kwargs)

Called after optimizer.update() by the BaseTemplate.

before_backward(strategy, **kwargs)

Called before criterion.backward() by the BaseTemplate.

before_eval(strategy, *args, **kwargs)

Called before eval by the BaseTemplate.

before_eval_dataset_adaptation(strategy, ...)

Called before eval_dataset_adaptation by the BaseTemplate.

before_eval_exp(strategy, *args, **kwargs)

Called before eval_exp by the BaseTemplate.

before_eval_forward(strategy, *args, **kwargs)

Called before model.forward() by the BaseTemplate.

before_eval_iteration(strategy, *args, **kwargs)

Called before the start of a training iteration by the BaseTemplate.

before_forward(strategy, *args, **kwargs)

Called before model.forward() by the BaseTemplate.

before_train_dataset_adaptation(strategy, ...)

Called before train_dataset_adapatation by the BaseTemplate.

before_training(strategy, *args, **kwargs)

Called before train by the BaseTemplate.

before_training_epoch(strategy, *args, **kwargs)

Called before train_epoch by the BaseTemplate.

before_training_exp(strategy, *args, **kwargs)

Called before train_exp by the BaseTemplate.

before_training_iteration(strategy, *args, ...)

Called before the start of a training iteration by the BaseTemplate.

before_update(strategy, *args, **kwargs)

Called before optimizer.update() by the BaseTemplate.

Attributes

supports_distributed

A flag describing whether this plugin supports distributed training.