Training module
training
Training Templates
Templates define the training/eval loop for each setting (supervised CL, online CL, RL, …). Each template supports a set of callback that can be used by a plugin to execute code inside the training/eval loops.
Templates
Templates are defined in the avalanche.training.templates module.
|
Base class for continual learning skeletons. |
|
Base SGD class for continual learning skeletons. |
|
Base class for continual learning strategies. |
|
Base class for continual learning strategies. |
Plugins ABCs
ABCs for plugins are available in avalanche.core.
ABC for BaseTemplate plugins. |
|
ABC for BaseSGDTemplate plugins. |
|
ABC for SupervisedTemplate plugins. |
Training Strategies
Ready-to-use continual learning strategies.
|
Cumulative training strategy. |
|
Joint training on the entire stream. |
|
Naive finetuning. |
|
AR1 with Latent Replay. |
|
Deep Streaming Linear Discriminant Analysis. |
|
iCaRL Strategy. |
|
Progressive Neural Network strategy. |
|
CWR* Strategy. |
|
Experience replay strategy. |
|
Experience replay strategy. |
|
GDumb strategy. |
|
Learning without Forgetting (LwF) strategy. |
|
Average Gradient Episodic Memory (A-GEM) strategy. |
|
Gradient Episodic Memory (GEM) strategy. |
|
Elastic Weight Consolidation (EWC) strategy. |
|
Synaptic Intelligence strategy. |
|
Continual Prototype Evolution strategy. |
|
Less Forgetful Learning strategy. |
|
Generative Replay Strategy |
|
Memory Aware Synapses (MAS) strategy. |
|
Bias Correction (BiC) strategy. |
|
Maximally Interfered Replay Strategy See ER_MIR plugin for details. |
Replay Buffers and Selection Strategies
Buffers to store past samples according to different policies and selection strategies.
Buffers
|
ABC for rehearsal buffers to store exemplars. |
|
Buffer updated with reservoir sampling. |
|
A buffer that stores exemplars for rehearsal in separate groups. |
|
Rehearsal buffer with samples balanced over experiences. |
|
Stores samples for replay, equally divided over classes. |
|
Stores samples for replay using a custom selection strategy and grouping. |
Selection strategies
Base class to define how to select a subset of exemplars from a dataset. |
|
Select the exemplars at random in the dataset |
|
Base class to select exemplars from their features |
|
|
The herding strategy as described in iCaRL. |
|
A greedy algorithm that selects the remaining exemplar that is the closest to the center of all elements (in feature space). |
Loss Functions
Similar to the Knowledge Distillation Loss. |
|
RegularizationMethod implement regularization strategies. |
|
|
Learning Without Forgetting. |
Training Plugins
Plugins can be added to any CL strategy to support additional behavior.
Utilities in avalanche.training.plugins.
|
Early stopping and model checkpoint plugin. |
|
Manager for logging and metrics. |
|
Learning Rate Scheduler Plugin. |
Strategy implemented as plugins in avalanche.training.plugins.
|
Average Gradient Episodic Memory Plugin. |
|
Continual Prototype Evolution plugin. |
|
CWR* Strategy. |
|
Elastic Weight Consolidation (EWC) plugin. |
|
GDumb plugin. |
|
Gradient Episodic Memory Plugin. |
|
GSSPlugin replay plugin. |
|
Less-Forgetful Learning (LFL) Plugin. |
|
Learning without Forgetting plugin. |
|
Experience replay plugin. |
|
Synaptic Intelligence plugin. |
|
Memory Aware Synapses (MAS) plugin. |
TrainGeneratorAfterExpPlugin makes sure that after each experience of training the solver of a scholar model, we also train the generator on the data of the current experience. |
|
|
Riemannian Walk (RWalk) plugin. |
|
Experience generative replay plugin. |
|
Bias Correction (BiC) plugin. |
|
Maximally Interfered Retrieval plugin, Implements the strategy defined in "Online Continual Learning with Maximally Interfered Retrieval" https://arxiv.org/abs/1908.04742 |