Training module

Training Strategies

Ready-to-use continual learning strategies.

BaseStrategy(model, optimizer[, criterion, ...])

Base class for continual learning strategies.

Cumulative(model, optimizer, criterion, ...)

Cumulative training strategy.

JointTraining(model, optimizer, criterion, ...)

Joint training on the entire stream.

Naive(model, optimizer[, criterion, device, ...])

Naive finetuning.

AR1([criterion, momentum, l2, ...])

AR1 with Latent Replay.

StreamingLDA(slda_model, criterion, ...[, ...])

Deep Streaming Linear Discriminant Analysis.

ICaRL(feature_extractor, classifier, ...[, ...])

iCaRL Strategy.

PNNStrategy(num_layers, in_features, ...[, ...])

Progressive Neural Network strategy.

CWRStar(model, optimizer, criterion, ...[, ...])

CWR* Strategy.

Replay(model, optimizer, criterion, ...[, ...])

Experience replay strategy.

GSS_greedy(model, optimizer, criterion, mem_size)

Experience replay strategy.

GDumb(model, optimizer, criterion, mem_size, ...)

GDumb strategy.

LwF(model, optimizer, criterion, alpha, ...)

Learning without Forgetting (LwF) strategy.

AGEM(model, optimizer, criterion, ...[, ...])

Average Gradient Episodic Memory (A-GEM) strategy.

GEM(model, optimizer, criterion, ...[, ...])

Gradient Episodic Memory (GEM) strategy.

EWC(model, optimizer, criterion, ewc_lambda, ...)

Elastic Weight Consolidation (EWC) strategy.

SynapticIntelligence(model, optimizer, ...)

Synaptic Intelligence strategy.

CoPE(model, optimizer, criterion, mem_size, ...)

Continual Prototype Evolution strategy.

LFL(model, optimizer, criterion, lambda_e, ...)

Less Forgetful Learning strategy.

Replay Buffers

Buffers to store past samples according to different policies and selection strategies.


ABC for rehearsal buffers to store exemplars.


Buffer updated with reservoir sampling.

BalancedExemplarsBuffer(max_size[, ...])

A buffer that stores exemplars for rehearsal in separate groups.

ExperienceBalancedBuffer(max_size[, ...])

Rehearsal buffer with samples balanced over experiences.

ClassBalancedBuffer(max_size[, ...])

Stores samples for replay, equally divided over classes.

ParametricBuffer(max_size[, groupby, ...])

Stores samples for replay using a custom selection strategy and grouping.

Selection strategies


Base class to define how to select a subset of exemplars from a dataset.


Select the exemplars at random in the dataset


Base class to select exemplars from their features

HerdingSelectionStrategy(model, layer_name)

The herding strategy as described in iCaRL.

ClosestToCenterSelectionStrategy(model, ...)

A greedy algorithm that selects the remaining exemplar that is the closest to the center of all elements (in feature space).

Loss Functions


Similar to the Knowledge Distillation Loss.

Training Plugins

Plugins can be added to any CL strategy to support additional behavior.



Base class for strategy plugins.

EarlyStoppingPlugin(patience, val_stream_name)

Early stopping plugin.

EvaluationPlugin(*metrics[, loggers, ...])

Manager for logging and metrics.

LRSchedulerPlugin(scheduler[, ...])

Learning Rate Scheduler Plugin.


AGEMPlugin(patterns_per_experience, sample_size)

Average Gradient Episodic Memory Plugin.

CoPEPlugin([mem_size, n_classes, p_size, ...])

Continual Prototype Evolution plugin.

CWRStarPlugin(model[, cwr_layer_name, ...])

CWR* Strategy.

EWCPlugin(ewc_lambda[, mode, decay_factor, ...])

Elastic Weight Consolidation (EWC) plugin.


GDumb plugin.

GEMPlugin(patterns_per_experience, ...)

Gradient Episodic Memory Plugin.

GSS_greedyPlugin([mem_size, mem_strength, ...])

GSSPlugin replay plugin.


Less-Forgetful Learning (LFL) Plugin.

LwFPlugin([alpha, temperature])

A Learning without Forgetting plugin.

ReplayPlugin([mem_size, storage_policy])

Experience replay plugin.

SynapticIntelligencePlugin(si_lambda[, eps, ...])

The Synaptic Intelligence plugin.