avalanche.training.supervised.lamaml_v2.LaMAML

class avalanche.training.supervised.lamaml_v2.LaMAML(model: ~torch.nn.modules.module.Module, optimizer: ~torch.optim.optimizer.Optimizer, criterion=CrossEntropyLoss(), n_inner_updates: int = 5, second_order: bool = True, grad_clip_norm: float = 1.0, learn_lr: bool = True, lr_alpha: float = 0.25, sync_update: bool = False, alpha_init: float = 0.1, max_buffer_size: int = 200, buffer_mb_size: int = 10, train_mb_size: int = 1, train_epochs: int = 1, eval_mb_size: int = 1, device: str | ~torch.device = 'cpu', plugins: ~typing.Sequence[~avalanche.core.SupervisedPlugin] | None = None, evaluator: ~avalanche.training.plugins.evaluation.EvaluationPlugin | ~typing.Callable[[], ~avalanche.training.plugins.evaluation.EvaluationPlugin] = <function default_evaluator>, eval_every=-1, peval_mode='epoch')[source]

__init__(model: ~torch.nn.modules.module.Module, optimizer: ~torch.optim.optimizer.Optimizer, criterion=CrossEntropyLoss(), n_inner_updates: int = 5, second_order: bool = True, grad_clip_norm: float = 1.0, learn_lr: bool = True, lr_alpha: float = 0.25, sync_update: bool = False, alpha_init: float = 0.1, max_buffer_size: int = 200, buffer_mb_size: int = 10, train_mb_size: int = 1, train_epochs: int = 1, eval_mb_size: int = 1, device: str | ~torch.device = 'cpu', plugins: ~typing.Sequence[~avalanche.core.SupervisedPlugin] | None = None, evaluator: ~avalanche.training.plugins.evaluation.EvaluationPlugin | ~typing.Callable[[], ~avalanche.training.plugins.evaluation.EvaluationPlugin] = <function default_evaluator>, eval_every=-1, peval_mode='epoch')[source]

Implementation of Look-ahead MAML (LaMAML) strategy.

Parameters:

model – PyTorch model.
optimizer – PyTorch optimizer.
criterion – loss function.
n_inner_updates – number of inner updates.
second_order – If True, it computes the second-order derivative of the inner update trajectory for the meta-loss. Otherwise, it computes the meta-loss with a first-order approximation.
grad_clip_norm – gradient clipping norm.
learn_lr – if True, it learns the LR for each batch of data.
lr_alpha – LR for learning the main update’s learning rate.
sync_update – if True, it updates the meta-model with a fixed learning rate. Mutually exclusive with learn_lr and lr_alpha.
alpha_init – initialization value for learnable LRs.
max_buffer_size – maximum buffer size. The default storage policy is reservoir-sampling.
buffer_mb_size – number of buffer samples in each step.

Methods

`__init__`(model, optimizer[, criterion, ...])	Implementation of Look-ahead MAML (LaMAML) strategy.
`backward`()	Run the backward pass.
`check_model_and_optimizer`([...])
`copy_grads`(params_1, params_2)
`criterion`()	Loss function for supervised problems.
`eval`(exp_list, **kwargs)	Evaluate the current model on a series of experiences and returns the last recorded value for each metric.
`eval_dataset_adaptation`(**kwargs)	Initialize self.adapted_dataset.
`eval_epoch`(**kwargs)	Evaluation loop over the current self.dataloader.
`forward`()	Compute the model's output given the current mini-batch.
`inner_update_step`(fast_params, x, y, t)	Update fast weights using current samples and return the updated fast model.
`make_eval_dataloader`([num_workers, shuffle, ...])	Initializes the eval data loader. :param num_workers: How many subprocesses to use for data loading. 0 means that the data will be loaded in the main process. (default: 0). :param pin_memory: If True, the data loader will copy Tensors into CUDA pinned memory before returning them. Defaults to True. :param kwargs: :return:.
`make_optimizer`([reset_optimizer_state])	Optimizer initialization.
`make_train_dataloader`([num_workers, ...])	Data loader initialization.
`model_adaptation`([model])	Adapts the model to the current data.
`optimizer_step`()	Execute the optimizer step (weights update).
`stop_training`()	Signals to stop training at the next iteration.
`train`(experiences[, eval_streams])	Training loop.
`train_dataset_adaptation`(**kwargs)	Initialize self.adapted_dataset.
`training_epoch`(**kwargs)	Training epoch.

Attributes

`is_eval`	True if the strategy is in evaluation mode.
`mb_task_id`	Current mini-batch task labels.
`mb_x`	Current mini-batch input.
`mb_y`	Current mini-batch target.
`mbatch`	Current mini-batch.
`mb_output`	Model's output computed on the current mini-batch.
`dataloader`	Dataloader.
`optimizer`	PyTorch optimizer.
`loss`	Loss of the current mini-batch.
`train_epochs`	Number of training epochs.
`train_mb_size`	Training mini-batch size.
`eval_mb_size`	Eval mini-batch size.
`retain_graph`	Retain graph when calling loss.backward().
`evaluator`	EvaluationPlugin used for logging and metric computations.
`clock`	Incremental counters for strategy events.
`adapted_dataset`	Data used to train.
`model`	PyTorch model.
`device`	PyTorch device where the model will be allocated.
`plugins`	List of `SupervisedPlugin`s. .
`experience`	Current experience.
`is_training`	True if the strategy is in training mode.
`current_eval_stream`	Current evaluation stream.