avalanche.models.dynamic_optimizers.update_optimizer

avalanche.models.dynamic_optimizers.update_optimizer(optimizer, new_params, optimized_params=None, reset_state=False, remove_params=False, verbose=False)

Update the optimizer by adding new parameters, removing removed parameters, and adding new parameters to the optimizer, for instance after model has been adapted to a new task. The state of the optimizer can also be reset, it will be reset for the modified parameters.

Newly added parameters are added by default to parameter group 0

WARNING: the first call to update_optimizer must be done before calling the model’s adaptation.

Parameters:

• optimizer – the Optimizer object.

• new_params – Dict (name, param) of new parameters.

• optimized_params – Dict (name, param) of currently optimized parameters. In most use cases, it will be None in the first call and the return value of the last `update_optimizer call for the subsequent calls.

• reset_state – Whether to reset the optimizer’s state (i.e momentum). Defaults to False.

• remove_params – Whether to remove parameters that were in the optimizer but are not found in new parameters. For safety reasons, defaults to False.

• verbose – If True, prints information about inferred parameter groups for new params.

Returns:

Dict (name, param) of optimized parameters