avalanche.training.plugins.EvaluationPlugin

class avalanche.training.plugins.EvaluationPlugin(*metrics: Union[PluginMetric, Sequence[PluginMetric]], loggers: Union[BaseLogger, Sequence[BaseLogger]] = None, collect_all=True, benchmark=None, strict_checks=False, suppress_warnings=False)[source]

Manager for logging and metrics.

An evaluation plugin that obtains relevant data from the training and eval loops of the strategy through callbacks. The plugin keeps a dictionary with the last recorded value for each metric. The dictionary will be returned by the train and eval methods of the strategies. It is also possible to keep a dictionary with all recorded metrics by specifying collect_all=True. The dictionary can be retrieved via the get_all_metrics method.

This plugin also logs metrics using the provided loggers.

__init__(*metrics: Union[PluginMetric, Sequence[PluginMetric]], loggers: Union[BaseLogger, Sequence[BaseLogger]] = None, collect_all=True, benchmark=None, strict_checks=False, suppress_warnings=False)[source]

Creates an instance of the evaluation plugin.

Parameters
  • metrics – The metrics to compute.

  • loggers – The loggers to be used to log the metric values.

  • collect_all – if True, collect in a separate dictionary all metric curves values. This dictionary is accessible with get_all_metrics method.

  • benchmark – DEPRECATED. Ignored argument.

  • strict_checks – if True, checks that the full evaluation streams is used when calling eval. An error will be raised otherwise. When False only warnings will be raised.

  • suppress_warnings – if True, warnings and errors will never be raised from the plugin. If False, warnings and errors will be raised following benchmark and strict_checks behavior.

Methods

__init__(*metrics[, loggers, collect_all, ...])

Creates an instance of the evaluation plugin.

before_eval(strategy, **kwargs)

get_all_metrics()

Return the dictionary of all collected metrics.

get_last_metrics()

Return a shallow copy of dictionary with metric names as keys and last metrics value as values.

publish_metric_value(mval)

Publish a MetricValue to be processed by the loggers.

reset_last_metrics()

Set the dictionary storing last value for each metric to be empty dict.

Attributes

active