avalanche.evaluation.metrics.StreamConfusionMatrix

class avalanche.evaluation.metrics.StreamConfusionMatrix(num_classes: typing.Optional[typing.Union[int, typing.Mapping[int, int]]] = None, normalize: typing.Optional[typing.Literal['true', 'pred', 'all']] = None, save_image: bool = True, image_creator: typing.Callable[[torch.Tensor, typing.Sequence], PIL.Image.Image] = <function default_cm_image_creator>, absolute_class_order: bool = False)[source]

The Stream Confusion Matrix metric. This plugin metric only works on the eval phase.

Confusion Matrix computation can be slow if you compute it for a large number of classes. We recommend to set save_image=False if the runtime is too large.

At the end of the eval phase, this metric logs the confusion matrix relative to all the patterns seen during eval.

The metric can log either a Tensor or a PIL Image representing the confusion matrix.

__init__(num_classes: typing.Optional[typing.Union[int, typing.Mapping[int, int]]] = None, normalize: typing.Optional[typing.Literal['true', 'pred', 'all']] = None, save_image: bool = True, image_creator: typing.Callable[[torch.Tensor, typing.Sequence], PIL.Image.Image] = <function default_cm_image_creator>, absolute_class_order: bool = False)[source]

Creates an instance of the Stream Confusion Matrix metric.

We recommend to set save_image=False if the runtime is too large. In fact, a large number of classes may increase the computation time of this metric.

Parameters
  • num_classes – The number of classes. Defaults to None, which means that the number of classes will be inferred from ground truth and prediction Tensors (see class description for more details). If not None, the confusion matrix will always be of size num_classes, num_classes and only the first num_classes values of output logits or target logits will be considered in the update. If the output or targets are provided as numerical labels, there can be no label greater than num_classes.

  • normalize – Normalizes confusion matrix over the true (rows), predicted (columns) conditions or all the population. If None, confusion matrix will not be normalized. Valid values are: ‘true’, ‘pred’ and ‘all’ or None.

  • save_image – If True, a graphical representation of the confusion matrix will be logged, too. If False, only the Tensor representation will be logged. Defaults to True.

  • image_creator – A callable that, given the tensor representation of the confusion matrix and the corresponding labels, returns a graphical representation of the matrix as a PIL Image. Defaults to default_cm_image_creator.

  • absolute_class_order – If true, the labels in the created image will be sorted by id, otherwise they will be sorted by order of encounter at training time. This parameter is ignored if save_image is False, or the scenario is not a NCScenario.

Methods

__init__([num_classes, normalize, ...])

Creates an instance of the Stream Confusion Matrix metric.

after_backward(strategy)

after_eval(strategy)

after_eval_dataset_adaptation(strategy)

after_eval_exp(strategy)

after_eval_forward(strategy)

after_eval_iteration(strategy)

after_forward(strategy)

after_train_dataset_adaptation(strategy)

after_training(strategy)

after_training_epoch(strategy)

after_training_exp(strategy)

after_training_iteration(strategy)

after_update(strategy)

before_backward(strategy)

before_eval(strategy)

before_eval_dataset_adaptation(strategy)

before_eval_exp(strategy)

before_eval_forward(strategy)

before_eval_iteration(strategy)

before_forward(strategy)

before_train_dataset_adaptation(strategy)

before_training(strategy)

before_training_epoch(strategy)

before_training_exp(strategy)

before_training_iteration(strategy)

before_update(strategy)

reset()

Resets the metric internal state.

result()

Obtains the value of the metric.

update(true_y, predicted_y)