They are used to see where we can prune the network to reduce the number of parameters without losing much accuracy.

They remove unlikely classes before the final classifier and remain part of the deployed network.

During training, their losses are added to encourage discriminative intermediate features, improve gradient propagation, and provide regularization; the classifiers are discarded at inference time.

During training, their losses are added to encourage discriminative intermediate features, improve gradient propagation, and provide regularization; their predicted classes are averaged with the main classifier at inference time.