The probabilities of all possible next tokens in the vocabulary.

The importance of each training example for the current prediction.

How strongly one query token attends to each key token when forming its updated representation.

The similarity between the final output embeddings of all words in the vocabulary.