Go to the 'Transformers - Text to Speech' tab. Record yourself with the text:

Good morning. We will now check how you can understand me.

and try to get a flawless transcription. Include a screenshot of the transcribed text with the spectrogram. Also try this with your native language (if it is not English) - can the model handle it too? Also include a screenshot of that attempt.

In the 'Transformers - Attention' tab, enter the following text:

AI agents might soon overtake the world

Set d_model = 16, leave scaling on enabled and set a seed of 2026. Which word in the sentence does the word 'agents' most strongly attend to?

What does one row of a self-attention matrix represent?

In self-attention, what happens after the attention weights have been computed?

Switch to probability sampling and set the temperature to 3.0. What output do you get now? Add the text here. Why do you think it looks like this?

Also explain the temperature parameter and how it affects the probability distribution of next generated token - what does a high/low temperature mean?

Set the following training hyperparameters:

• Epochs: 100
• Learning Rate: 0.001
• Grad clip: 0.25
• Seed: 2026
• Use early stopping: yes
• Patience: 3
• Minimum validation loss improvement: 0.001
• Save the model under a4_shakespeare.pt

Interpret your results:

• Attach the loss and perplexity plots here.
• Report your loss and perplexity on the test set.
• What does the perplexity value in general tell us and how do you now interpret your specific value?
• What would be the best possible perplexity value, which one the worst (random predictions)?

Generate some text.

• Use as start text Romeo: (don't forget to include the : - that is important)
• Length: 120 tokens
• k: 5
• Temperature: 0.7
• Seed: 2026

Generate three texts (probabilistic sampling, top-1 and top-k). List the three texts here. How do the texts differ? Do they capture the style and tone of Shakespearean language in your opinion? Which version manages to do that best?

Switch to top1 (greedy) generation and generate the text again. Then check out the next token probabilities. It is highly likely that the next predicted token after Romeo: is the <eos> token. Why do you think that is?

Enter just the word gleefully in the tokenization textbox. Which token(s) do you get? Why do you think is this word processed in this way?

Elaborate on the tradeoff between dictionary size and sequence length between tokenization schemes (word/char/BPE-subword level).