Attach the gradient magnitude plot here (initialization + after training, batches to sample = 10).

Comment on the plot, what can we say about the gradient magnitudes and how they develop when backpropagating through time?

Now select 'RNN' as model type. Choose the RNN - Simple preset and create the PyTorch model.

What is now the number of trainable parameters?

The results will be quite different. Discuss:

• How can we explain this difference?
• Why is one model so much better than the other?
• Is a CNN a viable architecture for sequential data then?

Now let's try a feed-forward network (FNN) first. Choose the FNN - 2xHidden (ReLU) preset and load the architecture.

How many trainable parameters does this model have?

Analyze the result:

• Compare the test accuracy with the mighty dice baseline. What can we say about the FNN performance?
• Comment and interpret the training/validation/test set accuracies.

Now try to catch up to our magic model! Try to find settings so that you are within 2% of the accuracy of the baseline magic model. In your answer, explain your reasoning for the settings you chose and upload screenshots that show your Dataset & loader, Model & optimization and Model architecture settings and the results, loss plot and confusion matrix/top-loss plots. You can upload several screenshot if not everything fits on one.

Now, with the same settings, switch to ReLU. Which model depth(s) now show at least some form of learning?

Go to the Vanishing Gradients tab. Use the following settings:

• seed: 2026
• learning rate 0.001
• activation: sigmoid
• hidden: 3072
• epochs: 1
• model depths: all
• image size: 32
• batch size: 32

Run the training and go to the Gradient magnitudes tab. Attach the plot that was generated. Describe in your own words what you see in the gradient magnitude plot when comparing the different model depths.

Try all activation functions once. Which one is the worst choice with regard to the vanishing gradient issue?

Now train with the following settings:

• seed: 2026
• learning rate 0.001
• activation: sigmoid
• hidden: 3072
• epochs: 5
• model depths: all
• image size: 32
• batch size: 32

Look at the loss plot. For which model depth(s) can you see that there is some learning taking place?