Homework 3
Note: * indicates a non-compulsory extra exercise. We won't penalize you if your notebooks won't contain solutions to these.
As for previous labs, please provide your solutions in a Jupyter Notebook, trying to interleave code cells with markdown cells to explain what you're doing (if not trivial).
1. Implement L1 norm regularization as a custom loss function
2. The third-to-last paragraph in the notebook is concerning early stopping, an "old" regularization technique which involves the stopping of training earlier than the number of epochs would suggest. Read the paragraph and download the paper from Prechelt et al.
a. Implement early stopping in the E_{opt} specification
b*. Implement early stopping in one of the additional specifications
3*. (from Lab 2) We have seen how to implement the Quadratic Loss for multinomial classification problems. Read the paper from Demirkaya et al. (in which the Quadratic Loss is introduced along with its issues) and try implementing Correct Class Quadratic Loss (CCQL) in PyTorch as well.