Homework 2
As for homework 1, try to write your solution in a Jupyter Notebook trying to explain your choices using comments or (preferably) the markdown cells. For this homework, Python scripts will be accepted as well.
Reconstruct in PyTorch the first experiment in Learning representations by back-propagating errors with learning rule in eq.8 (gradient descent without momentum) (alternative link to paper).
Try to be as close as possible to the original protocol, except for what regards the learning rule, and perhaps the random initialization method
-
Read the paper (don’t worry if you don’t understand the other experiments in detail, because our focus is on the first one)
-
Create the data, the model and everything is needed (do not use dataloaders if you don’t know yet how they work)
-
Train the model
-
Inspect the weights you obtained and check if they provide a solution to the problem
Compare the solution to the solution reported in the paper