SM35452SM2020: Homework 4

Opened: Thursday, 25 March 2021, 12:00 AM

Due: Thursday, 8 April 2021, 12:00 AM

Instructions:

As always, please provide your solution in a Jupyter Notebook.

Now that you have all the tools to train an MLP with high performance on MNIST, try reaching 0-loss (or 100% accuracy) on the training data (with a small epsilon, e.g. 99.99% training performance -- don't worry if you overfit!). The implementation is completely up to you. You just need to keep it an MLP without using fancy layers (e.g., keep the Linear layers, don't use Conv1d or something like this, don't use attention). You are free to use any LR scheduler or optimizer, any one of batchnorm/groupnorm, regularization methods... If you use something we haven't seen during lectures, please motivate your choice and explain (as briefly as possible) how it works.
Try reaching 0-loss on the training data with permuted labels. Assess the model on the test data (without permuted labels) and comment. Help yourself with 3. Tip: To permute the labels, act on the trainset.targets with an appropriate torch function. Then, you can pass this "permuted" Dataset to a DataLoader like so: trainloader_permuted = torch.utils.data.DataLoader(trainset_permuted, batch_size=batch_size_train, shuffle=True). You can now use this DataLoader inside the training function. Additional view for motivating this exercise:
.

P.S. I increased the number of files to upload from 1 to 5, so if you want you may include up to 4 local image files in your solution.