EECS4404/5327 Intr to ML/PR
Note: This assignment is mainly for you to review several important discriminative models.
You have to work individually. You must use the same mathematical notations in textbook
or lecture slides to answer these questions. You must use this latex template to write up your
solutions. Remember to fill in your information (name, student number, email) at above. No
handwriting is accepted.
(30 marks) Fully-Connected Neural Networks
(a) (5 marks) Q8.2 on page 199 (see the margin note on page 175 for some examples.)
(b) (25 marks) You will use the MNIST data set for this question. Implement the forward
and backward passes for fully-connected deep neural networks as in Figure 8.19. Use all
MNIST training data to learn a 10-class classifier using your own back-propagation implementation, investigate various network structures (such as different number of layers and nodes
per layer), and report the best possible classification performance in the held-out MNIST test
images. Note that you are only allowed to use libraries for linear algebra operations, such
as matrix multiplication, matrix inversion, and etc. You are not allowed to use any existing
machine learning or statistics toolkits or libraries or any open-source code for this question.
(30 marks) Convolutional Neural Networks
Note that 4404 students are required to do part (b) only (30 marks) while 5327 students need
to do both parts (a) and (b) (10 marks + 20 marks).
(a) Q8.6 on page 200
(b) Implement the forward and backward passes for the following convolutional neural network:
Use all MNIST training data to learn a 10-class classifier and report the best possible classification performance in the held-out MNIST test images. Note that you are allowed to use any
Department of Electrical Engineering and Computer Science 1
York University EECS4404/5327 Intr to ML/PR (Winter 2021)
machine learning or statistics toolkits or libraries for this question. Do some investigations to
ensure you use a suitable toolkit for this question.
(20 marks) Ensemble Learning
(a) (10 marks) Adaboost: Q9.1 on page 215
(b) (10 marks) Gradient Tree Boosting: Q9.5 on page 215
What to submit?
You must submit:
1. one PDF document (using this latex template) for your solutions to all written questions
and all results and discussions for your programming assignments;
2. one zip file that includes all of your Python codes (e.g., *.ipynb if you use Jupyter notebooks) and a readme file for TA to run your codes;
from eClass before the deadline. No late submission will be accepted.
Department of Electrical Engineering and Computer Science 2