Deep Learning with Tensorflow

update: you can also use PyTorch for the homeworks, if you already know it. It won't be taught in the course, though.

Instructor: Christoph Lampert

Teaching Assistant: Amelie Royer, Nikola Konstantinov, Mary Phuong (also for PyTorch)

Time and Location:

* Presentations: Mon 10:15-11:30 (Mondi 3 / Main Bldg)

* Hands-on sessions: Wed 10:15-11:30 (Mondi 3 / Main Bldg)

Duration: Mon, 26-Nov-2018 to Wed, 23-Jan-2019

Pre-Meeting: 19-Nov-2018, 13:15-14:00, Mondi 2

Registration: [for external participants] [for internal participants]

References:

Ian Goodfellow, Yoshua Bengio, Aaron Courville, "Deep Learning", MIT Press 2016 (www.deeplearningbook.org), also available at the IST Library
Aurélien Géron, "Hands-On Machine Learning with Scikit-Learn and TensorFlow", O'Reilly 2018, also available at the IST Library. And the corresponding github page with ipython notebooks.
Stanford Deep Learning courses:
- for NLP http://web.stanford.edu/class/cs224n/syllabus.html
- for Computer Vision http://cs231n.stanford.edu/syllabus.html

Description

Recent years have seen a revival of artificial neural networks for machine learning and data analysis under the name of “Deep Learning”. Tensorflow is one of the leading programming environments for deep learning models. The course will start by giving an introduction into the current state of deep learning. Afterwards, participants will introduce different models in seminar-like talks. A large part of the course will be hands-on homework, where participants implement the described models and apply them to real data.

Requirements

To benefit from the course, you will need

good Python/NumPy skills: the course will not refresh this. Just IST's "Intro to Python" will not be enough. Other programming languages are not possible.

basic knowledge of machine learning on the level of IST's ML course or the DSSC-TCC

the willingness and ability to spend several hours per week on implementing and running code (computational resources can be provided)

the ability to use IST's git server (easy to learn, but won't be covered in the course)

To pass the course, you will need to

regularly attend in the course

give a 30 minute presentation about a deep learning model

upload implementations of hands-on homework to the IST gitlab server

There will be no final exam.

Credits

3 ECTS (but it's more work than that...)

Final Grade

50% based on presentation, 50% based on homework

Schedule (tentative)

Date	Topic	Talk Title	Presenter	TA	References (tentative)
Mon 19-Nov-2018	pre-meeting	introduce format and distribute talk topics
Mon 26-Nov-2018	Introduction to tensorflow	Computation graphs / Linear Regression	Christoph Lampert		[1] [2]
Mon 26-Nov-2018		presentation slides (graphs work only with Chrome) complete (HTML)	Christoph Lampert		Hands-On book chapter 9
Wed 28-Nov-2018		Hands-on session Saving and restoring graphs (ipynb) (html) Placing variables on CPU/GPU manually (ipynb)
Mon 03-Dec-2018	no course	one extra week to familiarize yourself with everything
Mon 10-Dec-2018	Artificial neural networks	Multilayer Perceptrons (slides)	Lars Bollmann	Mary	Hands-On book chapter 10
Mon 10-Dec-2018		word2vec (slides) (notebook)	Divyansh Gupta	Amelie	[1] [2] [3] [4]
Wed 12-Dec-2018		Hands-on session 2018/19 colab document Document from 2017/18 for reference: Using Tensorboard (ipynb) (html) Variable scopes (ipynb)
Mon 17-Dec-2018	Convolutional neural networks 1	Convolutional neural networks (slides) (ConvNet demo)	Martin Töpfer	Mary	Hands-On book chapter 13
Mon 17-Dec-2018		AlexNet (slides)	Vyacheslav Li	Nikola	[1]
Wed 19-Dec-2018		hands-on session
	holiday break
Mon 7-Jan-2018		Neural networks tricks-of-the-trade (slides)	Joaquin Padilla Montani	Nikola	Hands-On book chapter 11
Mon 7-Jan-2018	Convolutional neural networks 2	Modern ConvNet architectures	Catalin Rusnac	Nikola	[1] [2]
Mon 14-Jan-2019	Deep generative models	Variational Autoencoders (slides)	Alexandra Peste	Mary	Hands-On book chapter 15, [1] [2] [3]
Mon 14-Jan-2019 (tenative)	Deep reinforcement learning	Deep Q-Learning (slides)	Matthias Lechner	Amelie	Hands-On book chapter 16, [1] [2 (the DQN part)]
Wed 16-Jan-2019		hands-on session
Mon 21-Jan-2019		hands-on session (if required)
Wed 23-Jan-2019		hands-on session

Homework

File	Due Date
HW1: Logistic Regression (Git page) PDF	Dec 10th 2018
HW2: Multilayer Perceptron (Git page) PDF	Dec 17th 2018
Results: padillamontani 97.6, Lars_Bollmann 97.3, Lechner 97.3, Vyacheslav_Li 95.9. -- outside of competition (wrong filename): alexandra_peste 97.7
HW3: Convolutional Neural Networks (Git page) PDF	Jan 7th 2019
Results: Lechner 99.2, Lars_Bollmann 99.1, padillamontani 99.1, Vyacheslav_Li 99.1, Divyansh 99.1, alexandra_peste 98.9, Martin_Topfer 98.9 -- outside of competition (after deadline): Catalin 99.0
HW4: Modern ConvNets (Git page) PDF	Jan 14th 2019
Results: padillamontani 65.2, Catalin 45.9, alexandra_peste 45.0, Vyacheslav Li 6.2, Lars_Bollmann 2.9, -- outside of competition (wrong file size): Lechner -- outside of competition (after deadline): Martin_Topfer 43.2
HW5: Image Generation (Git page) PDF	Jan 21st 2019
HW6: Reinforcement learning (Git page) PDF	Jan 28th 2019 (later on request)