Deep Learning with Tensorflow

Instructor: Christoph Lampert

Teaching Assistant: Alexander Kolesnikov, Amelie Royer

Time and Location:

* Tuesday lectures: 13:15-14:30 (large seminar room, ground floor / Lab Bldg West)

* Thursdays hands-on sessions: 13:15-14:30 (Mondi 2 / Main Bldg)

Duration: Tue, 28-Nov-2017 to Thu, 25-Jan-2018

Pre-Meeting: 14-Nov-2017, 13:15-14:00, Mondi 2

Registration: [for external participants] [for internal participants]

References:

Ian Goodfellow, Yoshua Bengio, Aaron Courville, "Deep Learning", MIT Press 2016 (www.deeplearningbook.org), also available at the IST Library
Aurélien Géron, "Hands-On Machine Learning with Scikit-Learn and TensorFlow", O'Reilly 2017, also available at the IST Library. And the corresponding github page with ipython notebooks.
Stanford Deep Learning courses:
- for NLP http://web.stanford.edu/class/cs224n/syllabus.html
- for Computer Vision http://cs231n.stanford.edu/syllabus.html

Description

Recent years have seen a revival of artificial neural networks for machine learning and data analysis under the name of “Deep Learning”. Tensorflow is one of the leading programming environments for deep learning models. The course will start by giving an introduction into the current state of deep learning. Afterwards, participants will introduce different models in seminar-like talks. A large part of the course will be hands-on homework, where participants implement the described models and apply them to real data.

Requirements

To benefit from the course, you will need

good Python/NumPy skills: the course will not refresh this. Just IST's "Intro to Python" will not be enough. Other programming languages are not possible.

basic knowledge of machine learning on the level of IST's ML course or the DSSC-TCC

the willingness and ability to spend several hours per week on implementing and running code (computational resources will be provided)

the ability to use IST's git server (easy to learn, but won't be covered in the course)

To pass the course, you will need to

regularly attend in the course

give a 30 minute presentation about a deep learning model

upload implementations of hands-on homework to the IST gitlab server

There will be no final exam.

Credits

3 ECTS

Final Grade

50% based on presentation, 50% based on homework

Schedule (tentative)

Date	Topic	Talk Title	Presenter	TA	References (tentative)
Tue 14-Nov-2017	pre-meeting	introduce format and distribute talk topics

Tue 28-Nov-2017	Introduction to tensorflow	Computation graphs / Linear Regression	Christoph Lampert		[1] [2]
Tue 28-Nov-2017		presentation slides (graphs work only with Chrome) complete (HTML) (don't cut-and-paste, code it yourself)	Christoph Lampert		Hands-On book chapter 9
Thu 30-Nov-2017		Hands-on session Saving and restoring graphs (ipynb) (html) Placing variables on CPU/GPU manually (ipynb)
Tue 5-Dec-2017	Artificial neural networks	Multilayer Perceptrons (slides)	Wojciech Rzadkowski	Amelie	Hands-On book chapter 10
Tue 5-Dec-2017		word2vec (slides)	Antoine Karam	Amelie	[1] [2] [3] [4]
Thu 7-Dec-2017		Hands-on session Using Tensorboard (ipynb) (html) Variable scopes (ipynb)
Tue 12-Dec-2017	Convolutional neural networks 1	Convolutional neural networks (ConvNet demo) (slides)	Dominik Schröder	Alex	Hands-On book chapter 13
Tue 12-Dec-2017		AlexNet (slides)	Djordje Slijepcevic	Alex	[1]
Thu 14-Dec-2017		hands-on session
Tue 19-Dec-2017	Convolutional neural networks 2	Modern ConvNet architectures (slides)	Nikola Konstantinov	Alex	[1] [2]
Tue 19-Dec-2017		Neural networks tricks-of-the-trade	Daniel Boocock	Alex	Hands-On book chapter 11
	holiday break
Tue 9-Jan-2018	Deep generative models	Variational Autoencoders (slides)	Georg Sperl	Amelie	Hands-On book chapter 15, [1] [2] [3]
Tue 9-Jan-2018		Generative Adversarial Networks (slides)	Mary Phuong	Amelie	[1] [2]
Thu 11-Jan-2018		hands-on session
Tue 16-Jan-2018	Deep reinforcement learning	Deep Q-Learning	Valentyn Boreiko	Alex	Hands-On book chapter 16, [1] [2 (the DQN part)]
Tue 16-Jan-2018		Playing Atari with Deep Reinforcement Learning	Patrick Blies	Alex	[1] [2]
Thu 18-Jan-2018		hands-on session
Tue 23-Jan-2018	time to complete homework
Thu 25-Jan-2018	time to complete homework

Homework

File	Due Date
HW1: Logistic Regression (Git page) PDF	Dec 5th 2017
HW2: Multilayer Perceptron (Git page) PDF	Dec 12th 2017
Results: bphuong 98.0, bvalentyn 97.7, nkonstan 97.5, dschroed 97.3, gsperl 97.2, dslijepc 97.2, vtoman 97.1, dboocock 97.0, koelsboe 96.8, wrzadkow 96.7
HW3: Convolutional Neural Networks (Git page) PDF	Dec 19th 2017
Results: nkonstan 99.0 (+1.5), bvalentyn 99.0 (+1.3), dboocock 99.0 (+1.9), dslijepc 99.0 (+1.8), dschroed 98.9 (+1.6), gsperl 98.9 (+1.7), bphuong 98.9 (+0.9), wrzadkow 98.3 (+1.7), koelsboe 98.3 (+1.5), vtoman 98.0 (+0.9)
HW4: Modern ConvNets (Git page) PDF	Jan 9th 2018
Results: nkonstan 66.6, gsperl 65.2, bphuong 63.6, dboocock 62.6, dslijepc 56.2, wrzadkow 52.2, dschroed 51.4, bvalentyn ~~23.4~~ 42.3, koelsboe ~~2.7~~ 22.8
HW5: Image Generation (Git page) PDF	Jan 16th 2018
Results:
HW6: Reinforcement learning (Git page) PDF	Jan 23rd/30th 2018