IST Austria Courses
IST Austria logo

Deep Learning with Tensorflow

update: you can also use PyTorch for the homeworks, if you already know it. It won't be taught in the course, though.

Instructor: Christoph Lampert

Teaching Assistant:  Amelie Royer, Nikola Konstantinov, Mary Phuong (also for PyTorch)

Time and Location:

* Presentations: Mon 10:15-11:30 (Mondi 3 / Main Bldg)

* Hands-on sessions: Wed 10:15-11:30 (Mondi 3 / Main Bldg)

Duration: Mon, 26-Nov-2018 to Wed, 23-Jan-2019

Pre-Meeting: 19-Nov-2018, 13:15-14:00, Mondi 2

Registration: [for external participants]   [for internal participants]

References:

Description

 

Recent years have seen a revival of artificial neural networks for machine learning and data analysis under the name of “Deep Learning”. Tensorflow is one of the leading programming environments for deep learning models. The course will start by giving an introduction into the current state of deep learning. Afterwards, participants will introduce different models in seminar-like talks. A large part of the course will be hands-on homework, where participants implement the described models and apply them to real data.


Requirements

To benefit from the course, you will need

  • the willingness and ability to spend several hours per week on implementing and running code (computational resources can be provided)

  • the ability to use IST's git server (easy to learn, but won't be covered in the course)

To pass the course, you will need to

  • regularly attend in the course

  • give a 30 minute presentation about a deep learning model

  • upload implementations of hands-on homework to the IST gitlab server

There will be no final exam.

 


Credits

Final Grade

 

 

Schedule (tentative)

Date Topic Talk Title Presenter TA References (tentative)
Mon 19-Nov-2018 pre-meeting introduce format and distribute talk topics      
Mon 26-Nov-2018 Introduction to tensorflow Computation graphs / Linear Regression Christoph Lampert   [1] [2]
Mon 26-Nov-2018   presentation slides (graphs work only with Chrome)
complete (HTML)
Christoph Lampert   Hands-On book chapter 9
Wed 28-Nov-2018   Hands-on session
Saving and restoring graphs (ipynb) (html)
Placing variables on CPU/GPU manually (ipynb)
     
Mon 03-Dec-2018 no course one extra week to familiarize
yourself with everything
   
Mon 10-Dec-2018 Artificial neural networks Multilayer Perceptrons (slides) Lars Bollmann Mary Hands-On book chapter 10
Mon 10-Dec-2018   word2vec (slides) (notebook) Divyansh Gupta Amelie [1] [2] [3] [4]
Wed 12-Dec-2018   Hands-on session
2018/19 colab document

Document from 2017/18 for reference:
Using Tensorboard (ipynb) (html)
Variable scopes (ipynb)
     
Mon 17-Dec-2018 Convolutional neural networks 1 Convolutional neural networks (slides) (ConvNet demo) Martin Töpfer Mary Hands-On book chapter 13
Mon 17-Dec-2018   AlexNet (slides) Vyacheslav Li Nikola [1]
Wed 19-Dec-2018   hands-on session      
  holiday break        
Mon 7-Jan-2018   Neural networks tricks-of-the-trade (slides) Joaquin Padilla Montani Nikola Hands-On book chapter 11
Mon 7-Jan-2018 Convolutional neural networks 2 Modern ConvNet architectures Catalin Rusnac Nikola [1] [2]
Mon 14-Jan-2019 Deep generative models Variational Autoencoders Alexandra Peste Mary Hands-On book chapter 15, [1] [2] [3]
Mon 14-Jan-2019 (tenative) Deep reinforcement learning Deep Q-Learning Matthias Lechner Amelie Hands-On book chapter 16, [1] [2 (the DQN part)]
Wed 16-Jan-2019   hands-on session      
Mon 21-Jan-2019 hands-on session (if required)
Wed 23-Jan-2019   hands-on session      

Homework

File Due Date
HW1: Logistic Regression (Git page) PDF Dec 10th 2018
HW2: Multilayer Perceptron (Git page) PDF Dec 17th 2018
Results: padillamontani 97.6, Lars_Bollmann 97.3, Lechner 97.3, Vyacheslav_Li 95.9.
-- outside of competition (wrong filename): alexandra_peste 97.7
 
HW3: Convolutional Neural Networks (Git page) PDF Jan 7th 2019
Results: Lechner 99.2, Lars_Bollmann 99.1, padillamontani 99.1, Vyacheslav_Li 99.1, Divyansh 99.1, alexandra_peste 98.9, Martin_Topfer 98.9 -- outside of competition (after deadline): Catalin 99.0  
HW4: Modern ConvNets (Git page) PDF Jan 14th 2019
Results: padillamontani 65.2, Catalin 45.9, alexandra_peste 45.0, Vyacheslav Li 6.2, Lars_Bollmann 2.9, -- outside of competition (wrong file size): Lechner -- outside of competition (after deadline): Martin_Topfer 43.2  
HW5: Image Generation (Git page) PDF Jan 21st 2019
HW6: Reinforcement learning (Git page) PDF Jan 28th 2019 (later on request)