Announcements
First Class
October 5, 2020, 3:30 PM.
Guidelines for Monthly Projects
Contains files to help with the format for your project submissions.
Monthly Project 2 Final Submissions
Upload to the shared folder. Follow the same formats for report and presentations as Monthly Project 1. Presentation dates: December 29, 30 (and 31).
Monthly Project 3
Upload the abstract by Jan 10, 2021. Presentations in the 1st week of Feb. (TBA).
Final Exam
January 23, 2021, afternoon. Same format as the Mid-Term.
Logistics
Teaching Assistants
Jaswanth Reddy K, Prachi Singh, Akshara Sonam
{jaswanthk, prachisingh, aksharas} aT iisc doT ac doT in
Lab: C 328 (2nd Floor)
Syllabus
- •Visual and Time Series Modeling - Semantic models, recurrent neural models and LSTM models, encoder-decoder models, attention models.
- •Representation Learning, Causality and Explainability - t-SNE visualization, hierarchical representation, semantic embeddings, gradient and perturbation analysis, topics in explainable learning, structural causal models.
- •Unsupervised Learning - Restricted Boltzmann machines, variational autoencoders, generative adversarial networks.
- •New Architectures - Capsule networks, end-to-end models, transformer networks.
- •Applications in NLP, speech, image/video domains across all modules.
Grading Details
60%
3 Monthly Research Projects
10%
Midterm Exam
30%
Final Exam
Monthly research projects span three different domains (Speech/Audio, Text, Images/Videos, Biomedical, Financial, Chemical/Physical/Mathematical Sciences).
Pre-requisites
Linear Algebra
Random Process
Basic Machine Learning/Pattern Recognition course
Good background in Python programming
References
Research Papers and Tutorials
A significant portion of the material comes from research papers and tutorials in the domain.
Lecture Notes
Provided in PDF format.
Slides
Date
Topic
Links
07-10-2020
Recap of deep learning - notations, model parameters, feed forward networks, learning rule with stochastic gradient descent, convolutional networks. Need for recurrence networks. Types of recurrence.
12-10-2020
Recurrent neural networks: forward and backward pass. Gradient propagation. Backpropagation through time (BPTT) algorithm. Vanishing gradients in recurrent networks.
19-10-2020
Recap of RNNs, LSTMs, bidirectional RNNs, encoder-decoder models, attention networks.
21-10-2020
Recap of encoder-decoder models, visualizing attention, multi-head attention, self-attention, transformers.
28-10-2020
Self and multi-head attention, issues in RNN/LSTM, introduction to transformer networks, transformer-encoder in detail.
02-11-2020
Transformer models in detail - encoder, self-attention and positional encoding.
04-11-2020
Transformer models in detail - encoder, self-attention and positional encoding (continued).
09-11-2020
t-SNE.
Reading: Visualizing Data using t-SNE
11-11-2020
Unsupervised representation learning, Boltzmann machine and restricted Boltzmann machine. Model parameters, conditional independence. Issues in RBM training.
Reading: Deep Generative Models
18-11-2020
Restricted Boltzmann machine training, approximating the negative phase with Gibbs sampling. Gaussian Bernoulli RBM - definition and properties. Deep belief networks (DBNs).
23-11-2020
Restricted Boltzmann machine training, deep belief networks (DBNs) for initialization and visualization, data generation using RBMs, variational autoencoders.
25-11-2020
Variational autoencoders (derivations of the loss functions). The variational lower bound. Model assumptions and approximations.
Reading: Auto-Encoding Variational Bayes
02-12-2020
Variational autoencoders examples. Generating data using VAEs. Introduction to generative adversarial networks. GANs - loss function.
07-12-2020
Introduction to generative adversarial networks, GANs - loss function, min-max game, deep convolutional GANs, conditional GANs, CycleGANs.
09-12-2020
Explainable deep learning - motivation, understanding hierarchical representations in deep learning. Transfer learning and representations.
14-12-2020
Explainable deep learning - t-SNE embeddings for visualization, understanding deep networks, representations.
16-12-2020
Explainable deep learning - architecture updates for interpretability, improving CAM without compromising architecture, relation between CAM and Grad-CAM, using attention mechanism for explainability.
21-12-2020
Causality, causal modeling, structural causal equations, causal inference and deep learning, pruning based analysis of neural networks, adversarial examples.
23-12-2020
Causal inference, pruning based approach to analyzing/compressing, criterion involved in identifying importance, approximating gradients, adversarial examples and learning, explainability with distillation, LIME model.
28-12-2020
Knowledge distillation, knowledge distillation for explainability, local interpretable model agnostic representation, LIME model, future research directions, capsule networks.
30-12-2020
Future research directions, problem with current deep learning networks, capsule networks, capsule vs neurons, routing algorithm, understanding the capsule output.
04-01-2021
Capsule networks, from a layer of neurons to layer of capsules, capsule network performance, automatic sign language detection task, comparing capsule networks with other architectures, deep learning on graphs.
06-01-2021
Deep learning on graphs, graph convolutional networks, semi-supervised learning using GCN.
13-01-2021
Modeling uncertainty in deep learning, Bayesian deep learning (basics), introduction to Gaussian processes, allowing for noise in the model, dropout and its Bayesian interpretation.