Singla Lab IIT Roorkee

Teaching · Autumn 2025–26

CSL-537

Deep Learning

The completed 2025–26 run: nineteen lectures from the perceptron through CNNs and transformers to variational autoencoders.

विद्या नाम नरस्य रूपमधिकं

Knowledge is the greatest attribute of a person.

Course information

Instructor

Jitin Singla · jsingla@bt.iitr.ac.in

Lectures

Tue · 11:05 AM–12:00 PM
Wed & Fri · 12:05–1:00 PM
GB-105

Tutorial

Wed · 3:00–3:55 PM
GB-004

Office hours

Tue · 4:00–5:00 PM
Room 211, BSBE Department

Discussion

Piazza · Microsoft Teams

Announcements

8 Aug 2025

HW-2 is available now.

30 Jul 2025

Venue changed to Room GB-105.

23 Jul 2025

Venue changed to Room GB-006.

20 Jul 2025

Venue changed to Room APJAKB-302.

15 Jul 2025

Course announcements will be posted here regularly. Email notifications will only be sent if the information is urgent.

Objectives

An introduction to deep learning, building a solid foundation in one of the most transformative areas of machine learning.

  • Establish mathematical foundations. The core principles underpinning deep learning models and algorithms.
  • Explore key architectures. The networks that have become standard in image analysis, language modelling and beyond.
  • Survey recent advances. Latest breakthroughs and trends, setting the stage for research in advanced topics.

Prerequisites

  • A working understanding of linear algebra and probability. Revision material: Linear algebra and Probability.
  • Python programming
  • A foundational course in AI or ML, covering: loss functions (MSE, negative log likelihood, cross entropy); optimization (GD, SGD, constrained convex optimization with Lagrangians); supervised methods (linear and logistic regression, perceptron, decision trees, random forests, SVMs, naive Bayes, k-NN); unsupervised methods (k-means, GMMs, EM, PCA).
  • Chapters 1–5 of Deep Learning by Ian Goodfellow

Course content

  • Introduction and motivation
  • Multi-layered perceptrons, neural networks, backpropagation
  • Loss functions, regularization: L1–L2 norms
  • Optimization: SGD, RMSProp, Adam, Adagrad
  • Dropout, batch and layer normalization
  • Convolutional networks (CNNs), residual networks
  • Recurrent neural networks (RNNs), LSTMs
  • Attention: Bahdanau attention, transformers
  • Word2Vec embeddings
  • Language modelling, BERT
  • GPT-1, GPT-2, GPT-3
  • Other LLMs: LLaMA, Mistral, Gemini, DeepSeek
  • Vision language models: ViT, CLIP

Schedule

#TopicEssential readingHomework
1Course info and outlineChapters 1–5, Deep LearningHW-1
2Deep learning applicationsInspirational applications—
3Basics of the perceptron, feedforward network notationChapter 6—
4XOR network and hidden layer transformationChapter 6—
5Output units and loss functionsChapter 6—
6BackpropagationChapter 6HW-2
7RegularizationChapter 7—
8L2 and L1 regularizationSection 7.1 · additional reading 7.2–7.5—
9Early stoppingSection 7.8HW-3
10DropoutSections 7.10–7.12—
11Optimization basicsSections 8.1–8.2—
12Batch normalizationSection 8.7.1 · BN paper—
13CNN basics and backpropagationSections 9.1–9.2 · AlexNet—
14ResNetsResNet paper—
15Recurrent neural networks (RNNs)——
16GRUs and LSTMsLSTM paper · Bahdanau attention—
17TransformersPaper · Video 1 · Video 2—
18Word2Vec, BERTWord2Vec · BERT—
19Variational autoencoders (VAEs)Paper · Video 1 · Video 2—

Assignments

  • Assignment 1 — due 31 October 2025.
  • Python is the default programming language for the course.
  • Submit via Moodle or GitHub, as specified in each assignment.
  • Honour code. Any case of copying is awarded a zero on the assignment. More severe penalties may follow.
  • Late submissions incur penalties, as announced with the assignment.

References and resources

Text

  • Deep Learning — Ian Goodfellow, Yoshua Bengio and Aaron Courville
  • Chapter 6 — multi-layered perceptrons, backpropagation
  • Chapter 7 — regularization: L1/L2 and other techniques
  • Chapter 8 — optimization techniques, normalization
  • Chapter 9 — deep learning for vision, basic models (CNNs)
  • Chapter 10 — deep learning for NLP, basic models (RNNs, LSTM)

Papers

Evaluation

30% Continuous assessment (CWS)Quizzes and assignments
30% Mid-term exam (MTE)
40% End-term exam (ETE)
All courses