Natural signals properties and the convolutional net
This lecture covers the concept of natural signals properties and the convolutional nets in practice and is a part of the Deep Learning Course at CDS, a course that covered the latest techniques in deep learning and representation learning, focusing on supervised and unsupervised deep learning, embedding methods, metric learning, convolutional and recurrent nets, with applications to computer vision, natural language understanding, and speech recognition. Prerequisites for this course include: Introduction to Deep Learning and Introduction to Data Science or a Graduate Level Machine Learning.
Chapters:
00:00 – Happy birthday to the TAs!
01:24 – Today topic: convolutional neural nets
2:37 – Input layer, points, and signals 1
9:21 – Natural signal properties
20:48 – 1D stationarity
21:30 – 1D locality
23:39 – 2D stationarity
25:04 – 2D locality
26:56 – 2D compositionality
31:17 – Fully connected recap
33:10 – Locality ⇒ sparsity
39:41 – Stationarity ⇒ parameter sharing
45:13 – 1D kernels
50:56 – 1D padding
53:18 – ConvNet for images and tensor reshaping
56:17 – Pooling
58:21 – Jupyter Notebook: fully connected vs. convnet
1:05:36 – Deterministic pixel shuffling: breaking signal properties
1:06:49 – Final comparison
1:08:35 – Goodbye
