Lecture 32: ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule
Description
Professor Strang begins the lecture talking about ImageNet, a large visual database used in visual object recognition software research. ImageNet is an example of a convolutional neural network (CNN). The rest of the lecture focuses on convolution.
Summary
Convolution matrices have \(\leq\) \(n\) parameters (not \(n\)2).
Fewer weights to compute in deep learning
Component \(k\) from convolution \(c*d\): Add all \(c(j)d(k-j)\)
Convolution Rule: \(F(c*d) = Fc\) times \(Fd\) (component by component)
\(F\) = Fourier matrix with \(j\), \(k\) entry \(= \exp (2 \pi i j k /n)\)
Related section in textbook: IV.2
Instructor: Prof. Gilbert Strang
Problems for Lecture 32
From textbook Section IV.2
4. Any two circulant matrices of the same size commute: \(CD=DC\). They have the same eigenvectors \(\boldsymbol{q}_k\) (the columns of the Fourier matrix \(F\)). Show that the eigenvalues \(\lambda_k(CD)\) are equal to \(\lambda_k(C)\) times \(\lambda_k(D)\).
