You'll learn how automatic differentiation packages and related concepts such as computational graphs, reverse-mode differentiation, backpropagation, nodes, variables, and tensors work. You'll then implement your own automatic differentiation library, a fork of Autograd, completely from scratch!

On top of your automatic differentiation library, you'll create a variety of neural network components which will integrate with it. These will include neural network layers, batch normalization, dropout, regularizers, activation functions, and more, all from scratch!

You'll implement an optimization module which will pull together your deep learning library. You'll learn how to code up gradient descent with and without momentum, implement loss functions, and create learning rate schedules. As before, this will all be done from scratch!