Backpropagation

Backpropagation is the method used to train artificial neural networks. The algorithm computes the network's output error, the difference between what the network produced and the correct answer, and then propagates this error backward through the layers, working out how much each connection contributed so its weight can be adjusted. Mathematically, it is the reverse mode of automatic differentiation: the chain rule applied backward through a sequence of simple operations, which yields every derivative at roughly the same cost as computing the output itself.

Backpropagation was first described…