Automatic differentiation

The chain rule connect different functions for calculus. Our neural network is based on it, but it includes infinity, so adding much data cause some troubles.

 We need to calculate faster. Symbolic differentiation faces the difficulty of converting a computer program into a single mathematical expression, and Numerical differentiation can introduce round-off errors in the discretization process and cancellation. Therefore, optimization process is needed.

●Forward accumulation is the chain rule from inside to outside.

●Reverse accumulation has the traversal from outside to inside.

This is NP-complete.