Derivatives are an important concept for understanding deep learning.
At a high level, the derivative of a function at a point is the “rate of change” of the output of the function with respect to its input at that point.
MATH
We can describe the number—change in the output of a function f as we change its input at a particular value ‘a’ of the input—as a limit:
The limit can be approximated by setting a very small value for Δ, such as 0.001. Now, we can compute the derivative as:
This is only one part of a full mental model of derivatives.
GEOMETRY
We draw a tangent to the graphical representation of the function f; the derivative of f at a point ‘a’ is the slope of this line at a.
We can either use calculus to calculate the slope of the line at that point or take the slope of the line connecting f at a — 0.001 and a + 0.001.
DERIVATIVES FOR HACKERS
This is an approximation to the derivative and not an exhaustive piece of code.
from typing import Callable
def deriv(func: Callable[[ndarray], ndarray], input_: ndarray, delta: float = 0.001) -> ndarray:
'''
Evaluates the derivative of a function "func" at every element in the "input_" array.
'''
return (func(input_ + delta) - func(input_ - delta)) / (2 * delta)Let’s connect and build a project together: 🐈⬛
