Derivatives and Gradients from Scratch
Understand rates of change, derivatives and gradients as the compass that guides learning
A taste of a lesson
For f(w1, w2) = w1^2 + 4*w1*w2, what is the gradient at (1, 2)?
Take each partial derivative, holding the other variable fixed. With respect to w1: w1^2 gives 2*w1, and 4*w1*w2 gives 4*w2, so 2*w1 + 4*w2. With respect to w2: w1^2 is constant, so 0, and 4*w1*w2 gives 4*w1. At (1, 2): first entry 2 + 8 = 10, second entry 4. The gradient is (10, 4), pointing steeply uphill mostly along w1. Check numerically: compute f(1.01, 2) minus f(1, 2), divided by 0.01. Do you get about 10?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Estimate derivatives numerically with small steps
- Apply the power, sum and constant rules with understanding
- Differentiate exponential and logarithmic functions
- Compute partial derivatives and assemble a gradient vector
- Explain why the negative gradient points towards lower loss
Lesson plan
- 1 Slopes and rates of change Understand a derivative as an instantaneous slope. Start
- 2 The basic rules Use the power, constant and sum rules and see why they work. Start
- 3 Exponentials and logarithms Differentiate e^x and ln(x) and spot where they appear in ML. Start
- 4 Reading the derivative Use sign and curvature to describe a function's shape. Start
- 5 Partial derivatives Differentiate functions of several variables one at a time. Start
- 6 The gradient Build the gradient vector and connect it to learning. Start
Try asking
About this tutor
A beginner tutor that teaches the calculus machine learning needs, starting from the idea of a slope. You will estimate rates of change from small steps, learn the basic derivative rules by seeing why they work, and compute derivatives of common functions used in models. Then you move to functions of several variables, partial derivatives and the gradient vector, which points uphill and tells a model how to change its parameters. Every idea is checked numerically with a calculator, and no prior calculus is assumed beyond comfort with simple algebra.
Reviews
4.7
3 ratingsSample
- Ines B.Sample
Checking every rule with a calculator made calculus feel trustworthy rather than magic. Gradients finally make sense.
- Hanna L.Sample
I failed calculus at school. This tutor showed every step and I never felt lost.
- Kwabena O.Sample
Patient and clear. The partial derivatives lesson was great. I would have liked a few more exercises with exponentials.
About the teacher
Calculus and optimisation for machine learning, done by hand
9 tutors 319 lessons taught Sample
I teach the calculus and optimisation that make models learn: derivatives, gradients, the chain rule, gradient descent and the loss functions it minimises. My background is in engineering and numerical computing, so I care about why things work and also about when they break, such as unstable learning rates or overflowing exponentials. I teach with pencil calculations first, then a...
See Leandro's profile and tutorsMore like this
Other tutors on the same or nearby topics.