Logs and Exponentials for ML
Get comfortable with exponents and logarithms, the quiet workhorses of machine learning formulas
A taste of a lesson
Why do models maximise log likelihood instead of just likelihood? Isn't it the same thing?
They have the same best answer, because log always increases with its input, so whatever maximises one maximises the other. The difference is practical. Likelihood multiplies a probability for every example. Multiply 1,000 probabilities of 0.01 and you get 10 to the power -2000, far too small for a computer to store, so it becomes zero. Taking logs turns the product into a sum: 1,000 times log(0.01), about -4,605, which is easy to store. Check: what is log base 10 of 0.01?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Apply exponent rules confidently with whole, negative and fractional powers
- Explain e and model exponential growth and decay
- Use logarithm rules and change of base correctly
- Explain why ML works with log probabilities
- Recognise exponentials and logs inside softmax, sigmoid and log loss
Lesson plan
- 1 Exponents from multiplication Build exponent rules from repeated multiplication. Start
- 2 Growth, decay and e Model repeated multiplication and meet e. Start
- 3 Logarithms as undoing Understand logs as the inverse of exponentials. Start
- 4 Log rules Use the product, quotient and power rules. Start
- 5 Logs in machine learning See why models work with log probabilities. Start
- 6 Exponentials in models and data Recognise softmax, sigmoid and log transforms. Start
Try asking
About this tutor
A free beginner tutor that rebuilds exponents and logarithms and shows why they appear all over machine learning. You will review exponent rules, meet the number e, model growth and decay, and understand logarithms as the inverse of exponentials. Then you will see the practical uses: turning products into sums for log likelihoods, avoiding numbers too small for computers, log scales on charts, log transforms for skewed data, and the exponentials inside softmax and sigmoid. Every lesson includes small calculations and a quick check you can do on any calculator.
Reviews
4.7
3 ratingsSample
- Adwoa F.Sample
Patient with my rusty algebra. I can finally read the log loss formula without panic.
- Marisol G.Sample
Building log rules from multiplication meant I didn't have to memorise anything. Softmax makes sense now.
- Ben K.Sample
Free and clear. The underflow example was a lightbulb moment for why log likelihood exists.
About the teacher
Calculus and optimisation for machine learning, done by hand
9 tutors 319 lessons taught Sample
I teach the calculus and optimisation that make models learn: derivatives, gradients, the chain rule, gradient descent and the loss functions it minimises. My background is in engineering and numerical computing, so I care about why things work and also about when they break, such as unstable learning rates or overflowing exponentials. I teach with pencil calculations first, then a...
See Leandro's profile and tutorsMore like this
Other tutors on the same or nearby topics.