Neurons and layers, built up by hand
Compute a small neural network on paper so every layer, weight and shape makes sense
A taste of a lesson
If a layer has 128 neurons and gets 784 inputs, how many weights is that?
Each of the 128 neurons has one weight per input, so that is 784 times 128, which is 100,352 weights. Then each neuron also has one bias, adding 128 more. Total parameters: 100,480. A handy way to remember it: the weight matrix has shape 128 by 784, and the bias vector has length 128. Your turn: the next layer goes from those 128 outputs to 10 outputs. How many parameters does it have, and what is the shape of its weight matrix?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Compute the output of a single neuron and a small dense layer by hand
- Write a dense layer as a matrix operation and state every shape involved
- Count the parameters of a multilayer network correctly
- Explain why nonlinear activations are needed between layers
Lesson plan
- 1 One neuron, computed by hand Calculate a neuron's output from inputs, weights, bias and an activation function. Start
- 2 From neurons to a layer Group several neurons into a layer and compute all outputs together. Start
- 3 Batches and shapes Handle many examples at once and track every tensor shape through a layer. Start
- 4 Stacking layers into a network Build a two layer network and see why the nonlinearity between layers matters. Start
- 5 Counting parameters and reading diagrams Count weights and biases in a network and read a typical architecture diagram. Start
- 6 What one layer can and cannot do Understand the limits of shallow networks and why depth helps in practice. Start
Try asking
About this tutor
For learners who know a little algebra or programming and want to understand what actually happens inside a neural network. You start with a single artificial neuron and calculate its output by hand, then group neurons into a layer, write that layer as a matrix operation, and stack layers into a small network. Along the way you count parameters, track tensor shapes and see why a nonlinearity is required between layers. Lessons are short, with one worked example and one exercise each, and the tutor checks your arithmetic and reasoning rather than just giving answers. By the end, a diagram of a dense network will read like a simple recipe.
Reviews
4.7
3 ratingsSample
- Fatima Z.Sample
Very methodical, maybe a bit slow in the first lesson if you already know matrix multiplication. The shape tracking habit has saved me real bugs since.
- Liam O.Sample
The tutor caught that I kept writing W as inputs by outputs and made me find it myself. Annoying in the moment, very useful later.
- Diego M.Sample
Doing XOR by hand with chosen weights was the moment it clicked. I had watched videos before but never computed anything myself.
About the teacher
I teach how neural networks learn, one small worked example at a time
9 tutors 424 lessons taught Sample
I teach the core mechanics of deep learning: what a neuron computes, how a loss turns mistakes into numbers, and how gradients and optimisers change weights. My background is in building and training models for applied research teams, which mostly meant staring at loss curves that refused to go down. That shaped how I teach. I start every idea with...
See Mira's profile and tutorsMore like this
Other tutors on the same or nearby topics.