Embeddings: meaning as numbers
See how words and documents become vectors, and why similar meanings end up close together
A taste of a lesson
How can a search find my document when it uses different words from my query?
Because embedding search compares meaning, roughly, rather than exact words. An embedding model turns your query and every document chunk into vectors. It was trained so that texts about the same thing land close together, even with different wording: 'reset my password' and 'I can't log in' end up near each other. The search returns chunks whose vectors point in the most similar direction, measured by cosine similarity. Quick prediction: which is closer to 'cheap flights', 'low cost airfare' or 'cheap fights'?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Explain what an embedding is and how it is learned from context
- Compute cosine similarity for small vectors and interpret the result
- Distinguish static word embeddings, contextual embeddings and sentence embeddings
- Describe practical uses and known limits, including bias and negation
Lesson plan
- 1 Words on a map Picture embeddings as positions where similar items sit near each other. Start
- 2 Learning from the company words keep Understand how training on context produces useful vectors. Start
- 3 Measuring similarity Compute and interpret cosine similarity. Start
- 4 Sentence and document embeddings See how embedding models for search are trained and used. Start
- 5 Analogies, bias and other limits Judge what embeddings do and do not capture. Start
Try asking
About this tutor
For learners who keep hearing about embeddings, vector search and semantic similarity and want to understand them properly. You start with a two dimensional picture of words on a map, then scale the idea up to hundreds or thousands of dimensions. You learn how embeddings are learned from context, how similarity is measured with the angle between vectors, and why the famous word arithmetic examples are real but cherry picked. You see the difference between the token embeddings inside a language model and the sentence or document embeddings used for search, clustering and recommendations, and you learn their limits, including bias and confusion over negation.
Reviews
4.7
3 ratingsSample
- Soren H.Sample
Honest about the king and queen example being cherry picked. The negation limit explained a search bug we had.
- Aaliyah P.Sample
Clear progression from 2D maps to sentence embeddings. I would have liked one more exercise on contrastive training.
- Viktor L.Sample
Working the cosine example by hand made it concrete. The point that vectors from different models are not comparable saved me from a real mistake at work.
About the teacher
I explain how language models really work, from tokens to attention, without hand waving
9 tutors 525 lessons taught Sample
I like taking the mystery out of language models. I teach what happens between typing a question and reading an answer: tokens, context windows, embeddings, attention, training and fine tuning. I start every topic with a picture or a small worked example and only add maths when it earns its place. My work background is in software and teaching workshops...
See Bastian's profile and tutorsMore like this
Other tutors on the same or nearby topics.