Word and sentence embeddings
Understand how meaning becomes vectors, compute similarity yourself and choose embeddings wisely
A taste of a lesson
My search finds 'cheap flights' when I search 'not cheap flights'. Why?
Most embedding models capture the topic of a text much better than its logic. 'Not cheap flights' and 'cheap flights' share almost all their words and context, so their vectors land very close, and the small word 'not' barely moves the point. This is a known weakness with negation, exact numbers and constraints. Practical fixes: combine embedding search with keyword filters, rerank results with a model that reads query and document together, or handle constraints such as price as structured filters. Quick check: what would happen with 'flights under 200 euros' versus 'flights over 200 euros'?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Explain the difference between one hot vectors and dense embeddings
- Compute cosine similarity between small vectors by hand
- Describe how contextual and sentence embeddings differ from static word vectors
- Recognise bias in embeddings and test models on your own data
Lesson plan
- 1 From one hot to dense vectors See why giving each word its own dimension fails and dense vectors work better. Start
- 2 Learning meaning from context Understand how word vectors are learned from the words around them. Start
- 3 Measuring similarity Compute cosine similarity and interpret scores sensibly. Start
- 4 Contextual and sentence embeddings See how modern models embed words in context and whole sentences for search. Start
- 5 Bias and limits Recognise social bias and other blind spots in embedding spaces. Start
- 6 Choosing and testing an embedding model Pick an embedding model by measuring it on your own queries and documents. Start
Try asking
About this tutor
For beginners who keep hearing about embeddings in search, chatbots and recommendations and want to know what they really are. You start with why one number per word is not enough, then see how word vectors learn meaning from the company words keep, including the famous analogy examples and why they are less tidy than they look. Next come contextual embeddings, where the same word gets different vectors in different sentences, and sentence embeddings used for semantic search. You will compute cosine similarity by hand, learn about bias inside embeddings, and finish by comparing embedding models on your own data rather than trusting leaderboards.
Reviews
4.7
3 ratingsSample
- Mohammed K.Sample
Computing cosine by hand removed the magic in a good way. The lesson on testing models with our own queries saved us from picking by leaderboard.
- Tendai M.Sample
I liked the bias lesson. We found gendered job associations in our candidate search and changed how we rank.
- Julia S.Sample
Clear and gentle. The analogy section was refreshingly honest. A little more on multilingual models would have helped me.
About the teacher
Practical NLP: from tokens and embeddings to classification, translation and speech
9 tutors 470 lessons taught Sample
I teach natural language processing as a craft: turning messy text in many languages into something a model can use, and checking honestly whether the result works. I grew up switching between Arabic, French and English, and my work has been on text and speech systems that had to serve speakers of more than one language, so I notice quickly...
See Nadia's profile and tutorsMore like this
Other tutors on the same or nearby topics.