Question answering, from extractive to generative
Understand how QA systems find, read and generate answers, and how to tell when they should abstain
A taste of a lesson
My RAG bot answers confidently even when the documents do not contain the answer. How do I fix that?
Treat it as two problems. First, tell the system explicitly that 'the documents do not say' is an acceptable answer, and require a cited passage for every claim; answers with no valid citation can be replaced with an abstention. Second, measure it: build a test set where about a fifth of the questions have no answer in your documents, and track how often the bot abstains correctly versus guesses. Also check retrieval, since weak retrieval makes guessing more tempting. Quick exercise: write two unanswerable questions for your own document set.
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Explain how extractive models predict answer spans and compute EM and F1 by hand
- Design a retriever plus reader pipeline and measure retrieval separately
- Ground generative answers in sources and verify their citations
- Handle unanswerable and multi hop questions with abstention and decomposition
- Avoid common evaluation traps when scoring QA systems
Lesson plan
- 1 Extractive QA and answer spans Understand how a model chooses start and end positions for an answer within a passage. Start
- 2 Exact match and F1 Compute the standard extractive QA metrics and know what they miss. Start
- 3 Open domain QA: retriever plus reader Build a two stage system and see how retrieval limits the whole pipeline. Start
- 4 Generative QA with grounding and citations Generate answers from retrieved passages and check that citations really support them. Start
- 5 Unanswerable and multi hop questions Teach a system when to abstain and how to combine evidence across passages. Start
- 6 Evaluating QA without fooling yourself Choose grading methods that match the answer type and spot contaminated tests. Start
Try asking
About this tutor
For learners who know basic NLP and want to understand question answering systems properly, from classic reading comprehension to retrieval backed generative answers. You will see how extractive models predict an answer span, compute exact match and F1 by hand, and build up the retriever plus reader design used for open domain questions. Then you move to generative QA: grounding answers in retrieved passages, adding citations, handling questions with no answer in the sources, and multi hop questions that need several pieces of evidence. The final lesson is about evaluation traps, such as scoring a correct answer as wrong or a fluent invented answer as right.
Reviews
4.7
3 ratingsSample
- Leah G.Sample
Adding unanswerable questions to our test set exposed how often the bot guessed. Practical and honest course.
- Hamid R.Sample
Clear explanation of span prediction and metrics. The multi hop lesson felt a bit short but the decomposition idea worked on my project.
- Valentina C.Sample
Measuring retrieval recall separately was the eye opener. Our reader was fine, retrieval was missing the right passage half the time.
About the teacher
Generative models and representation learning, explained with intuition first and maths second
9 tutors 335 lessons taught Sample
I teach how models learn useful representations and how they generate new data: autoencoders, GANs, diffusion models, self supervised learning and language model pretraining. I came to this through research engineering work where we had to decide which kind of model was worth the compute, so I teach with trade offs in mind. Each topic starts with a picture or...
See Mateo's profile and tutorsMore like this
Other tutors on the same or nearby topics.