Designing AI Roleplay and Tutor Scenarios
Build, test and evaluate AI roleplays and practice tutors that help learners without misleading them
A taste of a lesson
My Socratic maths tutor keeps solving the problem after two student messages. How do I fix it?
First, make the rule explicit and give it a reason: never state the next step or final answer; instead ask one question that helps the student find it. Second, add a short example exchange showing a good hint. Third, give it an escape valve, because pressure from students often causes the leak: after three failed attempts, offer a worked example of a different but similar problem. Then test with an adversarial persona who begs for the answer. How many runs will you do before trusting the fix?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Write a character brief, scenario rules and feedback rubric for an AI roleplay
- Identify and reduce failure modes such as sycophancy and answer leaking
- Run structured test conversations with varied learner personas
- Plan a pilot that compares AI feedback against expert judgement
- Set privacy, distress and human handover rules
Lesson plan
- 1 Objective, learner and scope Define the skill, the learner and a narrow scenario that a roleplay can support well. Start
- 2 Character brief and scenario rules Write instructions that make the AI character realistic, consistent and responsive to technique. Start
- 3 Feedback that cites evidence Build a rubric driven feedback step that quotes the learner's words against each criterion. Start
- 4 Guardrails and handover Plan responses to off topic use, distress, attempts to extract answers and requests for real advice. Start
- 5 Adversarial testing Test the scenario with varied personas, repeated runs and a scoring checklist. Start
- 6 Pilot and evaluation Pilot with real learners and compare AI feedback with expert ratings before scaling. Start
Try asking
About this tutor
For instructional designers, trainers and educators who want to create AI powered practice: customer conversation roleplays, interview simulations, Socratic tutors or feedback bots. We cover the design work behind a good scenario: learning objective, character brief, scenario rules, a rubric for feedback, guardrails, and when to hand over to a human. You learn to write and refine the instructions that steer the AI, run structured test conversations, catch failure modes like giving away answers, flattering weak work or drifting off topic, and plan a small pilot with privacy and evaluation built in. Tool neutral and focused on design.
Reviews
Students can review a tutor after a paid lesson. Nobody has yet.
About the teacher
Lecturer and learning designer for universities, workplace training and coaching practices
9 tutors 410 lessons taught Sample
My background is university teaching followed by learning design work, building courses with subject experts and turning them into something people can actually learn from. I have also run workshops for staff in organisations that wanted their teams to use AI sensibly. I teach lecturers, instructional designers, trainers and independent tutors how to bring AI into course design, practice and...
See Valentin's profile and tutorsMore like this
Other tutors on the same or nearby topics.