Classification and Routing for Tickets and Requests
Sort incoming requests into the right queue automatically, with a safe path for anything uncertain
A taste of a lesson
Our classifier mixes up 'billing' and 'account changes' constantly. Should I try a better model?
Check the categories before the model. Requests like 'change the card on my account' honestly belong to both. Look at twenty confused examples and ask whether two colleagues would agree on them. If they would not, rewrite the definitions with a clear rule, for example 'anything involving payment methods, invoices or charges is billing; profile details, users and permissions are account changes', and add a borderline example to each. Then re-measure. Could you pull twenty of the mixed up requests so we can sort them together?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Design a category list with definitions, borderline examples and an unclear option
- Check human agreement before measuring model accuracy
- Write classification instructions and route by rules after labelling
- Measure per category accuracy and confusion, weighted by error cost
- Monitor for drift as new request types appear
Lesson plan
- 1 Categories people agree on Build a category list with clear definitions and borderline examples. Start
- 2 Human agreement as the ceiling Check whether people label consistently before involving a model. Start
- 3 Classification instructions Write prompts that return one valid label with a short reason. Start
- 4 Priority and routing rules Combine rules and model output to set priority and destination. Start
- 5 Handling uncertainty Send unclear and risky cases to people without slowing everything down. Start
- 6 Measuring and watching for drift Evaluate accuracy and keep it from slipping over time. Start
Try asking
About this tutor
For support, IT, HR operations and shared service teams who triage a stream of requests by hand. You will learn to design a category list that people agree on, write classification instructions with clear definitions and examples, add priority and sentiment signals carefully, route by rules after classification, and handle uncertainty with a human triage queue. We build a labelled test set from past requests, measure accuracy and confusion between categories, and plan monitoring for drift when new request types appear.
Reviews
4.3
3 ratingsSample
- Samuel I.Sample
Good on routing and drift. I would have liked more on multi label tickets.
- Joaquin L.Sample
The human agreement exercise was humbling. Our team only agreed on about three quarters of tickets. Fixing definitions helped more than any model change.
- Ewa N.Sample
Clear, structured and practical. The point about tone not deciding priority was important for our multilingual customers.
About the teacher
I teach operations teams to build AI automations for documents, requests and data that fail safely
9 tutors 316 lessons taught Sample
I work with operations teams who handle volume: invoices, support tickets, forms, contracts and meeting notes. My background is in finance operations and process improvement, so I think about accuracy, audit trails and what happens at month end when something quietly broke two weeks ago. I teach how to put AI steps inside workflows so that extraction, classification and summaries...
See Hamza's profile and tutorsMore like this
Other tutors on the same or nearby topics.