Async Python for Many API Calls
Run hundreds of model or web API calls concurrently with asyncio, limits, retries and clean results.
A taste of a lesson
I switched to async but my 1,000 calls take just as long. What am I missing?
Two usual suspects. First, you may be awaiting each call inside a for loop, which still runs them one after another; build the coroutines first and pass them to asyncio.gather. Second, the HTTP client might be synchronous, so each call blocks the event loop while it waits. Check that your client is async and that nothing like time.sleep sits inside the coroutine. Once it runs concurrently, add a semaphore so you do not hit rate limits. Can you paste the loop that makes the calls?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Explain how coroutines and the event loop overlap network waiting
- Run calls concurrently with gather while capping concurrency with a semaphore
- Add per task timeouts and retries with backoff and jitter
- Write resumable jobs that keep results matched to their inputs
- Choose between async, a thread pool and a batch API for a workload
Lesson plan
- 1 Why waiting is the real cost Measure where time goes in a sequential API loop and see what async can save. Start
- 2 Coroutines and the event loop Write and run coroutines and explain what await actually does. Start
- 3 Concurrency with limits Run many calls at once while keeping parallelism under control. Start
- 4 Timeouts, retries and errors Make each task fail or recover cleanly without killing the batch. Start
- 5 Resumable jobs and result handling Write results as they finish and restart a crashed job without repeating work. Start
- 6 Choosing the right concurrency tool Pick async, a thread pool, processes or a batch API for a given job. Start
Try asking
About this tutor
For developers comfortable with Python who need to process thousands of texts through a model API, or fetch from many endpoints, without waiting for each call in turn. You learn how coroutines and the event loop work, how to run calls concurrently with gather, cap concurrency with a semaphore, add timeouts and retries per task, and keep results matched to their inputs. The tutor covers the traps: blocking libraries that freeze the loop, unbounded concurrency that triggers rate limits, notebooks with their own event loop, and CPU heavy work that async cannot speed up. You compare async with a simple thread pool so you can choose.
Reviews
4.7
3 ratingsSample
- Amara C.Sample
Very clear on the event loop and blocking calls. The honest comparison with thread pools was useful; for my case threads were enough.
- Viktor N.Sample
My job went from hours to minutes once I stopped awaiting inside the loop and added a semaphore. The resumable JSON Lines pattern saved me when the provider had an outage halfway.
- Ravi M.Sample
Explained why asyncio.run broke in Jupyter in one sentence. Retries with jitter and logging by input id are now in all my scripts.
About the teacher
Numerical Python and code quality for data and AI projects that have outgrown a single notebook
9 tutors 374 lessons taught Sample
I work on the part of AI projects nobody photographs: the arrays, dataframes, tests and packaging that let a prototype survive contact with real data. I came to this through scientific computing and later backend work, so I care about two things at once, getting the numbers right and keeping the code readable for the next person. My lessons are...
See Felix's profile and tutorsMore like this
Other tutors on the same or nearby topics.