tl  tr
  Home | Tutorials | Articles | Videos | Products | Tools | Search
Interviews | Open Source | Tag Cloud | Follow Us | Bookmark | Contact   
 Modern Python > Async Python > Python asyncio.gather and asyncio.wait

Python asyncio.gather and asyncio.wait

Author: Venkata Sudhakar

asyncio.gather() runs multiple coroutines concurrently and collects all their results. It is the simplest and most common way to run several async tasks at the same time - for example, fetching data from three different APIs in parallel instead of sequentially. All the coroutines start together and gather waits until every one of them is done, then returns all results in a list in the same order they were passed in.

asyncio.wait() gives more fine-grained control. Instead of waiting for all tasks, you can use return_when=asyncio.FIRST_COMPLETED to proceed as soon as any one task finishes, or FIRST_EXCEPTION to stop when a failure occurs. It returns two sets: done (finished tasks) and pending (still running). Use gather() for simple parallel fan-out, use wait() when you need to react to individual task completions.

The below example shows fetching table row counts from three endpoints concurrently with gather(), then demonstrates wait() with FIRST_COMPLETED to process results as they arrive.


It gives the following output,

All done in 1.50s   # Not 3.3s (1.0+1.5+0.8) - they ran in parallel!
  customers: 125000 rows
  orders: 125000 rows
  products: 125000 rows

It gives the following output,

Finished: products (125000 rows)   # 0.8s - fastest
Finished: customers (125000 rows)  # 1.0s
Finished: orders (125000 rows)     # 1.5s - slowest

# Results printed as each finishes, not all at the end

Use gather() for the common case of running N tasks and needing all their results. Use wait() with FIRST_COMPLETED when you want to process results as they come in or cancel remaining tasks after one succeeds. For error handling, gather(return_exceptions=True) captures exceptions as result values instead of raising them, letting you inspect which tasks succeeded and which failed after all have completed.


 
  


  
bl  br