Re: Anything better than asyncio.as_completed() and asyncio.wait() to manage execution of large amount of tasks?

On Thursday, July 17, 2014 7:09:02 AM UTC+8, Maxime Steisel wrote: > 2014-07-15 14:20 GMT+02:00 Valery Khamenya : > > > Hi, > > > > > > both asyncio.as_completed() and asyncio.wait() work with lists only. No > > > generators are accepted. Are there anything similar to those functions that > >

Re: Anything better than asyncio.as_completed() and asyncio.wait() to manage execution of large amount of tasks?

Hi Maxime, many thanks for your great solution. It would be so great to have it in stock asyncio and use it out-of-the-box... I've made 4 fixes to it that are rather of "cosmetic" nature. Here is the final version: import asyncio from concurrent import futures def as_completed_with_max_workers(

Re: Anything better than asyncio.as_completed() and asyncio.wait() to manage execution of large amount of tasks?

2014-07-15 14:20 GMT+02:00 Valery Khamenya : > Hi, > > both asyncio.as_completed() and asyncio.wait() work with lists only. No > generators are accepted. Are there anything similar to those functions that > pulls Tasks/Futures/coroutines one-by-one and processes them in a limited > task pool? Som

Anything better than asyncio.as_completed() and asyncio.wait() to manage execution of large amount of tasks?

Hi, both asyncio.as_completed() and asyncio.wait() work with lists only. No generators are accepted. Are there anything similar to those functions that pulls Tasks/Futures/coroutines one-by-one and processes them in a limited task pool? I have gazillion of Tasks, and do not want to instantiate th