Hi Justin, Yes, we want to have TaskGroups in asyncio, and the only reason we didn't add them to Python 3.8 is that we also need to support ExceptionGroups (or MultiErrors). Nathaniel and I plan to start working on that.
Here's am implementation of TaskGroups that I created for EdgeDB https://github.com/edgedb/edgedb/blob/master/edb/common/taskgroup.py — you can use it in the meantime, it works just fine. Yury On Tue, Nov 26, 2019 at 3:45 PM, Justin Mayfield < too...@gmail.com > wrote: > > I'm curious if any work has happened towards supporting a grouping concept > like Yury has discussed with TaskGroup? I've been away from python for a > while and I'm finding that the use of gather and wait is trickier than I > remember it. My code tends to be inelegant when done as the docs > recommend (using create_task first, etc). I think I benefit from a class > based context manager primitive like TaskGroup. > > > > Side note, I find I'm missing Javascript's Promise.race() quite a bit too > (as_completed is often an aesthetic mismatch for my code), so it would be > neat if the TaskGroup behaved like a set of outstanding work with methods > like `race()` and `wait()`. > > > Somewhat related, I recently watched Nathaniel's talk on the happy > eyeballs algo ( https:/ / www. youtube. com/ watch?v=oLkfnc_UMcE ( > https://www.youtube.com/watch?v=oLkfnc_UMcE ) ) and I enjoyed the use of > this very easy to understand algo as the test basis for concurrency > designs. However when he described the problem my first instinct was that > he would the write algo in a top-down style where the outer blocks handled > scheduling and the inner tasks simply tried to do work or failed. It > seemed to me the use of events objects, scheduling the next task from > within an existing task (with conditions) and handling cleanup manually > via cancel calls was less than perfect and I dare say not dissimilar to > the reviled GOTO. Could this algo be written in even simpler terms? > Namely, without calls the any sort of cancel, without the tasks themselves > knowing anything about the outside scheduling or work set, and perhaps > most importantly without the use of Event objects which are champions of > the preemptive world IMO. > > > Please permit me to indulge in some hypothetical code I would like to be > able to write someday to perhaps highlight my wishlist for asyncio... > > > async def might_fail_or_be_slow (): > await asyncio. sleep ( random. random () * 10 ) > if random. random () > 0.5 : > raise Exception ( "job failed" ) > return "job worked" > > > async def happy_eyeballs ( max_job_wait = 0.200 ): > async with asyncio. group () as ag : > for i in range ( 10 ): > ag. add_task ( might_fail_or_be_slow ()) > try : > task = await ag. race ( timeout = max_job_wait ) > except asyncio. TimeoutError : > print ( "Nothing is ready yet, add another.." ) > else : > try : > return await task > except Exception : > print ( "Somebody failed, add another.." ) > raise Exception ( "nobody worked" ) > > > > > Some notes. This is obviously just pseudo code and I've not thought this > through very carefully so it's obviously rough and incomplete. For one, > I'm making the assumption here that the TaskGroup will be somewhat queue > like. E.g. if you call `race()` the result is removed from the group; > This might be silly or need to be more explicit, or maybe there could be a > multitude of grouping classes for each use-case TaskQueue, TaskGroup, > TaskRace, etc (spitballing). LIkewise it might be useful for the above > TaskGroup class to have `remove_task(task)` , `wait(timeout=None) -> > [tasks...]`, and perhaps even something like `results(timeout) -> [results > from tasks...]` which is a shortcut method for `results = [await t for t > in await group.wait()]`. > > > Cheers. > > > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "python-tulip" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to python-tulip+unsubscribe@ googlegroups. com ( > python-tulip+unsubscr...@googlegroups.com ). > To view this discussion on the web visit https:/ / groups. google. com/ d/ > msgid/ python-tulip/ 54f410f6-b9b6-4af3-be29-c2e133fb7fe7%40googlegroups. com > ( > https://groups.google.com/d/msgid/python-tulip/54f410f6-b9b6-4af3-be29-c2e133fb7fe7%40googlegroups.com?utm_medium=email&utm_source=footer > ). > -- You received this message because you are subscribed to the Google Groups "python-tulip" group. To unsubscribe from this group and stop receiving emails from it, send an email to python-tulip+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/python-tulip/k3gf1sm6.3a8e58ab-eb46-4304-95b2-69f5e3418733%40we.are.superhuman.com.
