That's a great way of thinking about async structure - and perhaps
surprisingly (since switching from asyncio or trying fledgling
implementations as part of my day job is a no-go ;) immediately useful. A
large part of what I do is wrap existing libraries so they can be used with
asyncio- the idea being that once wrapped correctly it becomes easy to
throw together applications quickly and correctly.  For example 'cassandra'
or the python gRPC libraries. These both offer async-ish style APIs,
smattered with some callback style stuff and a handful of functions that
block but probably shouldn't. There are no standard examples of 'best
practice' in how to do this - the asyncio docs focus on using existing
asyncio components. As a result I end up with stuff that works until I need
to worry about error handling, cancellation, restarting components and then
I cry my heart out in the mailing lists and generally make a mess.

My key takeaway after skimming your blogs is that implementations should
'respect causality'. Aim for await my_implementation() to not spawn any
anonymous tasks that can't be controlled, to only complete when it really
has finished everything it started under the hood, and to correctly respect
cancellation. Limit APIs to coroutines only (ie limit yourself to a 'curio'
style) to make things simpler to reason about. If you must spawn tasks,
keep them in logical groups - eg within a single function (or nursery if
you have such an implementation) and make sure they are all finished before
the function ends.

It seems to me like these are good guiding principles to knock together
robust async/await APIs. At any rate, I'll keep them in mind and see if my
next attempt end up with less subtle problems to worry about.

Thanks - and I look forward to really getting to grips with the detail of
asynchronous design!

On Tue, 26 Feb 2019 at 12:14, Nathaniel Smith <n...@pobox.com> wrote:

> On Mon, Feb 25, 2019 at 4:15 PM Josh Quigley <0zer...@gmail.com> wrote:
> >
> > I've realised the error of my ways: because Task separates the
> scheduling from the response handling, you cannot know if an exception is
> unhandled until the task is deleted. So in my example the reference means
> the task is not deleted, so the exception is not yet unhandled.
> >
> > This is in contrast to APIs like call_soon(callable, success_callback,
> error_callback) where there the possibility of delayed error handling is
> not present. In that case the loop can reliably crash if either callback
> raises an exception.
> >
> > So, the 'solution' to this use-case is to always attach error handers to
> Tasks. A catch-all solution cannot catch every error case.
>
> That's right. There are other ways to structure async code to avoid
> running into these cases, that are implemented in Trio, and there are
> discussions happening (slowly) about adding them into asyncio as well.
> See:
>
>
> https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
>
> Also, I could swear I saw some library that tried to implement
> nurseries on asyncio, but I can't find it now... :-/ maybe someone
> else here knows?
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
_______________________________________________
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Reply via email to