On Thu, Apr 26, 2018 at 7:55 PM, Dima Tisnek <dim...@gmail.com> wrote: > My 2c after careful reading: > > restarting tasks automatically (custom nursery example) is quite questionable: > * it's unexpected > * it's not generally safe (argument reuse, side effects) > * user's coroutine can be decorated to achieve same effect
It's an example of something that a user could implement. I guess if you go to the trouble of implementing this behavior, then it is no longer unexpected and you can also cope with handling the edge cases :-).There may be some reason why it turns out to be a bad idea specifically in the context of Python, but it's one of the features that's famously helpful for making Erlang work so well, so it seemed worth mentioning. > It's very nice to have the escape hatch of posting tasks to "someone > else's" nursery. > I feel there are more caveats to posting a task to parent's or global > nursery though. > Consider that local tasks typically await on other local tasks. > What happens when N1-task1 waits on N2-task2 and N2-task9 encounters an error? > My guess is N2-task2 is cancelled, which by default cancels N1-task1 too, > right? > That kinda break the abstraction, doesn't it? "Await on a task" is not a verb that Trio has. (We don't even have task objects, except in some low-level plumbing/introspection APIs.) You can do 'await queue.get()' to wait for another task to send you something, but if the other task gets cancelled then the data will just... never arrive. There is some discussion here of moving from a queue.Queue-like model to a model with separate send- and receive-channels: https://github.com/python-trio/trio/issues/497 If we do this (which I suspect we will), then probably the task that gets cancelled was holding the only reference to the send-channel (or even better, did 'with send_channel: ...'), so the channel will get closed, and then the call to get() will raise an error which it can handle or not... But yes, you do need to spend some time thinking about what kind of task tree topology makes sense for your problem. Trio can give you tools but it's not a replacement for thoughtful design :-). > If the escape hatch is available, how about allowing tasks to be moved > between nurseries? That would be possible (and in fact there's one special case internally where we do it!), but I haven't seen a good reason yet to implement it as a standard feature. If someone shows up with use cases then we could talk about it :-). > Is dependency inversion allowed? > (as in given parent N1 and child N1.N2, can N1.N2.t2 await on N1.t1 ?) > If that's the case, I guess it's not a "tree of tasks", as in the > graph is arbitrary, not DAG. See above re: not having "wait on a task" as a verb. > I've seen [proprietary] strict DAG task frameworks. > while they are useful to e.g. perform sub-requests in parallel, > they are not general enough to be useful at large. > Thus I'm assuming trio does not enforce DAG... The task tree itself is in fact a tree, not a DAG. But that tree doesn't control which tasks can talk to each other. It's just used for exception propagation, and for enforcing that all children have to finish before the parent can continue. (Just like how in a regular function call, the caller stops while the callee is running.) Does that help? > Finally, slob programmers like me occasionally want fire-and-forget > tasks, aka daemonic threads. > Some are long-lived, e.g. "battery status poller", others short-lived, > e.g. "tail part of low-latency logging". > Obv., a careful programmer would keep track of those, but we want > things simple :) > Perhaps in line with batteries included principle, trio could include > a standard way to accomplish that? Well, what semantics do you want? If the battery status poller crashes, what should happen? If the "tail part of low-latency logging" command is still running when you go to shut down, do you want to wait a bit for it to finish, or cancel it, or ...? You can certainly implement some helper like: async with open_throwaway_nursery() as throwaway_nursery: # If this crashes, we ignore the problem, maybe log it or something throwaway_nursery.start_soon(some_fn) ... # When we exit the with block, it gets cancelled ... if that's what you want. Before adding anything like this to trio itself though I'd like to see some evidence of how it's being used in real-ish projects. > Thanks again for the great post! > I think you could publish an article on this, it would be good to have > wider discussion, academic, ES6, etc. Thanks for the vote of confidence :-). And, we'll see... -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
