Yep On Sat, Feb 12, 2022 at 3:12 PM Eugen Kosteev <[email protected]> wrote:
> list(dag.tasks) >> watcher() > > may be more accurate to not rely on the implementation details of tasks > property (it can be generator in the future). > > Wdyt? > > On Thu, 10 Feb 2022 at 23:00, Jarek Potiuk <[email protected]> wrote: > >> One more thing, callbacks (daniel) - also bad thing is that we really >> need it to set the "dag status" not run anything. The "watcher" is mainly >> there to propagate the "failure" status to Dag to get the Dag "fail" when >> any of the tasks fail - if we have callbacks for all tasks, The group >> approach also asks for more "noise" to the example, The nice thing about >> adding watcher is that it's just "completely separate" part of the DAGand >> we can easily mark it as "boilerplate" without raising too many questions >> from the users watching the examples. >> >> Dennis -> I think the cases you described are already there in >> "all_done", and other triggering rules. I guess what I only aimed at here >> is how to set the dependencies to "all other" tasks (no matter what the >> triggering rule is) and the proposal by Ash seems to "just work". >> >> J. >> >> >> On Thu, Feb 10, 2022 at 9:48 PM Jarek Potiuk <[email protected]> wrote: >> >>> @dstandish.- Yeah - callback is a bit too "invasive". I thought about it >>> but at least some examples use default_args and adding "callback" there >>> would mix the concerns and make it difficult to extract parts of the >>> examples in our documentation. >>> >>> @ash - yeah. I realized that when you posted it that the >> will work in >>> this case only because of the sequence of processing of the operator :). I >>> have not thought about it before, but when you posted it, it was kinda >>> obvious :). >>> It is a bit "implicit" and probably I would not rely on this behaviour >>> elsewhere, but it actually plays very well with the use-case of system >>> tests. >>> >>> J. >>> >>> >>> On Thu, Feb 10, 2022 at 8:08 PM Daniel Standish >>> <[email protected]> wrote: >>> >>>> The other thing that comes to mind is you can add your "normal" tasks >>>> to a task group and then do `my_group >> watcher` >>>> >>>> Also I noticed that dag can take success / failure callbacks. Maybe >>>> the "watcher" task makes sense as a callback. >>>> >>>> -- > Eugene >
