One more thing, callbacks (daniel) - also bad thing is that we really need it to set the "dag status" not run anything. The "watcher" is mainly there to propagate the "failure" status to Dag to get the Dag "fail" when any of the tasks fail - if we have callbacks for all tasks, The group approach also asks for more "noise" to the example, The nice thing about adding watcher is that it's just "completely separate" part of the DAGand we can easily mark it as "boilerplate" without raising too many questions from the users watching the examples.
Dennis -> I think the cases you described are already there in "all_done", and other triggering rules. I guess what I only aimed at here is how to set the dependencies to "all other" tasks (no matter what the triggering rule is) and the proposal by Ash seems to "just work". J. On Thu, Feb 10, 2022 at 9:48 PM Jarek Potiuk <[email protected]> wrote: > @dstandish.- Yeah - callback is a bit too "invasive". I thought about it > but at least some examples use default_args and adding "callback" there > would mix the concerns and make it difficult to extract parts of the > examples in our documentation. > > @ash - yeah. I realized that when you posted it that the >> will work in > this case only because of the sequence of processing of the operator :). I > have not thought about it before, but when you posted it, it was kinda > obvious :). > It is a bit "implicit" and probably I would not rely on this behaviour > elsewhere, but it actually plays very well with the use-case of system > tests. > > J. > > > On Thu, Feb 10, 2022 at 8:08 PM Daniel Standish > <[email protected]> wrote: > >> The other thing that comes to mind is you can add your "normal" tasks to >> a task group and then do `my_group >> watcher` >> >> Also I noticed that dag can take success / failure callbacks. Maybe the >> "watcher" task makes sense as a callback. >> >>
