Rather than a BashOperator I'd suggest using DummyOperator.
The other thing you might want to look at is priority_weight attribute of a
task. By default the priority of a task includes the weight of all it's down
stream tasks, but you can configure this with the priority_weight and
weight_rule attributes to the operator constructor - see
http://airflow.apache.org/docs/stable/_api/airflow/operators/index.html?highlight=priority#airflow.operators.BaseOperator
for the possible values and meanings.
So priority_weight=20, weight_rule=absolute for your blue tasks should make the
scheduler run those before the other tasks in your dag.
-Ash
On Jan 30 2020, at 1:46 am, Reed Villanueva <[email protected]> wrote:
> After some more thought, here is my workaround...
>
>
> Anyone see any holes with this?
>
>
> On Wed, Jan 29, 2020 at 3:26 PM Reed Villanueva <[email protected]
> (mailto:[email protected])> wrote:
> > Kamil,
> >
> > One thing about my case is that the next tasks (greens) in each branch
> > should only run if the blue task in that same branch completes successfully
> > (should not care about the success/failure status of the other blue tasks,
> > only that they have been run). Thus I don't think the ALL_DONE trigger rule
> > will help the greens and ALL_SUCCESS would be too strict.
> >
> > Any ideas for such a thing?
> >
> >
> > On Wed, Jan 29, 2020 at 2:14 PM Kamil Breguła <[email protected]
> > (mailto:[email protected])> wrote:
> > > white
> > > blue = [blue_a, blue_b, blue_c]
> > > green = [green_a, green_b, green_c]
> > > yellow = [yellow_a, yellow_b]
> > >
> > > cross_downstream(from_tasks=[white], to_tasks=[blue])
> > > cross_downstream(from_tasks=blue, to_tasks=green)
> > > cross_downstream(from_tasks=green to_tasks=yellow)
> > >
> > > This should create the required network of dependencies between tasks.
> > > Here is visualization available:
> > > https://imgur.com/a/2jqyqQO
> > >
> > > This is the easiest solution and in my opinion the correct one.
> > > However, if you don't want a dependencies then you can create a new
> > > schedule rule by editing the BaseOperator.deps property.
> >
> >
>
>
>
> This electronic message is intended only for the named
> recipient, and may contain information that is confidential or
> privileged. If you are not the intended recipient, you are
> hereby notified that any disclosure, copying, distribution or
> use of the contents of this message is strictly prohibited. If
> you have received this message in error or are not the named
> recipient, please notify us immediately by contacting the
> sender at the electronic mail address noted above, and delete
> and destroy all copies of this message. Thank you.
>
