Bharath Palaksha created AIRFLOW-4512:
-----------------------------------------
Summary: New trigger rule for tasks - At least one successful
Key: AIRFLOW-4512
URL: https://issues.apache.org/jira/browse/AIRFLOW-4512
Project: Apache Airflow
Issue Type: New Feature
Components: core, dependencies
Affects Versions: 1.10.3
Reporter: Bharath Palaksha
Assignee: Bharath Palaksha
{{New trigger rule - *atleast_one_success*}}
{{Trigger rules help in defining better dependencies on up stream tasks. There
are good number of rules currently defined which are as below}}
* {{all_success}}: (default) all parents have succeeded
* {{all_failed}}: all parents are in a {{failed}} or {{upstream_failed}} state
* {{all_done}}: all parents are done with their execution
* {{one_failed}}: fires as soon as at least one parent has failed, it does not
wait for all parents to be done
* {{one_success}}: fires as soon as at least one parent succeeds, it does not
wait for all parents to be done
* {{none_failed}}: all parents have not failed ({{failed}} or
{{upstream_failed}}) i.e. all parents have succeeded or been skipped
* {{none_skipped}}: no parent is in a {{skipped}} state, i.e. all parents are
in a {{success}}, {{failed}}, or {{upstream_failed}} state
* {{dummy}}: dependencies are just for show, trigger at will
There can be another rule added here which is *atleast_one_success* - This
waits for all parent tasks to be complete and checks if at least one parent is
successful and triggers current task. It differs from one_success as it does
not wait for all parents to be done.
Consider a very common scenario in data pipelines where you have a number of
parallel tasks generating some data. As a downstream task to all these generate
tasks, you have a task to collate all data into one collection which has to run
if any of the upstream generate is successful and also has to wait for all of
them to be done. one_success can't be used as it doesn't wait for other tasks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
