The GitHub Actions job "Tests" on airflow.git has failed. Run started by GitHub user ephraimbuddy (triggered by ephraimbuddy).
Head commit for run: 28c93200c33832bb711f6a1772ecd26b797a1f97 / Ephraim Anierobi <[email protected]> Fix mapped task not waiting for upstream mapped to complete Mapped task with an upstream mapped task doesn't wait for the mapped upstream task instances to complete before attempting to run. This results in inconsistent behaviour. For example, if you have an upstream task with 3 mapped task, once any of the task instances finishes, the downstream task will try to run and will depend on the state with which the first finished task in the upstream finished. If the state is failed, then the downstream will be upstream failed. Also, if the state is successful, then the downstream trigger rule will evaluate using this success state of one of the upstream tasks. If eventually, other tasks in the upstream are completed with other states other than success, those are not used in the downstream mapped evaluation. To fix this, I had to check if all the upstreams were done and only proceed to evaluate the trigger rule if the upstreams were done. This affected tests in tests/models/test_taskinstance.py because to evaluate most, we have to make the UpstreamStates.done to be really be done in tests so the trigger rules can be evaluated To test the bug, run the below task and notice the inconsistent state of the finished dagrun then run the dag using this commit to see the consistency: ```python with DAG( dag_id="AAAbug", start_date=datetime(2023, 7, 4), schedule="@daily", tags=["setup_teardown"] ) as dag: @task def my_work(val): print(val) @task def my_setup(val): if val == "data2.json": raise ValueError("fail!") elif val == "data3.json": raise AirflowSkipException("skip!") print(f"setup: {val}") return val s = my_setup.expand(val=["data1.json", "data2.json", "data3.json"]) my_work(s) ``` Report URL: https://github.com/apache/airflow/actions/runs/6099150677 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
