noamelf commented on a change in pull request #4751: [AIRFLOW-3607] collected trigger rule dep check per dag run URL: https://github.com/apache/airflow/pull/4751#discussion_r347107741
########## File path: airflow/ti_deps/deps/trigger_rule_dep.py ########## @@ -34,6 +35,31 @@ class TriggerRuleDep(BaseTIDep): IGNOREABLE = True IS_TASK_DEP = True + @staticmethod + def _get_states_count_upstream_ti(ti, finished_tasks): + """ + :param ti the ti that we want to calculate deps for + :type ti airflow.models.TaskInstance + :param finished_tasks all the finished tasks of the dag_run + :type finished_tasks of finished ti's + """ + successes, skipped, failed, upstream_failed, done = 0, 0, 0, 0, 0 + upstream_tasks = [finished_task for finished_task in finished_tasks + if finished_task.task_id in ti.task.upstream_task_ids] + if upstream_tasks: + upstream_tasks_sorted = sorted(upstream_tasks, key=lambda x: x.state) + for k, g in groupby(upstream_tasks_sorted, key=lambda x: x.state): + if k == State.SUCCESS: + successes = len(list(g)) + elif k == State.SKIPPED: + skipped = len(list(g)) + elif k == State.FAILED: + failed = len(list(g)) + elif k == State.UPSTREAM_FAILED: + upstream_failed = len(list(g)) Review comment: I think that by using a counter you can get the same results with simpler code: ```python Counter(task.state for task in upstream_tasks_sorted) ``` And then organize the results this way: ```python counter[State.SKIPPED], counter[State.UPSTREAM_FAILED] # ... ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services