mik-laj commented on pull request #8718: URL: https://github.com/apache/airflow/pull/8718#issuecomment-624179448
Can you check if DAG cleaning works if it contains ExternalTaskMarker? In my opinion, this can still cause problems in some cases. Here is a potential path that can cause problems. 1. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/www/views.py#L1138 or https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/www/views.py#L1156 2. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/www/views.py#L1138 3. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/www/views.py#L1072 4. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/models/dag.py#L956 5. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/models/dag.py#L1055 6. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/models/dag.py#L1083-L1084 Can you try running this DAG with serialization enabled and disabled? https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/example_dags/example_external_task_marker_dag.py If you will test on different versions of Airflow, can you also check how long the whole DAG Run will be done? I suspect there will be a big difference between Airflow 1.10.10 and older versions, but I haven't checked it yet. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
