[
https://issues.apache.org/jira/browse/AIRFLOW-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896066#comment-16896066
]
ASF subversion and git services commented on AIRFLOW-4822:
----------------------------------------------------------
Commit 05c01a97497e992c7d8b05a39a7855343dee1603 in airflow's branch
refs/heads/master from Du Jiangfan
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=05c01a9 ]
[AIRFLOW-4822] Fix bug where parent-dag task instances are wrongly cleared
(#5444)
Full matching required in this case, so the regex should start and end
with "^$". Blurred matching might result in irrelevant task instances be
cleared.
Also in this commit:
* Added independent test dag: `clear_subdag_test_dag`
* Polished related unit test: `test_subdag_clear_parentdag_downstream_clear`
> When clearing task instance of subdag, irrelevant tasks within parent dag
> might be cleared as well.
> ---------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-4822
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4822
> Project: Apache Airflow
> Issue Type: Bug
> Components: DagRun
> Affects Versions: 1.10.3
> Reporter: Du Jiangfan
> Assignee: Du Jiangfan
> Priority: Major
> Fix For: 1.10.4
>
> Attachments: image-2019-06-20-15-20-39-565.png,
> image-2019-06-20-15-20-48-949.png, image-2019-06-20-15-21-08-614.png,
> image-2019-06-20-15-21-28-744.png
>
>
> Dag file:
> {code}
> import datetime
> from airflow.models import DAG
> from airflow.operators.bash_operator import BashOperator
> from airflow.operators.subdag_operator import SubDagOperator
> def create_subdag_opt(main_dag):
> subdag_name = "daily_job"
> subdag = DAG(
> dag_id='.'.join([dag_name, subdag_name]),
> start_date=start_date,
> schedule_interval=None,
> concurrency=2,
> )
> subdag_task = BashOperator(
> bash_command="echo 1",
> task_id="daily_job_subdag_task",
> dag=subdag
> )
> return SubDagOperator(
> task_id=subdag_name,
> subdag=subdag,
> dag=main_dag,
> )
> dag_name = "daily_job_parent"
> start_date = datetime.datetime(2019, 6, 18)
> dag = DAG(
> dag_id=dag_name,
> concurrency=3,
> start_date=start_date,
> schedule_interval="45 18 * * *"
> )
> daily_job_irrelevant = BashOperator(
> bash_command="echo 1",
> task_id="daily_job_irrelevant",
> dag=dag,
> )
> daily_job = create_subdag_opt(main_dag=dag)
> {code}
> Graph view:
> !image-2019-06-20-15-20-39-565.png!
>
> !image-2019-06-20-15-21-08-614.png!
> When clearing dagrun of task: daily_job_subdag_task,
> !image-2019-06-20-15-21-28-744.png!
> irrelevant task daily_job_parent.daily_job_irrelevant is listed in the
> to-clear list, which is not expected.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
