[
https://issues.apache.org/jira/browse/AIRFLOW-52?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-52:
-------------------------------
Description:
Release the airflow 1.7.1 tag.
Current status:
There are two issues blocking this release caused by this commit:
https://github.com/apache/incubator-airflow/commit/fb0c5775cda4f84c07d8d5c0e6277fc387c172e6
1. DAGs with a lot of tasks take much longer to parse (~25x slowdown)
2. The following kind of patterns fail:
{code}
email.set_upstream(dag.roots)
dag.add_task(email)
{code}
This is because set_upstream now calls add_task and a task can't be added more
than once.
I'm working with the owner of the commit to resolve these issues.
The way to catch (1) in the future is an integration test that asserts a given
non-trivial DAG parses under X seconds.
was:
Release the airflow 1.7.1 tag.
Current status:
I found the commit that caused the slowdown in 1.7.1rc3 yesterday (some DAGs
were timing out when they got parsed via collect_dags).
Working with the owner of the commit to decide whether it's better to roll
forward with a fix or revert the commit for now.
The way to catch these issues in the future is an integration test that asserts
a given non-trivial DAG parses under X seconds.
> Release airflow 1.7.1
> ---------------------
>
> Key: AIRFLOW-52
> URL: https://issues.apache.org/jira/browse/AIRFLOW-52
> Project: Apache Airflow
> Issue Type: Task
> Components: ease
> Reporter: Dan Davydov
> Assignee: Dan Davydov
> Labels: release
>
> Release the airflow 1.7.1 tag.
> Current status:
> There are two issues blocking this release caused by this commit:
> https://github.com/apache/incubator-airflow/commit/fb0c5775cda4f84c07d8d5c0e6277fc387c172e6
> 1. DAGs with a lot of tasks take much longer to parse (~25x slowdown)
> 2. The following kind of patterns fail:
> {code}
> email.set_upstream(dag.roots)
> dag.add_task(email)
> {code}
> This is because set_upstream now calls add_task and a task can't be added
> more than once.
> I'm working with the owner of the commit to resolve these issues.
> The way to catch (1) in the future is an integration test that asserts a
> given non-trivial DAG parses under X seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
