[
https://issues.apache.org/jira/browse/AIRFLOW-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AIRFLOW-3001.
---------------------------------------
Resolution: Fixed
Fix Version/s: 2.0.0
> Accumulative tis slow allocation of new schedule
> ------------------------------------------------
>
> Key: AIRFLOW-3001
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3001
> Project: Apache Airflow
> Issue Type: Improvement
> Components: scheduler
> Affects Versions: 1.10.0
> Reporter: Jason Kim
> Assignee: Jason Kim
> Priority: Major
> Fix For: 2.0.0
>
>
> I have created very long term schedule in short interval. (2~3 years as 10
> min interval)
> So, dag could be bigger and bigger as scheduling goes on.
> Finally, at critical point (I don't know exactly when it is), the allocation
> of new task_instances get slow and then almost stop.
> I found that in this point, many slow query logs had occurred. (I was using
> mysql as meta repository)
> queries like this
> "SELECT * FROM task_instance WHERE dag_id = 'some_dag_id' AND execution_date
> = ''2018-09-01 00:00:00"
> I could resolve this issue by adding new index consists of dag_id and
> execution_date.
> So, I wanted 1.10 branch to be modified to create task_instance table with
> the index.
> Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
