[jira] [Created] (AIRFLOW-7000) Allow passing in env var JSON dict in task_test
Kevin Yang created AIRFLOW-7000:
---
Summary: Allow passing in env var JSON dict in task_test
Key: AIRFLOW-7000
URL: https://issues.apache.org/jira/browse/AIRFLOW-7000
Project: Apache Airflow
Issue Type: Improvement
Components: cli
Affects Versions: 1.10.9
Reporter: Kevin Yang
Assignee: Kevin Yang
Add a convinient way to batch add env vars, e.g. {'test_mode': True,
"write_to_production": True}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AIRFLOW-5897) Allow setting -1 as pool slots value in webserver
Kevin Yang created AIRFLOW-5897: --- Summary: Allow setting -1 as pool slots value in webserver Key: AIRFLOW-5897 URL: https://issues.apache.org/jira/browse/AIRFLOW-5897 Project: Apache Airflow Issue Type: Improvement Components: webserver Affects Versions: 2.0.0 Reporter: Kevin Yang Assignee: Kevin Yang Given https://github.com/apache/airflow/pull/6520, we should also allow -1 as pool slots value in the webserver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-5417) Webserver won't start with long DagBag parsing
Kevin Yang created AIRFLOW-5417: --- Summary: Webserver won't start with long DagBag parsing Key: AIRFLOW-5417 URL: https://issues.apache.org/jira/browse/AIRFLOW-5417 Project: Apache Airflow Issue Type: Improvement Components: webserver Affects Versions: 1.10.5 Reporter: Kevin Yang Assignee: Kevin Yang When DB disconnect during DagBag creation, webserver will fail to start. This becomes a big issue when DagBag is large enough that creating it takes more time than DB server side connection expire limit. As it will fail webserver start up determinisitically. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (AIRFLOW-5362) Imports are not ordered properly
Kevin Yang created AIRFLOW-5362: --- Summary: Imports are not ordered properly Key: AIRFLOW-5362 URL: https://issues.apache.org/jira/browse/AIRFLOW-5362 Project: Apache Airflow Issue Type: Improvement Components: PR tool Affects Versions: 1.10.4 Reporter: Kevin Yang Assignee: Kevin Yang Imports in the projects are not ordered properly and there's no unified way to sort them. This causes big headache resolving conflicts and the lookings of the code base :D -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (AIRFLOW-4956) LocalTaskJob heartbeat can create spamming log
Kevin Yang created AIRFLOW-4956: --- Summary: LocalTaskJob heartbeat can create spamming log Key: AIRFLOW-4956 URL: https://issues.apache.org/jira/browse/AIRFLOW-4956 Project: Apache Airflow Issue Type: Improvement Components: logging, worker Affects Versions: 1.10.3 Reporter: Kevin Yang Assignee: Kevin Yang When DB is unavailable and heartbeat() fails, LocalTaskJob will keep trying to heartbeat--causing DB load and spamming log. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (AIRFLOW-4870) Gantt chart can have wrong order
Kevin Yang created AIRFLOW-4870: --- Summary: Gantt chart can have wrong order Key: AIRFLOW-4870 URL: https://issues.apache.org/jira/browse/AIRFLOW-4870 Project: Apache Airflow Issue Type: Improvement Components: webserver Affects Versions: 1.10.3 Reporter: Kevin Yang Order in [ti_fails|https://github.com/apache/airflow/blob/65eef1ca3d848ecf02eb79c41a72d736e40a9108/airflow/www/views.py#L1763] is [not guaranteed|https://dba.stackexchange.com/questions/5774/why-is-ssms-inserting-new-rows-at-the-top-of-a-table-not-the-bottom/5775#5775https://dba.stackexchange.com/questions/5774/why-is-ssms-inserting-new-rows-at-the-top-of-a-table-not-the-bottom/5775#5775]. Resulting in the Gantt Chart, we can have a try with later start date appear on the left side of a try with earlier start date -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4419) Concurrency calculation in scheduler is slow and has a bug
Kevin Yang created AIRFLOW-4419: --- Summary: Concurrency calculation in scheduler is slow and has a bug Key: AIRFLOW-4419 URL: https://issues.apache.org/jira/browse/AIRFLOW-4419 Project: Apache Airflow Issue Type: Improvement Reporter: Kevin Yang Assignee: Kevin Yang The pool, dag and task concurrency calculation logic is slow and can be optimized. Also current task concurrency should not increase if the executor already has the task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-4285) Current TI DEPS are confusing and not up to date
[ https://issues.apache.org/jira/browse/AIRFLOW-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Yang reassigned AIRFLOW-4285: --- Assignee: Kevin Yang > Current TI DEPS are confusing and not up to date > > > Key: AIRFLOW-4285 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4285 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Minor > > The name of the TI DEPS are not representative, not used efficiently and not > up to date. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4285) Current TI DEPS are confusing and not up to date
Kevin Yang created AIRFLOW-4285: --- Summary: Current TI DEPS are confusing and not up to date Key: AIRFLOW-4285 URL: https://issues.apache.org/jira/browse/AIRFLOW-4285 Project: Apache Airflow Issue Type: Improvement Reporter: Kevin Yang The name of the TI DEPS are not representative, not used efficiently and not up to date. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2061) Remove expensive redundant async.state calls in celery executor
[ https://issues.apache.org/jira/browse/AIRFLOW-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789987#comment-16789987 ] Kevin Yang commented on AIRFLOW-2061: - Sry I meant to link this PR https://github.com/apache/airflow/pull/3830 But Jack is correct that anyway we no longer have that redundant call > Remove expensive redundant async.state calls in celery executor > --- > > Key: AIRFLOW-2061 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2061 > Project: Apache Airflow > Issue Type: Improvement > Components: executor >Reporter: Dan Davydov >Assignee: Dan Davydov >Priority: Major > > The Celery Executor makes the expense calls async.state multiple times, when > it could reuse the initial async.state call. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2061) Remove expensive redundant async.state calls in celery executor
[ https://issues.apache.org/jira/browse/AIRFLOW-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789259#comment-16789259 ] Kevin Yang commented on AIRFLOW-2061: - Think this is a dup ticket dealing with the same problem this PR is solving. Should be safe to resolve. > Remove expensive redundant async.state calls in celery executor > --- > > Key: AIRFLOW-2061 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2061 > Project: Apache Airflow > Issue Type: Improvement > Components: executor >Reporter: Dan Davydov >Assignee: Dan Davydov >Priority: Major > > The Celery Executor makes the expense calls async.state multiple times, when > it could reuse the initial async.state call. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3393) Fix bug in usage of reload_module
[ https://issues.apache.org/jira/browse/AIRFLOW-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770773#comment-16770773 ] Kevin Yang commented on AIRFLOW-3393: - Think it was a duplicate of[ this ticket|https://issues.apache.org/jira/browse/AIRFLOW-3414], closing. > Fix bug in usage of reload_module > - > > Key: AIRFLOW-3393 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3393 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Major > > The[ reload_module > usage|https://github.com/apache/incubator-airflow/blob/master/airflow/utils/dag_processing.py#L479] > is wrong. Need to remove the last section in the package string. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (AIRFLOW-3393) Fix bug in usage of reload_module
[ https://issues.apache.org/jira/browse/AIRFLOW-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Yang closed AIRFLOW-3393. --- Resolution: Duplicate > Fix bug in usage of reload_module > - > > Key: AIRFLOW-3393 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3393 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Major > > The[ reload_module > usage|https://github.com/apache/incubator-airflow/blob/master/airflow/utils/dag_processing.py#L479] > is wrong. Need to remove the last section in the package string. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
