[jira] [Resolved] (AIRFLOW-2183) Refactor DruidHook to able to issue arbitrary query to druid broker

2018-03-14 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-2183. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request #3105

[jira] [Closed] (AIRFLOW-2203) Dag import too slow for very large dags

2018-03-14 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong closed AIRFLOW-2203. - Resolution: Fixed Fix Version/s: 2.0.0 > Dag import too slow for very large dags

[6/6] incubator-airflow git commit: [AIRFLOW-2203] Defer cycle detection

2018-03-14 Thread fokko
[AIRFLOW-2203] Defer cycle detection Moved from adding_task to when dag is being bagged. This changes import dag runtime from polynomial to somewhat linear. Closes #3116 from wongwill86:dag_import_speed Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[4/6] incubator-airflow git commit: [AIRFLOW-2203] Cache signature in apply_defaults

2018-03-14 Thread fokko
[AIRFLOW-2203] Cache signature in apply_defaults Cache inspect.signature for the wrapper closure to avoid calling it at every decorated invocation. This is separate sig_cache created per decoration, i.e. each function decorated using apply_defaults will have a different sig_cache. Project:

incubator-airflow git commit: [AIRFLOW-2183] Refactor DruidHook to enable sql

2018-03-14 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master c3730650c -> 7a880a7e9 [AIRFLOW-2183] Refactor DruidHook to enable sql Refactor DruidHook to be able to issue druid sql query to druid broker Closes #3105 from feng-tao/airflow-2183 Project:

[jira] [Commented] (AIRFLOW-2183) Refactor DruidHook to able to issue arbitrary query to druid broker

2018-03-14 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398233#comment-16398233 ] ASF subversion and git services commented on AIRFLOW-2183: -- Commit

[jira] [Assigned] (AIRFLOW-2211) Fix contributed HDFS sensors' pkg name

2018-03-14 Thread Tao Feng (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Feng reassigned AIRFLOW-2211: - Assignee: Tao Feng > Fix contributed HDFS sensors' pkg name >

[2/6] incubator-airflow git commit: [AIRFLOW-2203] Cache static rules (trigger/weight)

2018-03-14 Thread fokko
[AIRFLOW-2203] Cache static rules (trigger/weight) No need to recalculate them everytime just to see if they are valid Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/16ab314c Tree:

[1/6] incubator-airflow git commit: [AIRFLOW-2203] Store task ids as sets not lists

2018-03-14 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master 4cf2fba19 -> c3730650c [AIRFLOW-2203] Store task ids as sets not lists Massively improve performance by using sets to represent a task's upstream and downstream task ids. Project:

[5/6] incubator-airflow git commit: [AIRFLOW-2203] Remove Useless Commands.

2018-03-14 Thread fokko
[AIRFLOW-2203] Remove Useless Commands. self.tasks is a temp list gen from self.task_dict. no reason to append to it Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/6f0a0d2f Tree:

[jira] [Commented] (AIRFLOW-2203) Dag import too slow for very large dags

2018-03-14 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398224#comment-16398224 ] ASF subversion and git services commented on AIRFLOW-2203: -- Commit

[jira] [Commented] (AIRFLOW-2203) Dag import too slow for very large dags

2018-03-14 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398225#comment-16398225 ] ASF subversion and git services commented on AIRFLOW-2203: -- Commit

[jira] [Commented] (AIRFLOW-2203) Dag import too slow for very large dags

2018-03-14 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398223#comment-16398223 ] ASF subversion and git services commented on AIRFLOW-2203: -- Commit

[jira] [Commented] (AIRFLOW-2203) Dag import too slow for very large dags

2018-03-14 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398222#comment-16398222 ] ASF subversion and git services commented on AIRFLOW-2203: -- Commit

[3/6] incubator-airflow git commit: [AIRFLOW-2203] Speed up Operator Resources

2018-03-14 Thread fokko
[AIRFLOW-2203] Speed up Operator Resources Set default values of Resources loaded from the configuration to prevent 4x Config lookups for every task created. Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[jira] [Commented] (AIRFLOW-2203) Dag import too slow for very large dags

2018-03-14 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398221#comment-16398221 ] ASF subversion and git services commented on AIRFLOW-2203: -- Commit

[jira] [Commented] (AIRFLOW-2203) Dag import too slow for very large dags

2018-03-14 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398220#comment-16398220 ] ASF subversion and git services commented on AIRFLOW-2203: -- Commit

[jira] [Commented] (AIRFLOW-1235) Odd behaviour when all gunicorn workers die

2018-03-14 Thread yuanyifang (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398316#comment-16398316 ] yuanyifang commented on AIRFLOW-1235: - I send a get request to the webserver to check if the

[jira] [Commented] (AIRFLOW-1418) Implement google cloud storage checker

2018-03-14 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398287#comment-16398287 ] Fokko Driesprong commented on AIRFLOW-1418: --- This is in there:

[jira] [Resolved] (AIRFLOW-1418) Implement google cloud storage checker

2018-03-14 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-1418. --- Resolution: Fixed Assignee: Kaxil Naik > Implement google cloud storage

[jira] [Resolved] (AIRFLOW-22) Google cloud code needs tests

2018-03-14 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-22?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-22. - Resolution: Fixed A lot of work has been done on the GC stuff > Google cloud code needs

[jira] [Resolved] (AIRFLOW-1501) Google Cloud Storage delete operator

2018-03-14 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-1501. --- Resolution: Fixed > Google Cloud Storage delete operator >

[jira] [Commented] (AIRFLOW-1501) Google Cloud Storage delete operator

2018-03-14 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398288#comment-16398288 ] Fokko Driesprong commented on AIRFLOW-1501: --- This is in there:

[jira] [Commented] (AIRFLOW-2137) Google Cloud Storage to BigQuery Operator to allow wildcards

2018-03-14 Thread Yannick Einsweiler (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398904#comment-16398904 ] Yannick Einsweiler commented on AIRFLOW-2137: - Silly me. It's just a question of passing a

[jira] [Commented] (AIRFLOW-2137) Google Cloud Storage to BigQuery Operator to allow wildcards

2018-03-14 Thread Yannick Einsweiler (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398809#comment-16398809 ] Yannick Einsweiler commented on AIRFLOW-2137: - Any target release or code fix for this we

[jira] [Created] (AIRFLOW-2216) Cannot specify a profile for AWS Hook to load with s3 config file

2018-03-14 Thread Lorena Mesa (JIRA)
Lorena Mesa created AIRFLOW-2216: Summary: Cannot specify a profile for AWS Hook to load with s3 config file Key: AIRFLOW-2216 URL: https://issues.apache.org/jira/browse/AIRFLOW-2216 Project: Apache