[jira] [Commented] (AIRFLOW-1157) Assigning a task to a pool that doesn't exist crashes the scheduler

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358187#comment-16358187 ] ASF subversion and git services commented on AIRFLOW-1157: -- Commit

[jira] [Resolved] (AIRFLOW-1157) Assigning a task to a pool that doesn't exist crashes the scheduler

2018-02-09 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-1157. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request #3002

[jira] [Created] (AIRFLOW-2088) Duplicate keys in MySQL to GCS Operator

2018-02-09 Thread Kaxil Naik (JIRA)
Kaxil Naik created AIRFLOW-2088: --- Summary: Duplicate keys in MySQL to GCS Operator Key: AIRFLOW-2088 URL: https://issues.apache.org/jira/browse/AIRFLOW-2088 Project: Apache Airflow Issue Type:

[jira] [Created] (AIRFLOW-2089) Add on kill for SparkSubmit in Standalone Cluster

2018-02-09 Thread Milan van der Meer (JIRA)
Milan van der Meer created AIRFLOW-2089: --- Summary: Add on kill for SparkSubmit in Standalone Cluster Key: AIRFLOW-2089 URL: https://issues.apache.org/jira/browse/AIRFLOW-2089 Project: Apache

[jira] [Created] (AIRFLOW-2087) Scheduler Report shows incorrect "Total task number"

2018-02-09 Thread I don't want an account (JIRA)
I don't want an account created AIRFLOW-2087: Summary: Scheduler Report shows incorrect "Total task number" Key: AIRFLOW-2087 URL: https://issues.apache.org/jira/browse/AIRFLOW-2087

incubator-airflow git commit: [AIRFLOW-2066] Add operator to create empty BQ table

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master 4751abf8a -> 2920d0475 [AIRFLOW-2066] Add operator to create empty BQ table - Add operator that creates a new, empty table in the specified BigQuery dataset, optionally with schema. Closes #3006 from kaxil/bq_empty_table_op

[jira] [Commented] (AIRFLOW-713) EmrCreateJobFlowOperator and EmrAddStepsOperatordoes attributes are not jinjafied

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358147#comment-16358147 ] ASF subversion and git services commented on AIRFLOW-713: - Commit

incubator-airflow git commit: [AIRFLOW-1157] Fix missing pools crashing the scheduler

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master 44551e249 -> e6973b159 [AIRFLOW-1157] Fix missing pools crashing the scheduler Throw a warning when a pool associated with a Task Instance doesn't exist instead of crashing the scheduler. Use the default value of 0 slots for

[jira] [Commented] (AIRFLOW-1667) Remote log handlers don't upload logs on task finish

2018-02-09 Thread Josh Bacon (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358749#comment-16358749 ] Josh Bacon commented on AIRFLOW-1667: - +1 We are using CeleryExecutors and notice that our logs 

incubator-airflow git commit: [AIRFLOW-2094] Jinjafied project_id, region & zone in DataProc{*} Operators

2018-02-09 Thread bolke
Repository: incubator-airflow Updated Branches: refs/heads/master bf1296fbd -> 556c9ec5b [AIRFLOW-2094] Jinjafied project_id, region & zone in DataProc{*} Operators - Minor docstring change - Jinjafied project_id, region & zone in DataProc{*} Operators to allow using Airflow variables Closes

[jira] [Created] (AIRFLOW-2094) Jinjafy project_id, region & zone in DataProc{*} Operators

2018-02-09 Thread Kaxil Naik (JIRA)
Kaxil Naik created AIRFLOW-2094: --- Summary: Jinjafy project_id, region & zone in DataProc{*} Operators Key: AIRFLOW-2094 URL: https://issues.apache.org/jira/browse/AIRFLOW-2094 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-1667) Remote log handlers don't upload logs on task finish

2018-02-09 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358757#comment-16358757 ] Ash Berlin-Taylor commented on AIRFLOW-1667: The process that writes to the log files is a

[jira] [Resolved] (AIRFLOW-2094) Jinjafy project_id, region & zone in DataProc{*} Operators

2018-02-09 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved AIRFLOW-2094. - Resolution: Fixed Fix Version/s: 1.10.0 Issue resolved by pull request #3027

[jira] [Commented] (AIRFLOW-2094) Jinjafy project_id, region & zone in DataProc{*} Operators

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358878#comment-16358878 ] ASF subversion and git services commented on AIRFLOW-2094: -- Commit

[jira] [Created] (AIRFLOW-2093) Feature DagBag loading from S3 (and others)

2018-02-09 Thread Bruno Bonagura (JIRA)
Bruno Bonagura created AIRFLOW-2093: --- Summary: Feature DagBag loading from S3 (and others) Key: AIRFLOW-2093 URL: https://issues.apache.org/jira/browse/AIRFLOW-2093 Project: Apache Airflow

[jira] [Created] (AIRFLOW-2096) SparkSubmitOperator should expose yarn_application_id in the templates

2018-02-09 Thread Azhagu Selvan SP (JIRA)
Azhagu Selvan SP created AIRFLOW-2096: - Summary: SparkSubmitOperator should expose yarn_application_id in the templates Key: AIRFLOW-2096 URL: https://issues.apache.org/jira/browse/AIRFLOW-2096

[jira] [Created] (AIRFLOW-2095) Add operator to create External BigQuery Table

2018-02-09 Thread Kaxil Naik (JIRA)
Kaxil Naik created AIRFLOW-2095: --- Summary: Add operator to create External BigQuery Table Key: AIRFLOW-2095 URL: https://issues.apache.org/jira/browse/AIRFLOW-2095 Project: Apache Airflow

[jira] [Created] (AIRFLOW-2097) UnboundLocalError: local variable 'tz' referenced before assignment

2018-02-09 Thread Bryce Drennnan (JIRA)
Bryce Drennnan created AIRFLOW-2097: --- Summary: UnboundLocalError: local variable 'tz' referenced before assignment Key: AIRFLOW-2097 URL: https://issues.apache.org/jira/browse/AIRFLOW-2097 Project:

[jira] [Created] (AIRFLOW-2090) Fix typo in DataStore Hook

2018-02-09 Thread Kaxil Naik (JIRA)
Kaxil Naik created AIRFLOW-2090: --- Summary: Fix typo in DataStore Hook Key: AIRFLOW-2090 URL: https://issues.apache.org/jira/browse/AIRFLOW-2090 Project: Apache Airflow Issue Type: Task

[jira] [Commented] (AIRFLOW-2091) Fix incorrect parameter in BigQueryCursor docstring

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358326#comment-16358326 ] ASF subversion and git services commented on AIRFLOW-2091: -- Commit

incubator-airflow git commit: [AIRFLOW-2088] Fix duplicate keys in MySQL to GCS Helper function

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master 0a71370d0 -> 759d8f83e [AIRFLOW-2088] Fix duplicate keys in MySQL to GCS Helper function - Remove the duplicate key in `MySqlToGoogleCloudStorageOperator` in the `type_map` helper function that maps from MySQL fields to

[jira] [Commented] (AIRFLOW-2088) Duplicate keys in MySQL to GCS Operator

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358329#comment-16358329 ] ASF subversion and git services commented on AIRFLOW-2088: -- Commit

[jira] [Resolved] (AIRFLOW-2088) Duplicate keys in MySQL to GCS Operator

2018-02-09 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-2088. --- Resolution: Fixed Issue resolved by pull request #3022

[jira] [Created] (AIRFLOW-2092) Fixed incorrect parameter in docstring for FTPHook

2018-02-09 Thread Kaxil Naik (JIRA)
Kaxil Naik created AIRFLOW-2092: --- Summary: Fixed incorrect parameter in docstring for FTPHook Key: AIRFLOW-2092 URL: https://issues.apache.org/jira/browse/AIRFLOW-2092 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-2092) Fixed incorrect parameter in docstring for FTPHook

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358343#comment-16358343 ] ASF subversion and git services commented on AIRFLOW-2092: -- Commit

[jira] [Created] (AIRFLOW-2091) Fix incorrect parameter in BigQueryCursor docstring

2018-02-09 Thread Kaxil Naik (JIRA)
Kaxil Naik created AIRFLOW-2091: --- Summary: Fix incorrect parameter in BigQueryCursor docstring Key: AIRFLOW-2091 URL: https://issues.apache.org/jira/browse/AIRFLOW-2091 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-2086) The tree view page is too slow when display big dag.

2018-02-09 Thread Andrew Maguire (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358307#comment-16358307 ] Andrew Maguire commented on AIRFLOW-2086: - I found this too. I also found that for big dags with

incubator-airflow git commit: [AIRFLOW-2091] Fix incorrect docstring parameter in BigQuery Hook

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master 4c7ae420a -> 0a71370d0 [AIRFLOW-2091] Fix incorrect docstring parameter in BigQuery Hook - Instead of `seq_of_parameters`, the docstring contains `parameters`. Fixed this Closes #3025 from kaxil/fix-parameter Project:

[jira] [Resolved] (AIRFLOW-2091) Fix incorrect parameter in BigQueryCursor docstring

2018-02-09 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-2091. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request #3025

incubator-airflow git commit: [AIRFLOW-2090] Fix typo in DataStore Hook

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master e6973b159 -> 4c7ae420a [AIRFLOW-2090] Fix typo in DataStore Hook Spelling mistake in the word `simultaneously` in DataStore docs Closes #3024 from kaxil/patch-1 Project:

[jira] [Commented] (AIRFLOW-2090) Fix typo in DataStore Hook

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358281#comment-16358281 ] ASF subversion and git services commented on AIRFLOW-2090: -- Commit

[jira] [Resolved] (AIRFLOW-2090) Fix typo in DataStore Hook

2018-02-09 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-2090. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request #3024

incubator-airflow git commit: [AIRFLOW-XXX] Add SocialCops to Airflow users

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master 759d8f83e -> a1e5a075c [AIRFLOW-XXX] Add SocialCops to Airflow users Closes #3018 from vinayak-mehta/update_readme Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

incubator-airflow git commit: [AIRFLOW-2092] Fixed incorrect parameter in docstring for FTPHook

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master a1e5a075c -> bf1296fbd [AIRFLOW-2092] Fixed incorrect parameter in docstring for FTPHook Fixed the datatype of parameter `local_full_path_or_buffer` in `retrieve_file` method for FTPHook Closes #3026 from kaxil/patch-2

[jira] [Resolved] (AIRFLOW-2092) Fixed incorrect parameter in docstring for FTPHook

2018-02-09 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-2092. --- Resolution: Fixed Issue resolved by pull request #3026

[jira] [Commented] (AIRFLOW-2066) Add an operator to create an Empty BigQuery Table

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358133#comment-16358133 ] ASF subversion and git services commented on AIRFLOW-2066: -- Commit

[jira] [Resolved] (AIRFLOW-2083) Incorrect usage of "it's" appears throughout the documentation

2018-02-09 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-2083. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request #3020

incubator-airflow git commit: [AIRFLOW-2083] Docs: Use "its" instead of "it's" where appropriate

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master 2920d0475 -> 15b8a36b9 [AIRFLOW-2083] Docs: Use "its" instead of "it's" where appropriate Closes #3020 from wrp/spelling Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[jira] [Commented] (AIRFLOW-2083) Incorrect usage of "it's" appears throughout the documentation

2018-02-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358138#comment-16358138 ] ASF subversion and git services commented on AIRFLOW-2083: -- Commit

[1/2] incubator-airflow git commit: [AIRFLOW-XXX] add Karmic to list of companies

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master 15b8a36b9 -> fd6772116 [AIRFLOW-XXX] add Karmic to list of companies Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/822296af Tree:

[2/2] incubator-airflow git commit: Merge pull request #3019 from hyw/master

2018-02-09 Thread fokko
Merge pull request #3019 from hyw/master Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/fd677211 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/fd677211 Diff:

incubator-airflow git commit: [AIRFLOW-713] Jinjafy {EmrCreateJobFlow, EmrAddSteps}Operator attributes

2018-02-09 Thread fokko
Repository: incubator-airflow Updated Branches: refs/heads/master fd6772116 -> 44551e249 [AIRFLOW-713] Jinjafy {EmrCreateJobFlow,EmrAddSteps}Operator attributes To dynamically templat the fields of the Emr Operators, we need to pass the fields to jinja Closes #3016 from

[jira] [Resolved] (AIRFLOW-713) EmrCreateJobFlowOperator and EmrAddStepsOperatordoes attributes are not jinjafied

2018-02-09 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-713. -- Resolution: Fixed Fix Version/s: (was: Airflow 2.0) 2.0.0