[jira] [Updated] (AIRFLOW-809) SqlAlchemy is_ ColumnOperator Causing Errors in MSSQL

2017-06-21 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated AIRFLOW-809: --- Fix Version/s: 1.8.2 > SqlAlchemy is_ ColumnOperator Causing Errors in MSSQL > --

[jira] [Updated] (AIRFLOW-1338) gcp_dataflow_hook is incompatible with the recent google-cloud-dataflow(apache-beam) release.

2017-06-21 Thread Feng Lu (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Lu updated AIRFLOW-1338: - Summary: gcp_dataflow_hook is incompatible with the recent google-cloud-dataflow(apache-beam) release.

[jira] [Created] (AIRFLOW-1338) gcp_dataflow_hook is compatible with the recent google-cloud-dataflow(apache-beam) release.

2017-06-21 Thread Feng Lu (JIRA)
Feng Lu created AIRFLOW-1338: Summary: gcp_dataflow_hook is compatible with the recent google-cloud-dataflow(apache-beam) release. Key: AIRFLOW-1338 URL: https://issues.apache.org/jira/browse/AIRFLOW-1338

[jira] [Created] (AIRFLOW-1337) Customize log format via config file

2017-06-21 Thread Ron Fung (JIRA)
Ron Fung created AIRFLOW-1337: - Summary: Customize log format via config file Key: AIRFLOW-1337 URL: https://issues.apache.org/jira/browse/AIRFLOW-1337 Project: Apache Airflow Issue Type: Improve

[jira] [Created] (AIRFLOW-1336) Fix the default runner of DataFlowPythonOperator

2017-06-21 Thread Younghee Kwon (JIRA)
Younghee Kwon created AIRFLOW-1336: -- Summary: Fix the default runner of DataFlowPythonOperator Key: AIRFLOW-1336 URL: https://issues.apache.org/jira/browse/AIRFLOW-1336 Project: Apache Airflow

[jira] [Created] (AIRFLOW-1335) Use buffered logger

2017-06-21 Thread Alex Guziel (JIRA)
Alex Guziel created AIRFLOW-1335: Summary: Use buffered logger Key: AIRFLOW-1335 URL: https://issues.apache.org/jira/browse/AIRFLOW-1335 Project: Apache Airflow Issue Type: Improvement

[incubator-airflow] Git Push Summary

2017-06-21 Thread saguziel
Repository: incubator-airflow Updated Branches: refs/heads/aguziel-use-join-apache [deleted] 436fe71c8

[jira] [Commented] (AIRFLOW-1334) Improve efficiency of checking for backfills on scheduler loop

2017-06-21 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058325#comment-16058325 ] ASF subversion and git services commented on AIRFLOW-1334: -- Com

incubator-airflow git commit: [AIRFLOW-1334] Check if tasks are backfill on scheduler in a join

2017-06-21 Thread saguziel
Repository: incubator-airflow Updated Branches: refs/heads/aguziel-use-join-apache [created] 436fe71c8 [AIRFLOW-1334] Check if tasks are backfill on scheduler in a join Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incuba

[jira] [Created] (AIRFLOW-1334) Improve efficiency of checking for backfills on scheduler loop

2017-06-21 Thread Alex Guziel (JIRA)
Alex Guziel created AIRFLOW-1334: Summary: Improve efficiency of checking for backfills on scheduler loop Key: AIRFLOW-1334 URL: https://issues.apache.org/jira/browse/AIRFLOW-1334 Project: Apache Airf

[jira] [Updated] (AIRFLOW-1325) Make Airflow Logging Backed By Elasticsearch

2017-06-21 Thread Allison Wang (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated AIRFLOW-1325: -- Description: Currently, Airflow uses S3/GCS as the log storage backend. Workers, when executi

[jira] [Updated] (AIRFLOW-1325) Make Airflow Logging Backed By Elasticsearch

2017-06-21 Thread Allison Wang (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated AIRFLOW-1325: -- Description: Currently, Airflow uses S3/GCS as the log storage backend. Workers, when executi

[jira] [Updated] (AIRFLOW-1325) Make Airflow Logging Backed By Elasticsearch

2017-06-21 Thread Allison Wang (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated AIRFLOW-1325: -- Description: Currently, Airflow uses S3/GCS as the log storage backend. Workers, when executi

[jira] [Updated] (AIRFLOW-1329) Problematic DAG cause worker queue saturated

2017-06-21 Thread Zhen Zhang (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Zhang updated AIRFLOW-1329: Description: We see this weird issue in our production airflow cluster: # User has a problematic

[jira] [Updated] (AIRFLOW-1333) Enable copy function for Google Cloud Storage Hook

2017-06-21 Thread Younghee Kwon (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Younghee Kwon updated AIRFLOW-1333: --- Summary: Enable copy function for Google Cloud Storage Hook (was: GoogleCloudStorageHook sh

[jira] [Created] (AIRFLOW-1333) GoogleCloudStorageHook should support copy()

2017-06-21 Thread Younghee Kwon (JIRA)
Younghee Kwon created AIRFLOW-1333: -- Summary: GoogleCloudStorageHook should support copy() Key: AIRFLOW-1333 URL: https://issues.apache.org/jira/browse/AIRFLOW-1333 Project: Apache Airflow I

[jira] [Created] (AIRFLOW-1332) Add attempt column to task instance

2017-06-21 Thread Allison Wang (JIRA)
Allison Wang created AIRFLOW-1332: - Summary: Add attempt column to task instance Key: AIRFLOW-1332 URL: https://issues.apache.org/jira/browse/AIRFLOW-1332 Project: Apache Airflow Issue Type:

[jira] [Work started] (AIRFLOW-1332) Add attempt column to task instance

2017-06-21 Thread Allison Wang (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-1332 started by Allison Wang. - > Add attempt column to task instance > --- >

[jira] [Commented] (AIRFLOW-1330) Connection.parse_from_uri doesn't work for google_cloud_platform and so on

2017-06-21 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057990#comment-16057990 ] Chris Riccomini commented on AIRFLOW-1330: -- I agree (2) is the right way to go.

[jira] [Commented] (AIRFLOW-1330) Connection.parse_from_uri doesn't work for google_cloud_platform and so on

2017-06-21 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057929#comment-16057929 ] Yu Ishikawa commented on AIRFLOW-1330: -- I think we need to change the command line

[jira] [Commented] (AIRFLOW-1330) Connection.parse_from_uri doesn't work for google_cloud_platform and so on

2017-06-21 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057857#comment-16057857 ] Yu Ishikawa commented on AIRFLOW-1330: -- If we want to any connection whose connecti

[jira] [Created] (AIRFLOW-1331) Contrib.SparkSubmitOperator should allow --packages parameter

2017-06-21 Thread manuel garrido (JIRA)
manuel garrido created AIRFLOW-1331: --- Summary: Contrib.SparkSubmitOperator should allow --packages parameter Key: AIRFLOW-1331 URL: https://issues.apache.org/jira/browse/AIRFLOW-1331 Project: Apache

[jira] [Commented] (AIRFLOW-1275) Fix `airflow pool` command exception

2017-06-21 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057618#comment-16057618 ] ASF subversion and git services commented on AIRFLOW-1275: -- Com

[jira] [Resolved] (AIRFLOW-1275) Fix `airflow pool` command exception

2017-06-21 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved AIRFLOW-1275. - Resolution: Fixed Issue resolved by pull request #2346 [https://github.com/apache/incuba

incubator-airflow git commit: [AIRFLOW-1275] Put 'airflow pool' into API

2017-06-21 Thread bolke
Repository: incubator-airflow Updated Branches: refs/heads/master a45e2d188 -> 9958aa9d5 [AIRFLOW-1275] Put 'airflow pool' into API Closes #2346 from skudriashev/airflow-1275 Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf

[jira] [Comment Edited] (AIRFLOW-1328) Daily DAG execute the past day

2017-06-21 Thread Pierre-Antoine Tible (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057198#comment-16057198 ] Pierre-Antoine Tible edited comment on AIRFLOW-1328 at 6/21/17 8:53 AM: --

[jira] [Comment Edited] (AIRFLOW-1328) Daily DAG execute the past day

2017-06-21 Thread Pierre-Antoine Tible (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057198#comment-16057198 ] Pierre-Antoine Tible edited comment on AIRFLOW-1328 at 6/21/17 8:52 AM: --

[jira] [Commented] (AIRFLOW-1328) Daily DAG execute the past day

2017-06-21 Thread Pierre-Antoine Tible (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057198#comment-16057198 ] Pierre-Antoine Tible commented on AIRFLOW-1328: --- Okay, it's quite strange,

[jira] [Commented] (AIRFLOW-1296) DAGs using operators involving cascading skipped tasks fail prematurely

2017-06-21 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057152#comment-16057152 ] ASF subversion and git services commented on AIRFLOW-1296: -- Com

[jira] [Resolved] (AIRFLOW-1296) DAGs using operators involving cascading skipped tasks fail prematurely

2017-06-21 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin resolved AIRFLOW-1296. - Resolution: Fixed Issue resolved by pull request #2365 [https://github.com/apache/incuba

[jira] [Commented] (AIRFLOW-719) Skipped operations make DAG finish prematurely

2017-06-21 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057153#comment-16057153 ] ASF subversion and git services commented on AIRFLOW-719: - Commit

incubator-airflow git commit: [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks

2017-06-21 Thread bolke
Repository: incubator-airflow Updated Branches: refs/heads/v1-8-test 93825d507 -> 333e0b3eb [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks The ShortCircuitOperator and LatestOnlyOperator did not mark all downstream tasks as skipped, but only direct downstream tasks. Closes #2365 fro

[jira] [Commented] (AIRFLOW-1296) DAGs using operators involving cascading skipped tasks fail prematurely

2017-06-21 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057145#comment-16057145 ] ASF subversion and git services commented on AIRFLOW-1296: -- Com

[jira] [Commented] (AIRFLOW-719) Skipped operations make DAG finish prematurely

2017-06-21 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057146#comment-16057146 ] ASF subversion and git services commented on AIRFLOW-719: - Commit

incubator-airflow git commit: [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks

2017-06-21 Thread bolke
Repository: incubator-airflow Updated Branches: refs/heads/master ea240cd1d -> a45e2d188 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks The ShortCircuitOperator and LatestOnlyOperator did not mark all downstream tasks as skipped, but only direct downstream tasks. Closes #2365 from b

[jira] [Updated] (AIRFLOW-935) Impossible to use plugin executors

2017-06-21 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated AIRFLOW-935: --- Fix Version/s: 1.8.2 > Impossible to use plugin executors > -

[jira] [Updated] (AIRFLOW-860) Circular module dependency prevents loading of custom executor plugin

2017-06-21 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated AIRFLOW-860: --- Fix Version/s: (was: 1.8.,2) 1.8.2 > Circular module dependency preven

[jira] [Updated] (AIRFLOW-860) Circular module dependency prevents loading of custom executor plugin

2017-06-21 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated AIRFLOW-860: --- Fix Version/s: 1.8.,2 > Circular module dependency prevents loading of custom executor plugin

incubator-airflow git commit: Updating README.md to add Pandora Media Inc. as a user

2017-06-21 Thread bolke
Repository: incubator-airflow Updated Branches: refs/heads/master 7b620391a -> ea240cd1d Updating README.md to add Pandora Media Inc. as a user Closes #2371 from Acehaidrey/master Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repo