[GitHub] [airflow] potiuk commented on issue #7727: [AIRFLOW-XXXX] Updated supported versions of backends

2020-03-14 Thread GitBox
potiuk commented on issue #7727: [AIRFLOW-] Updated supported versions of backends URL: https://github.com/apache/airflow/pull/7727#issuecomment-599171665 I updated the supported versions -> I am adding more matrix components in 1.10 including python 3.7. I think we can add more build

[GitHub] [airflow] potiuk opened a new pull request #7727: [AIRFLOW-XXXX] Updated supported versions of backends

2020-03-14 Thread GitBox
potiuk opened a new pull request #7727: [AIRFLOW-] Updated supported versions of backends URL: https://github.com/apache/airflow/pull/7727 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before

[GitHub] [airflow] codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics URL: https://github.com/apache/airflow/pull/6342#issuecomment-547121627 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6342?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics URL: https://github.com/apache/airflow/pull/6342#issuecomment-547121627 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6342?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics URL: https://github.com/apache/airflow/pull/6342#issuecomment-547121627 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6342?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics URL: https://github.com/apache/airflow/pull/6342#issuecomment-547121627 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6342?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #6342: [AIRFLOW-5662] fix incorrect naming and batch db call for scheduler metrics URL: https://github.com/apache/airflow/pull/6342#issuecomment-547121627 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6342?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-7065) Give StreamLogHandler ability to tee to log and console

2020-03-14 Thread Christopher (Jira)
Christopher created AIRFLOW-7065: Summary: Give StreamLogHandler ability to tee to log and console Key: AIRFLOW-7065 URL: https://issues.apache.org/jira/browse/AIRFLOW-7065 Project: Apache Airflow

[GitHub] [airflow] nuclearpinguin closed pull request #7192: [AIRFLOW-XXXX] Fix BigQuery example DAG

2020-03-14 Thread GitBox
nuclearpinguin closed pull request #7192: [AIRFLOW-] Fix BigQuery example DAG URL: https://github.com/apache/airflow/pull/7192 This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (AIRFLOW-7058) Add support for DB versions in CI

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059514#comment-17059514 ] ASF GitHub Bot commented on AIRFLOW-7058: - potiuk commented on pull request #7717:

[GitHub] [airflow] potiuk merged pull request #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
potiuk merged pull request #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Assigned] (AIRFLOW-6295) Create guide for BigQuery service operator

2020-03-14 Thread Alexandra Abbas (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandra Abbas reassigned AIRFLOW-6295: Assignee: Alexandra Abbas > Create guide for BigQuery service operator >

[jira] [Commented] (AIRFLOW-4175) S3Hook load_file should support ACL policy parameter

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059503#comment-17059503 ] ASF GitHub Bot commented on AIRFLOW-4175: - OmairK commented on pull request #7726:

[GitHub] [airflow] OmairK opened a new pull request #7726: [AIRFLOW-4175] S3Hook load_file should support ACL policy parameter

2020-03-14 Thread GitBox
OmairK opened a new pull request #7726: [AIRFLOW-4175] S3Hook load_file should support ACL policy parameter URL: https://github.com/apache/airflow/pull/7726 - Added acl_policy parameter to all the S3Hook.load_*() and S3Hook.copy_object() functions - Added unittest to test the

[jira] [Commented] (AIRFLOW-6295) Create guide for BigQuery service operator

2020-03-14 Thread Alexandra Maha Abbas (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059498#comment-17059498 ] Alexandra Maha Abbas commented on AIRFLOW-6295: --- Hey [~kamil.bregula], I'm happy to pick

[jira] [Commented] (AIRFLOW-6946) Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059490#comment-17059490 ] ASF GitHub Bot commented on AIRFLOW-6946: - potiuk commented on pull request #7570:

[jira] [Resolved] (AIRFLOW-6946) Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6946. --- Fix Version/s: 2.0.0 Resolution: Fixed > Switch to MySQL 5.7 in 2.0 as base >

[GitHub] [airflow] potiuk merged pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
potiuk merged pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] potiuk commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
potiuk commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#discussion_r392612429 ## File path: airflow/utils/cli.py ## @@ -238,3 +238,12 @@ def sigquit_handler(sig,

[jira] [Commented] (AIRFLOW-6518) Task did not retry when there was temporary metastore db connectivity loss

2020-03-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059433#comment-17059433 ] Jarek Potiuk commented on AIRFLOW-6518: --- I wonder if this is a real problem that we should take

[GitHub] [airflow] aggarwaldeepak commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
aggarwaldeepak commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#discussion_r392611916 ## File path: airflow/utils/cli.py ## @@ -238,3 +238,12 @@ def

[GitHub] [airflow] aggarwaldeepak commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
aggarwaldeepak commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#discussion_r392611916 ## File path: airflow/utils/cli.py ## @@ -238,3 +238,12 @@ def

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[GitHub] [airflow] potiuk commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
potiuk commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#discussion_r392602040 ## File path: airflow/utils/cli.py ## @@ -238,3 +238,12 @@ def sigquit_handler(sig,

[GitHub] [airflow] potiuk commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
potiuk commented on a change in pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#discussion_r392602337 ## File path: airflow/cli/commands/celery_command.py ## @@ -135,6 +135,7 @@ def

[jira] [Resolved] (AIRFLOW-7063) dag.clear() slowness caused by multiple UNION statements and tis.count()

2020-03-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-7063. --- Fix Version/s: 1.10.10 Resolution: Fixed > dag.clear() slowness caused by multiple

[GitHub] [airflow] potiuk commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
potiuk commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723#issuecomment-599095395 Thanks @yuqian90 ! This is an automated message from the Apache

[jira] [Commented] (AIRFLOW-7063) dag.clear() slowness caused by multiple UNION statements and tis.count()

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059404#comment-17059404 ] ASF GitHub Bot commented on AIRFLOW-7063: - potiuk commented on pull request #7723:

[GitHub] [airflow] potiuk merged pull request #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
potiuk merged pull request #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] codecov-io edited a comment on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723#issuecomment-599089710 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7723?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
codecov-io commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723#issuecomment-599089710 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7723?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7227: [AIRFLOW-6530] Allow Custom Statsd Client

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7227: [AIRFLOW-6530] Allow Custom Statsd Client URL: https://github.com/apache/airflow/pull/7227#issuecomment-577151472 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7227?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7227: [AIRFLOW-6530] Allow Custom Statsd Client

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7227: [AIRFLOW-6530] Allow Custom Statsd Client URL: https://github.com/apache/airflow/pull/7227#issuecomment-577151472 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7227?src=pr=h1) Report > Merging

[GitHub] [airflow] potiuk commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
potiuk commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723#issuecomment-599088010 Indeed. Interesting one :) This is an automated message from

[GitHub] [airflow] potiuk commented on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
potiuk commented on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-599087668 @nuclearpinguin -> fixed all :) This is an automated message from the

[GitHub] [airflow] potiuk commented on a change in pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
potiuk commented on a change in pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#discussion_r392599301 ## File path: airflow/models/base.py ## @@ -38,3 +38,11 @@ # used for typing class Operator:

[GitHub] [airflow] yuqian90 commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
yuqian90 commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723#issuecomment-599082749 > > Thanks for the review. The test I added does not play well with sqlite. I'll fix soon > > The error is really

[GitHub] [airflow] codecov-io commented on issue #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
codecov-io commented on issue #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#issuecomment-599077938 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7724?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#issuecomment-599077938 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7724?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#issuecomment-599077938 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7724?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-7064) Add CloudFirestoreExportDatabaseOperator

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059384#comment-17059384 ] ASF GitHub Bot commented on AIRFLOW-7064: - mik-laj commented on pull request #7725:

[GitHub] [airflow] mik-laj opened a new pull request #7725: [AIRFLOW-7064][WIP] Add CloudFirestoreExportDatabaseOperator

2020-03-14 Thread GitBox
mik-laj opened a new pull request #7725: [AIRFLOW-7064][WIP] Add CloudFirestoreExportDatabaseOperator URL: https://github.com/apache/airflow/pull/7725 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below

[GitHub] [airflow] nuclearpinguin commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
nuclearpinguin commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723#issuecomment-599076360 > Thanks for the review. The test I added does not play well with sqlite. I'll fix soon The error is really

[jira] [Created] (AIRFLOW-7064) Add CloudFirestoreExportDatabaseOperator

2020-03-14 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-7064: -- Summary: Add CloudFirestoreExportDatabaseOperator Key: AIRFLOW-7064 URL: https://issues.apache.org/jira/browse/AIRFLOW-7064 Project: Apache Airflow

[jira] [Updated] (AIRFLOW-7064) Add CloudFirestoreExportDatabaseOperator

2020-03-14 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-7064: --- Issue Type: New Feature (was: Improvement) > Add CloudFirestoreExportDatabaseOperator >

[jira] [Commented] (AIRFLOW-1536) DaemonContext uses default umask 0

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059377#comment-17059377 ] ASF GitHub Bot commented on AIRFLOW-1536: - aggarwaldeepak commented on pull request #7724:

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
boring-cyborg[bot] commented on issue #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724#issuecomment-599071486 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any

[GitHub] [airflow] aggarwaldeepak opened a new pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode

2020-03-14 Thread GitBox
aggarwaldeepak opened a new pull request #7724: [AIRFLOW-1536] Inherit umask from parent process in daemon mode URL: https://github.com/apache/airflow/pull/7724 When celery workers are run in daemon mode, umask was set to default(0) which create dirs/files with 0777/0666 permissions.

[jira] [Commented] (AIRFLOW-7063) dag.clear() slowness caused by multiple UNION statements and tis.count()

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059362#comment-17059362 ] ASF GitHub Bot commented on AIRFLOW-7063: - yuqian90 commented on pull request #7723:

[jira] [Commented] (AIRFLOW-7063) dag.clear() slowness caused by multiple UNION statements and tis.count()

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059361#comment-17059361 ] ASF GitHub Bot commented on AIRFLOW-7063: - yuqian90 commented on pull request #7723:

[GitHub] [airflow] yuqian90 edited a comment on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
yuqian90 edited a comment on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723#issuecomment-599065844 Thanks for the review. The test I added does not play well with sqlite. I'll fix soon

[GitHub] [airflow] yuqian90 commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
yuqian90 commented on issue #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723#issuecomment-599065844 Thanks for the review. The test I added does not play well with sqlite

[GitHub] [airflow] yuqian90 opened a new pull request #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
yuqian90 opened a new pull request #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723 Calling `tis.count()` makes `dag.clear()` much slower than just retrieving all the `tis` and call `len()` because of some issues with how

[GitHub] [airflow] yuqian90 closed pull request #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
yuqian90 closed pull request #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
nuclearpinguin commented on a change in pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#discussion_r392590665 ## File path: airflow/models/base.py ## @@ -38,3 +38,11 @@ # used for typing class

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
nuclearpinguin commented on a change in pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#discussion_r392590528 ## File path: airflow/migrations/versions/e3a246e0dc1_current_schema.py ## @@ -224,16

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
nuclearpinguin commented on a change in pull request #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#discussion_r392590462 ## File path: airflow/models/base.py ## @@ -38,3 +38,11 @@ # used for typing class

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7689: [AIRFLOW-7038] Cassandra sensors tests

2020-03-14 Thread GitBox
nuclearpinguin commented on a change in pull request #7689: [AIRFLOW-7038] Cassandra sensors tests URL: https://github.com/apache/airflow/pull/7689#discussion_r392589839 ## File path: tests/providers/apache/cassandra/sensors/test_record.py ## @@ -0,0 +1,71 @@ +# +#

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
nuclearpinguin commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392589021 ## File path: airflow/providers/presto/hooks/presto.py ## @@ -28,7 +28,7 @@ class

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588743 ## File path: tests/providers/google/cloud/operators/test_presto_to_gcs_system.py ## @@ -0,0

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588701 ## File path: airflow/providers/google/cloud/operators/presto_to_gcs.py ## @@ -0,0 +1,178 @@ +#

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588688 ## File path: airflow/providers/google/cloud/operators/sql_to_gcs.py ## @@ -177,7 +183,7 @@ def

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588706 ## File path: airflow/providers/google/cloud/operators/presto_to_gcs.py ## @@ -0,0 +1,178 @@ +#

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588693 ## File path: airflow/providers/google/cloud/operators/presto_to_gcs.py ## @@ -0,0 +1,178 @@ +#

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588660 ## File path: docs/operators-and-hooks-ref.rst ## @@ -840,6 +840,11 @@ These integrations allow

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588638 ## File path: docs/howto/operator/gcp/presto_to_gcs.rst ## @@ -0,0 +1,142 @@ + .. Licensed to the

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588623 ## File path: docs/howto/operator/gcp/presto_to_gcs.rst ## @@ -0,0 +1,142 @@ + .. Licensed to the

[GitHub] [airflow] mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator

2020-03-14 Thread GitBox
mik-laj commented on a change in pull request #7718: [AIRFLOW-5421] Add Presto to GCS transfer operator URL: https://github.com/apache/airflow/pull/7718#discussion_r392588497 ## File path: airflow/providers/google/cloud/example_dags/example_presto_to_gcs.py ## @@ -0,0

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-1536) DaemonContext uses default umask 0

2020-03-14 Thread Deepak Aggarwal (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059347#comment-17059347 ] Deepak Aggarwal commented on AIRFLOW-1536: -- [~tokeefe] I have started working on the issue. I

[GitHub] [airflow] blcksrx commented on issue #7707: [AIRFLOW-7051] Implement Vertica Hook Tests

2020-03-14 Thread GitBox
blcksrx commented on issue #7707: [AIRFLOW-7051] Implement Vertica Hook Tests URL: https://github.com/apache/airflow/pull/7707#issuecomment-599059690 @nuclearpinguin Thanks! please merge this one https://github.com/apache/airflow/pull/7689

[jira] [Resolved] (AIRFLOW-7051) Implement Vertica Hook Tests

2020-03-14 Thread Tomasz Urbaszek (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomasz Urbaszek resolved AIRFLOW-7051. -- Fix Version/s: 2.0.0 Resolution: Done > Implement Vertica Hook Tests >

[GitHub] [airflow] nuclearpinguin merged pull request #7707: [AIRFLOW-7051] Implement Vertica Hook Tests

2020-03-14 Thread GitBox
nuclearpinguin merged pull request #7707: [AIRFLOW-7051] Implement Vertica Hook Tests URL: https://github.com/apache/airflow/pull/7707 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-7051) Implement Vertica Hook Tests

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059344#comment-17059344 ] ASF GitHub Bot commented on AIRFLOW-7051: - nuclearpinguin commented on pull request #7707:

[jira] [Commented] (AIRFLOW-7063) dag.clear() slowness caused by multiple UNION statements and tis.count()

2020-03-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059326#comment-17059326 ] Jarek Potiuk commented on AIRFLOW-7063: --- Interesting threads at SO ! > dag.clear() slowness

[jira] [Commented] (AIRFLOW-7063) dag.clear() slowness caused by multiple UNION statements and tis.count()

2020-03-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059325#comment-17059325 ] Jarek Potiuk commented on AIRFLOW-7063: --- Whoa. What a surprise :) > dag.clear() slowness caused

[jira] [Commented] (AIRFLOW-7063) dag.clear() slowness caused by multiple UNION statements and tis.count()

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059323#comment-17059323 ] ASF GitHub Bot commented on AIRFLOW-7063: - yuqian90 commented on pull request #7723:

[jira] [Resolved] (AIRFLOW-5705) add option for alternative creds backend

2020-03-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5705. --- Fix Version/s: 1.10.10 Resolution: Fixed > add option for alternative creds backend

[GitHub] [airflow] yuqian90 opened a new pull request #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count

2020-03-14 Thread GitBox
yuqian90 opened a new pull request #7723: [AIRFLOW-7063] Fix dag.clear() slowness caused by count URL: https://github.com/apache/airflow/pull/7723 Calling `tis.count()` makes `dag.clear()` much slower than just retrieving all the `tis` and call `len()` because of some issues with how

[GitHub] [airflow] potiuk commented on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM

2020-03-14 Thread GitBox
potiuk commented on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM URL: https://github.com/apache/airflow/pull/6376#issuecomment-599050292 Coool! This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
potiuk commented on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599047964 > Last time I looked at mysql 8 we failed cos of some bulk export type thing we were using I will try both python 3.8 and

[jira] [Commented] (AIRFLOW-7063) dag.clear() slowness caused by multiple UNION statements and tis.count()

2020-03-14 Thread Qian Yu (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059319#comment-17059319 ] Qian Yu commented on AIRFLOW-7063: -- Here's a self-contained test to reproduce this slowness. 

[GitHub] [airflow] potiuk commented on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
potiuk commented on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599047739 @ashb @kaxil - I've added description to README to make it clear which Python/MySQL/Postgres DB versions are currently tested with

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7570: [AIRFLOW-6946] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions

2020-03-14 Thread GitBox
codecov-io edited a comment on issue #7717: [AIRFLOW-7058] Add support for different DB versions URL: https://github.com/apache/airflow/pull/7717#issuecomment-599046718 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7717?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-5705) add option for alternative creds backend

2020-03-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059314#comment-17059314 ] ASF GitHub Bot commented on AIRFLOW-5705: - kaxil commented on pull request #6376:

  1   2   >