[airflow] tag nightly-master updated (94a7673 -> f107338)

2020-05-22 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 94a7673 (commit) to f107338 (commit) from

[GitHub] [airflow] tanjinP commented on issue #8970: Improve guide for KubernetesPodOperator

2020-05-22 Thread GitBox
tanjinP commented on issue #8970: URL: https://github.com/apache/airflow/issues/8970#issuecomment-632979988 I can take this on as it relates to #8883 This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] kaxil commented on a change in pull request #8974: Vault has now VaultHook not only SecretBackend

2020-05-22 Thread GitBox
kaxil commented on a change in pull request #8974: URL: https://github.com/apache/airflow/pull/8974#discussion_r429505660 ## File path: airflow/providers/hashicorp/secrets/vault.py ## @@ -72,98 +68,90 @@ class VaultBackend(BaseSecretsBackend, LoggingMixin): :type

[GitHub] [airflow] VijayantSoni commented on pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-22 Thread GitBox
VijayantSoni commented on pull request #8701: URL: https://github.com/apache/airflow/pull/8701#issuecomment-632564576 @ashb made the changes. Test are failing due to moto missing support for mocking ElastiCache. Do I need to write custom mock for this ? Let me know how should we test

[GitHub] [airflow] ashb commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
ashb commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r429139507 ## File path: airflow/operators/python.py ## @@ -145,6 +148,142 @@ def execute_callable(self): return self.python_callable(*self.op_args,

[GitHub] [airflow] stale[bot] closed pull request #7452: [AIRFLOW-6831] Support for various ssh keys and certificates

2020-05-22 Thread GitBox
stale[bot] closed pull request #7452: URL: https://github.com/apache/airflow/pull/7452 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (AIRFLOW-7086) Change the dags-paused API endpoint to a POST

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113785#comment-17113785 ] ASF GitHub Bot commented on AIRFLOW-7086: - stale[bot] closed pull request #7761: URL:

[GitHub] [airflow] stale[bot] closed pull request #7761: [AIRFLOW-7086] /dags/paused should be a POST

2020-05-22 Thread GitBox
stale[bot] closed pull request #7761: URL: https://github.com/apache/airflow/pull/7761 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (AIRFLOW-6831) Support for various ssh keys and certificates

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113786#comment-17113786 ] ASF GitHub Bot commented on AIRFLOW-6831: - stale[bot] closed pull request #7452: URL:

[GitHub] [airflow] fuyi opened a new issue #8965: use separate pod_mutation_hook for Kubernetes Executor pod

2020-05-22 Thread GitBox
fuyi opened a new issue #8965: URL: https://github.com/apache/airflow/issues/8965 **Description** According to the [Kubernetes Executor doc](https://airflow.apache.org/docs/stable/kubernetes.html), pod_mutation_hook is used to mutate pod object before sending to the

[GitHub] [airflow] j-y-matsubara edited a comment on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-22 Thread GitBox
j-y-matsubara edited a comment on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632606590 @yuqian90 Thank you for your advise. I have corrected the document that you pointed out. And.. The continuous-integration/travis-ci/pr failure

[GitHub] [airflow] j-y-matsubara edited a comment on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-22 Thread GitBox
j-y-matsubara edited a comment on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632606590 @yuqian90 Thank you for your advise. I have corrected the document that you pointed out. And.. The continuous-integration/travis-ci/pr failure

[jira] [Commented] (AIRFLOW-5262) Update timeout exception to include dag name

2020-05-22 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113912#comment-17113912 ] ASF subversion and git services commented on AIRFLOW-5262: -- Commit

[jira] [Commented] (AIRFLOW-4512) New trigger rule for tasks - At least one successful

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113932#comment-17113932 ] ASF GitHub Bot commented on AIRFLOW-4512: - psuanm5030 commented on pull request #5288: URL:

[GitHub] [airflow] ashb commented on pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-05-22 Thread GitBox
ashb commented on pull request #8701: URL: https://github.com/apache/airflow/pull/8701#issuecomment-632633027 @VijayantSoni Probably just using `unittest.mock` is enough here, yes please -- we've done that in a few other places I think

[GitHub] [airflow] j-y-matsubara edited a comment on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-22 Thread GitBox
j-y-matsubara edited a comment on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632606590 @yuqian90 Thank you for your advise. I have corrected the document that you pointed out. And... The continuous-integration/travis-ci/pr failure

[airflow] 02/02: Fix race in Celery tests by pre-creating result tables (#8909)

2020-05-22 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 3f3a4b5e2a0fb256c023d6087026608697f40a2f Author: Ash Berlin-Taylor AuthorDate: Tue May 19 13:21:44 2020 +0100

[airflow] branch v1-10-test updated (24b7416 -> 3f3a4b5)

2020-05-22 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. discard 24b7416 Fix race in Celery tests by pre-creating result tables (#8909) discard d13c681 [AIRFLOW-3367] Run

[airflow] 01/02: [AIRFLOW-3367] Run celery integration test with redis broker. (#4207)

2020-05-22 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 229f631f2dafa7a698e634a614c8fa4102f68b38 Author: Joshua Carp AuthorDate: Mon Dec 3 04:19:25 2018 -0500

[GitHub] [airflow] dimitrijepetrovic opened a new issue #8967: 2.0.0 - db upgrade fails on missing dag_stats

2020-05-22 Thread GitBox
dimitrijepetrovic opened a new issue #8967: URL: https://github.com/apache/airflow/issues/8967 the error happens because of "drop table " like in all other places This is an automated message from the Apache Git Service.

[GitHub] [airflow] kaxil commented on pull request #7335: [AIRFLOW-6711] Drop plugin support for stat_name_handler

2020-05-22 Thread GitBox
kaxil commented on pull request #7335: URL: https://github.com/apache/airflow/pull/7335#issuecomment-632628871 > I've put this in for 1.10.12, but we should do that in a backwards compatible way (i.e. use the plugin method, but warn) I think the stat_name_handler thing never made it

[GitHub] [airflow] ashb commented on pull request #8901: Getting the connections correctly in AWS Secrets Manager backend

2020-05-22 Thread GitBox
ashb commented on pull request #8901: URL: https://github.com/apache/airflow/pull/8901#issuecomment-632632193 Hi @JavierLopezT The screenshot you posted looks like a nice feature -- is there a reason you closed this PR?

[GitHub] [airflow] j-y-matsubara edited a comment on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-22 Thread GitBox
j-y-matsubara edited a comment on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632606590 @yuqian90 Thank you for your advise. I have corrected the document that you pointed out. And.. The continuous-integration/travis-ci/pr failure

[GitHub] [airflow] j-y-matsubara edited a comment on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-22 Thread GitBox
j-y-matsubara edited a comment on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632606590 @yuqian90 Thank you for your advise. I have corrected the document that you pointed out. And.. The continuous-integration/travis-ci/pr failure

[jira] [Updated] (AIRFLOW-3367) Test celery with redis broker

2020-05-22 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor updated AIRFLOW-3367: --- Fix Version/s: (was: 2.0.0) 1.10.11 > Test celery with

[GitHub] [airflow] U-Ozdemir opened a new pull request #8968: Create guide for Machine Learning Engine operators #8207

2020-05-22 Thread GitBox
U-Ozdemir opened a new pull request #8968: URL: https://github.com/apache/airflow/pull/8968 This my first time working with an open source project and I posting here the first attempt for an ML operator guide. It is still in progress now, but some feedback is always welcome. #

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8968: Create guide for Machine Learning Engine operators #8207

2020-05-22 Thread GitBox
boring-cyborg[bot] commented on pull request #8968: URL: https://github.com/apache/airflow/pull/8968#issuecomment-632661432 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] U-Ozdemir commented on issue #8207: Create guide for Machine Learning Engine operators

2020-05-22 Thread GitBox
U-Ozdemir commented on issue #8207: URL: https://github.com/apache/airflow/issues/8207#issuecomment-632669193 Hello, I have submitted a part of the guide. It can be used by others here working on it aswell. Any feedback is always welcome. Best regards, Ozdemir

[GitHub] [airflow] mik-laj opened a new issue #8970: Improve guide for Kubernetes Pod Operators

2020-05-22 Thread GitBox
mik-laj opened a new issue #8970: URL: https://github.com/apache/airflow/issues/8970 Hello, A better guide that describes how to use KubernetesPodOperator would be useful. This is one of the most frequently used Airflow operators, so many users have problems with this operator.

[jira] [Commented] (AIRFLOW-4816) add MySQLToS3Operator

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113894#comment-17113894 ] ASF GitHub Bot commented on AIRFLOW-4816: - JavierLopezT closed pull request #6670: URL:

[GitHub] [airflow] JavierLopezT closed pull request #8901: Getting the connections correctly in AWS Secrets Manager backend

2020-05-22 Thread GitBox
JavierLopezT closed pull request #8901: URL: https://github.com/apache/airflow/pull/8901 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[airflow] branch master updated (94a7673 -> 9a4a2d1)

2020-05-22 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 94a7673 Pin google-cloud-datacatalog to <0.8 (#8957) add 9a4a2d1 [AIRFLOW-5262] Update timeout exception

[jira] [Commented] (AIRFLOW-5262) Update timeout exception to include dag name

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113911#comment-17113911 ] ASF GitHub Bot commented on AIRFLOW-5262: - turbaszek merged pull request #8466: URL:

[GitHub] [airflow] ashb commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
ashb commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r429140874 ## File path: airflow/operators/python.py ## @@ -145,6 +148,142 @@ def execute_callable(self): return self.python_callable(*self.op_args,

[jira] [Commented] (AIRFLOW-6711) Drop plugin support for stat_name_handler

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113938#comment-17113938 ] ASF GitHub Bot commented on AIRFLOW-6711: - kaxil commented on pull request #7335: URL:

[GitHub] [airflow] JavierLopezT commented on pull request #8901: Getting the connections correctly in AWS Secrets Manager backend

2020-05-22 Thread GitBox
JavierLopezT commented on pull request #8901: URL: https://github.com/apache/airflow/pull/8901#issuecomment-632633845 Hi @ashb, thank you. The only reason is that there is a huge mess with git and I was thinking about a fresh start. It is explained a little deeper here

[airflow] branch v1-10-test updated (179fbb6 -> 24b7416)

2020-05-22 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from 179fbb6 Cache 1 10 ci images (#8955) (#8959) new d13c681 [AIRFLOW-3367] Run celery integration test with

[airflow] 02/02: Fix race in Celery tests by pre-creating result tables (#8909)

2020-05-22 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 24b74168afe26500abaf82fd388eccee18611c4e Author: Ash Berlin-Taylor AuthorDate: Tue May 19 13:21:44 2020 +0100

[airflow] 01/02: [AIRFLOW-3367] Run celery integration test with redis broker. (#4207)

2020-05-22 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit d13c6810674dae4abbc572116852e6b331c9f40e Author: Joshua Carp AuthorDate: Mon Dec 3 04:19:25 2018 -0500

[jira] [Commented] (AIRFLOW-3367) Test celery with redis broker

2020-05-22 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113950#comment-17113950 ] ASF subversion and git services commented on AIRFLOW-3367: -- Commit

[jira] [Commented] (AIRFLOW-3367) Test celery with redis broker

2020-05-22 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113951#comment-17113951 ] ASF subversion and git services commented on AIRFLOW-3367: -- Commit

[GitHub] [airflow] nodyke opened a new issue #8969: Atlas Backend module improvements

2020-05-22 Thread GitBox
nodyke opened a new issue #8969: URL: https://github.com/apache/airflow/issues/8969 **Description** As part of data lineage implementation, there is requirement to sending lineage data from airflow to Apache Atlas. Current module, what represented in 1.10 stable version,

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8969: Atlas Backend module improvements

2020-05-22 Thread GitBox
boring-cyborg[bot] commented on issue #8969: URL: https://github.com/apache/airflow/issues/8969#issuecomment-632671185 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] j-y-matsubara commented on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-22 Thread GitBox
j-y-matsubara commented on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632606590 @yuqian90 Thank you your advise. I have corrected the document that you pointed out. And.. The continuous-integration/travis-ci/pr failure doesn't seem to

[jira] [Commented] (AIRFLOW-5262) Update timeout exception to include dag name

2020-05-22 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113913#comment-17113913 ] ASF subversion and git services commented on AIRFLOW-5262: -- Commit

[GitHub] [airflow] ashb commented on pull request #8901: Getting the connections correctly in AWS Secrets Manager backend

2020-05-22 Thread GitBox
ashb commented on pull request #8901: URL: https://github.com/apache/airflow/pull/8901#issuecomment-632634642 Ahha, cool. yes. `git rebase` can be tricky the first thousand times you run it - I've been there! Don't worry to much about closed PRs, do what is easiest for you to not

[GitHub] [airflow] turbaszek commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-05-22 Thread GitBox
turbaszek commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r429236013 ## File path: airflow/providers/google/cloud/hooks/datafusion.py ## @@ -357,12 +418,45 @@ def list_pipelines( ) return

[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-22 Thread GitBox
j-y-matsubara commented on a change in pull request #8867: URL: https://github.com/apache/airflow/pull/8867#discussion_r429157021 ## File path: UPDATING.md ## @@ -62,6 +62,17 @@ https://developers.google.com/style/inclusive-documentation --> +### BaseSensorOperator to

[GitHub] [airflow] ashb commented on pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
ashb commented on pull request #8962: URL: https://github.com/apache/airflow/pull/8962#issuecomment-632623113 Do copy what I put in slack, Thanks to Stackoverflow I've got a way of having `from airflow import task` be callable, and still work as a submodule: ```diff diff --git

[jira] [Commented] (AIRFLOW-6711) Drop plugin support for stat_name_handler

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113935#comment-17113935 ] ASF GitHub Bot commented on AIRFLOW-6711: - ashb commented on pull request #7335: URL:

[GitHub] [airflow] ashb commented on pull request #7335: [AIRFLOW-6711] Drop plugin support for stat_name_handler

2020-05-22 Thread GitBox
ashb commented on pull request #7335: URL: https://github.com/apache/airflow/pull/7335#issuecomment-632627339 I've put this in for 1.10.12, but we should do that in a backwards compatible way (i.e. use the plugin method, but warn)

[GitHub] [airflow] ashb commented on issue #8967: 2.0.0 - db upgrade fails on missing dag_stats

2020-05-22 Thread GitBox
ashb commented on issue #8967: URL: https://github.com/apache/airflow/issues/8967#issuecomment-632642806 migrations are recorded when they are performed, and are only run once. It should be impossible for the table to be dropped without also recording that the migration was deleted.

[GitHub] [airflow] ashb commented on pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
ashb commented on pull request #8962: URL: https://github.com/apache/airflow/pull/8962#issuecomment-632646688 > The main question is: do we want to support that? Is this "hack" worth it, and do IDEs get massively confused by this? I think if IDEs support it, and we add tests for it

[GitHub] [airflow] JavierLopezT closed pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-05-22 Thread GitBox
JavierLopezT closed pull request #6670: URL: https://github.com/apache/airflow/pull/6670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] turbaszek commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
turbaszek commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r429151128 ## File path: airflow/operators/python.py ## @@ -145,6 +147,141 @@ def execute_callable(self): return self.python_callable(*self.op_args,

[GitHub] [airflow] j-y-matsubara edited a comment on pull request #8867: Fixed BaseSensorOperator to make respect the trigger_rule in downstream tasks, when setting soft_fail="True"

2020-05-22 Thread GitBox
j-y-matsubara edited a comment on pull request #8867: URL: https://github.com/apache/airflow/pull/8867#issuecomment-632606590 @yuqian90 Thank you your advise. I have corrected the document that you pointed out. And.. The continuous-integration/travis-ci/pr failure doesn't

[GitHub] [airflow] turbaszek merged pull request #8466: [AIRFLOW-5262] Update timeout exception to include dag

2020-05-22 Thread GitBox
turbaszek merged pull request #8466: URL: https://github.com/apache/airflow/pull/8466 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8966: 2.0.0-dev - Webserver front page missing dag_id for dag runs and task instances

2020-05-22 Thread GitBox
boring-cyborg[bot] commented on issue #8966: URL: https://github.com/apache/airflow/issues/8966#issuecomment-632615701 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] dimitrijepetrovic opened a new issue #8966: 2.0.0-dev - Webserver front page missing dag_id for dag runs and task instances

2020-05-22 Thread GitBox
dimitrijepetrovic opened a new issue #8966: URL: https://github.com/apache/airflow/issues/8966 When trying to view task_isntance list or dag_run (by clicking on circles on the front page) dag_id field is passed as undefined dags.html (line 430) - value of "d" is undefined `

[GitHub] [airflow] psuanm5030 commented on pull request #5288: AIRFLOW-4512 - New trigger rule for tasks - At least one successful

2020-05-22 Thread GitBox
psuanm5030 commented on pull request #5288: URL: https://github.com/apache/airflow/pull/5288#issuecomment-632626255 Curious if there was any progress or other methods of getting this result? This is an automated message from

[GitHub] [airflow] ashb commented on pull request #7335: [AIRFLOW-6711] Drop plugin support for stat_name_handler

2020-05-22 Thread GitBox
ashb commented on pull request #7335: URL: https://github.com/apache/airflow/pull/7335#issuecomment-632631751 Ohhh, yeah, you may be right. This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-6711) Drop plugin support for stat_name_handler

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113943#comment-17113943 ] ASF GitHub Bot commented on AIRFLOW-6711: - ashb commented on pull request #7335: URL:

[GitHub] [airflow] ashb closed issue #8967: 2.0.0 - db upgrade fails on missing dag_stats

2020-05-22 Thread GitBox
ashb closed issue #8967: URL: https://github.com/apache/airflow/issues/8967 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] ashb edited a comment on pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
ashb edited a comment on pull request #8962: URL: https://github.com/apache/airflow/pull/8962#issuecomment-632646688 > The main question is: do we want to support that? Is this "hack" worth it, and do IDEs get massively confused by this? I think if IDEs support it, and we add tests

[GitHub] [airflow] klsnreddy opened a new pull request #8983: 8876 Provide option to time delta on upstream tasks in TimeDeltaSensor

2020-05-22 Thread GitBox
klsnreddy opened a new pull request #8983: URL: https://github.com/apache/airflow/pull/8983 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes (not needed for documentation

[GitHub] [airflow] dimberman merged pull request #8702: Add context to execution_date_fn in ExternalTaskSensor

2020-05-22 Thread GitBox
dimberman merged pull request #8702: URL: https://github.com/apache/airflow/pull/8702 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (AIRFLOW-6569) Broken sentry integration

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114118#comment-17114118 ] ASF GitHub Bot commented on AIRFLOW-6569: - maxcountryman commented on pull request #7232: URL:

[airflow] branch v1-10-test updated: Pin Version of Azure Cosmos to <4 (#8956)

2020-05-22 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 17150ae Pin Version of Azure Cosmos to

[GitHub] [airflow] tylux commented on pull request #4660: [AIRFLOW-3819] - Allow the configuration of a global default for work…

2020-05-22 Thread GitBox
tylux commented on pull request #4660: URL: https://github.com/apache/airflow/pull/4660#issuecomment-632749722 Can this get revisited? This goes against Kubernetes best practices to rely on a different mechanism i.e dag code to set the resource limits instead of standard pod limits.

[GitHub] [airflow] turbaszek commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-05-22 Thread GitBox
turbaszek commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r429236013 ## File path: airflow/providers/google/cloud/hooks/datafusion.py ## @@ -357,12 +418,45 @@ def list_pipelines( ) return

[GitHub] [airflow] maxcountryman commented on pull request #7232: [AIRFLOW-6569] Flush pending Sentry exceptions before exiting forked process

2020-05-22 Thread GitBox
maxcountryman commented on pull request #7232: URL: https://github.com/apache/airflow/pull/7232#issuecomment-632730483 It seems like the tests that are failing are unrelated to this patch and seem to be the same tests that are also failing on master. What's blocking merging this in

[GitHub] [airflow] turbaszek commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-05-22 Thread GitBox
turbaszek commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r429307560 ## File path: airflow/providers/google/cloud/operators/datafusion.py ## @@ -674,14 +676,23 @@ def execute(self, context: Dict):

[GitHub] [airflow] MichaelJDrK opened a new issue #8972: Duplicate logging

2020-05-22 Thread GitBox
MichaelJDrK opened a new issue #8972: URL: https://github.com/apache/airflow/issues/8972 **Apache Airflow version**: 1.10.10 **What happened**: Use airflow out of the box, create any DAG, run the DAG. Log will contain nested log messages like this (one line):

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8972: Duplicate logging

2020-05-22 Thread GitBox
boring-cyborg[bot] commented on issue #8972: URL: https://github.com/apache/airflow/issues/8972#issuecomment-632754220 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] dimberman closed pull request #8958: Add caching

2020-05-22 Thread GitBox
dimberman closed pull request #8958: URL: https://github.com/apache/airflow/pull/8958 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] ashb opened a new pull request #8973: Run static checks on Python3.6

2020-05-22 Thread GitBox
ashb opened a new pull request #8973: URL: https://github.com/apache/airflow/pull/8973 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes (not needed for documentation

[GitHub] [airflow] potiuk opened a new pull request #8974: Vault has now VaultHook not only SecretBackend

2020-05-22 Thread GitBox
potiuk opened a new pull request #8974: URL: https://github.com/apache/airflow/pull/8974 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes (not needed for documentation

[GitHub] [airflow] yerrysherry opened a new issue #8971: Problem when rotating the fernet keys

2020-05-22 Thread GitBox
yerrysherry opened a new issue #8971: URL: https://github.com/apache/airflow/issues/8971 $ airflow version 1.10.10 Database: CentOS Linux release 7.6.1810 (Core), Webserver - scheduler: lsb_release -a No LSB modules are available. Distributor ID: Ubuntu

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8971: Problem when rotating the fernet keys

2020-05-22 Thread GitBox
boring-cyborg[bot] commented on issue #8971: URL: https://github.com/apache/airflow/issues/8971#issuecomment-632701306 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] turbaszek commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-05-22 Thread GitBox
turbaszek commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r429292921 ## File path: airflow/providers/google/cloud/hooks/datafusion.py ## @@ -386,15 +480,29 @@ def start_pipeline( pipeline_name,

[jira] [Commented] (AIRFLOW-3819) k8s executor - Allow the configuration of a global default for pod resource request/limits

2020-05-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114144#comment-17114144 ] ASF GitHub Bot commented on AIRFLOW-3819: - tylux commented on pull request #4660: URL:

[GitHub] [airflow] ashb commented on pull request #8973: Run static checks on Python3.6

2020-05-22 Thread GitBox
ashb commented on pull request #8973: URL: https://github.com/apache/airflow/pull/8973#issuecomment-632759670 (I'm not sure at all if this is the right fix or not, but thought I'd try it) This is an automated message from

[GitHub] [airflow] potiuk commented on pull request #8974: Vault has now VaultHook not only SecretBackend

2020-05-22 Thread GitBox
potiuk commented on pull request #8974: URL: https://github.com/apache/airflow/pull/8974#issuecomment-632766569 Hey @ashb @kaxil -> here is the first of the Hooks. This one is implemented for Vault. I extracted the client out of the Backend and added all the missing authentication

[GitHub] [airflow] dimberman commented on pull request #8702: Add context to execution_date_fn in ExternalTaskSensor

2020-05-22 Thread GitBox
dimberman commented on pull request #8702: URL: https://github.com/apache/airflow/pull/8702#issuecomment-632705967 Hi @Acehaidrey apologies for the delay. I've merged the PR and will work to backport it on 1.10.11 :) This

[airflow] branch master updated (9a4a2d1 -> b055151)

2020-05-22 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 9a4a2d1 [AIRFLOW-5262] Update timeout exception to include dag (#8466) add b055151 Add context to

[GitHub] [airflow] jhtimmins commented on pull request #8680: Add metric for start/end task run

2020-05-22 Thread GitBox
jhtimmins commented on pull request #8680: URL: https://github.com/apache/airflow/pull/8680#issuecomment-632960556 @Acehaidrey Not sure about increasing the memory, but several of us are actively working to reduce the image size to solve that issue. Hard to give an exact timeline, but can

[GitHub] [airflow] kaxil opened a new pull request #8981: Remove duplicate line from CONTRIBUTING.rst

2020-05-22 Thread GitBox
kaxil opened a new pull request #8981: URL: https://github.com/apache/airflow/pull/8981 Remove duplicate line from CONTRIBUTING.rst --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests

[GitHub] [airflow] Acehaidrey commented on pull request #8680: Add metric for start/end task run

2020-05-22 Thread GitBox
Acehaidrey commented on pull request #8680: URL: https://github.com/apache/airflow/pull/8680#issuecomment-632969149 hey @jhtimmins thanks for the update! That is okay with me. I just don't know how to make this test pass otherwise and @dimberman has asked for it

[GitHub] [airflow] Acehaidrey edited a comment on pull request #8680: Add metric for start/end task run

2020-05-22 Thread GitBox
Acehaidrey edited a comment on pull request #8680: URL: https://github.com/apache/airflow/pull/8680#issuecomment-632969149 hey @jhtimmins thanks for the update! That is okay with me. I just don't know how to make this test pass otherwise and @dimberman has asked for it. do you think we

[GitHub] [airflow] kaxil opened a new pull request #8982: Remove defunct code from setup.py

2020-05-22 Thread GitBox
kaxil opened a new pull request #8982: URL: https://github.com/apache/airflow/pull/8982 Remove defunct code from setup.py --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for

[GitHub] [airflow] samuelkhtu commented on issue #8525: SQLBranchOperator

2020-05-22 Thread GitBox
samuelkhtu commented on issue #8525: URL: https://github.com/apache/airflow/issues/8525#issuecomment-632786479 Quick update. The PR is under review. This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] ashb commented on pull request #8973: Run static checks on Python3.6 for v1-10 branches

2020-05-22 Thread GitBox
ashb commented on pull request #8973: URL: https://github.com/apache/airflow/pull/8973#issuecomment-632802828 yeah, I thought this might be deliberate, bit wondered if it was a pre-commit 1 vs 2 problem. This is an

[GitHub] [airflow] casassg commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
casassg commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r429363622 ## File path: docs/concepts.rst ## @@ -116,6 +116,47 @@ DAGs can be used as context managers to automatically assign new operators to th op.dag

[GitHub] [airflow] jaketf commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-05-22 Thread GitBox
jaketf commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r429358908 ## File path: airflow/providers/google/cloud/hooks/datafusion.py ## @@ -435,20 +435,52 @@ def _get_pipeline( "workflows",

[GitHub] [airflow] jaketf commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-05-22 Thread GitBox
jaketf commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r429345281 ## File path: airflow/providers/google/cloud/hooks/datafusion.py ## @@ -386,15 +480,29 @@ def start_pipeline( pipeline_name,

[GitHub] [airflow] casassg commented on pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
casassg commented on pull request #8962: URL: https://github.com/apache/airflow/pull/8962#issuecomment-632819478 I prefer to avoid doing that. Seems to add quite a bit of code complexity and I doubt it will impact significantly the user experience. I think it should be fine from users to

[GitHub] [airflow] ashb commented on pull request #8973: Run static checks on Python3.6

2020-05-22 Thread GitBox
ashb commented on pull request #8973: URL: https://github.com/apache/airflow/pull/8973#issuecomment-632774494 https://github.com/apache/airflow/runs/699984569?check_suite_focus=true is the problem I was dealing with This is

[GitHub] [airflow] casassg commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-22 Thread GitBox
casassg commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r429339659 ## File path: airflow/operators/python.py ## @@ -145,6 +148,142 @@ def execute_callable(self): return self.python_callable(*self.op_args,

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8846: Adding support for databricks spark python and submit tasks.

2020-05-22 Thread GitBox
boring-cyborg[bot] commented on pull request #8846: URL: https://github.com/apache/airflow/pull/8846#issuecomment-632825269 Awesome work, congrats on your first merged pull request! This is an automated message from the

[GitHub] [airflow] mik-laj merged pull request #8846: Adding support for databricks spark python and submit tasks.

2020-05-22 Thread GitBox
mik-laj merged pull request #8846: URL: https://github.com/apache/airflow/pull/8846 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

  1   2   >