[GitHub] [airflow] brandonwillard opened a new pull request #7405: [AIRFLOW-6780] Add option to use args instead of command in K8s pod spec

2020-02-11 Thread GitBox
brandonwillard opened a new pull request #7405: [AIRFLOW-6780] Add option to use args instead of command in K8s pod spec URL: https://github.com/apache/airflow/pull/7405 This PR adds an option, `kubernetes.worker_pod_uses_args`, that makes `AirflowKubernetesScheduler.run_next` use the

[jira] [Commented] (AIRFLOW-6780) Use args Instead of command in Kubernetes Executor Worker Pods

2020-02-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035097#comment-17035097 ] ASF GitHub Bot commented on AIRFLOW-6780: - brandonwillard commented on pull request #7405:

[GitHub] [airflow] dimberman commented on issue #7403: can launch KubernetesExecutor pods from CeleryExecutor

2020-02-11 Thread GitBox
dimberman commented on issue #7403: can launch KubernetesExecutor pods from CeleryExecutor URL: https://github.com/apache/airflow/pull/7403#issuecomment-585049197 @mik-laj It does. I think there are certain use-cases that the CeleryExecutor doesn't cover, but between the KEDA autoscaler

[jira] [Closed] (AIRFLOW-6742) Task instance state set to failed even though the Pod succeeded when using KubernetesExecutor

2020-02-11 Thread Andrew Cleland (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Cleland closed AIRFLOW-6742. --- Resolution: Not A Problem > Task instance state set to failed even though the Pod

[jira] [Commented] (AIRFLOW-6742) Task instance state set to failed even though the Pod succeeded when using KubernetesExecutor

2020-02-11 Thread Andrew Cleland (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035040#comment-17035040 ] Andrew Cleland commented on AIRFLOW-6742: - This issue was occurring due to incorrect

[jira] [Commented] (AIRFLOW-6781) Enforce example_dags and guide for Google integrations

2020-02-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035038#comment-17035038 ] ASF GitHub Bot commented on AIRFLOW-6781: - mik-laj commented on pull request #7404:

[GitHub] [airflow] mik-laj opened a new pull request #7404: [AIRFLOW-6781] Enforce example_dags and guide for Google integrations

2020-02-11 Thread GitBox
mik-laj opened a new pull request #7404: [AIRFLOW-6781] Enforce example_dags and guide for Google integrations URL: https://github.com/apache/airflow/pull/7404 Hello, For operators from the Google package we have examples of DAGs and guides for the majority of integrations. Ther

[jira] [Created] (AIRFLOW-6781) Enforce example_dags and guide for Google integrations

2020-02-11 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6781: -- Summary: Enforce example_dags and guide for Google integrations Key: AIRFLOW-6781 URL: https://issues.apache.org/jira/browse/AIRFLOW-6781 Project: Apache Airflow

[GitHub] [airflow] mik-laj commented on issue #7403: can launch KubernetesExecutor pods from CeleryExecutor

2020-02-11 Thread GitBox
mik-laj commented on issue #7403: can launch KubernetesExecutor pods from CeleryExecutor URL: https://github.com/apache/airflow/pull/7403#issuecomment-585019626 @dimberman Does this PR solve the same use case? https://github.com/apache/airflow/pull/6725

[GitHub] [airflow] dimberman commented on issue #7403: can launch KubernetesExecutor pods from CeleryExecutor

2020-02-11 Thread GitBox
dimberman commented on issue #7403: can launch KubernetesExecutor pods from CeleryExecutor URL: https://github.com/apache/airflow/pull/7403#issuecomment-585016788 @ashb I have no idea how we're gonna integration test this but I think this would be a really valuable feature.

[GitHub] [airflow] dimberman opened a new pull request #7403: can launch KubernetesExecutor pods from CeleryExecutor

2020-02-11 Thread GitBox
dimberman opened a new pull request #7403: can launch KubernetesExecutor pods from CeleryExecutor URL: https://github.com/apache/airflow/pull/7403 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Created] (AIRFLOW-6780) Use args Instead of command in Kubernetes Executor Worker Pods

2020-02-11 Thread Brandon Willard (Jira)
Brandon Willard created AIRFLOW-6780: Summary: Use args Instead of command in Kubernetes Executor Worker Pods Key: AIRFLOW-6780 URL: https://issues.apache.org/jira/browse/AIRFLOW-6780 Project:

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Commented] (AIRFLOW-6779) Schedule stopped scheduling tasks with concurrency limit or depends on past

2020-02-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034969#comment-17034969 ] ASF GitHub Bot commented on AIRFLOW-6779: - houqp commented on pull request #7402:

[GitHub] [airflow] houqp opened a new pull request #7402: [AIRFLOW-6779] fix scheduler bug related to concurrency and depends on past

2020-02-11 Thread GitBox
houqp opened a new pull request #7402: [AIRFLOW-6779] fix scheduler bug related to concurrency and depends on past URL: https://github.com/apache/airflow/pull/7402 commit 50efda5c69c1ddfaa869b408540182fb19f1a286 introduced a bug that prevents scheduler from scheduling tasks with the

[jira] [Created] (AIRFLOW-6779) Schedule stopped scheduling tasks with concurrency limit or depends on past

2020-02-11 Thread QP Hou (Jira)
QP Hou created AIRFLOW-6779: --- Summary: Schedule stopped scheduling tasks with concurrency limit or depends on past Key: AIRFLOW-6779 URL: https://issues.apache.org/jira/browse/AIRFLOW-6779 Project: Apache

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Labels: kubernetes options (was: ) > Add a DAGs PVC Mount Point Option for Workers

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Updated] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard updated AIRFLOW-6778: - Description: The worker pods generated by the Kubernetes Executor force the DAGs PVC

[jira] [Assigned] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Willard reassigned AIRFLOW-6778: Assignee: Brandon Willard > Add a DAGs PVC Mount Point Option for Workers under

[GitHub] [airflow] brandonwillard opened a new pull request #7401: Add a configurable DAGs volume mount path for Kubernetes

2020-02-11 Thread GitBox
brandonwillard opened a new pull request #7401: Add a configurable DAGs volume mount path for Kubernetes URL: https://github.com/apache/airflow/pull/7401 Added a `kubernetes.dags_volume_mount_point` option. --- Issue link: WILL BE INSERTED BY

[jira] [Created] (AIRFLOW-6778) Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor

2020-02-11 Thread Brandon Willard (Jira)
Brandon Willard created AIRFLOW-6778: Summary: Add a DAGs PVC Mount Point Option for Workers under Kubernetes Executor Key: AIRFLOW-6778 URL: https://issues.apache.org/jira/browse/AIRFLOW-6778

[jira] [Created] (AIRFLOW-6777) Missing Hyperparameter Tuning Operator/Hook for MLEngine

2020-02-11 Thread Mark Zhang (Jira)
Mark Zhang created AIRFLOW-6777: --- Summary: Missing Hyperparameter Tuning Operator/Hook for MLEngine Key: AIRFLOW-6777 URL: https://issues.apache.org/jira/browse/AIRFLOW-6777 Project: Apache Airflow

[GitHub] [airflow] codecov-io edited a comment on issue #7251: [AIRFLOW-6628] Fix search auto complete behaviour

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #7251: [AIRFLOW-6628] Fix search auto complete behaviour URL: https://github.com/apache/airflow/pull/7251#issuecomment-578727497 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7251?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7251: [AIRFLOW-6628] Fix search auto complete behaviour

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #7251: [AIRFLOW-6628] Fix search auto complete behaviour URL: https://github.com/apache/airflow/pull/7251#issuecomment-578727497 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7251?src=pr=h1) Report > Merging

[GitHub] [airflow] mik-laj commented on issue #5519: [AIRFLOW-4543] Update slack operator to support slackclient v2

2020-02-11 Thread GitBox
mik-laj commented on issue #5519: [AIRFLOW-4543] Update slack operator to support slackclient v2 URL: https://github.com/apache/airflow/pull/5519#issuecomment-584855613 Airflow 2.0 will only support Python 3+, so we can now update this library. If you do it quite quickly, they will be

[GitHub] [airflow] mrbungie edited a comment on issue #5519: [AIRFLOW-4543] Update slack operator to support slackclient v2

2020-02-11 Thread GitBox
mrbungie edited a comment on issue #5519: [AIRFLOW-4543] Update slack operator to support slackclient v2 URL: https://github.com/apache/airflow/pull/5519#issuecomment-584847056 > Closing this PR as https://pypi.org/project/slackclient/ requires py36. Is there a roadmap regarding

[GitHub] [airflow] mrbungie edited a comment on issue #5519: [AIRFLOW-4543] Update slack operator to support slackclient v2

2020-02-11 Thread GitBox
mrbungie edited a comment on issue #5519: [AIRFLOW-4543] Update slack operator to support slackclient v2 URL: https://github.com/apache/airflow/pull/5519#issuecomment-584847056 > Closing this PR as https://pypi.org/project/slackclient/ requires py36. Is there a roadmap regarding

[GitHub] [airflow] mrbungie commented on issue #5519: [AIRFLOW-4543] Update slack operator to support slackclient v2

2020-02-11 Thread GitBox
mrbungie commented on issue #5519: [AIRFLOW-4543] Update slack operator to support slackclient v2 URL: https://github.com/apache/airflow/pull/5519#issuecomment-584847056 > Closing this PR as https://pypi.org/project/slackclient/ requires py36. Is there a roadmap regarding this

[jira] [Assigned] (AIRFLOW-5548) REST API: get DAGs

2020-02-11 Thread Matt Buell (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Buell reassigned AIRFLOW-5548: --- Assignee: Matt Buell (was: Norbert Biczo) > REST API: get DAGs > -- > >

[GitHub] [airflow] stale[bot] commented on issue #6837: [AIRFLOW-6266] Better asserts in test_views.py

2020-02-11 Thread GitBox
stale[bot] commented on issue #6837: [AIRFLOW-6266] Better asserts in test_views.py URL: https://github.com/apache/airflow/pull/6837#issuecomment-584829462 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity

[GitHub] [airflow] stale[bot] commented on issue #6820: [AIRFLOW-6255] Redis Hook Refactor

2020-02-11 Thread GitBox
stale[bot] commented on issue #6820: [AIRFLOW-6255] Redis Hook Refactor URL: https://github.com/apache/airflow/pull/6820#issuecomment-584803026 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank

[jira] [Created] (AIRFLOW-6776) Celery Executor Creates Zombie Tasks on Worker Death

2020-02-11 Thread Kimberly Orr (Jira)
Kimberly Orr created AIRFLOW-6776: - Summary: Celery Executor Creates Zombie Tasks on Worker Death Key: AIRFLOW-6776 URL: https://issues.apache.org/jira/browse/AIRFLOW-6776 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-4238) Experimental API sometimes responds with Bad Request

2020-02-11 Thread Colin Ingarfield (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034690#comment-17034690 ] Colin Ingarfield commented on AIRFLOW-4238: --- Can you explain the need for

[jira] [Comment Edited] (AIRFLOW-4238) Experimental API sometimes responds with Bad Request

2020-02-11 Thread Colin Ingarfield (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034690#comment-17034690 ] Colin Ingarfield edited comment on AIRFLOW-4238 at 2/11/20 6:20 PM:

[GitHub] [airflow] davlum edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
davlum edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-584765424 Thanks guys! This is an automated

[GitHub] [airflow] davlum commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
davlum commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-584765424 thx guise! This is an automated message from

[jira] [Commented] (AIRFLOW-5413) Get pod configuration from JSON or Yaml

2020-02-11 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034660#comment-17034660 ] ASF subversion and git services commented on AIRFLOW-5413: -- Commit

[GitHub] [airflow] kaxil commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
kaxil commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-584764110 Good one @davlum This is an automated message

[jira] [Commented] (AIRFLOW-5413) Get pod configuration from JSON or Yaml

2020-02-11 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034659#comment-17034659 ] ASF subversion and git services commented on AIRFLOW-5413: -- Commit

[jira] [Resolved] (AIRFLOW-5413) Get pod configuration from JSON or Yaml

2020-02-11 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman resolved AIRFLOW-5413. -- Resolution: Fixed > Get pod configuration from JSON or Yaml >

[jira] [Commented] (AIRFLOW-5413) Get pod configuration from JSON or Yaml

2020-02-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034657#comment-17034657 ] ASF GitHub Bot commented on AIRFLOW-5413: - dimberman commented on pull request #6230:

[GitHub] [airflow] dimberman commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
dimberman commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-584763785 @davlum congrats! This is an automated

[GitHub] [airflow] dimberman merged pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
dimberman merged pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230 This is an automated message from the Apache Git Service. To

[GitHub] [airflow] codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-537654940 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6230?src=pr=h1) Report > :exclamation: No

[GitHub] [airflow] codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-537654940 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6230?src=pr=h1) Report > :exclamation: No

[GitHub] [airflow] codecov-io commented on issue #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
codecov-io commented on issue #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#issuecomment-584739074 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7400?src=pr=h1) Report > Merging

[GitHub] [airflow] MarkYHZhang commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
MarkYHZhang commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377767498 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@

[GitHub] [airflow] MarkYHZhang commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
MarkYHZhang commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377767498 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@

[GitHub] [airflow] MarkYHZhang commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
MarkYHZhang commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377766618 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@

[GitHub] [airflow] ashb commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-11 Thread GitBox
ashb commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-584735108 I've got the wheel files built by CI and am going to check out they install and uninstall in all the different permutations I am think of

[GitHub] [airflow] mik-laj commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
mik-laj commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377755518 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@ -1015,3

[GitHub] [airflow] codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-537654940 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6230?src=pr=h1) Report > :exclamation: No

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377745677 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377745225 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@

[GitHub] [airflow] ashb commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
ashb commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r377745002 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ##

[GitHub] [airflow] dimberman commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
dimberman commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r377744737 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377744740 ## File path: airflow/providers/google/cloud/hooks/mlengine.py ## @@

[GitHub] [airflow] joeyfreund commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
joeyfreund commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377743755 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@

[GitHub] [airflow] joeyfreund commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
joeyfreund commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377742121 ## File path: airflow/providers/google/cloud/hooks/mlengine.py ## @@ -148,6

[GitHub] [airflow] codecov-io edited a comment on issue #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-584201606 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7391?src=pr=h1) Report > Merging

[GitHub] [airflow] ashb commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
ashb commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r377740018 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ##

[GitHub] [airflow] codecov-io edited a comment on issue #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#issuecomment-583511253 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7370?src=pr=h1) Report > :exclamation: No coverage uploaded for pull

[GitHub] [airflow] ashb commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
ashb commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r377739066 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ##

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r377737240 ## File path: airflow/providers/google/cloud/hooks/mlengine.py ## @@

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
boring-cyborg[bot] commented on issue #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#issuecomment-584712021 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any

[jira] [Commented] (AIRFLOW-6759) Missing cancel job operator/hook for MLEngine

2020-02-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034569#comment-17034569 ] ASF GitHub Bot commented on AIRFLOW-6759: - MarkYHZhang commented on pull request #7400:

[GitHub] [airflow] MarkYHZhang opened a new pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-11 Thread GitBox
MarkYHZhang opened a new pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400 Added `MLEngineTrainingJobFailureOperator` with `cancel_job` hook for MLEngine. --- Issue link: WILL BE INSERTED BY

[GitHub] [airflow] codecov-io edited a comment on issue #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#issuecomment-583511253 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7370?src=pr=h1) Report > :exclamation: No coverage uploaded for pull

[jira] [Created] (AIRFLOW-6775) ExecutorConfig for KubernetesExecutor does not allow for setting SecurityContext

2020-02-11 Thread Daniel Imberman (Jira)
Daniel Imberman created AIRFLOW-6775: Summary: ExecutorConfig for KubernetesExecutor does not allow for setting SecurityContext Key: AIRFLOW-6775 URL: https://issues.apache.org/jira/browse/AIRFLOW-6775

[GitHub] [airflow] potiuk commented on issue #7252: [AIRFLOW-6531] Initial Yandex.Cloud Dataproc support

2020-02-11 Thread GitBox
potiuk commented on issue #7252: [AIRFLOW-6531] Initial Yandex.Cloud Dataproc support URL: https://github.com/apache/airflow/pull/7252#issuecomment-584699844 Sorry - I've been swamped recently, I will take a look soon (unless @nuclearpinguin finishes his review ;) )

[GitHub] [airflow] ashb commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-11 Thread GitBox
ashb commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r377716910 ## File path: airflow/kubernetes/pod_generator.py ## @@ -445,12 +424,31 @@ def

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-11 Thread GitBox
nuclearpinguin commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#discussion_r377711286 ## File path: airflow/config_templates/config.yml ## @@ -495,6 +495,13 @@ type: string

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-11 Thread GitBox
nuclearpinguin commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#discussion_r377710664 ## File path: scripts/perf/sql_queries.py ## @@ -0,0 +1,178 @@ +# Licensed to the Apache

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-11 Thread GitBox
nuclearpinguin commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#discussion_r377698012 ## File path: tests/models/test_dagrun.py ## @@ -227,15 +227,14 @@ def

[jira] [Commented] (AIRFLOW-6773) Creating users with Airflow CLI leaves the password in clear text in the logs

2020-02-11 Thread Jira
[ https://issues.apache.org/jira/browse/AIRFLOW-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034458#comment-17034458 ] Noël BARDELOT commented on AIRFLOW-6773: See also AIRFLOW-6774 > Creating users with Airflow

[jira] [Created] (AIRFLOW-6774) Creating connections with Airflow CLI leaves sensitive data in clear text in the logs

2020-02-11 Thread Jira
Noël BARDELOT created AIRFLOW-6774: -- Summary: Creating connections with Airflow CLI leaves sensitive data in clear text in the logs Key: AIRFLOW-6774 URL: https://issues.apache.org/jira/browse/AIRFLOW-6774

[jira] [Created] (AIRFLOW-6773) Creating users with Airflow CLI leaves the password in clear text in the logs

2020-02-11 Thread Jira
Noël BARDELOT created AIRFLOW-6773: -- Summary: Creating users with Airflow CLI leaves the password in clear text in the logs Key: AIRFLOW-6773 URL: https://issues.apache.org/jira/browse/AIRFLOW-6773

[jira] [Created] (AIRFLOW-6772) {celery_executor.py:224} ERROR - Error sending Celery task:No module named 'MySQLdb'

2020-02-11 Thread Nitin Vashisth (Jira)
Nitin Vashisth created AIRFLOW-6772: --- Summary: {celery_executor.py:224} ERROR - Error sending Celery task:No module named 'MySQLdb' Key: AIRFLOW-6772 URL: https://issues.apache.org/jira/browse/AIRFLOW-6772

[GitHub] [airflow] ashb commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-11 Thread GitBox
ashb commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#discussion_r377595801 ## File path: airflow/utils/sqlalchemy.py ## @@ -20,48 +20,73 @@ import json import logging import os

[GitHub] [airflow] ashb commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-11 Thread GitBox
ashb commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#discussion_r377594151 ## File path: scripts/perf/sql_queries.py ## @@ -0,0 +1,178 @@ +# Licensed to the Apache Software

[GitHub] [airflow-site] mik-laj commented on a change in pull request #246: Expand the guide for contributors

2020-02-11 Thread GitBox
mik-laj commented on a change in pull request #246: Expand the guide for contributors URL: https://github.com/apache/airflow-site/pull/246#discussion_r377593539 ## File path: CONTRIBUTE.md ## @@ -54,49 +55,120 @@ └── sphinx_airflow_theme ``` -## Working with the

[jira] [Commented] (AIRFLOW-6649) Google storage to Snowflake

2020-02-11 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034360#comment-17034360 ] Kamil Bregula commented on AIRFLOW-6649: I agree, but I know that the internal implementation

[GitHub] [airflow] codecov-io edited a comment on issue #6660: [AIRFLOW-6065] Add Stackdriver Task Handler

2020-02-11 Thread GitBox
codecov-io edited a comment on issue #6660: [AIRFLOW-6065] Add Stackdriver Task Handler URL: https://github.com/apache/airflow/pull/6660#issuecomment-561766454 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6660?src=pr=h1) Report > Merging

[GitHub] [airflow] stale[bot] commented on issue #6507: [DO NOT MERGE] POC of adding backportable packages

2020-02-11 Thread GitBox
stale[bot] commented on issue #6507: [DO NOT MERGE] POC of adding backportable packages URL: https://github.com/apache/airflow/pull/6507#issuecomment-584586399 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further

[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-11 Thread GitBox
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-584570933 ``` root@1b70ba426ae0:/opt/airflow# python3 -c 'import hashlib; import mysql.connector'

[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-11 Thread GitBox
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-584570933 ``` root@1b70ba426ae0:/opt/airflow# python3 -c 'import hashlib; import mysql.connector'

[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-11 Thread GitBox
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-584570933 ``` root@1b70ba426ae0:/opt/airflow# python3 -c 'import hashlib; import mysql.connector'

[GitHub] [airflow] feluelle commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-11 Thread GitBox
feluelle commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-584570933 ``` root@1b70ba426ae0:/opt/airflow# python3 -c 'import hashlib; import mysql.connector'

[GitHub] [airflow] feluelle commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-11 Thread GitBox
feluelle commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-584565979 Will try that, thanks. This is an

[GitHub] [airflow] potiuk edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-11 Thread GitBox
potiuk edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-584564040 > @potiuk I tested it with breeze and it does not occur there - only in the fresh travis breeze

[GitHub] [airflow] potiuk commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-11 Thread GitBox
potiuk commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-584564040 > @potiuk I tested it with breeze and it does not occur there - only in the fresh travis breeze env. :/

[jira] [Commented] (AIRFLOW-6770) Unable to run specific test using breeze CLI

2020-02-11 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034323#comment-17034323 ] ASF subversion and git services commented on AIRFLOW-6770: -- Commit

[jira] [Commented] (AIRFLOW-6770) Unable to run specific test using breeze CLI

2020-02-11 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034321#comment-17034321 ] ASF subversion and git services commented on AIRFLOW-6770: -- Commit

  1   2   >