[jira] [Work started] (AIRFLOW-5165) Make Dataproc highly available

[ https://issues.apache.org/jira/browse/AIRFLOW-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-5165 started by Maulik Soneji. -- > Make Dataproc highly available > -- > >

[jira] [Commented] (AIRFLOW-5165) Make Dataproc highly available

[ https://issues.apache.org/jira/browse/AIRFLOW-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904851#comment-16904851 ] ASF GitHub Bot commented on AIRFLOW-5165: - mauliksoneji commented on pull request #5781:

[GitHub] [airflow] mauliksoneji opened a new pull request #5781: [AIRFLOW-5165] make number of dataproc masters configurable

mauliksoneji opened a new pull request #5781: [AIRFLOW-5165] make number of dataproc masters configurable URL: https://github.com/apache/airflow/pull/5781 Make sure you have checked _all_ steps below. ### Jira https://issues.apache.org/jira/browse/AIRFLOW-5165 ###

[jira] [Created] (AIRFLOW-5165) Make Dataproc highly available

Maulik Soneji created AIRFLOW-5165: -- Summary: Make Dataproc highly available Key: AIRFLOW-5165 URL: https://issues.apache.org/jira/browse/AIRFLOW-5165 Project: Apache Airflow Issue Type:

[jira] [Assigned] (AIRFLOW-5165) Make Dataproc highly available

[ https://issues.apache.org/jira/browse/AIRFLOW-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maulik Soneji reassigned AIRFLOW-5165: -- Assignee: Maulik Soneji > Make Dataproc highly available >

[jira] [Created] (AIRFLOW-5164) MongoHook fails to parse MongoDB replica set URIs

Ayla Khan created AIRFLOW-5164: -- Summary: MongoHook fails to parse MongoDB replica set URIs Key: AIRFLOW-5164 URL: https://issues.apache.org/jira/browse/AIRFLOW-5164 Project: Apache Airflow

[GitHub] [airflow] coufon edited a comment on issue #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability

coufon edited a comment on issue #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#issuecomment-520182466 > Pending Issues: > > * Issue displaying SubDags: This bug is because subdags are

[GitHub] [airflow] houqp commented on issue #5731: [AIRFLOW-5117] support refreshing EKS api tokens

houqp commented on issue #5731: [AIRFLOW-5117] support refreshing EKS api tokens URL: https://github.com/apache/airflow/pull/5731#issuecomment-520271784 @dimberman my original patch added comment for every function and Class that can be safely removed once patch lands in upstream. I have

[jira] [Assigned] (AIRFLOW-1665) Airflow webserver/scheduler don't handle database disconnects (mysql)

[ https://issues.apache.org/jira/browse/AIRFLOW-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ry Walker reassigned AIRFLOW-1665: -- Assignee: (was: Ry Walker) > Airflow webserver/scheduler don't handle database

[jira] [Assigned] (AIRFLOW-1665) Airflow webserver/scheduler don't handle database disconnects (mysql)

[ https://issues.apache.org/jira/browse/AIRFLOW-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ry Walker reassigned AIRFLOW-1665: -- Assignee: Ry Walker (was: zgl) > Airflow webserver/scheduler don't handle database

[GitHub] [airflow] ryw commented on issue #2744: [AIRFLOW-1665] Database reconnect logic

ryw commented on issue #2744: [AIRFLOW-1665] Database reconnect logic URL: https://github.com/apache/airflow/pull/2744#issuecomment-520264939 @henrym2 which component(s) of Airflow are you experiencing this issue with? This

[GitHub] [airflow] mik-laj commented on issue #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement

mik-laj commented on issue #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement URL: https://github.com/apache/airflow/pull/5770#issuecomment-520261036 @steven-freed I recommend configuring Travis on your own fork.

[GitHub] [airflow] steven-freed commented on issue #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement

steven-freed commented on issue #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement URL: https://github.com/apache/airflow/pull/5770#issuecomment-520257848 My PR Travis tests were canceled. I may have ran too many test (this is my first PR). Is there any way that I can have my

[GitHub] [airflow] steven-freed edited a comment on issue #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement

steven-freed edited a comment on issue #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement URL: https://github.com/apache/airflow/pull/5770#issuecomment-520257848 My PR Travis tests were canceled. I may have ran too many tests (this is my first PR). Is there any way that I can have

[GitHub] [airflow] maxcountryman opened a new pull request #5780: allow callers to specify custom `ExtraArgs`

maxcountryman opened a new pull request #5780: allow callers to specify custom `ExtraArgs` URL: https://github.com/apache/airflow/pull/5780 This patch allows callers to specify their own `ExtraArgs` dictionary. This is an

[GitHub] [airflow] dimberman commented on issue #5680: [AIRFLOW-5066] allow k8s fieldref substitution

dimberman commented on issue #5680: [AIRFLOW-5066] allow k8s fieldref substitution URL: https://github.com/apache/airflow/pull/5680#issuecomment-520254969 @jsurloppe I agree with @potiuk. Besides that I'm good to merge this.

[GitHub] [airflow] dimberman commented on issue #5765: [AIRFLOW-4412] PodLauncher drops log lines printed by the pod

dimberman commented on issue #5765: [AIRFLOW-4412] PodLauncher drops log lines printed by the pod URL: https://github.com/apache/airflow/pull/5765#issuecomment-520254925 @smrt28 could you rebase/fix failures? otherwise I'm good to merge this.

[GitHub] [airflow] dimberman commented on issue #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

dimberman commented on issue #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#issuecomment-520254645 @potiuk No worries at all! As I've said before, if I had to write documents in Polish the results would not be

[jira] [Commented] (AIRFLOW-5162) GCS Uploads for local files and file data (string or bytes)

[ https://issues.apache.org/jira/browse/AIRFLOW-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904726#comment-16904726 ] ASF GitHub Bot commented on AIRFLOW-5162: - steven-freed commented on pull request #5770:

[GitHub] [airflow] steven-freed opened a new pull request #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement

steven-freed opened a new pull request #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement URL: https://github.com/apache/airflow/pull/5770 **Jira** - [x] My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My

[GitHub] [airflow] steven-freed closed pull request #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement

steven-freed closed pull request #5770: [AIRFLOW-5162] GCS Hook Upload Method Improvement URL: https://github.com/apache/airflow/pull/5770 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-5162) GCS Uploads for local files and file data (string or bytes)

[ https://issues.apache.org/jira/browse/AIRFLOW-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904724#comment-16904724 ] ASF GitHub Bot commented on AIRFLOW-5162: - steven-freed commented on pull request #5770:

[GitHub] [airflow] BasPH opened a new pull request #5779: [AIRLFOW-5163] Default do_xcom_push to False

BasPH opened a new pull request #5779: [AIRLFOW-5163] Default do_xcom_push to False URL: https://github.com/apache/airflow/pull/5779 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[GitHub] [airflow] dimberman commented on issue #5731: [AIRFLOW-5117] support refreshing EKS api tokens

dimberman commented on issue #5731: [AIRFLOW-5117] support refreshing EKS api tokens URL: https://github.com/apache/airflow/pull/5731#issuecomment-520248280 @houqp I'm a bit conflicted about this. On the one hand this is a very important bug that we need to address, on the other hand I'd

[jira] [Updated] (AIRFLOW-5162) GCS Uploads for local files and file data (string or bytes)

[ https://issues.apache.org/jira/browse/AIRFLOW-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Freed updated AIRFLOW-5162: -- Affects Version/s: (was: 1.10.3) (was: 1.10.2)

[jira] [Updated] (AIRFLOW-5163) Default do_xcom_push to False

[ https://issues.apache.org/jira/browse/AIRFLOW-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bas Harenslak updated AIRFLOW-5163: --- Description: By default BaseOperator.do_xcom_push is set to True. This is most of the time

[jira] [Updated] (AIRFLOW-5163) Default do_xcom_push to False and give meaningful name

[ https://issues.apache.org/jira/browse/AIRFLOW-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bas Harenslak updated AIRFLOW-5163: --- Description: By default BaseOperator.do_xcom_push is set to True. This is most of the time

[jira] [Updated] (AIRFLOW-5163) Default do_xcom_push to False and give meaningful name

[ https://issues.apache.org/jira/browse/AIRFLOW-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bas Harenslak updated AIRFLOW-5163: --- Summary: Default do_xcom_push to False and give meaningful name (was: Default

[jira] [Updated] (AIRFLOW-5163) Default do_xcom_push to False

[ https://issues.apache.org/jira/browse/AIRFLOW-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bas Harenslak updated AIRFLOW-5163: --- Summary: Default do_xcom_push to False (was: Default do_xcom_push to False and give

[GitHub] [airflow] stale[bot] commented on issue #5183: [AIRFLOW-4410]Add Non-ssl ldap server support

stale[bot] commented on issue #5183: [AIRFLOW-4410]Add Non-ssl ldap server support URL: https://github.com/apache/airflow/pull/5183#issuecomment-520239222 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity

[jira] [Created] (AIRFLOW-5163) Default do_xcom_push to False

Bas Harenslak created AIRFLOW-5163: -- Summary: Default do_xcom_push to False Key: AIRFLOW-5163 URL: https://issues.apache.org/jira/browse/AIRFLOW-5163 Project: Apache Airflow Issue Type:

[jira] [Updated] (AIRFLOW-5162) GCS Uploads for local files and file data (string or bytes)

[ https://issues.apache.org/jira/browse/AIRFLOW-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Freed updated AIRFLOW-5162: -- Description: Allows for gcs uploads to be done by specifying a local file name or by using

[jira] [Created] (AIRFLOW-5162) GCS Uploads for local files and file data (string or bytes)

Steven Freed created AIRFLOW-5162: - Summary: GCS Uploads for local files and file data (string or bytes) Key: AIRFLOW-5162 URL: https://issues.apache.org/jira/browse/AIRFLOW-5162 Project: Apache

[jira] [Comment Edited] (AIRFLOW-5158) Google Sheet hook

[ https://issues.apache.org/jira/browse/AIRFLOW-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904666#comment-16904666 ] Omar Al-Jadda edited comment on AIRFLOW-5158 at 8/11/19 2:20 PM: -

[jira] [Commented] (AIRFLOW-5158) Google Sheet hook

[ https://issues.apache.org/jira/browse/AIRFLOW-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904666#comment-16904666 ] Omar Al-Jadda commented on AIRFLOW-5158: correct, I've updated. I'm planning to code this, if

[jira] [Updated] (AIRFLOW-5158) Google Sheet hook

[ https://issues.apache.org/jira/browse/AIRFLOW-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omar Al-Jadda updated AIRFLOW-5158: --- Description: Need a hook that can both read and write from a google sheet (was: Need an

[jira] [Updated] (AIRFLOW-5158) Google Sheet hook

[ https://issues.apache.org/jira/browse/AIRFLOW-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omar Al-Jadda updated AIRFLOW-5158: --- Summary: Google Sheet hook (was: Google Sheet Operator) > Google Sheet hook >

[GitHub] [airflow] potiuk commented on issue #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on issue #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#issuecomment-520221392 Thanks @dimberman! Very good comments. I improved the docs quite a bit now. Note also that I am not too

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312734802 ## File path: CONTRIBUTING.md ## @@ -348,17 +365,20 @@ they should give the

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312734433 ## File path: CONTRIBUTING.md ## @@ -583,67 +611,84 @@ class LoginForm(Form):

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312734413 ## File path: CONTRIBUTING.md ## @@ -583,67 +611,84 @@ class LoginForm(Form):

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312734329 ## File path: CONTRIBUTING.md ## @@ -583,67 +611,84 @@ class LoginForm(Form):

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312734333 ## File path: CONTRIBUTING.md ## @@ -583,67 +611,84 @@ class LoginForm(Form):

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312734089 ## File path: CONTRIBUTING.md ## @@ -583,67 +611,84 @@ class LoginForm(Form):

[GitHub] [airflow] mik-laj commented on issue #5770: GCS Hook Upload Method(s)

mik-laj commented on issue #5770: GCS Hook Upload Method(s) URL: https://github.com/apache/airflow/pull/5770#issuecomment-520219486 Please follow pull request's guidelines: https://github.com/apache/airflow/blob/master/CONTRIBUTING.md#pull-request-guidelines This is required for your PR

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312734023 ## File path: CONTRIBUTING.md ## @@ -583,67 +611,84 @@ class LoginForm(Form):

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312733872 ## File path: CONTRIBUTING.md ## @@ -430,6 +450,14 @@ See [Troubleshooting

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312733715 ## File path: CONTRIBUTING.md ## @@ -430,6 +450,14 @@ See [Troubleshooting

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312733679 ## File path: CONTRIBUTING.md ## @@ -430,6 +450,14 @@ See [Troubleshooting

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312733256 ## File path: CONTRIBUTING.md ## @@ -430,6 +450,14 @@ See [Troubleshooting

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312733236 ## File path: CONTRIBUTING.md ## @@ -430,6 +450,14 @@ See [Troubleshooting

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312733150 ## File path: CONTRIBUTING.md ## @@ -430,6 +450,14 @@ See [Troubleshooting

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312733083 ## File path: CONTRIBUTING.md ## @@ -348,17 +365,20 @@ they should give the

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312733090 ## File path: CONTRIBUTING.md ## @@ -348,17 +365,20 @@ they should give the

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312732512 ## File path: CONTRIBUTING.md ## @@ -174,7 +184,14 @@ mkvirtualenv

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312732525 ## File path: CONTRIBUTING.md ## @@ -348,17 +365,20 @@ they should give the

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312732470 ## File path: CONTRIBUTING.md ## @@ -174,7 +184,14 @@ mkvirtualenv

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312732448 ## File path: CONTRIBUTING.md ## @@ -174,7 +184,14 @@ mkvirtualenv

[GitHub] [airflow] potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks

potiuk commented on a change in pull request #5777: [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks URL: https://github.com/apache/airflow/pull/5777#discussion_r312732343 ## File path: CONTRIBUTING.md ## @@ -142,6 +149,9 @@ environment and you can

[jira] [Commented] (AIRFLOW-5158) Google Sheet Operator

[ https://issues.apache.org/jira/browse/AIRFLOW-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904595#comment-16904595 ] jack commented on AIRFLOW-5158: --- I think you mean hook? > Google Sheet Operator > -