[GitHub] [airflow] mik-laj commented on issue #5597: [WIP][AIRFLOW-4964] Add BigQuery Data Transfer Hook and Operator

mik-laj commented on issue #5597: [WIP][AIRFLOW-4964] Add BigQuery Data Transfer Hook and Operator URL: https://github.com/apache/airflow/pull/5597#issuecomment-511676116 Hi. I have operator for this service: https://github.com/PolideaInternal/airflow/pull/86/files But I found one

[GitHub] [airflow] codecov-io commented on issue #5597: [WIP][AIRFLOW-4964] Add BigQuery Data Transfer Hook and Operator

codecov-io commented on issue #5597: [WIP][AIRFLOW-4964] Add BigQuery Data Transfer Hook and Operator URL: https://github.com/apache/airflow/pull/5597#issuecomment-511672797 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5597?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #5597: [WIP][AIRFLOW-4964] Add BigQuery Data Transfer Hook and Operator

codecov-io edited a comment on issue #5597: [WIP][AIRFLOW-4964] Add BigQuery Data Transfer Hook and Operator URL: https://github.com/apache/airflow/pull/5597#issuecomment-511672797 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5597?src=pr=h1) Report > Merging

[GitHub] [airflow] potiuk edited a comment on issue #5461: [AIRFLOW-4835] Refactor render_template

potiuk edited a comment on issue #5461: [AIRFLOW-4835] Refactor render_template URL: https://github.com/apache/airflow/pull/5461#issuecomment-51187 Side comment @nuclearpinguin @BasPH -> I think we should adapt our approach for pylint changes a bit after first experiences with it. This

[GitHub] [airflow] potiuk commented on issue #5461: [AIRFLOW-4835] Refactor render_template

potiuk commented on issue #5461: [AIRFLOW-4835] Refactor render_template URL: https://github.com/apache/airflow/pull/5461#issuecomment-51187 Side comment @nuclearpinguin @BasPH -> I think we should adapt our approach for pylint changes a bit after first experiences with it. This (and

[jira] [Commented] (AIRFLOW-4964) Add BigQuery Data Transfer Hook and Operator

[ https://issues.apache.org/jira/browse/AIRFLOW-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885846#comment-16885846 ] ASF GitHub Bot commented on AIRFLOW-4964: - ryanyuan commented on pull request #5597:

[GitHub] [airflow] ryanyuan opened a new pull request #5597: [WIP][AIRFLOW-4964] Add BigQuery Data Transfer Hook and Operator

ryanyuan opened a new pull request #5597: [WIP][AIRFLOW-4964] Add BigQuery Data Transfer Hook and Operator URL: https://github.com/apache/airflow/pull/5597 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[jira] [Created] (AIRFLOW-4964) Add BigQuery Data Transfer Hook and Operator

Ryan Yuan created AIRFLOW-4964: -- Summary: Add BigQuery Data Transfer Hook and Operator Key: AIRFLOW-4964 URL: https://issues.apache.org/jira/browse/AIRFLOW-4964 Project: Apache Airflow Issue

[jira] [Work started] (AIRFLOW-4964) Add BigQuery Data Transfer Hook and Operator

[ https://issues.apache.org/jira/browse/AIRFLOW-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4964 started by Ryan Yuan. -- > Add BigQuery Data Transfer Hook and Operator >

[GitHub] [airflow] jmcarp commented on issue #5569: [AIRFLOW-4936] Add context manager to set temporary config variables.

jmcarp commented on issue #5569: [AIRFLOW-4936] Add context manager to set temporary config variables. URL: https://github.com/apache/airflow/pull/5569#issuecomment-511664915 Ready for another look @ashb. This is an

[GitHub] [airflow] potiuk edited a comment on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

potiuk edited a comment on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#issuecomment-511661597 And I would not worry about multiprocessing queue problems in this case. In this case we have simple Producer ->

[GitHub] [airflow] potiuk commented on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

potiuk commented on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#issuecomment-511661597 And I would not worry about multiprocessing queue problems in this case. In this case we have simple Producer ->

[GitHub] [airflow] potiuk commented on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

potiuk commented on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#issuecomment-511660265 I like the idea. As @ashb mentioned in JIRA - it likely won't be needed in this form when we implement persisting the

[GitHub] [airflow] codecov-io edited a comment on issue #5569: [AIRFLOW-4936] Add context manager to set temporary config variables.

codecov-io edited a comment on issue #5569: [AIRFLOW-4936] Add context manager to set temporary config variables. URL: https://github.com/apache/airflow/pull/5569#issuecomment-510755850 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5569?src=pr=h1) Report > Merging

[GitHub] [airflow] coufon commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

coufon commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#discussion_r303715835 ## File path: airflow/dag/stringified_dags.py ## @@ -0,0 +1,137 @@ +# -*- coding:

[GitHub] [airflow] mik-laj commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

mik-laj commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#discussion_r303701174 ## File path: airflow/dag/stringified_dags.py ## @@ -0,0 +1,137 @@ +# -*- coding:

[GitHub] [airflow] mik-laj commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

mik-laj commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#discussion_r303700466 ## File path: airflow/dag/stringified_dags.py ## @@ -0,0 +1,137 @@ +# -*- coding:

[GitHub] [airflow] mik-laj commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

mik-laj commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#discussion_r303700466 ## File path: airflow/dag/stringified_dags.py ## @@ -0,0 +1,137 @@ +# -*- coding:

[GitHub] [airflow] coufon commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

coufon commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#discussion_r303684267 ## File path: airflow/dag/stringified_dags.py ## @@ -0,0 +1,137 @@ +# -*- coding:

[GitHub] [airflow] codecov-io commented on issue #5591: AIRFLOW-4959: Add .hql support for the DataProcHiveOperator

codecov-io commented on issue #5591: AIRFLOW-4959: Add .hql support for the DataProcHiveOperator URL: https://github.com/apache/airflow/pull/5591#issuecomment-511594593 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5591?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #5591: AIRFLOW-4959: Add .hql support for the DataProcHiveOperator

codecov-io edited a comment on issue #5591: AIRFLOW-4959: Add .hql support for the DataProcHiveOperator URL: https://github.com/apache/airflow/pull/5591#issuecomment-511594593 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5591?src=pr=h1) Report > Merging

[jira] [Comment Edited] (AIRFLOW-4593) Memory leak in Airflow scheduler

[ https://issues.apache.org/jira/browse/AIRFLOW-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885411#comment-16885411 ] Grant Nicholas edited comment on AIRFLOW-4593 at 7/15/19 10:10 PM: ---

[GitHub] [airflow] Fokko commented on issue #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs

Fokko commented on issue #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs URL: https://github.com/apache/airflow/pull/4633#issuecomment-511587220 @chaimt A few more remarks, sorry for the late response.

[GitHub] [airflow] Fokko commented on a change in pull request #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs

Fokko commented on a change in pull request #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs URL: https://github.com/apache/airflow/pull/4633#discussion_r303654687 ## File path: airflow/contrib/hooks/gcp_dataflow_hook.py ## @@ -47,58

[GitHub] [airflow] Fokko commented on a change in pull request #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs

Fokko commented on a change in pull request #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs URL: https://github.com/apache/airflow/pull/4633#discussion_r303655507 ## File path: airflow/contrib/hooks/gcp_dataflow_hook.py ## @@ -47,58

[GitHub] [airflow] Fokko commented on a change in pull request #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs

Fokko commented on a change in pull request #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs URL: https://github.com/apache/airflow/pull/4633#discussion_r303654008 ## File path: airflow/contrib/operators/dataflow_operator.py ## @@

[GitHub] [airflow] Fokko commented on a change in pull request #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs

Fokko commented on a change in pull request #4633: AIRFLOW-3791: Dataflow - Support check status if pipeline spans on multiple jobs URL: https://github.com/apache/airflow/pull/4633#discussion_r303655911 ## File path: airflow/contrib/hooks/gcp_dataflow_hook.py ## @@ -47,58

[GitHub] [airflow] nuclearpinguin commented on issue #5461: [AIRFLOW-4835] Refactor render_template

nuclearpinguin commented on issue #5461: [AIRFLOW-4835] Refactor render_template URL: https://github.com/apache/airflow/pull/5461#issuecomment-511574298 I just wanted to suggest that maybe after your refactor the files could be remove from the todo list. If there's only few errors left

[GitHub] [airflow] codecov-io edited a comment on issue #5589: [WIP][AIRFLOW-4956] Fix LocalTaskJob heartbeat log spamming

codecov-io edited a comment on issue #5589: [WIP][AIRFLOW-4956] Fix LocalTaskJob heartbeat log spamming URL: https://github.com/apache/airflow/pull/5589#issuecomment-511549148 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5589?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #5589: [WIP][AIRFLOW-4956] Fix LocalTaskJob heartbeat log spamming

codecov-io edited a comment on issue #5589: [WIP][AIRFLOW-4956] Fix LocalTaskJob heartbeat log spamming URL: https://github.com/apache/airflow/pull/5589#issuecomment-511549148 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5589?src=pr=h1) Report > Merging

[jira] [Assigned] (AIRFLOW-4910) KuberenetesExecutor - KubernetesJobWatcher can silently fail

[ https://issues.apache.org/jira/browse/AIRFLOW-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman reassigned AIRFLOW-4910: Assignee: Daniel Imberman > KuberenetesExecutor - KubernetesJobWatcher can

[jira] [Commented] (AIRFLOW-4963) Avoid recreating task context

[ https://issues.apache.org/jira/browse/AIRFLOW-4963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885574#comment-16885574 ] ASF GitHub Bot commented on AIRFLOW-4963: - BasPH commented on pull request #5596:

[GitHub] [airflow] BasPH opened a new pull request #5596: [AIRFLOW-4963] Avoid recreating task context

BasPH opened a new pull request #5596: [AIRFLOW-4963] Avoid recreating task context URL: https://github.com/apache/airflow/pull/5596 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[jira] [Created] (AIRFLOW-4963) Avoid recreating task context

Bas Harenslak created AIRFLOW-4963: -- Summary: Avoid recreating task context Key: AIRFLOW-4963 URL: https://issues.apache.org/jira/browse/AIRFLOW-4963 Project: Apache Airflow Issue Type:

[jira] [Updated] (AIRFLOW-4963) Avoid recreating task context

[ https://issues.apache.org/jira/browse/AIRFLOW-4963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bas Harenslak updated AIRFLOW-4963: --- Labels: performance (was: ) > Avoid recreating task context >

[GitHub] [airflow] codecov-io commented on issue #5589: [WIP][AIRFLOW-4956] Fix LocalTaskJob heartbeat log spamming

codecov-io commented on issue #5589: [WIP][AIRFLOW-4956] Fix LocalTaskJob heartbeat log spamming URL: https://github.com/apache/airflow/pull/5589#issuecomment-511549148 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5589?src=pr=h1) Report > Merging

[GitHub] [airflow] BasPH edited a comment on issue #5461: [AIRFLOW-4835] Refactor render_template

BasPH edited a comment on issue #5461: [AIRFLOW-4835] Refactor render_template URL: https://github.com/apache/airflow/pull/5461#issuecomment-511546744 > Maybe you can remove refactored models from pylint TODO list? @nuclearpinguin Maybe I don't understand correctly, but this only

[jira] [Commented] (AIRFLOW-4954) Remove unused variables from tests

[ https://issues.apache.org/jira/browse/AIRFLOW-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885558#comment-16885558 ] ASF subversion and git services commented on AIRFLOW-4954: -- Commit

[jira] [Commented] (AIRFLOW-4954) Remove unused variables from tests

[ https://issues.apache.org/jira/browse/AIRFLOW-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885557#comment-16885557 ] ASF GitHub Bot commented on AIRFLOW-4954: - potiuk commented on pull request #5588:

[jira] [Resolved] (AIRFLOW-4954) Remove unused variables from tests

[ https://issues.apache.org/jira/browse/AIRFLOW-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4954. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove unused variables from tests >

[GitHub] [airflow] potiuk merged pull request #5588: [AIRFLOW-4954] Remove unused variables in tests

potiuk merged pull request #5588: [AIRFLOW-4954] Remove unused variables in tests URL: https://github.com/apache/airflow/pull/5588 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests

nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests URL: https://github.com/apache/airflow/pull/5586#discussion_r303592391 ## File path: tests/contrib/hooks/test_grpc_hook.py ## @@ -48,21 +48,21 @@ def

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests

nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests URL: https://github.com/apache/airflow/pull/5586#discussion_r303595914 ## File path: tests/operators/test_virtualenv_operator.py ## @@ -194,7 +194,7 @@ def

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests

nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests URL: https://github.com/apache/airflow/pull/5586#discussion_r303593882 ## File path: tests/contrib/operators/test_mysql_to_gcs_operator.py ## @@ -262,7 +262,7 @@

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests

nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests URL: https://github.com/apache/airflow/pull/5586#discussion_r303595609 ## File path: tests/dags/test_task_view_type_check.py ## @@ -40,7 +40,7 @@ def

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests

nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests URL: https://github.com/apache/airflow/pull/5586#discussion_r303590717 ## File path: pylintrc ## @@ -324,7 +324,7 @@

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests

nuclearpinguin commented on a change in pull request #5586: [AIRFLOW-4952] Remove unused arguments in tests URL: https://github.com/apache/airflow/pull/5586#discussion_r303592391 ## File path: tests/contrib/hooks/test_grpc_hook.py ## @@ -48,21 +48,21 @@ def

[GitHub] [airflow] codecov-io commented on issue #5595: [AIRFLOW-4962] Fix Werkzeug v0.15 deprecation notice for DispatcherMi…

codecov-io commented on issue #5595: [AIRFLOW-4962] Fix Werkzeug v0.15 deprecation notice for DispatcherMi… URL: https://github.com/apache/airflow/pull/5595#issuecomment-511525842 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5595?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #5595: [AIRFLOW-4962] Fix Werkzeug v0.15 deprecation notice for DispatcherMi…

codecov-io edited a comment on issue #5595: [AIRFLOW-4962] Fix Werkzeug v0.15 deprecation notice for DispatcherMi… URL: https://github.com/apache/airflow/pull/5595#issuecomment-511525842 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5595?src=pr=h1) Report > Merging

[GitHub] [airflow] mik-laj commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

mik-laj commented on a change in pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#discussion_r303581952 ## File path: airflow/dag/stringified_dags.py ## @@ -0,0 +1,137 @@ +# -*- coding:

[GitHub] [airflow] mik-laj edited a comment on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

mik-laj edited a comment on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#issuecomment-511513081 Thanks. Good job. I will look at it more closely in the near future, but now I would like to point out that we

[GitHub] [airflow] coufon commented on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

coufon commented on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#issuecomment-511521781 > Thanks. Well done. I will look at it more closely in the near future, but now I would like to point out that we only

[GitHub] [airflow] potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3]

potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3] URL: https://github.com/apache/airflow/pull/4938#discussion_r303576669 ## File path: Dockerfile ## @@ -398,6 +398,10 @@ WORKDIR ${AIRFLOW_SOURCES} ENV

[GitHub] [airflow] mik-laj edited a comment on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

mik-laj edited a comment on issue #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594#issuecomment-511513081 Thanks. Well done. I will look at it more closely in the near future, but now I would like to point out that

[jira] [Commented] (AIRFLOW-4962) Fix Werkzeug v0.15 deprecation notice for DispatcherMiddleware import

[ https://issues.apache.org/jira/browse/AIRFLOW-4962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885455#comment-16885455 ] ASF GitHub Bot commented on AIRFLOW-4962: - nritholtz commented on pull request #5595:

[GitHub] [airflow] nritholtz opened a new pull request #5595: [AIRFLOW-4962] Fix Werkzeug v0.15 deprecation notice for DispatcherMi…

nritholtz opened a new pull request #5595: [AIRFLOW-4962] Fix Werkzeug v0.15 deprecation notice for DispatcherMi… URL: https://github.com/apache/airflow/pull/5595 …ddleware import Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the

[jira] [Created] (AIRFLOW-4962) Fix Werkzeug v0.15 deprecation notice for DispatcherMiddleware import

Nathaniel Ritholtz created AIRFLOW-4962: --- Summary: Fix Werkzeug v0.15 deprecation notice for DispatcherMiddleware import Key: AIRFLOW-4962 URL: https://issues.apache.org/jira/browse/AIRFLOW-4962

[jira] [Commented] (AIRFLOW-4724) Make params dict to be templated for operators

[ https://issues.apache.org/jira/browse/AIRFLOW-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885444#comment-16885444 ] Srivignessh Pacham Sri Srinivasan commented on AIRFLOW-4724: I am

[jira] [Commented] (AIRFLOW-4941) default_args not applied when dag is assigned to task through setter

[ https://issues.apache.org/jira/browse/AIRFLOW-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885437#comment-16885437 ] ASF GitHub Bot commented on AIRFLOW-4941: - milton0825 commented on pull request #5574:

[GitHub] [airflow] milton0825 closed pull request #5574: [AIRFLOW-4941] Warn that default_args not applied via setter

milton0825 closed pull request #5574: [AIRFLOW-4941] Warn that default_args not applied via setter URL: https://github.com/apache/airflow/pull/5574 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] milton0825 commented on issue #5574: [AIRFLOW-4941] Warn that default_args not applied via setter

milton0825 commented on issue #5574: [AIRFLOW-4941] Warn that default_args not applied via setter URL: https://github.com/apache/airflow/pull/5574#issuecomment-511499886 @ashb I think actually that is the expected behavior right? Operators not decorated with `apply_default` will not apply

[jira] [Commented] (AIRFLOW-4924) Loading DAGs asynchronously in Airflow webserver

[ https://issues.apache.org/jira/browse/AIRFLOW-4924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885436#comment-16885436 ] ASF GitHub Bot commented on AIRFLOW-4924: - coufon commented on pull request #5594:

[GitHub] [airflow] coufon opened a new pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver

coufon opened a new pull request #5594: [AIRFLOW-4924] Loading DAGs asynchronously in Airflow webserver URL: https://github.com/apache/airflow/pull/5594 ### Jira - [(/)] My PR addresses the following issues: - [Airflow

[jira] [Comment Edited] (AIRFLOW-4584) Error when using ssh operateur to execute a sh script from an remote server

[ https://issues.apache.org/jira/browse/AIRFLOW-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882915#comment-16882915 ] W Hasnaoui edited comment on AIRFLOW-4584 at 7/15/19 5:35 PM: -- Hi [~ash],

[GitHub] [airflow] msumit commented on issue #5570: [AIRFLOW-4939] Add default_task_retries config

msumit commented on issue #5570: [AIRFLOW-4939] Add default_task_retries config URL: https://github.com/apache/airflow/pull/5570#issuecomment-511492468 @ashb tried `importlib.reload` as well, but unfortunately, it's not working as well. Should we just skip the failing test?

[jira] [Comment Edited] (AIRFLOW-4593) Memory leak in Airflow scheduler

[ https://issues.apache.org/jira/browse/AIRFLOW-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885411#comment-16885411 ] Grant Nicholas edited comment on AIRFLOW-4593 at 7/15/19 5:01 PM: -- +1

[jira] [Commented] (AIRFLOW-4593) Memory leak in Airflow scheduler

[ https://issues.apache.org/jira/browse/AIRFLOW-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885411#comment-16885411 ] Grant Nicholas commented on AIRFLOW-4593: - +1 on here. We upgraded airflow  from 1.10.0 to

[GitHub] [airflow] dimberman commented on issue #5481: [AIRFLOW-4851] Refactor K8S codebase with k8s API models

dimberman commented on issue #5481: [AIRFLOW-4851] Refactor K8S codebase with k8s API models URL: https://github.com/apache/airflow/pull/5481#issuecomment-511480075 @davlum I think this looks great. It will make it easier for us to keep up with the k8s API changes/features user request us

[jira] [Commented] (AIRFLOW-4961) SQL Error when writing Task Failure (using CockroachDB)

[ https://issues.apache.org/jira/browse/AIRFLOW-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885399#comment-16885399 ] Fahran Wallace commented on AIRFLOW-4961: - [~ash] yep, that works too, and is a bit neater.

[GitHub] [airflow] ashb commented on issue #5573: [AIRFLOW-4929] Pretty print JSON Variables in UI

ashb commented on issue #5573: [AIRFLOW-4929] Pretty print JSON Variables in UI URL: https://github.com/apache/airflow/pull/5573#issuecomment-511477370 Ah yeah, just add a comment in AIRFLOW-310 pointing at AIRFLOW-4929 then please.

[GitHub] [airflow] eeshugerman commented on a change in pull request #5573: [AIRFLOW-4929] Pretty print JSON Variables in UI

eeshugerman commented on a change in pull request #5573: [AIRFLOW-4929] Pretty print JSON Variables in UI URL: https://github.com/apache/airflow/pull/5573#discussion_r303527698 ## File path: airflow/www/templates/airflow/variable_edit.html ## @@ -0,0 +1,28 @@ +{# +

[GitHub] [airflow] Fokko commented on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill

Fokko commented on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill URL: https://github.com/apache/airflow/pull/5498#issuecomment-511473203 My best guess would be @KevinYang21, otherwise @mistercrunch 

[jira] [Commented] (AIRFLOW-4961) SQL Error when writing Task Failure (using CockroachDB)

[ https://issues.apache.org/jira/browse/AIRFLOW-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885368#comment-16885368 ] ASF GitHub Bot commented on AIRFLOW-4961: - fahran commented on pull request #5593:

[GitHub] [airflow] fahran opened a new pull request #5593: [AIRFLOW-4961] Prevent cast when writing TaskFail model to DB

fahran opened a new pull request #5593: [AIRFLOW-4961] Prevent cast when writing TaskFail model to DB URL: https://github.com/apache/airflow/pull/5593 When writing a task failure record, convert 'duration' decimal value -> an int before persistence, to remove reliance on the database

[GitHub] [airflow] codecov-io commented on issue #5592: [AIRFLOW-4867] check if table exists before creating

codecov-io commented on issue #5592: [AIRFLOW-4867] check if table exists before creating URL: https://github.com/apache/airflow/pull/5592#issuecomment-511462870 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5592?src=pr=h1) Report > Merging

[GitHub] [airflow] dszakallas edited a comment on issue #5592: [AIRFLOW-4867] check if table exists before creating

dszakallas edited a comment on issue #5592: [AIRFLOW-4867] check if table exists before creating URL: https://github.com/apache/airflow/pull/5592#issuecomment-511451534 I updated the airflow version from 1.10.2 to 1.10.4b2. I didn't do any manual migrations. initdb should have taken care

[GitHub] [airflow] ashb commented on issue #5592: [AIRFLOW-4867] check if table exists before creating

ashb commented on issue #5592: [AIRFLOW-4867] check if table exists before creating URL: https://github.com/apache/airflow/pull/5592#issuecomment-511449557 The "up" migration for this table drops those two tables. Do you know how you ended up with this migration being recorded as applied

[jira] [Commented] (AIRFLOW-4867) Cannot downgrade drop_user_and_chart migration script

[ https://issues.apache.org/jira/browse/AIRFLOW-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885320#comment-16885320 ] ASF GitHub Bot commented on AIRFLOW-4867: - dszakallas commented on pull request #5592:

[GitHub] [airflow] dszakallas opened a new pull request #5592: [AIRFLOW-4867] migration script should check if table exists

dszakallas opened a new pull request #5592: [AIRFLOW-4867] migration script should check if table exists URL: https://github.com/apache/airflow/pull/5592 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following

svn commit: r34886 - /dev/airflow/1.10.4rc3/

Author: ash Date: Mon Jul 15 14:18:27 2019 New Revision: 34886 Log: Add artefacts for Airflow 1.10.4rc3 Added: dev/airflow/1.10.4rc3/ dev/airflow/1.10.4rc3/apache-airflow-1.10.4rc3-bin.tar.gz (with props) dev/airflow/1.10.4rc3/apache-airflow-1.10.4rc3-bin.tar.gz.asc

[jira] [Commented] (AIRFLOW-4797) Zombie detection and killing is not deterministic

[ https://issues.apache.org/jira/browse/AIRFLOW-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885271#comment-16885271 ] ASF subversion and git services commented on AIRFLOW-4797: -- Commit

[jira] [Commented] (AIRFLOW-4911) Silence the FORBIDDEN errors from the KubernetesExecutor

[ https://issues.apache.org/jira/browse/AIRFLOW-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885270#comment-16885270 ] ASF subversion and git services commented on AIRFLOW-4911: -- Commit

[jira] [Commented] (AIRFLOW-4884) Roll up import_errors in UI

[ https://issues.apache.org/jira/browse/AIRFLOW-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885268#comment-16885268 ] ASF subversion and git services commented on AIRFLOW-4884: -- Commit

[jira] [Commented] (AIRFLOW-4911) Silence the FORBIDDEN errors from the KubernetesExecutor

[ https://issues.apache.org/jira/browse/AIRFLOW-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885269#comment-16885269 ] ASF subversion and git services commented on AIRFLOW-4911: -- Commit

[jira] [Commented] (AIRFLOW-4961) SQL Error when writing Task Failure (using CockroachDB)

[ https://issues.apache.org/jira/browse/AIRFLOW-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885266#comment-16885266 ] Ash Berlin-Taylor commented on AIRFLOW-4961: We probably don't care about sub-second

[jira] [Updated] (AIRFLOW-4961) SQL Error when writing Task Failure (using CockroachDB)

[ https://issues.apache.org/jira/browse/AIRFLOW-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor updated AIRFLOW-4961: --- Affects Version/s: (was: 1.10.5) (was: 1.10.4) >

[jira] [Updated] (AIRFLOW-4961) SQL Error when writing Task Failure (using CockroachDB)

[ https://issues.apache.org/jira/browse/AIRFLOW-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fahran Wallace updated AIRFLOW-4961: Summary: SQL Error when writing Task Failure (using CockroachDB) (was: SQL Error when

[jira] [Updated] (AIRFLOW-4961) SQL Error when writing Task Failure

[ https://issues.apache.org/jira/browse/AIRFLOW-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fahran Wallace updated AIRFLOW-4961: Description: We're running Airflow against CockroachDB (which is designed to be

[jira] [Updated] (AIRFLOW-4961) SQL Error when writing Task Failure

[ https://issues.apache.org/jira/browse/AIRFLOW-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fahran Wallace updated AIRFLOW-4961: Summary: SQL Error when writing Task Failure (was: Error when writing Task) > SQL Error

[jira] [Created] (AIRFLOW-4961) Error when writing Task

Fahran Wallace created AIRFLOW-4961: --- Summary: Error when writing Task Key: AIRFLOW-4961 URL: https://issues.apache.org/jira/browse/AIRFLOW-4961 Project: Apache Airflow Issue Type: Bug

[jira] [Closed] (AIRFLOW-4921) scheduler stuck with schedule is 2.30am and timezone is daylight saving time

[ https://issues.apache.org/jira/browse/AIRFLOW-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dungnguyen closed AIRFLOW-4921. --- Resolution: Fixed Thanks for looking at this issue. Confirmed that it's working with 1.10.4rc2, I

[jira] [Commented] (AIRFLOW-4748) dagbag exception catching does not print a stack trace

[ https://issues.apache.org/jira/browse/AIRFLOW-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885244#comment-16885244 ] ASF GitHub Bot commented on AIRFLOW-4748: - ashb commented on pull request #5386: AIRFLOW-4748

[GitHub] [airflow] ashb commented on issue #5386: AIRFLOW-4748 Print stacktrace for dagbag process exceptions

ashb commented on issue #5386: AIRFLOW-4748 Print stacktrace for dagbag process exceptions URL: https://github.com/apache/airflow/pull/5386#issuecomment-511408834 Going to close this PR for now as the messages it prints will likely be too long and will make a mess of the UI. Sorry. It'll

[GitHub] [airflow] ashb closed pull request #5386: AIRFLOW-4748 Print stacktrace for dagbag process exceptions

ashb closed pull request #5386: AIRFLOW-4748 Print stacktrace for dagbag process exceptions URL: https://github.com/apache/airflow/pull/5386 This is an automated message from the Apache Git Service. To respond to the

[jira] [Created] (AIRFLOW-4960) Create OpenFaaS Sensor

Itai Yaffe created AIRFLOW-4960: --- Summary: Create OpenFaaS Sensor Key: AIRFLOW-4960 URL: https://issues.apache.org/jira/browse/AIRFLOW-4960 Project: Apache Airflow Issue Type: Improvement

[jira] [Commented] (AIRFLOW-137) Airflow does not respect 'max_active_runs' when task from multiple dag runs cleared

[ https://issues.apache.org/jira/browse/AIRFLOW-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885227#comment-16885227 ] Michael Kyriacou commented on AIRFLOW-137: -- Thanks [~ash], however I am aware of this. Prior to

[jira] [Commented] (AIRFLOW-4797) Zombie detection and killing is not deterministic

[ https://issues.apache.org/jira/browse/AIRFLOW-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885213#comment-16885213 ] ASF subversion and git services commented on AIRFLOW-4797: -- Commit

[GitHub] [airflow] ashb commented on issue #5574: [AIRFLOW-4941] Warn that default_args not applied via setter

ashb commented on issue #5574: [AIRFLOW-4941] Warn that default_args not applied via setter URL: https://github.com/apache/airflow/pull/5574#issuecomment-511393939 Okay I tried out my idea and it works with one caveat. If you define a custom operator and _don't_ decorate with

[GitHub] [airflow] codecov-io edited a comment on issue #5590: [AIRFLOW-XXX] Disable intersphinx loading of `requests` modules

codecov-io edited a comment on issue #5590: [AIRFLOW-XXX] Disable intersphinx loading of `requests` modules URL: https://github.com/apache/airflow/pull/5590#issuecomment-511383488 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5590?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #5590: [AIRFLOW-XXX] Disable intersphinx loading of `requests` modules

codecov-io commented on issue #5590: [AIRFLOW-XXX] Disable intersphinx loading of `requests` modules URL: https://github.com/apache/airflow/pull/5590#issuecomment-511383488 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5590?src=pr=h1) Report > Merging

  1   2   >