[GitHub] [airflow] potiuk commented on a change in pull request #6681: [AIRFLOW-6089] Reorder setup.py dependencies and add ci

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6681: [AIRFLOW-6089] Reorder setup.py dependencies and add ci URL: https://github.com/apache/airflow/pull/6681#discussion_r352274461 ## File path: BREEZE.rst ## @@ -886,7 +886,7 @@ This is the current syntax for `./breeze

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352292059 ## File path: airflow/executors/celery_executor.py ## @@ -256,6 +260,12 @@ def

[GitHub] [airflow] codecov-io edited a comment on issue #6694: AIRFLOW-6132 - Allow to pass in tags

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6694: AIRFLOW-6132 - Allow to pass in tags URL: https://github.com/apache/airflow/pull/6694#issuecomment-559975397 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6694?src=pr=h1) Report > Merging

[GitHub] [airflow] zhongjiajie commented on issue #6681: [AIRFLOW-6089] Reorder setup.py dependencies and add ci

2019-11-30 Thread GitBox
zhongjiajie commented on issue #6681: [AIRFLOW-6089] Reorder setup.py dependencies and add ci URL: https://github.com/apache/airflow/pull/6681#issuecomment-560003116 Thanks Jarek's review and merge This is an automated

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299109 ## File path: airflow/executors/base_executor.py ## @@ -16,67 +14,82 @@ # KIND, either

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299235 ## File path: airflow/executors/base_executor.py ## @@ -16,67 +14,82 @@ # KIND, either

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352300164 ## File path: docs/plugins.rst ## @@ -167,7 +167,7 @@ definitions in Airflow. from

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299475 ## File path: airflow/executors/kubernetes_executor.py ## @@ -715,6 +749,10 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299255 ## File path: airflow/executors/base_executor.py ## @@ -132,47 +151,64 @@ def

[jira] [Assigned] (AIRFLOW-6010) Remove cyclic imports and pylint hacks

2019-11-30 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk reassigned AIRFLOW-6010: - Assignee: Jarek Potiuk > Remove cyclic imports and pylint hacks >

[jira] [Assigned] (AIRFLOW-6003) Cyclic imports detected by pylint randomly

2019-11-30 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk reassigned AIRFLOW-6003: - Assignee: Jarek Potiuk > Cyclic imports detected by pylint randomly >

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299639 ## File path: airflow/executors/local_executor.py ## @@ -201,31 +273,50 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352298971 ## File path: airflow/executors/all_executors.py ## @@ -0,0 +1,88 @@ +# Licensed to the

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299315 ## File path: airflow/executors/base_executor.py ## @@ -185,25 +221,22 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #6696: [AIRFLOW-6128] Untangle plugins_manager and web from cyclic dependencies. Based on [AIRFLOW-6004]

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6696: [AIRFLOW-6128] Untangle plugins_manager and web from cyclic dependencies. Based on [AIRFLOW-6004] URL: https://github.com/apache/airflow/pull/6696#discussion_r352300952 ## File path:

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352302856 ## File path: airflow/executors/base_executor.py ## @@ -132,47 +151,64 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352303531 ## File path: airflow/executors/kubernetes_executor.py ## @@ -703,10 +735,12 @@ def

[GitHub] [airflow] yk5 commented on a change in pull request #6264: [AIRFLOW-5600] Add MLEngine system tests

2019-11-30 Thread GitBox
yk5 commented on a change in pull request #6264: [AIRFLOW-5600] Add MLEngine system tests URL: https://github.com/apache/airflow/pull/6264#discussion_r352304842 ## File path: airflow/gcp/example_dags/example_mlengine.py ## @@ -0,0 +1,219 @@ +# -*- coding: utf-8 -*- +# +#

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352307296 ## File path: airflow/kubernetes/pod_generator.py ## @@ -26,35 +26,7 @@ import

[jira] [Created] (AIRFLOW-6137) Remove refuse after CLI refactor

2019-11-30 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6137: -- Summary: Remove refuse after CLI refactor Key: AIRFLOW-6137 URL: https://issues.apache.org/jira/browse/AIRFLOW-6137 Project: Apache Airflow Issue Type:

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352307650 ## File path: airflow/utils/dag_processing.py ## @@ -1394,3 +1196,8 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352308822 ## File path: airflow/executors/dask_executor.py ## @@ -84,15 +91,18 @@ def

[GitHub] [airflow] codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-558211492 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6627?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-558211492 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6627?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-558211492 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6627?src=pr=h1) Report > Merging

[jira] [Updated] (AIRFLOW-6128) Untangle "plugins_manager" and "web" cyclic dependencies

2019-11-30 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6128: -- Summary: Untangle "plugins_manager" and "web" cyclic dependencies (was: Untangle

[jira] [Updated] (AIRFLOW-6128) Untangle "plugins_manager" and "web" cyclic dependencies

2019-11-30 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6128: -- Description: Plugins manager was deeply entangled via cyclic dependencies with

[GitHub] [airflow] codecov-io edited a comment on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#issuecomment-559224555 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6596?src=pr=h1) Report > Merging

[GitHub] [airflow] potiuk commented on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#issuecomment-560017527 > Thank you very much - I ask because I've used git bisect a couple of times over the last few weeks on Airflow to track

[GitHub] [airflow] potiuk edited a comment on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk edited a comment on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#issuecomment-560017527 > Thank you very much - I ask because I've used git bisect a couple of times over the last few weeks on Airflow

[GitHub] [airflow] yk5 commented on a change in pull request #6264: [AIRFLOW-5600] Add MLEngine system tests

2019-11-30 Thread GitBox
yk5 commented on a change in pull request #6264: [AIRFLOW-5600] Add MLEngine system tests URL: https://github.com/apache/airflow/pull/6264#discussion_r352304900 ## File path: airflow/gcp/example_dags/example_mlengine.py ## @@ -0,0 +1,219 @@ +# -*- coding: utf-8 -*- +# +#

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352308004 ## File path: docs/plugins.rst ## @@ -167,7 +167,7 @@ definitions in Airflow.

[GitHub] [airflow] stale[bot] commented on issue #5914: [AIRFLOW-5311] Add an AWS Lambda Operator

2019-11-30 Thread GitBox
stale[bot] commented on issue #5914: [AIRFLOW-5311] Add an AWS Lambda Operator URL: https://github.com/apache/airflow/pull/5914#issuecomment-560008785 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299444 ## File path: airflow/executors/kubernetes_executor.py ## @@ -703,10 +735,12 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299553 ## File path: airflow/executors/local_executor.py ## @@ -43,40 +42,44 @@ This option

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299790 ## File path: airflow/kubernetes/pod_generator.py ## @@ -26,35 +26,7 @@ import

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299527 ## File path: airflow/executors/kubernetes_executor.py ## @@ -820,8 +877,14 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299335 ## File path: airflow/executors/celery_executor.py ## @@ -57,17 +61,17 @@

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299726 ## File path: airflow/jobs/scheduler_job.py ## @@ -35,21 +35,24 @@ from sqlalchemy

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352300093 ## File path: airflow/utils/dag_processing.py ## @@ -1394,3 +1196,8 @@ def

[GitHub] [airflow] ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
ashb commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352299897 ## File path: airflow/models/dagbag.py ## @@ -317,9 +317,7 @@ def kill_zombies(self,

[GitHub] [airflow-site] ashb commented on issue #215: [#2] Add .asf.yaml to add repo description

2019-11-30 Thread GitBox
ashb commented on issue #215: [#2] Add .asf.yaml to add repo description URL: https://github.com/apache/airflow-site/pull/215#issuecomment-560014786 Oh man, I didn't know asf would automatically publish from pelican for us

[GitHub] [airflow] potiuk commented on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#issuecomment-560014934 > It also seems that about 2/3 of the files have change s unrelated to removing cyclic imports! > > I am a strong

[GitHub] [airflow] mik-laj commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
mik-laj commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352308080 ## File path: airflow/executors/dask_executor.py ## @@ -84,15 +91,18 @@ def

[GitHub] [airflow] mik-laj commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
mik-laj commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352308163 ## File path: airflow/executors/kubernetes_executor.py ## @@ -703,10 +735,12 @@ def

[GitHub] [airflow] codecov-io commented on issue #6698: [AIRFLOW-6136] Remove hardcoded project ID in example_dag

2019-11-30 Thread GitBox
codecov-io commented on issue #6698: [AIRFLOW-6136] Remove hardcoded project ID in example_dag URL: https://github.com/apache/airflow/pull/6698#issuecomment-560028522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6698?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6698: [AIRFLOW-6136] Remove hardcoded project ID in example_dag

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6698: [AIRFLOW-6136] Remove hardcoded project ID in example_dag URL: https://github.com/apache/airflow/pull/6698#issuecomment-560028522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6698?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6698: [AIRFLOW-6136] Remove hardcoded project ID in example_dag

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6698: [AIRFLOW-6136] Remove hardcoded project ID in example_dag URL: https://github.com/apache/airflow/pull/6698#issuecomment-560028522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6698?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-6134) Scheduler hanging every 45 minutes

2019-11-30 Thread Brent Driskill (Jira)
Brent Driskill created AIRFLOW-6134: --- Summary: Scheduler hanging every 45 minutes Key: AIRFLOW-6134 URL: https://issues.apache.org/jira/browse/AIRFLOW-6134 Project: Apache Airflow Issue

[GitHub] [airflow] potiuk opened a new pull request #6696: [AIRFLOW-6128] Untangle plugins_manager and web from cyclic dependencies. Based on [AIRFLOW-6004]

2019-11-30 Thread GitBox
potiuk opened a new pull request #6696: [AIRFLOW-6128] Untangle plugins_manager and web from cyclic dependencies. Based on [AIRFLOW-6004] URL: https://github.com/apache/airflow/pull/6696 NOTE FOR REVIEWER - this change is based on #6596 so please take a look only at the last commit.

[jira] [Created] (AIRFLOW-6135) Extracct Dag processing from SchedulerJob into separate class

2019-11-30 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6135: -- Summary: Extracct Dag processing from SchedulerJob into separate class Key: AIRFLOW-6135 URL: https://issues.apache.org/jira/browse/AIRFLOW-6135 Project: Apache

[jira] [Updated] (AIRFLOW-6135) Extract DAG processing from SchedulerJob into separate class

2019-11-30 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-6135: --- Summary: Extract DAG processing from SchedulerJob into separate class (was: Extracct Dag

[jira] [Commented] (AIRFLOW-6128) Untangle "plugins_manager" and "web" cyclic dependencies

2019-11-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985415#comment-16985415 ] ASF GitHub Bot commented on AIRFLOW-6128: - potiuk commented on pull request #6696:

[GitHub] [airflow] potiuk commented on issue #6696: [AIRFLOW-6128] Untangle plugins_manager and web from cyclic dependencies. Based on [AIRFLOW-6004]

2019-11-30 Thread GitBox
potiuk commented on issue #6696: [AIRFLOW-6128] Untangle plugins_manager and web from cyclic dependencies. Based on [AIRFLOW-6004] URL: https://github.com/apache/airflow/pull/6696#issuecomment-560014121 While working on adding more type annotations I've found more cyclic dependencies in

[jira] [Commented] (AIRFLOW-6135) Extract DAG processing from SchedulerJob into separate class

2019-11-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985417#comment-16985417 ] ASF GitHub Bot commented on AIRFLOW-6135: - mik-laj commented on pull request #6697:

[GitHub] [airflow] mik-laj opened a new pull request #6697: [AIRFLOW-6135] Extract DAG processing from SchedulerJob into separate class

2019-11-30 Thread GitBox
mik-laj opened a new pull request #6697: [AIRFLOW-6135] Extract DAG processing from SchedulerJob into separate class URL: https://github.com/apache/airflow/pull/6697 Hello, Currently, ScheduleJob class has two uses cases: as a scheduler loop - started as the main process

[GitHub] [airflow] potiuk commented on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#issuecomment-560017256 > Is this going to make it slower -- i.e. does this genrate a list of keys and then do a list search for the key vs just

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352302966 ## File path: airflow/executors/base_executor.py ## @@ -185,25 +221,22 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352304421 ## File path: airflow/executors/local_executor.py ## @@ -201,31 +273,50 @@ def

[jira] [Comment Edited] (AIRFLOW-2540) Create an example dag using Google Cloud ML Engine operators

2019-11-30 Thread Younghee Kwon (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985437#comment-16985437 ] Younghee Kwon edited comment on AIRFLOW-2540 at 11/30/19 9:23 PM: --

[GitHub] [airflow] potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352307296 ## File path: airflow/kubernetes/pod_generator.py ## @@ -26,35 +26,7 @@ import

[jira] [Commented] (AIRFLOW-6131) Make Cassandra hooks/sensors pylint compatible

2019-11-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985342#comment-16985342 ] ASF GitHub Bot commented on AIRFLOW-6131: - nuclearpinguin commented on pull request #6693:

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352285880 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -209,29 +201,38 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352286033 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352285919 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352286047 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352285854 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -209,29 +201,38 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352286085 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352286092 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352286062 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352285840 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -209,29 +201,38 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352286074 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352285938 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#discussion_r352286011 ## File path: airflow/operators/cassandra_to_gcs.py ## @@ -247,120 +248,140 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6695: [AIRFLOW-6133] Make Hive transfer operators pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6695: [AIRFLOW-6133] Make Hive transfer operators pylint compatible URL: https://github.com/apache/airflow/pull/6695#discussion_r352289225 ## File path: airflow/operators/hive_to_druid.py ## @@ -124,7 +129,7 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6695: [AIRFLOW-6133] Make Hive transfer operators pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6695: [AIRFLOW-6133] Make Hive transfer operators pylint compatible URL: https://github.com/apache/airflow/pull/6695#discussion_r352289193 ## File path: airflow/contrib/operators/vertica_to_hive.py ## @@ -93,19 +97,20 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #6695: [AIRFLOW-6133] Make Hive transfer operators pylint compatible

2019-11-30 Thread GitBox
potiuk commented on a change in pull request #6695: [AIRFLOW-6133] Make Hive transfer operators pylint compatible URL: https://github.com/apache/airflow/pull/6695#discussion_r352289268 ## File path: airflow/operators/s3_to_hive_operator.py ## @@ -148,20 +152,20 @@ def

[jira] [Commented] (AIRFLOW-6080) New version of mypy catches more type errors

2019-11-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985391#comment-16985391 ] ASF GitHub Bot commented on AIRFLOW-6080: - ashb commented on pull request #6674: [AIRFLOW-6080]

[GitHub] [airflow] ashb merged pull request #6674: [AIRFLOW-6080] fix type error from new version of mypy

2019-11-30 Thread GitBox
ashb merged pull request #6674: [AIRFLOW-6080] fix type error from new version of mypy URL: https://github.com/apache/airflow/pull/6674 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-6129) Pylint fails on master with no-member issues

2019-11-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985325#comment-16985325 ] ASF GitHub Bot commented on AIRFLOW-6129: - potiuk commented on pull request #6691:

[GitHub] [airflow] potiuk opened a new pull request #6691: [AIRFLOW-6129] Fix pylint errors

2019-11-30 Thread GitBox
potiuk opened a new pull request #6691: [AIRFLOW-6129] Fix pylint errors URL: https://github.com/apache/airflow/pull/6691 Make sure you have checked _all_ steps below. There ia failure in CRON job

[jira] [Created] (AIRFLOW-6130) Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread Tomasz Urbaszek (Jira)
Tomasz Urbaszek created AIRFLOW-6130: Summary: Make Cassandra to GCS operator pylint compatible Key: AIRFLOW-6130 URL: https://issues.apache.org/jira/browse/AIRFLOW-6130 Project: Apache Airflow

[jira] [Updated] (AIRFLOW-6128) Untangle "plugins_manager" cyclic dependencies

2019-11-30 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6128: -- Description: Plugins manager was deeply entangled via cyclic dependencies with

[jira] [Assigned] (AIRFLOW-6128) Untangle "plugins_manager" cyclic dependencies

2019-11-30 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk reassigned AIRFLOW-6128: - Assignee: Jarek Potiuk > Untangle "plugins_manager" cyclic dependencies >

[GitHub] [airflow] nuclearpinguin opened a new pull request #6695: [AIRFLOW-6133] Make Hive transfer operators pylint compatible

2019-11-30 Thread GitBox
nuclearpinguin opened a new pull request #6695: [AIRFLOW-6133] Make Hive transfer operators pylint compatible URL: https://github.com/apache/airflow/pull/6695 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[jira] [Commented] (AIRFLOW-6133) Make Hive transfer operators pylint compatible

2019-11-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985373#comment-16985373 ] ASF GitHub Bot commented on AIRFLOW-6133: - nuclearpinguin commented on pull request #6695:

[GitHub] [airflow] codecov-io edited a comment on issue #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6692: [AIRFLOW-6130] Make Cassandra to GCS operator pylint compatible URL: https://github.com/apache/airflow/pull/6692#issuecomment-559966254 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6692?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports

2019-11-30 Thread GitBox
codecov-io edited a comment on issue #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#issuecomment-559224555 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6596?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-6131) Make Cassandra hooks/sensors pylint compatible

2019-11-30 Thread Tomasz Urbaszek (Jira)
Tomasz Urbaszek created AIRFLOW-6131: Summary: Make Cassandra hooks/sensors pylint compatible Key: AIRFLOW-6131 URL: https://issues.apache.org/jira/browse/AIRFLOW-6131 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-6129) Pylint fails on master with no-member issues

2019-11-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985339#comment-16985339 ] ASF GitHub Bot commented on AIRFLOW-6129: - potiuk commented on pull request #6691:

[jira] [Commented] (AIRFLOW-6129) Pylint fails on master with no-member issues

2019-11-30 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985340#comment-16985340 ] ASF subversion and git services commented on AIRFLOW-6129: -- Commit

[jira] [Resolved] (AIRFLOW-6129) Pylint fails on master with no-member issues

2019-11-30 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6129. --- Fix Version/s: 2.0.0 Resolution: Fixed > Pylint fails on master with no-member

[jira] [Created] (AIRFLOW-6132) AzureContainerInstancesOperator should allow to pass in tags

2019-11-30 Thread Jorge Machado (Jira)
Jorge Machado created AIRFLOW-6132: -- Summary: AzureContainerInstancesOperator should allow to pass in tags Key: AIRFLOW-6132 URL: https://issues.apache.org/jira/browse/AIRFLOW-6132 Project: Apache

[GitHub] [airflow] codecov-io commented on issue #6694: AIRFLOW-6132 - Allow to pass in tags

2019-11-30 Thread GitBox
codecov-io commented on issue #6694: AIRFLOW-6132 - Allow to pass in tags URL: https://github.com/apache/airflow/pull/6694#issuecomment-559975397 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6694?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-6131) Make Cassandra hooks/sensors pylint compatible

2019-11-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985385#comment-16985385 ] ASF GitHub Bot commented on AIRFLOW-6131: - potiuk commented on pull request #6693:

[jira] [Commented] (AIRFLOW-6131) Make Cassandra hooks/sensors pylint compatible

2019-11-30 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985387#comment-16985387 ] ASF subversion and git services commented on AIRFLOW-6131: -- Commit

[jira] [Commented] (AIRFLOW-6131) Make Cassandra hooks/sensors pylint compatible

2019-11-30 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985389#comment-16985389 ] ASF subversion and git services commented on AIRFLOW-6131: -- Commit

[jira] [Resolved] (AIRFLOW-6131) Make Cassandra hooks/sensors pylint compatible

2019-11-30 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6131. --- Fix Version/s: 2.0.0 Resolution: Fixed > Make Cassandra hooks/sensors pylint

[jira] [Commented] (AIRFLOW-6080) New version of mypy catches more type errors

2019-11-30 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985393#comment-16985393 ] ASF subversion and git services commented on AIRFLOW-6080: -- Commit

[jira] [Resolved] (AIRFLOW-6080) New version of mypy catches more type errors

2019-11-30 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-6080. Fix Version/s: 2.0.0 Resolution: Fixed > New version of mypy catches more

  1   2   >