[GitHub] xnuinside opened a new pull request #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators

2018-08-07 Thread GitBox
xnuinside opened a new pull request #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators URL: https://github.com/apache/incubator-airflow/pull/3717 Hi everyone! I saw the old task relative to my current work scope and decide to do this little PR ) Just adding

[GitHub] yrqls21 commented on issue #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file()

2018-08-07 Thread GitBox
yrqls21 commented on issue #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file() URL: https://github.com/apache/incubator-airflow/pull/3698#issuecomment-411255565 TY @feng-tao! This is an automated

[GitHub] tedmiston commented on issue #3703: [AIRFLOW-2857] Fix broken RTD env

2018-08-07 Thread GitBox
tedmiston commented on issue #3703: [AIRFLOW-2857] Fix broken RTD env URL: https://github.com/apache/incubator-airflow/pull/3703#issuecomment-411272356 Hey all - It looks like our RTD build post-merge was successful.  I dug into the mock dependency etc in the RTD build environment

[GitHub] r39132 edited a comment on issue #3716: [AIRFLOW-XXX] Remove smart quote from default config

2018-08-07 Thread GitBox
r39132 edited a comment on issue #3716: [AIRFLOW-XXX] Remove smart quote from default config URL: https://github.com/apache/incubator-airflow/pull/3716#issuecomment-411222958 +1 -- please follow contributor guidelines in the future. I'll create a tracking JIRA and update the commit

[GitHub] feng-tao commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
feng-tao commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208408841 ## File path: airflow/contrib/operators/oracle_to_azure_data_lake_transfer.py

[GitHub] feng-tao commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
feng-tao commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208408365 ## File path: airflow/contrib/hooks/databricks_hook.py ## @@ -65,7

[GitHub] r39132 edited a comment on issue #3716: [AIRFLOW-2869] Remove smart quote from default config

2018-08-07 Thread GitBox
r39132 edited a comment on issue #3716: [AIRFLOW-2869] Remove smart quote from default config URL: https://github.com/apache/incubator-airflow/pull/3716#issuecomment-411222958 +1 -- please follow contributor guidelines in the future. I'll create a tracking JIRA and update the JIRA &

[GitHub] codecov-io edited a comment on issue #3586: [AIRFLOW-2733] Reconcile psutil and subprocess in webserver cli

2018-08-07 Thread GitBox
codecov-io edited a comment on issue #3586: [AIRFLOW-2733] Reconcile psutil and subprocess in webserver cli URL: https://github.com/apache/incubator-airflow/pull/3586#issuecomment-403631506 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3586?src=pr=h1) Report >

[GitHub] kaxil commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
kaxil commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208411271 ## File path: airflow/contrib/operators/oracle_to_azure_data_lake_transfer.py

[GitHub] troychen728 commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training

2018-08-07 Thread GitBox
troychen728 commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training URL: https://github.com/apache/incubator-airflow/pull/3658#discussion_r208422276 ## File path: airflow/contrib/operators/sagemaker_create_training_job_operator.py ## @@

[GitHub] feng-tao commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
feng-tao commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208408365 ## File path: airflow/contrib/hooks/databricks_hook.py ## @@ -65,7

[GitHub] feng-tao commented on issue #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
feng-tao commented on issue #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#issuecomment-411230444 lgtm This is an automated message

[GitHub] codecov-io commented on issue #3716: [AIRFLOW-2869] Remove smart quote from default config

2018-08-07 Thread GitBox
codecov-io commented on issue #3716: [AIRFLOW-2869] Remove smart quote from default config URL: https://github.com/apache/incubator-airflow/pull/3716#issuecomment-411232064 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3716?src=pr=h1) Report > Merging

[GitHub] codecov-io edited a comment on issue #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file()

2018-08-07 Thread GitBox
codecov-io edited a comment on issue #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file() URL: https://github.com/apache/incubator-airflow/pull/3698#issuecomment-410575101 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=h1) Report

[GitHub] codecov-io edited a comment on issue #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file()

2018-08-07 Thread GitBox
codecov-io edited a comment on issue #3698: [AIRFLOW-2855] Check Cron Expression Validity in DagBag.process_file() URL: https://github.com/apache/incubator-airflow/pull/3698#issuecomment-410575101 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3698?src=pr=h1) Report

[GitHub] ashb opened a new pull request #3715: [AIRFLOW-XXX] Updating instructions about logging changes in 1.10

2018-08-07 Thread GitBox
ashb opened a new pull request #3715: [AIRFLOW-XXX] Updating instructions about logging changes in 1.10 URL: https://github.com/apache/incubator-airflow/pull/3715 We had a few other logging changes that weren't mentioned in here that meant previous logs were not viewable anymore.

[GitHub] bolkedebruin edited a comment on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime

2018-08-07 Thread GitBox
bolkedebruin edited a comment on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime URL: https://github.com/apache/incubator-airflow/pull/3708#issuecomment-411195033 @ashb I think I finally nailed it. The issue was in my understanding how sqlalchemy was dealing with timezone

[GitHub] ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208371759 ## File path: airflow/contrib/operators/oracle_to_oracle_transfer.py ##

[GitHub] ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208372051 ## File path: airflow/operators/s3_to_hive_operator.py ## @@ -261,9

[GitHub] ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208371591 ## File path: airflow/contrib/operators/dataflow_operator.py ## @@

[GitHub] ashb commented on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime

2018-08-07 Thread GitBox
ashb commented on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime URL: https://github.com/apache/incubator-airflow/pull/3708#issuecomment-411189623 Can I help? This is an automated message from the Apache Git Service.

[GitHub] ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208371167 ## File path: airflow/configuration.py ## @@ -186,7 +186,8 @@ def

[GitHub] gwax commented on issue #2482: AIRFLOW-1460 clear "REMOVED" tis on DagRun update

2018-08-07 Thread GitBox
gwax commented on issue #2482: AIRFLOW-1460 clear "REMOVED" tis on DagRun update URL: https://github.com/apache/incubator-airflow/pull/2482#issuecomment-411221356 This Issue has been resolved by https://github.com/apache/incubator-airflow/pull/3137 I will close this PR as a

[GitHub] gwax closed pull request #2482: AIRFLOW-1460 clear "REMOVED" tis on DagRun update

2018-08-07 Thread GitBox
gwax closed pull request #2482: AIRFLOW-1460 clear "REMOVED" tis on DagRun update URL: https://github.com/apache/incubator-airflow/pull/2482 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As

[GitHub] bolkedebruin closed pull request #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook

2018-08-07 Thread GitBox
bolkedebruin closed pull request #3700: [AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook URL: https://github.com/apache/incubator-airflow/pull/3700 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake

[GitHub] kaxil commented on issue #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-07 Thread GitBox
kaxil commented on issue #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#issuecomment-411194032 @ashb This is now ready for review. :)

[GitHub] jakahn commented on issue #3677: [AIRFLOW-2826] Add GoogleCloudKMSHook

2018-08-07 Thread GitBox
jakahn commented on issue #3677: [AIRFLOW-2826] Add GoogleCloudKMSHook URL: https://github.com/apache/incubator-airflow/pull/3677#issuecomment-411156427 @Fokko PTAL when you can This is an automated message from the Apache

[GitHub] bolkedebruin commented on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime

2018-08-07 Thread GitBox
bolkedebruin commented on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime URL: https://github.com/apache/incubator-airflow/pull/3708#issuecomment-411195033 @ashb I think I finally nailed it. The issue was in my understanding how sqlalchemy was dealing with timezone information

[GitHub] ashb commented on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime

2018-08-07 Thread GitBox
ashb commented on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime URL: https://github.com/apache/incubator-airflow/pull/3708#issuecomment-411195448 AIP-3: Drop support for Mysql :D This is an automated message

[GitHub] feng-tao commented on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime

2018-08-07 Thread GitBox
feng-tao commented on issue #3708: [AIRFLOW-2859] Implement own UtcDateTime URL: https://github.com/apache/incubator-airflow/pull/3708#issuecomment-411222194 flake8 fails. This is an automated message from the Apache Git

[GitHub] kaxil edited a comment on issue #3696: [AIRFLOW-2851] Canonicalize "as _..." etc imports

2018-08-07 Thread GitBox
kaxil edited a comment on issue #3696: [AIRFLOW-2851] Canonicalize "as _..." etc imports URL: https://github.com/apache/incubator-airflow/pull/3696#issuecomment-411169973 @mistercrunch 樂 I might have definitely done it on `gcs_hook`. Sorry about that. Not sure if I ever used `_os`.

[GitHub] bolkedebruin closed pull request #3715: [AIRFLOW-XXX] Updating instructions about logging changes in 1.10

2018-08-07 Thread GitBox
bolkedebruin closed pull request #3715: [AIRFLOW-XXX] Updating instructions about logging changes in 1.10 URL: https://github.com/apache/incubator-airflow/pull/3715 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the

[GitHub] wdhorton opened a new pull request #3716: [AIRFLOW-XXX] Remove smart quote from default config

2018-08-07 Thread GitBox
wdhorton opened a new pull request #3716: [AIRFLOW-XXX] Remove smart quote from default config URL: https://github.com/apache/incubator-airflow/pull/3716 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] Fokko commented on a change in pull request #3703: [AIRFLOW-2857] Fix broken RTD env

2018-08-08 Thread GitBox
Fokko commented on a change in pull request #3703: [AIRFLOW-2857] Fix broken RTD env URL: https://github.com/apache/incubator-airflow/pull/3703#discussion_r208478397 ## File path: setup.py ## @@ -161,6 +164,7 @@ def write_version(filename=os.path.join(*['airflow',

[GitHub] kaxil commented on a change in pull request #3703: [AIRFLOW-2857] Fix broken RTD env

2018-08-08 Thread GitBox
kaxil commented on a change in pull request #3703: [AIRFLOW-2857] Fix broken RTD env URL: https://github.com/apache/incubator-airflow/pull/3703#discussion_r208502954 ## File path: setup.py ## @@ -161,6 +164,7 @@ def write_version(filename=os.path.join(*['airflow',

[GitHub] xnuinside commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-08 Thread GitBox
xnuinside commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208514928 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -238,6

[GitHub] ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-08 Thread GitBox
ashb commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208515928 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -238,6

[GitHub] kaxil commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-08 Thread GitBox
kaxil commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208519487 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -238,6

[GitHub] kaxil commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-08 Thread GitBox
kaxil commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208519487 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -238,6

[GitHub] xnuinside commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-08 Thread GitBox
xnuinside commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208520636 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -238,6

[GitHub] xnuinside edited a comment on issue #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators

2018-08-08 Thread GitBox
xnuinside edited a comment on issue #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators URL: https://github.com/apache/incubator-airflow/pull/3717#issuecomment-411349555 @kaxil , yeah, sure, first time I have done it the way as you write, but then found close PR with

[GitHub] xnuinside commented on issue #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators

2018-08-08 Thread GitBox
xnuinside commented on issue #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators URL: https://github.com/apache/incubator-airflow/pull/3717#issuecomment-411349555 @kaxil , yeah, sure, first time I have done it the way as you write, but then found close PR with the

[GitHub] verdan opened a new pull request #3719: [AIRFLOW-2874] Enables FAB's theme support

2018-08-08 Thread GitBox
verdan opened a new pull request #3719: [AIRFLOW-2874] Enables FAB's theme support URL: https://github.com/apache/incubator-airflow/pull/3719 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] xnuinside commented on issue #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators

2018-08-08 Thread GitBox
xnuinside commented on issue #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators URL: https://github.com/apache/incubator-airflow/pull/3717#issuecomment-411360602 tests failed with ERROR [airflow.models.DagBag] Failed to bag_dag:

[GitHub] bolkedebruin closed pull request #3708: [AIRFLOW-2859] Implement own UtcDateTime

2018-08-08 Thread GitBox
bolkedebruin closed pull request #3708: [AIRFLOW-2859] Implement own UtcDateTime URL: https://github.com/apache/incubator-airflow/pull/3708 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this

[GitHub] Fokko closed pull request #3677: [AIRFLOW-2826] Add GoogleCloudKMSHook

2018-08-08 Thread GitBox
Fokko closed pull request #3677: [AIRFLOW-2826] Add GoogleCloudKMSHook URL: https://github.com/apache/incubator-airflow/pull/3677 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a

[GitHub] kaxil commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines

2018-08-08 Thread GitBox
kaxil commented on a change in pull request #3714: [AIRFLOW-2867] Refactor code to conform Python standards & guidelines URL: https://github.com/apache/incubator-airflow/pull/3714#discussion_r208498985 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -238,6

[GitHub] codecov-io commented on issue #3718: [AIRFLOW-2872] Fix and Refine 'Ad Hoc Query' View

2018-08-08 Thread GitBox
codecov-io commented on issue #3718: [AIRFLOW-2872] Fix and Refine 'Ad Hoc Query' View URL: https://github.com/apache/incubator-airflow/pull/3718#issuecomment-411331261 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3718?src=pr=h1) Report > Merging

[GitHub] Fokko commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training

2018-08-08 Thread GitBox
Fokko commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training URL: https://github.com/apache/incubator-airflow/pull/3658#discussion_r208485615 ## File path: airflow/contrib/hooks/sagemaker_hook.py ## @@ -0,0 +1,239 @@ +# -*- coding: utf-8

[GitHub] Fokko commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training

2018-08-08 Thread GitBox
Fokko commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training URL: https://github.com/apache/incubator-airflow/pull/3658#discussion_r208486176 ## File path: airflow/contrib/hooks/sagemaker_hook.py ## @@ -0,0 +1,239 @@ +# -*- coding: utf-8

[GitHub] XD-DENG opened a new pull request #3718: [AIRFLOW-2872] Fix and Refine 'Ad Hoc Query' View

2018-08-08 Thread GitBox
XD-DENG opened a new pull request #3718: [AIRFLOW-2872] Fix and Refine 'Ad Hoc Query' View URL: https://github.com/apache/incubator-airflow/pull/3718 ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and

[GitHub] kaxil commented on issue #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators

2018-08-08 Thread GitBox
kaxil commented on issue #3717: [AIRFLOW-1874] use_legacy_sql added to BigQueryCheck operators URL: https://github.com/apache/incubator-airflow/pull/3717#issuecomment-411331977 As far as I know, there is a plan to deprecate args and kwargs keywords in Airflow. Can you instead just add

[GitHub] bolkedebruin commented on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration

2018-08-08 Thread GitBox
bolkedebruin commented on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration URL: https://github.com/apache/incubator-airflow/pull/3720#issuecomment-411468352 Ah let me fix that This is an automated message

[GitHub] codecov-io edited a comment on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration

2018-08-08 Thread GitBox
codecov-io edited a comment on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration URL: https://github.com/apache/incubator-airflow/pull/3720#issuecomment-411401924 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3720?src=pr=h1) Report > Merging

[GitHub] bolkedebruin closed pull request #3709: [AIRFLOW-2861] Added index on log table

2018-08-08 Thread GitBox
bolkedebruin closed pull request #3709: [AIRFLOW-2861] Added index on log table URL: https://github.com/apache/incubator-airflow/pull/3709 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this

[GitHub] wdhorton opened a new pull request #3721: [AIRFLOW-2875] Escape env vars in tmp config

2018-08-08 Thread GitBox
wdhorton opened a new pull request #3721: [AIRFLOW-2875] Escape env vars in tmp config URL: https://github.com/apache/incubator-airflow/pull/3721 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] troychen728 commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training

2018-08-08 Thread GitBox
troychen728 commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training URL: https://github.com/apache/incubator-airflow/pull/3658#discussion_r208672024 ## File path: airflow/contrib/hooks/sagemaker_hook.py ## @@ -0,0 +1,239 @@ +# -*- coding:

[GitHub] bolkedebruin commented on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration

2018-08-08 Thread GitBox
bolkedebruin commented on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration URL: https://github.com/apache/incubator-airflow/pull/3720#issuecomment-411469023 @gwax done. PTAL This is an automated message

[GitHub] bolkedebruin edited a comment on issue #3560: [AIRFLOW-2697] Drop snakebite in favour of hdfs3

2018-08-08 Thread GitBox
bolkedebruin edited a comment on issue #3560: [AIRFLOW-2697] Drop snakebite in favour of hdfs3 URL: https://github.com/apache/incubator-airflow/pull/3560#issuecomment-411484838 Can you revise and squash your commits? The we can (most likely :-) ) merge.

[GitHub] bolkedebruin edited a comment on issue #3560: [AIRFLOW-2697] Drop snakebite in favour of hdfs3

2018-08-08 Thread GitBox
bolkedebruin edited a comment on issue #3560: [AIRFLOW-2697] Drop snakebite in favour of hdfs3 URL: https://github.com/apache/incubator-airflow/pull/3560#issuecomment-411484838 Can you rebase and squash your commits? The we can (most likely :-) ) merge.

[GitHub] troychen728 commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training

2018-08-08 Thread GitBox
troychen728 commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training URL: https://github.com/apache/incubator-airflow/pull/3658#discussion_r208671964 ## File path: airflow/contrib/hooks/sagemaker_hook.py ## @@ -0,0 +1,239 @@ +# -*- coding:

[GitHub] troychen728 commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training

2018-08-08 Thread GitBox
troychen728 commented on a change in pull request #3658: [AIRFLOW-2524] Add Amazon SageMaker Training URL: https://github.com/apache/incubator-airflow/pull/3658#discussion_r208671902 ## File path: airflow/contrib/hooks/sagemaker_hook.py ## @@ -0,0 +1,239 @@ +# -*- coding:

[GitHub] gwax commented on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration

2018-08-08 Thread GitBox
gwax commented on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration URL: https://github.com/apache/incubator-airflow/pull/3720#issuecomment-411461955 Clever approach; thanks for taking this on @bolkedebruin On attempting `upgradedb`, I run into the following

[GitHub] gwax commented on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration

2018-08-08 Thread GitBox
gwax commented on issue #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration URL: https://github.com/apache/incubator-airflow/pull/3720#issuecomment-411480640 That works for me now. :shipit: ! This is an

[GitHub] bolkedebruin closed pull request #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration

2018-08-08 Thread GitBox
bolkedebruin closed pull request #3720: [AIRFLOW-2870] Use abstract TaskInstance for migration URL: https://github.com/apache/incubator-airflow/pull/3720 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of

[GitHub] bolkedebruin commented on issue #3560: [AIRFLOW-2697] Drop snakebite in favour of hdfs3

2018-08-08 Thread GitBox
bolkedebruin commented on issue #3560: [AIRFLOW-2697] Drop snakebite in favour of hdfs3 URL: https://github.com/apache/incubator-airflow/pull/3560#issuecomment-411484838 Can you squash your commits? The we can (most likely :-) ) merge.

[GitHub] feng-tao commented on a change in pull request #3747: [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs

2018-08-13 Thread GitBox
feng-tao commented on a change in pull request #3747: [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs URL: https://github.com/apache/incubator-airflow/pull/3747#discussion_r209707193 ## File path: UPDATING.md ## @@ -420,10 +420,6 @@ indefinitely. This is

[GitHub] yrqls21 commented on issue #2986: [AIRFLOW-2027] Only trigger sleep in scheduler after all files have parsed

2018-08-13 Thread GitBox
yrqls21 commented on issue #2986: [AIRFLOW-2027] Only trigger sleep in scheduler after all files have parsed URL: https://github.com/apache/incubator-airflow/pull/2986#issuecomment-412637527 @aoen Agree on sleep for a minimal time interval, it should not be a huge problem to sleep at

[GitHub] feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY

2018-08-13 Thread GitBox
feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY URL: https://github.com/apache/incubator-airflow/pull/3738#issuecomment-412609075 This pr currently can't address the issue when we have a cluster for webserver. IMO, I would prefer to revert the orignal prs

[GitHub] TrevorEdwards commented on a change in pull request #3532: [AIRFLOW-2658] Add GCP specific k8s pod operator

2018-08-13 Thread GitBox
TrevorEdwards commented on a change in pull request #3532: [AIRFLOW-2658] Add GCP specific k8s pod operator URL: https://github.com/apache/incubator-airflow/pull/3532#discussion_r209703493 ## File path: airflow/contrib/operators/gcp_container_operator.py ## @@ -170,3

[GitHub] aoen opened a new pull request #3747: [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs

2018-08-13 Thread GitBox
aoen opened a new pull request #3747: [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs URL: https://github.com/apache/incubator-airflow/pull/3747 Reverts most of AIRFLOW-2027 until the issues with it can be fixed. ### Jira - [X] My PR addresses the following

[GitHub] feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY

2018-08-13 Thread GitBox
feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY URL: https://github.com/apache/incubator-airflow/pull/3738#issuecomment-412608140 looking at this https://github.com/apache/incubator-airflow/pull/3651 pr, I am wondering whether we could do better than use

[GitHub] feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY

2018-08-13 Thread GitBox
feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY URL: https://github.com/apache/incubator-airflow/pull/3738#issuecomment-412609075 This pr currently can't address the issue when we have a cluster for webserver. IMO, I would prefer to revert the orignal prs

[GitHub] feng-tao commented on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY

2018-08-13 Thread GitBox
feng-tao commented on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY URL: https://github.com/apache/incubator-airflow/pull/3738#issuecomment-412608140 looking at this https://github.com/apache/incubator-airflow/pull/3651 pr, I am wondering whether we could do better than use random

[GitHub] codecov-io commented on issue #3747: [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs

2018-08-13 Thread GitBox
codecov-io commented on issue #3747: [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs URL: https://github.com/apache/incubator-airflow/pull/3747#issuecomment-412669040 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3747?src=pr=h1) Report > Merging

[GitHub] kaxil commented on a change in pull request #3725: [AIRFLOW-2877] Make docs site URL consistent everywhere

2018-08-13 Thread GitBox
kaxil commented on a change in pull request #3725: [AIRFLOW-2877] Make docs site URL consistent everywhere URL: https://github.com/apache/incubator-airflow/pull/3725#discussion_r209730842 ## File path: README.md ## @@ -24,9 +24,9 @@ monitor progress, and troubleshoot

[GitHub] feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY

2018-08-13 Thread GitBox
feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY URL: https://github.com/apache/incubator-airflow/pull/3738#issuecomment-412608140 looking at this https://github.com/apache/incubator-airflow/pull/3651 pr, I am wondering whether we could do better than use

[GitHub] feng-tao commented on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY

2018-08-13 Thread GitBox
feng-tao commented on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY URL: https://github.com/apache/incubator-airflow/pull/3738#issuecomment-412609075 IMO, I would prefer to revert the orignal prs and improve the secret_key config documentation.

[GitHub] feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY

2018-08-13 Thread GitBox
feng-tao edited a comment on issue #3738: [AIRFLOW-2886] Secure Flask SECRET_KEY URL: https://github.com/apache/incubator-airflow/pull/3738#issuecomment-412608140 looking at this https://github.com/apache/incubator-airflow/pull/3651 pr, I am wondering whether we could do better than use

[GitHub] Fokko commented on issue #3745: [AIRFLOW-2860] DruidHook: time check is wrong

2018-08-13 Thread GitBox
Fokko commented on issue #3745: [AIRFLOW-2860] DruidHook: time check is wrong URL: https://github.com/apache/incubator-airflow/pull/3745#issuecomment-412612939 Thanks again @awelsh93 This is an automated message from the

[GitHub] aoen commented on a change in pull request #3747: [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs

2018-08-13 Thread GitBox
aoen commented on a change in pull request #3747: [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs URL: https://github.com/apache/incubator-airflow/pull/3747#discussion_r209744335 ## File path: UPDATING.md ## @@ -420,10 +420,6 @@ indefinitely. This is only

[GitHub] tedmiston commented on a change in pull request #3725: [AIRFLOW-2877] Make docs site URL consistent everywhere

2018-08-13 Thread GitBox
tedmiston commented on a change in pull request #3725: [AIRFLOW-2877] Make docs site URL consistent everywhere URL: https://github.com/apache/incubator-airflow/pull/3725#discussion_r209729859 ## File path: README.md ## @@ -24,9 +24,9 @@ monitor progress, and troubleshoot

[GitHub] tedmiston commented on a change in pull request #3725: [AIRFLOW-2877] Make docs site URL consistent everywhere

2018-08-13 Thread GitBox
tedmiston commented on a change in pull request #3725: [AIRFLOW-2877] Make docs site URL consistent everywhere URL: https://github.com/apache/incubator-airflow/pull/3725#discussion_r209729859 ## File path: README.md ## @@ -24,9 +24,9 @@ monitor progress, and troubleshoot

[GitHub] yrqls21 edited a comment on issue #2986: [AIRFLOW-2027] Only trigger sleep in scheduler after all files have parsed

2018-08-13 Thread GitBox
yrqls21 edited a comment on issue #2986: [AIRFLOW-2027] Only trigger sleep in scheduler after all files have parsed URL: https://github.com/apache/incubator-airflow/pull/2986#issuecomment-412637527 @aoen Agree on sleep for a minimal time interval(max(0, 1s-duration)), it should not

[GitHub] kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413134052 @ashb No worries. The aim of this PR was to obfuscate sensitive variables from UI export but it would still allow

[GitHub] ashb commented on a change in pull request #3743: [AIRFLOW-2799] Filtering UI objects by datetime is broken

2018-08-15 Thread GitBox
ashb commented on a change in pull request #3743: [AIRFLOW-2799] Filtering UI objects by datetime is broken URL: https://github.com/apache/incubator-airflow/pull/3743#discussion_r210211080 ## File path: airflow/www/utils.py ## @@ -428,7 +437,26 @@ def __call__(self,

[GitHub] ron819 commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
ron819 commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413140291 Why not allow the user to decide per Variable if it's sensitive or not (addition of Boolean column)? Then you

[GitHub] kaxil closed pull request #3744: [AIRFLOW-2893] fix stuck dataflow job due to name mismatch

2018-08-15 Thread GitBox
kaxil closed pull request #3744: [AIRFLOW-2893] fix stuck dataflow job due to name mismatch URL: https://github.com/apache/incubator-airflow/pull/3744 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of

[GitHub] kaxil commented on issue #3744: [AIRFLOW-2893] fix stuck dataflow job due to name mismatch

2018-08-15 Thread GitBox
kaxil commented on issue #3744: [AIRFLOW-2893] fix stuck dataflow job due to name mismatch URL: https://github.com/apache/incubator-airflow/pull/3744#issuecomment-41311 Thanks @fenglu-g @TrevorEdwards This is an

[GitHub] ashb commented on issue #3749: [AIRFLOW-2900] Show code for packaged DAGs

2018-08-15 Thread GitBox
ashb commented on issue #3749: [AIRFLOW-2900] Show code for packaged DAGs URL: https://github.com/apache/incubator-airflow/pull/3749#issuecomment-413136390 > Is there a way to force the www_rbac code path in a local airflow Check out UPDATING.md for 1.10 - I think it's `rbac =

[GitHub] kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413141931 @ron819 Even if a user decides whether a field is sensitive or not, you would still need to add the functionality

[GitHub] xnuinside commented on issue #3733: [AIRFLOW-491] Add cache parameter in BigQuery query method - with 'src_fmt_configs'

2018-08-15 Thread GitBox
xnuinside commented on issue #3733: [AIRFLOW-491] Add cache parameter in BigQuery query method - with 'src_fmt_configs' URL: https://github.com/apache/incubator-airflow/pull/3733#issuecomment-413147879 @kaxil, pls review

[GitHub] kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413132605 @ashb Thoughts? This is an automated message from

[GitHub] kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413132540 @Fokko Fixing this on API level will mean we won't be able to get the variables in plain text through CLI as well.

[GitHub] kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
kaxil commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413151970 @ron819 I didn't get mad at all, apologies and sorry if you felt so. I seriously think that it should have been a

[GitHub] kaxil edited a comment on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
kaxil edited a comment on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413141931 @ron819 Even if a user decides whether a field is sensitive or not, you would still need to add the

[GitHub] ashb commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
ashb commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413133170 Yes. And No? Yes, because by default it probably shouldn't be possible to get the secrets back out. But the

[GitHub] ashb commented on a change in pull request #3725: [AIRFLOW-2877] Make docs site URL consistent everywhere

2018-08-15 Thread GitBox
ashb commented on a change in pull request #3725: [AIRFLOW-2877] Make docs site URL consistent everywhere URL: https://github.com/apache/incubator-airflow/pull/3725#discussion_r210212865 ## File path: CONTRIBUTING.md ## @@ -61,8 +61,16 @@ If you are proposing a feature:

[GitHub] Fokko commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables

2018-08-15 Thread GitBox
Fokko commented on issue #3748: [AIRFLOW-2899] Hide sensitive data when Exporting Variables URL: https://github.com/apache/incubator-airflow/pull/3748#issuecomment-413122811 I might even argue that we need to fix this on an API level instead of the view level. In this case the API will

[GitHub] XD-DENG commented on a change in pull request #3560: [AIRFLOW-2697] Drop snakebite in favour of hdfs3

2018-08-15 Thread GitBox
XD-DENG commented on a change in pull request #3560: [AIRFLOW-2697] Drop snakebite in favour of hdfs3 URL: https://github.com/apache/incubator-airflow/pull/3560#discussion_r210199659 ## File path: airflow/sensors/hdfs_sensor.py ## @@ -17,103 +17,231 @@ # specific

<    1   2   3   4   5   6   7   8   9   10   >