[GitHub] feng-tao commented on issue #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists

2019-01-16 Thread GitBox
feng-tao commented on issue #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists URL: https://github.com/apache/airflow/pull/3688#issuecomment-455068969 @XD-DENG agree. I think we should rescan the dag and check whether the task still exists. I can't think of other way

[GitHub] feng-tao edited a comment on issue #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists

2019-01-16 Thread GitBox
feng-tao edited a comment on issue #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists URL: https://github.com/apache/airflow/pull/3688#issuecomment-455068969 @XD-DENG agree. I think we should rescan the dag and check whether the task still exists. I can't think of

[GitHub] potiuk commented on issue #4541: [AIRFLOW-3713] Updated documentation for GCP optional project_id

2019-01-16 Thread GitBox
potiuk commented on issue #4541: [AIRFLOW-3713] Updated documentation for GCP optional project_id URL: https://github.com/apache/airflow/pull/4541#issuecomment-455071991 @kaxil -> all fixed. I checked what was released in 1.10.1 and removed all the hooks/operators that were not released

[GitHub] 3vivekb opened a new pull request #4544: [AIRFLOW-XXX] Fixing a typo of config

2019-01-16 Thread GitBox
3vivekb opened a new pull request #4544: [AIRFLOW-XXX] Fixing a typo of config URL: https://github.com/apache/airflow/pull/4544 Fixing a tiny documentation typo. connfig -> config This is an automated message

[GitHub] codecov-io commented on issue #4544: [AIRFLOW-XXX] Fixing a typo of config

2019-01-16 Thread GitBox
codecov-io commented on issue #4544: [AIRFLOW-XXX] Fixing a typo of config URL: https://github.com/apache/airflow/pull/4544#issuecomment-455057812 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4544?src=pr=h1) Report > Merging

[GitHub] XD-DENG commented on issue #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists

2019-01-16 Thread GitBox
XD-DENG commented on issue #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists URL: https://github.com/apache/airflow/pull/3688#issuecomment-455069576 @feng-tao Scanning a DAG may take up to a few seconds. If it's more than that, to me the DAG is written wrongly

[GitHub] laakejs commented on a change in pull request #4465: [AIRFLOW-3682] Rename s3_default to aws_default

2019-01-17 Thread GitBox
laakejs commented on a change in pull request #4465: [AIRFLOW-3682] Rename s3_default to aws_default URL: https://github.com/apache/airflow/pull/4465#discussion_r248584759 ## File path: UPDATING.md ## @@ -24,13 +24,17 @@ assists users migrating to a new version. ##

[GitHub] feng-tao commented on issue #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists

2019-01-16 Thread GitBox
feng-tao commented on issue #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists URL: https://github.com/apache/airflow/pull/3688#issuecomment-455058719 Thanks @XD-DENG , but I think this pr needs more work. E.g I think the goal is that the sensor shouldn't wait

[GitHub] mik-laj edited a comment on issue #4523: [AIRFLOW-3616] Add aliases for schema with underscore

2019-01-17 Thread GitBox
mik-laj edited a comment on issue #4523: [AIRFLOW-3616] Add aliases for schema with underscore URL: https://github.com/apache/airflow/pull/4523#issuecomment-455198427 @ashb I introduced all expected changes. This is an

[GitHub] mik-laj commented on a change in pull request #2455: [AIRFLOW-1423] Add logs to the scheduler DAG run decision logic

2019-01-17 Thread GitBox
mik-laj commented on a change in pull request #2455: [AIRFLOW-1423] Add logs to the scheduler DAG run decision logic URL: https://github.com/apache/airflow/pull/2455#discussion_r240830440 ## File path: airflow/jobs.py ## @@ -786,6 +788,9 @@ def create_dag_run(self, dag,

[GitHub] codecov-io commented on issue #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI

2019-01-17 Thread GitBox
codecov-io commented on issue #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI URL: https://github.com/apache/airflow/pull/4548#issuecomment-455333956 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4548?src=pr=h1) Report > Merging

[GitHub] feng-tao commented on issue #4502: [AIRFLOW-3591] Fix start date, end date, duration for rescheduled tasks

2019-01-17 Thread GitBox
feng-tao commented on issue #4502: [AIRFLOW-3591] Fix start date, end date, duration for rescheduled tasks URL: https://github.com/apache/airflow/pull/4502#issuecomment-455325072 LGTM, PTAL @ashb and see if he has any other comments.

[GitHub] mik-laj commented on a change in pull request #4494: [AIRFLOW-3284] Azure Batch AI Operator

2019-01-17 Thread GitBox
mik-laj commented on a change in pull request #4494: [AIRFLOW-3284] Azure Batch AI Operator URL: https://github.com/apache/airflow/pull/4494#discussion_r248836913 ## File path: airflow/contrib/hooks/azure_batchai_hook.py ## @@ -0,0 +1,111 @@ +# -*- coding: utf-8 -*- +# +#

[GitHub] mik-laj commented on a change in pull request #4494: [AIRFLOW-3284] Azure Batch AI Operator

2019-01-17 Thread GitBox
mik-laj commented on a change in pull request #4494: [AIRFLOW-3284] Azure Batch AI Operator URL: https://github.com/apache/airflow/pull/4494#discussion_r248833770 ## File path: airflow/utils/db.py ## @@ -280,6 +280,14 @@ def initdb(rbac=False): Connection(

[GitHub] kaxil merged pull request #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI

2019-01-17 Thread GitBox
kaxil merged pull request #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI URL: https://github.com/apache/airflow/pull/4548 This is an automated message from the Apache Git Service. To respond to

[GitHub] idavison commented on issue #4294: [AIRFLOW-3451] Add UI button to refresh all dags

2019-01-17 Thread GitBox
idavison commented on issue #4294: [AIRFLOW-3451] Add UI button to refresh all dags URL: https://github.com/apache/airflow/pull/4294#issuecomment-455342669 @ron819 rebased This is an automated message from the Apache Git

[GitHub] kaxil commented on a change in pull request #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI

2019-01-17 Thread GitBox
kaxil commented on a change in pull request #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI URL: https://github.com/apache/airflow/pull/4548#discussion_r248829454 ## File path: airflow/www/templates/airflow/graph.html ## @@ -112,8 +112,8 @@

[GitHub] feng-tao commented on issue #4459: [AIRFLOW-3649] Feature to add extra labels to kubernetes worker pods

2019-01-17 Thread GitBox
feng-tao commented on issue #4459: [AIRFLOW-3649] Feature to add extra labels to kubernetes worker pods URL: https://github.com/apache/airflow/pull/4459#issuecomment-455347596 once you rebase with master, it should be good @wyndhblb

[GitHub] danabananarama commented on issue #4514: [AIRFLOW-3698] Add documentation for AWS Connection

2019-01-17 Thread GitBox
danabananarama commented on issue #4514: [AIRFLOW-3698] Add documentation for AWS Connection URL: https://github.com/apache/airflow/pull/4514#issuecomment-455352968 @mik-laj Oh cool, I didn't know you could do that! I'm afraid I haven't set up the AWS Connection that way before so I'm not

[GitHub] dargueta edited a comment on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12

2019-01-17 Thread GitBox
dargueta edited a comment on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12 URL: https://github.com/apache/airflow/pull/3723#issuecomment-455369056 I also can't reproduce the crash. `apache-airflow`: 1.10.1 Python: 3.7.2 I installed `tenacity` using the constraints

[GitHub] dargueta edited a comment on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12

2019-01-17 Thread GitBox
dargueta edited a comment on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12 URL: https://github.com/apache/airflow/pull/3723#issuecomment-455369056 I also can't reproduce the crash. `apache-airflow`: 1.10.1 `tenacity`: I tried 4.11 _and_ 4.12. Python: 3.7.2

[GitHub] dargueta edited a comment on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12

2019-01-17 Thread GitBox
dargueta edited a comment on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12 URL: https://github.com/apache/airflow/pull/3723#issuecomment-455369056 I also can't reproduce the crash. `apache-airflow`: 1.10.1 Python: 3.7.2 tenacity: I tried 4.11 _and_ 4.12.

[GitHub] feng-tao commented on issue #4547: [AIRFLOW-2843] Add flag in ExternalTaskSensor to check if external DAG/task exists

2019-01-17 Thread GitBox
feng-tao commented on issue #4547: [AIRFLOW-2843] Add flag in ExternalTaskSensor to check if external DAG/task exists URL: https://github.com/apache/airflow/pull/4547#issuecomment-455260837 LGTM, thanks @XD-DENG This is an

[GitHub] feng-tao merged pull request #4547: [AIRFLOW-2843] Add flag in ExternalTaskSensor to check if external DAG/task exists

2019-01-17 Thread GitBox
feng-tao merged pull request #4547: [AIRFLOW-2843] Add flag in ExternalTaskSensor to check if external DAG/task exists URL: https://github.com/apache/airflow/pull/4547 This is an automated message from the Apache Git

[GitHub] feng-tao merged pull request #4541: [AIRFLOW-3713] Updated documentation for GCP optional project_id

2019-01-17 Thread GitBox
feng-tao merged pull request #4541: [AIRFLOW-3713] Updated documentation for GCP optional project_id URL: https://github.com/apache/airflow/pull/4541 This is an automated message from the Apache Git Service. To respond to

[GitHub] asawitt opened a new pull request #4549: [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df

2019-01-17 Thread GitBox
asawitt opened a new pull request #4549: [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df URL: https://github.com/apache/airflow/pull/4549 [AIRFLOW-3725] Bigquery Hook authentication currently defaults to Google User-account credentials, and the user is asked to

[GitHub] mik-laj commented on issue #4514: [AIRFLOW-3698] Add documentation for AWS Connection

2019-01-17 Thread GitBox
mik-laj commented on issue #4514: [AIRFLOW-3698] Add documentation for AWS Connection URL: https://github.com/apache/airflow/pull/4514#issuecomment-455378999 @mik-laj What specific things do you need? `schema` part must be equals `aws`. Query parameters must cotanins extra parameters.

[GitHub] mik-laj commented on issue #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-17 Thread GitBox
mik-laj commented on issue #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#issuecomment-455137371 @ashb I added note in `UPDATING.md`. This is an automated message

[GitHub] chaimt opened a new pull request #4546: AIRFLOW-3720

2019-01-17 Thread GitBox
chaimt opened a new pull request #4546: AIRFLOW-3720 URL: https://github.com/apache/airflow/pull/4546 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow Jira] - https://issues.apache.org/jira/browse/AIRFLOW-3720 ###

[GitHub] potiuk commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-17 Thread GitBox
potiuk commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#discussion_r248617340 ## File path: Dockerfile ## @@ -16,26 +16,83 @@ FROM python:3.6-slim -COPY .

[GitHub] codecov-io commented on issue #4545: [AIRFLOW-XXX] Correct Typo in sensor's exception

2019-01-17 Thread GitBox
codecov-io commented on issue #4545: [AIRFLOW-XXX] Correct Typo in sensor's exception URL: https://github.com/apache/airflow/pull/4545#issuecomment-455145178 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4545?src=pr=h1) Report > Merging

[GitHub] dlamblin opened a new pull request #4545: [AIRFLOW-XXX] Correct Typo in sensor's exception

2019-01-17 Thread GitBox
dlamblin opened a new pull request #4545: [AIRFLOW-XXX] Correct Typo in sensor's exception URL: https://github.com/apache/airflow/pull/4545 ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references them in the

[GitHub] potiuk commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-17 Thread GitBox
potiuk commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#discussion_r248618573 ## File path: Dockerfile ## @@ -16,26 +16,83 @@ FROM python:3.6-slim -COPY .

[GitHub] mik-laj commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-17 Thread GitBox
mik-laj commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r248631253 ## File path: airflow/utils/log/s3_task_handler.py ## @@ -45,7 +45,7 @@ def _build_hook(self):

[GitHub] ashb commented on a change in pull request #4225: [AIRFLOW-3383] Rotate fernet keys.

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/airflow/pull/4225#discussion_r249108558 ## File path: airflow/bin/cli.py ## @@ -2109,7 +2120,12 @@ class CLIFactory(object): 'func':

[GitHub] potiuk commented on issue #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-18 Thread GitBox
potiuk commented on issue #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#issuecomment-455621176 Ok. Cool. It's good to disagree sometimes as it might lead to better final results :). I can do those things now:

[GitHub] Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database

2019-01-18 Thread GitBox
Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database URL: https://github.com/apache/airflow/pull/4396#discussion_r249078245 ## File path: tests/core.py ## @@ -1807,6 +1819,11 @@ def tearDown(self): @unittest.skip(reason="Skipping

[GitHub] Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database

2019-01-18 Thread GitBox
Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database URL: https://github.com/apache/airflow/pull/4396#discussion_r249076362 ## File path: airflow/models/__init__.py ## @@ -703,6 +644,57 @@ def __init__(self, task, execution_date,

[GitHub] Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database

2019-01-18 Thread GitBox
Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database URL: https://github.com/apache/airflow/pull/4396#discussion_r249076129 ## File path: airflow/models/__init__.py ## @@ -703,6 +644,57 @@ def __init__(self, task, execution_date,

[GitHub] bbcbruno opened a new pull request #4554: [AIRFLOW-3729] Support env variables with pod runtime information in KubernetesPodOperator

2019-01-18 Thread GitBox
bbcbruno opened a new pull request #4554: [AIRFLOW-3729] Support env variables with pod runtime information in KubernetesPodOperator URL: https://github.com/apache/airflow/pull/4554 ### Jira - [X] https://issues.apache.org/jira/browse/AIRFLOW-3729 ### Description - [X]

[GitHub] ashb merged pull request #4502: [AIRFLOW-3591] Fix start date, end date, duration for rescheduled tasks

2019-01-18 Thread GitBox
ashb merged pull request #4502: [AIRFLOW-3591] Fix start date, end date, duration for rescheduled tasks URL: https://github.com/apache/airflow/pull/4502 This is an automated message from the Apache Git Service. To respond

[GitHub] ashb commented on a change in pull request #4502: [AIRFLOW-3591] Fix start date, end date, duration for rescheduled tasks

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4502: [AIRFLOW-3591] Fix start date, end date, duration for rescheduled tasks URL: https://github.com/apache/airflow/pull/4502#discussion_r249115654 ## File path: airflow/www/views.py ## @@ -1728,33 +1728,15 @@ def gantt(self,

[GitHub] Fokko commented on issue #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-18 Thread GitBox
Fokko commented on issue #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#issuecomment-455506851 Having to regularly invalidate the cache would take away a couple of my concerns. Good to see that we can script the advanced

[GitHub] BrechtDeVlieger opened a new pull request #4553: [AIRFLOW-3541] Add Avro logical type conversion to bigquery hook

2019-01-18 Thread GitBox
BrechtDeVlieger opened a new pull request #4553: [AIRFLOW-3541] Add Avro logical type conversion to bigquery hook URL: https://github.com/apache/airflow/pull/4553 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[GitHub] ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#discussion_r249112432 ## File path: Dockerfile ## @@ -16,26 +16,77 @@ FROM python:3.6-slim -COPY .

[GitHub] ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#discussion_r249112331 ## File path: Dockerfile ## @@ -16,26 +16,77 @@ FROM python:3.6-slim -COPY .

[GitHub] ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#discussion_r249112795 ## File path: Dockerfile ## @@ -16,26 +16,77 @@ FROM python:3.6-slim -COPY .

[GitHub] ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#discussion_r249112971 ## File path: Dockerfile ## @@ -16,26 +16,77 @@ FROM python:3.6-slim -COPY .

[GitHub] ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#discussion_r249111627 ## File path: .dockerignore ## @@ -0,0 +1,23 @@ +# +# Licensed to the Apache Software

[GitHub] codecov-io edited a comment on issue #4523: [AIRFLOW-3616] Add aliases for schema with underscore

2019-01-18 Thread GitBox
codecov-io edited a comment on issue #4523: [AIRFLOW-3616] Add aliases for schema with underscore URL: https://github.com/apache/airflow/pull/4523#issuecomment-454115923 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4523?src=pr=h1) Report > Merging

[GitHub] r39132 merged pull request #4552: [Airflow 3728] remove double comma in hive_hooks.py

2019-01-18 Thread GitBox
r39132 merged pull request #4552: [Airflow 3728] remove double comma in hive_hooks.py URL: https://github.com/apache/airflow/pull/4552 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] ashb commented on issue #4475: [AIRFLOW-3602] Changes ImapHook to raise an exception on no attachments

2019-01-18 Thread GitBox
ashb commented on issue #4475: [AIRFLOW-3602] Changes ImapHook to raise an exception on no attachments URL: https://github.com/apache/airflow/pull/4475#issuecomment-455605712 ```bash $ mkdir tmp $ ls tmp/ $ echo $? 0 ```

[GitHub] r39132 commented on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12

2019-01-18 Thread GitBox
r39132 commented on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12 URL: https://github.com/apache/airflow/pull/3723#issuecomment-455617713 @dargueta I'll try to find some time to verify this on a linux VM and get back to you. In the meantime, can you resolve the conflicts? if it

[GitHub] codecov-io edited a comment on issue #4396: [AIRFLOW-3585] - Adding edges to database

2019-01-18 Thread GitBox
codecov-io edited a comment on issue #4396: [AIRFLOW-3585] - Adding edges to database URL: https://github.com/apache/airflow/pull/4396#issuecomment-450633941 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4396?src=pr=h1) Report > Merging

[GitHub] codecov-io edited a comment on issue #4396: [AIRFLOW-3585] - Adding edges to database

2019-01-18 Thread GitBox
codecov-io edited a comment on issue #4396: [AIRFLOW-3585] - Adding edges to database URL: https://github.com/apache/airflow/pull/4396#issuecomment-450633941 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4396?src=pr=h1) Report > Merging

[GitHub] Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database

2019-01-18 Thread GitBox
Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database URL: https://github.com/apache/airflow/pull/4396#discussion_r249077794 ## File path: airflow/models/__init__.py ## @@ -4189,16 +4243,18 @@ def create_dagrun(self,

[GitHub] Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database

2019-01-18 Thread GitBox
Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database URL: https://github.com/apache/airflow/pull/4396#discussion_r249078557 ## File path: tests/www/test_views.py ## @@ -728,15 +728,22 @@ def __init__(self, test, endpoint): def

[GitHub] Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database

2019-01-18 Thread GitBox
Fokko commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to database URL: https://github.com/apache/airflow/pull/4396#discussion_r249078144 ## File path: tests/core.py ## @@ -107,7 +107,17 @@ def execute(*args, **kwargs): class

[GitHub] ashb commented on a change in pull request #4225: [AIRFLOW-3383] Rotate fernet keys.

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/airflow/pull/4225#discussion_r249108075 ## File path: tests/models.py ## @@ -2991,27 +2993,118 @@ def test_xcom_get_many(self):

[GitHub] ashb commented on a change in pull request #4523: [AIRFLOW-3616] Add aliases for schema with underscore

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4523: [AIRFLOW-3616] Add aliases for schema with underscore URL: https://github.com/apache/airflow/pull/4523#discussion_r249109494 ## File path: docs/howto/manage-connections.rst ## @@ -151,6 +156,29 @@ Scopes (comma separated)

[GitHub] ashb commented on a change in pull request #4523: [AIRFLOW-3616] Add aliases for schema with underscore

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4523: [AIRFLOW-3616] Add aliases for schema with underscore URL: https://github.com/apache/airflow/pull/4523#discussion_r249109716 ## File path: tests/models.py ## @@ -3077,6 +3077,23 @@ def

[GitHub] ashb commented on a change in pull request #4523: [AIRFLOW-3616] Add aliases for schema with underscore

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4523: [AIRFLOW-3616] Add aliases for schema with underscore URL: https://github.com/apache/airflow/pull/4523#discussion_r249109381 ## File path: docs/howto/manage-connections.rst ## @@ -70,10 +70,15 @@ When referencing the

[GitHub] abitrolly commented on issue #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI

2019-01-18 Thread GitBox
abitrolly commented on issue #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI URL: https://github.com/apache/airflow/pull/4548#issuecomment-455504778 > We don't test UI stuff currently Hmm..

[GitHub] abitrolly edited a comment on issue #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI

2019-01-18 Thread GitBox
abitrolly edited a comment on issue #4548: [AIRFLOW-3724] Fix the broken refresh button on Graph View in RBAC UI URL: https://github.com/apache/airflow/pull/4548#issuecomment-455504778 > We don't test UI stuff currently Hmm..

[GitHub] suensummit opened a new pull request #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods

2019-01-18 Thread GitBox
suensummit opened a new pull request #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods URL: https://github.com/apache/airflow/pull/4551 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[GitHub] Fokko edited a comment on issue #4543: [AIRFLOW-3718] Multi-layered version of the docker image

2019-01-18 Thread GitBox
Fokko edited a comment on issue #4543: [AIRFLOW-3718] Multi-layered version of the docker image URL: https://github.com/apache/airflow/pull/4543#issuecomment-455506851 Having to regularly invalidate the cache would take away a couple of my concerns. Good to see that we can script the

[GitHub] zhongjiajie opened a new pull request #4552: [Airflow 3728] remove double comma in hive_hooks.py

2019-01-18 Thread GitBox
zhongjiajie opened a new pull request #4552: [Airflow 3728] remove double comma in hive_hooks.py URL: https://github.com/apache/airflow/pull/4552 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[GitHub] stale[bot] commented on issue #3367: [AIRFLOW-2475] Reference triggered dag when using TriggerDagRunOperator

2019-01-18 Thread GitBox
stale[bot] commented on issue #3367: [AIRFLOW-2475] Reference triggered dag when using TriggerDagRunOperator URL: https://github.com/apache/airflow/pull/3367#issuecomment-43114 This issue has been automatically marked as stale because it has not had recent activity. It will be closed

[GitHub] codecov-io edited a comment on issue #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods

2019-01-18 Thread GitBox
codecov-io edited a comment on issue #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods URL: https://github.com/apache/airflow/pull/4551#issuecomment-455531251 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4551?src=pr=h1) Report > Merging

[GitHub] codecov-io commented on issue #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods

2019-01-18 Thread GitBox
codecov-io commented on issue #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods URL: https://github.com/apache/airflow/pull/4551#issuecomment-455531251 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4551?src=pr=h1) Report > Merging

[GitHub] codecov-io edited a comment on issue #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods

2019-01-18 Thread GitBox
codecov-io edited a comment on issue #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods URL: https://github.com/apache/airflow/pull/4551#issuecomment-455531251 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4551?src=pr=h1) Report > Merging

[GitHub] dargueta commented on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12

2019-01-18 Thread GitBox
dargueta commented on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12 URL: https://github.com/apache/airflow/pull/3723#issuecomment-455650114 This isn't my PR, unfortunately. @Fokko could you fix the conflicts and change the pinning to `>= 4.10.0, <5.0`? I think 5.x will work but it's

[GitHub] ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r249178779 ## File path: setup.py ## @@ -335,10 +383,8 @@ def do_setup(): 'all_dbs': all_dbs,

[GitHub] ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r249177730 ## File path: UPDATING.md ## @@ -24,6 +24,15 @@ assists users migrating to a new version. ##

[GitHub] ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r249178310 ## File path: setup.py ## @@ -157,30 +160,29 @@ def

[GitHub] ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r249178692 ## File path: setup.py ## @@ -211,27 +217,44 @@ def

[GitHub] ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r249177814 ## File path: UPDATING.md ## @@ -24,6 +24,15 @@ assists users migrating to a new version. ##

[GitHub] ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r249178124 ## File path: docs/installation.rst ## @@ -56,6 +56,10 @@ Here's the list of the subpackages and

[GitHub] ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r249177942 ## File path: airflow/utils/log/s3_task_handler.py ## @@ -45,7 +45,7 @@ def _build_hook(self):

[GitHub] ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers

2019-01-18 Thread GitBox
ashb commented on a change in pull request #4524: [AIRFLOW-3707] Group subpackages by cloud providers URL: https://github.com/apache/airflow/pull/4524#discussion_r249177607 ## File path: UPDATING.md ## @@ -24,6 +24,15 @@ assists users migrating to a new version. ##

[GitHub] codecov-io edited a comment on issue #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods

2019-01-18 Thread GitBox
codecov-io edited a comment on issue #4551: [AIRFLOW-2955] Fix kubernetes pod operator to set requests and limits on task pods URL: https://github.com/apache/airflow/pull/4551#issuecomment-455531251 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4551?src=pr=h1) Report > Merging

[GitHub] criccomini commented on issue #4549: [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df

2019-01-18 Thread GitBox
criccomini commented on issue #4549: [AIRFLOW-3725] Add private_key to bigquery_hook get_pandas_df URL: https://github.com/apache/airflow/pull/4549#issuecomment-455658442 Failures in CI test are K8s issues that are unrelated to this PR and are currently failing on master:

[GitHub] zihengCat opened a new pull request #4555: [AIRFLOW-XXX] Add Firestone Inventing as a Big Data & AI company uses Apache Airflow into README.md

2019-01-18 Thread GitBox
zihengCat opened a new pull request #4555: [AIRFLOW-XXX] Add Firestone Inventing as a Big Data & AI company uses Apache Airflow into README.md URL: https://github.com/apache/airflow/pull/4555 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the

[GitHub] dargueta edited a comment on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12

2019-01-18 Thread GitBox
dargueta edited a comment on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12 URL: https://github.com/apache/airflow/pull/3723#issuecomment-455650114 This isn't my PR, unfortunately. @Fokko could you fix the conflicts and change the pinning to `>= 4.11.0, <5.0`? I think 5.x will work

[GitHub] max-sixty commented on issue #4511: [AIRFLOW-3695] Replace Flake8 by Pylint

2019-01-18 Thread GitBox
max-sixty commented on issue #4511: [AIRFLOW-3695] Replace Flake8 by Pylint URL: https://github.com/apache/airflow/pull/4511#issuecomment-455671280 I haven't been involved as an airflow developer, but saw this on the mailing list and thought it would be helpful to comment given my

[GitHub] villasv commented on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12

2019-01-18 Thread GitBox
villasv commented on issue #3723: [AIRFLOW-2876] Update Tenacity to 4.12 URL: https://github.com/apache/airflow/pull/3723#issuecomment-455660200 In #4217 I’ve tried to use tenacity>5 and tests didn’t pass. Also the example HttpOperator DAG breaks.

[GitHub] Licht-T commented on issue #2906: [AIRFLOW-1956] Add parameter whether the navbar clock time is UTC

2019-01-18 Thread GitBox
Licht-T commented on issue #2906: [AIRFLOW-1956] Add parameter whether the navbar clock time is UTC URL: https://github.com/apache/airflow/pull/2906#issuecomment-455737905 @nmcalabroso @ron819 Now the navbar clock follows the Airflow internal timezone setting. Should I change the

[GitHub] zhongjiajie commented on issue #4552: [Airflow 3728] remove double comma in hive_hooks.py

2019-01-18 Thread GitBox
zhongjiajie commented on issue #4552: [Airflow 3728] remove double comma in hive_hooks.py URL: https://github.com/apache/airflow/pull/4552#issuecomment-455742265 > For minor changes, you can just use AIRFLOW-XXX as the jira - no need to create a JIRA. @r39132 Ok, next time I will

[GitHub] codecov-io edited a comment on issue #4550: [Airflow 3714] Correct DAG name in docs/start.rst

2019-01-18 Thread GitBox
codecov-io edited a comment on issue #4550: [Airflow 3714] Correct DAG name in docs/start.rst URL: https://github.com/apache/airflow/pull/4550#issuecomment-455434092 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4550?src=pr=h1) Report > Merging

[GitHub] mik-laj commented on a change in pull request #4460: [AIRFLOW-3604] Add Azure Hdinsight operators for creating cluster and submitting

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4460: [AIRFLOW-3604] Add Azure Hdinsight operators for creating cluster and submitting URL: https://github.com/apache/airflow/pull/4460#discussion_r249216549 ## File path: airflow/contrib/hooks/azure_hdinsight_hook.py ## @@

[GitHub] mik-laj commented on a change in pull request #4460: [AIRFLOW-3604] Add Azure Hdinsight operators for creating cluster and submitting

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4460: [AIRFLOW-3604] Add Azure Hdinsight operators for creating cluster and submitting URL: https://github.com/apache/airflow/pull/4460#discussion_r247306177 ## File path: setup.py ## @@ -349,6 +353,7 @@ def do_setup():

[GitHub] mik-laj commented on a change in pull request #4460: [AIRFLOW-3604] Add Azure Hdinsight operators for creating cluster and submitting

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4460: [AIRFLOW-3604] Add Azure Hdinsight operators for creating cluster and submitting URL: https://github.com/apache/airflow/pull/4460#discussion_r247306177 ## File path: setup.py ## @@ -349,6 +353,7 @@ def do_setup():

[GitHub] mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator URL: https://github.com/apache/airflow/pull/4530#discussion_r249239029 ## File path: airflow/contrib/hooks/azure_kubernetes_hook.py ## @@ -0,0 +1,75 @@ +# -*-

[GitHub] mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator URL: https://github.com/apache/airflow/pull/4530#discussion_r249205052 ## File path: airflow/contrib/operators/aks_operator.py ## @@ -0,0 +1,200 @@ +# -*-

[GitHub] mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator URL: https://github.com/apache/airflow/pull/4530#discussion_r249203936 ## File path: airflow/contrib/operators/aks_operator.py ## @@ -0,0 +1,200 @@ +# -*-

[GitHub] mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator URL: https://github.com/apache/airflow/pull/4530#discussion_r249205052 ## File path: airflow/contrib/operators/aks_operator.py ## @@ -0,0 +1,200 @@ +# -*-

[GitHub] echo-ray commented on issue #4339: [AIRFLOW-3303] Deprecate old UI in favor of FAB

2019-01-19 Thread GitBox
echo-ray commented on issue #4339: [AIRFLOW-3303] Deprecate old UI in favor of FAB URL: https://github.com/apache/airflow/pull/4339#issuecomment-455761779 the new UI with role manager is pretty nice, but I have took lot of time to get it there. Hope the new UI would be default in the

[GitHub] mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4530: [AIRFLOW-3282] Implement Azure Kubernetes Service Operator URL: https://github.com/apache/airflow/pull/4530#discussion_r249239186 ## File path: airflow/contrib/utils/aks_utils.py ## @@ -0,0 +1,92 @@ +# -*- coding: utf-8

[GitHub] mik-laj commented on a change in pull request #4303: [AIRFLOW-3370] Elasticsearch log task handler additional features

2019-01-19 Thread GitBox
mik-laj commented on a change in pull request #4303: [AIRFLOW-3370] Elasticsearch log task handler additional features URL: https://github.com/apache/airflow/pull/4303#discussion_r249239393 ## File path: docs/howto/write-logs.rst ## @@ -135,3 +135,103 @@ example:

[GitHub] ffinfo commented on issue #4396: [AIRFLOW-3585] - Add edges to database

2019-01-19 Thread GitBox
ffinfo commented on issue #4396: [AIRFLOW-3585] - Add edges to database URL: https://github.com/apache/airflow/pull/4396#issuecomment-455764359 @feng-tao In the end this is part of this bigger issue: https://jira.apache.org/jira/browse/AIRFLOW-3562 In the end this would solve the

<    1   2   3   4   5   6   7   8   9   10   >