[jira] [Commented] (AIRFLOW-6704) TaskInstance.operator is not set when task is marked success or failed in the Web UI

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132976#comment-17132976 ] ASF GitHub Bot commented on AIRFLOW-6704: - yuqian90 commented on pull request #

[GitHub] [airflow] yuqian90 commented on pull request #7324: [AIRFLOW-6704] Copy common TaskInstance attributes from Task

2020-06-10 Thread GitBox
yuqian90 commented on pull request #7324: URL: https://github.com/apache/airflow/pull/7324#issuecomment-642451211 > @yuqian90 : Thank You ! for your response. > If so be the case then just wondering the same code with `execution_date` works perfectly fine with the earlier version of Airf

[airflow] tag nightly-master updated (82c8343 -> fb1c8b8)

2020-06-10 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 82c8343 (commit) to fb1c8b8 (commit) from 82

[jira] [Commented] (AIRFLOW-6704) TaskInstance.operator is not set when task is marked success or failed in the Web UI

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132875#comment-17132875 ] ASF GitHub Bot commented on AIRFLOW-6704: - shanit-saha commented on pull reques

[GitHub] [airflow] shanit-saha commented on pull request #7324: [AIRFLOW-6704] Copy common TaskInstance attributes from Task

2020-06-10 Thread GitBox
shanit-saha commented on pull request #7324: URL: https://github.com/apache/airflow/pull/7324#issuecomment-642381084 @yuqian90 : Thank You ! for your response. If so be the case then just wondering the same code with `execution_date` works perfectly fine with the earlier version of Airfl

[GitHub] [airflow] kaxil commented on a change in pull request #8974: Add VaultHook to hashicorp provider

2020-06-10 Thread GitBox
kaxil commented on a change in pull request #8974: URL: https://github.com/apache/airflow/pull/8974#discussion_r438491756 ## File path: airflow/providers/hashicorp/common/vault_client.py ## @@ -0,0 +1,458 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or mo

[jira] [Commented] (AIRFLOW-6704) TaskInstance.operator is not set when task is marked success or failed in the Web UI

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132817#comment-17132817 ] ASF GitHub Bot commented on AIRFLOW-6704: - yuqian90 commented on pull request #

[GitHub] [airflow] yuqian90 commented on pull request #7324: [AIRFLOW-6704] Copy common TaskInstance attributes from Task

2020-06-10 Thread GitBox
yuqian90 commented on pull request #7324: URL: https://github.com/apache/airflow/pull/7324#issuecomment-642350478 Hi @shanit-saha > execution_date=datetime.now() I think this line is the cause of the problem. I believe what you actually want to express is to trigger `dag_proc

[GitHub] [airflow] kaxil commented on a change in pull request #8974: Add VaultHook to hashicorp provider

2020-06-10 Thread GitBox
kaxil commented on a change in pull request #8974: URL: https://github.com/apache/airflow/pull/8974#discussion_r438491649 ## File path: airflow/providers/hashicorp/common/vault_client.py ## @@ -0,0 +1,458 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or mo

[GitHub] [airflow] ashb opened a new pull request #9224: Fix up coredns pods in KIND CI

2020-06-10 Thread GitBox
ashb opened a new pull request #9224: URL: https://github.com/apache/airflow/pull/9224 These were passing for a while, but seemingly broke after _something_ outside of the airflow code base changed (DNS of Github Actions runners? Phase of the moon) so adding this config map that we had

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438477694 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -59,9 +68,60 @@ GCLOUD_IMAGE="gcr.io/google.com/cloudsdktool/cloud-sdk:latest" prepare_tool_scr

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438476917 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -72,25 +132,36 @@ function prepare_terraform_script() { cat >"${TARGET_TOOL_PATH}" <

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438476274 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -29,19 +29,28 @@ function prepare_tool_script() { cat >"${TARGET_TOOL_PATH}" &1

[GitHub] [airflow] ashb commented on a change in pull request #9220: Get all pod logs on Kube CI failure

2020-06-10 Thread GitBox
ashb commented on a change in pull request #9220: URL: https://github.com/apache/airflow/pull/9220#discussion_r438468236 ## File path: scripts/ci/libraries/_kind.sh ## @@ -365,19 +365,23 @@ function apply_kubernetes_resources() { function dump_kubernetes_logs() { POD=$(ku

[GitHub] [airflow] ashb commented on issue #9221: UI/TZ issue: incorectly mapping DAGs

2020-06-10 Thread GitBox
ashb commented on issue #9221: URL: https://github.com/apache/airflow/issues/9221#issuecomment-642328422 What frequency is your dag? Hourly? (This is not part of the problem, but would help in reproducing the issue) This is

[GitHub] [airflow] mik-laj opened a new pull request #9223: Add generic CLI tool wrappers

2020-06-10 Thread GitBox
mik-laj opened a new pull request #9223: URL: https://github.com/apache/airflow/pull/9223 NOTE TO REVIEWERS. This commit is based on https://github.com/apache/airflow/pull/9219 so please review only last commit. I wanted to present a generic wrapper for CLI tools that uses only one

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #9153: [WIP] add readonly endpoints for dagruns

2020-06-10 Thread GitBox
ephraimbuddy commented on a change in pull request #9153: URL: https://github.com/apache/airflow/pull/9153#discussion_r438468200 ## File path: airflow/api_connexion/endpoints/dag_run_endpoint.py ## @@ -26,18 +34,66 @@ def delete_dag_run(): raise NotImplementedError("Not im

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #9153: [WIP] add readonly endpoints for dagruns

2020-06-10 Thread GitBox
ephraimbuddy commented on a change in pull request #9153: URL: https://github.com/apache/airflow/pull/9153#discussion_r438468343 ## File path: airflow/api_connexion/endpoints/dag_run_endpoint.py ## @@ -26,18 +34,66 @@ def delete_dag_run(): raise NotImplementedError("Not im

[GitHub] [airflow] ashb commented on a change in pull request #9220: Get all pod logs on Kube CI failure

2020-06-10 Thread GitBox
ashb commented on a change in pull request #9220: URL: https://github.com/apache/airflow/pull/9220#discussion_r438468236 ## File path: scripts/ci/libraries/_kind.sh ## @@ -365,19 +365,23 @@ function apply_kubernetes_resources() { function dump_kubernetes_logs() { POD=$(ku

[GitHub] [airflow] Acehaidrey commented on pull request #8598: make hive macros return string type vs bytes

2020-06-10 Thread GitBox
Acehaidrey commented on pull request #8598: URL: https://github.com/apache/airflow/pull/8598#issuecomment-642322974 @ashb wanted too knoow if there were any more concerns or if we merge this This is an automated message from

[GitHub] [airflow] ashb commented on a change in pull request #9220: Get all pod logs on Kube CI failure

2020-06-10 Thread GitBox
ashb commented on a change in pull request #9220: URL: https://github.com/apache/airflow/pull/9220#discussion_r438465926 ## File path: scripts/ci/libraries/_kind.sh ## @@ -365,19 +365,23 @@ function apply_kubernetes_resources() { function dump_kubernetes_logs() { POD=$(ku

[GitHub] [airflow] ashb commented on a change in pull request #9220: Get all pod logs on Kube CI failure

2020-06-10 Thread GitBox
ashb commented on a change in pull request #9220: URL: https://github.com/apache/airflow/pull/9220#discussion_r438465611 ## File path: scripts/ci/libraries/_kind.sh ## @@ -365,19 +365,23 @@ function apply_kubernetes_resources() { function dump_kubernetes_logs() { POD=$(ku

[GitHub] [airflow] dimberman commented on pull request #9220: Get all logs on failure

2020-06-10 Thread GitBox
dimberman commented on pull request #9220: URL: https://github.com/apache/airflow/pull/9220#issuecomment-642311700 @ashb yes was just thinking that This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [airflow] ashb commented on pull request #9220: Get all logs on failure

2020-06-10 Thread GitBox
ashb commented on pull request #9220: URL: https://github.com/apache/airflow/pull/9220#issuecomment-642311498 @dimberman Can you also make it dump the logs/status of postgres, and I guess kubedns/coredns/whatever DNS is actually in use?

[GitHub] [airflow] potiuk commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438434218 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -59,9 +68,60 @@ GCLOUD_IMAGE="gcr.io/google.com/cloudsdktool/cloud-sdk:latest" prepare_tool_scri

[GitHub] [airflow] potiuk commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438435306 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -72,25 +132,36 @@ function prepare_terraform_script() { cat >"${TARGET_TOOL_PATH}" <

[GitHub] [airflow] potiuk commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438434218 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -59,9 +68,60 @@ GCLOUD_IMAGE="gcr.io/google.com/cloudsdktool/cloud-sdk:latest" prepare_tool_scri

[GitHub] [airflow] potiuk commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438432062 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -29,19 +29,28 @@ function prepare_tool_script() { cat >"${TARGET_TOOL_PATH}" <

[GitHub] [airflow] svetli-n commented on issue #9197: dagrun_operator in Airflow Version 1.10.10 ERRORS _run_raw_task result = task_copy.execute(context=context)

2020-06-10 Thread GitBox
svetli-n commented on issue #9197: URL: https://github.com/apache/airflow/issues/9197#issuecomment-642289763 There is no `python_callable` in 1.10.10. It was removed with f3c3812908fe6bb802b7a24011fe21e4b46530bf Would it work if you extracted `set_up_dag_run_preprocessing` to new/ex

[GitHub] [airflow] potiuk commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438430965 ## File path: scripts/ci/docker-compose/local.yml ## @@ -56,7 +56,7 @@ services: - ../../../setup.py:/opt/airflow/setup.py:cached - ../../../

[GitHub] [airflow] potiuk commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438430716 ## File path: scripts/ci/docker-compose/local-prod.yml ## @@ -35,7 +35,7 @@ services: - ../../../setup.cfg:/opt/airflow/setup.cfg:cached - ..

[GitHub] [airflow] ashb opened a new pull request #9222: Correctly colour full warning/error log message with parameters.

2020-06-10 Thread GitBox
ashb opened a new pull request #9222: URL: https://github.com/apache/airflow/pull/9222 Previously after the first parameter the colour was lost, as `color()` included the reset code always. I have fixed it by using `\e[22m` -- which resets the bold/thin state, but not the colour.

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9221: UI/TZ issue: incorectly mapping DAGs

2020-06-10 Thread GitBox
boring-cyborg[bot] commented on issue #9221: URL: https://github.com/apache/airflow/issues/9221#issuecomment-642271824 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] ebacal opened a new issue #9221: UI/TZ issue: incorectly mapping DAGs

2020-06-10 Thread GitBox
ebacal opened a new issue #9221: URL: https://github.com/apache/airflow/issues/9221 **1.10.9 and 1.10.10**: **MysQL**: - **5.7**: - **TZ=PST** - **airflow.cfg tested with time_zone=system, Los_Angeles/America, utc** **What happened**:

[GitHub] [airflow] dimberman opened a new pull request #9220: Get all logs on failure

2020-06-10 Thread GitBox
dimberman opened a new pull request #9220: URL: https://github.com/apache/airflow/pull/9220 uses `--all-containers=true` flag to get all k8s logs --- Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Uni

[GitHub] [airflow] dimberman commented on pull request #9220: Get all logs on failure

2020-06-10 Thread GitBox
dimberman commented on pull request #9220: URL: https://github.com/apache/airflow/pull/9220#issuecomment-642270936 @potiuk @ashb this should help us figure out why the init-db container is failing. This is an automated messa

[GitHub] [airflow] samuelkhtu commented on pull request #9124: [AIRFLOW-9099] Merging multiple sql operators

2020-06-10 Thread GitBox
samuelkhtu commented on pull request #9124: URL: https://github.com/apache/airflow/pull/9124#issuecomment-642267078 Yes @mik-lajThank you for the feedback. I will work on this ASAP. This is an automated message from the

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438404044 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -29,19 +29,28 @@ function prepare_tool_script() { cat >"${TARGET_TOOL_PATH}" <

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438403264 ## File path: scripts/ci/docker-compose/local-prod.yml ## @@ -35,7 +35,7 @@ services: - ../../../setup.cfg:/opt/airflow/setup.cfg:cached - .

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438403264 ## File path: scripts/ci/docker-compose/local-prod.yml ## @@ -35,7 +35,7 @@ services: - ../../../setup.cfg:/opt/airflow/setup.cfg:cached - .

[GitHub] [airflow] mik-laj commented on pull request #9216: Fixes lack of terminal in cli tools

2020-06-10 Thread GitBox
mik-laj commented on pull request #9216: URL: https://github.com/apache/airflow/pull/9216#issuecomment-642257901 Hello. I proposed a similar change. We found a solution at the same time, but I still had to solve other problems. https://github.com/apache/airflow/pull/9219/files Do

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438405110 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -72,25 +132,36 @@ function prepare_terraform_script() { cat >"${TARGET_TOOL_PATH}" <

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438404330 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -59,9 +68,60 @@ GCLOUD_IMAGE="gcr.io/google.com/cloudsdktool/cloud-sdk:latest" prepare_tool_scr

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438404330 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -59,9 +68,60 @@ GCLOUD_IMAGE="gcr.io/google.com/cloudsdktool/cloud-sdk:latest" prepare_tool_scr

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438404044 ## File path: scripts/ci/prepare_tool_scripts.sh ## @@ -29,19 +29,28 @@ function prepare_tool_script() { cat >"${TARGET_TOOL_PATH}" <

[GitHub] [airflow] mik-laj commented on a change in pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj commented on a change in pull request #9219: URL: https://github.com/apache/airflow/pull/9219#discussion_r438403264 ## File path: scripts/ci/docker-compose/local-prod.yml ## @@ -35,7 +35,7 @@ services: - ../../../setup.cfg:/opt/airflow/setup.cfg:cached - .

[GitHub] [airflow] mik-laj opened a new pull request #9219: Fix system tests for DataflowCreateJavaJobOperator

2020-06-10 Thread GitBox
mik-laj opened a new pull request #9219: URL: https://github.com/apache/airflow/pull/9219 --- Make sure to mark the boxes below before creating PR: [x] - [X] Description above provides context of the change - [X] Unit tests coverage for changes (not needed for documentation chan

[GitHub] [airflow] tooptoop4 commented on pull request #7637: [AIRFLOW-6994] SparkSubmitOperator re-launches spark driver even when original driver still running

2020-06-10 Thread GitBox
tooptoop4 commented on pull request #7637: URL: https://github.com/apache/airflow/pull/7637#issuecomment-642237702 it is trivial change tho This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-6994) SparkSubmitOperator re launches spark driver even when original driver still running

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132695#comment-17132695 ] ASF GitHub Bot commented on AIRFLOW-6994: - tooptoop4 commented on pull request

[GitHub] [airflow] ashb commented on pull request #9208: Make it possible to silence warnings from Airflow

2020-06-10 Thread GitBox
ashb commented on pull request #9208: URL: https://github.com/apache/airflow/pull/9208#issuecomment-642219594 we could, I kind of don't want it easily discoverable - I'd rather people fix the warnings. (I want this for the exe)c based health check output to be cleaner ---

[GitHub] [airflow] turbaszek commented on pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
turbaszek commented on pull request #9215: URL: https://github.com/apache/airflow/pull/9215#issuecomment-642217410 @dimberman as expected, this wasn't an issue. I've rebased to the first PR. Any ideas about what could have happened? ---

[GitHub] [airflow] turbaszek edited a comment on pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
turbaszek edited a comment on pull request #9215: URL: https://github.com/apache/airflow/pull/9215#issuecomment-642217410 @dimberman as expected, this wasn't an issue. I've rebased to the first commit. Any ideas about what could have happened?

[GitHub] [airflow] mik-laj commented on issue #8144: API Endpoint - Health - Spec and impelementataion

2020-06-10 Thread GitBox
mik-laj commented on issue #8144: URL: https://github.com/apache/airflow/issues/8144#issuecomment-642214912 Hello @OmairK Sure. You can rely on https://github.com/apache/airflow/blob/e405be0141a996c5bb3659e3f19cab0e1ac8dc8d/docs/howto/check-health.rst --

[GitHub] [airflow] jkbngl opened a new pull request #9218: Mssql to oracle operator

2020-06-10 Thread GitBox
jkbngl opened a new pull request #9218: URL: https://github.com/apache/airflow/pull/9218 --- Make sure to mark the boxes below before creating PR: [x] Added Operator to transfer data from MSSql to Oracle, building on top of oracleToOracleTransfer operator. - [X] Description

[GitHub] [airflow] OmairK commented on issue #8144: API Endpoint - Health - Spec and impelementataion

2020-06-10 Thread GitBox
OmairK commented on issue #8144: URL: https://github.com/apache/airflow/issues/8144#issuecomment-642210452 Hey @mik-laj I find this issue interesting and I would like to work on it. I will start by preparing the health endpoint specification and getting it approved before implementing it.

[GitHub] [airflow] dimberman commented on pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
dimberman commented on pull request #9215: URL: https://github.com/apache/airflow/pull/9215#issuecomment-642207592 @turbaszek please let me know if this is the solution. cc: @mik-laj. This is an automated message from the Apa

[GitHub] [airflow] OmairK opened a new pull request #9217: [WIP] import error endpoints

2020-06-10 Thread GitBox
OmairK opened a new pull request #9217: URL: https://github.com/apache/airflow/pull/9217 Closes #8130 --- Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Unit tests coverage for changes (not needed for do

[GitHub] [airflow] turbaszek commented on pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
turbaszek commented on pull request #9215: URL: https://github.com/apache/airflow/pull/9215#issuecomment-642204414 This seems to be not related change but looking at commit history this problem persists since https://github.com/apache/airflow/pull/9207 -

[GitHub] [airflow] dimberman commented on pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
dimberman commented on pull request #9215: URL: https://github.com/apache/airflow/pull/9215#issuecomment-642201517 @potiuk could this have to do with moving to host? This is an automated message from the Apache Git Service.

[GitHub] [airflow] turbaszek commented on pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
turbaszek commented on pull request #9215: URL: https://github.com/apache/airflow/pull/9215#issuecomment-642200812 Hi @dimberman @potiuk it seems we have a problem with k8s ci builds: ``` psycopg2.OperationalError: could not translate host name "postgres-airflow" to address: Temporary

[GitHub] [airflow] potiuk commented on pull request #9216: Fixes lack of terminal in cli tools

2020-06-10 Thread GitBox
potiuk commented on pull request #9216: URL: https://github.com/apache/airflow/pull/9216#issuecomment-642198237 But I think the second part is not correct --interactive is about stdin, not stdout and has nothing to do with terminal allocation.

[GitHub] [airflow] jkbngl closed pull request #9201: Added MSSql To Oracle Transfer Operator

2020-06-10 Thread GitBox
jkbngl closed pull request #9201: URL: https://github.com/apache/airflow/pull/9201 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [airflow] potiuk commented on pull request #9216: Fixes lack of terminal in cli tools

2020-06-10 Thread GitBox
potiuk commented on pull request #9216: URL: https://github.com/apache/airflow/pull/9216#issuecomment-642195178 Please check @mik-laj if it solves your problem. It worked in my tests. This is an automated message from the Ap

[GitHub] [airflow] potiuk commented on issue #9193: Breaks production Docker image: Celery 4.4.4 (known bug) lacks "future" module thus breaks airflow

2020-06-10 Thread GitBox
potiuk commented on issue #9193: URL: https://github.com/apache/airflow/issues/9193#issuecomment-642175919 Glad it helped :) This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [airflow] BasPH commented on pull request #9214: flipped boolean to support KMS decryption for ssm ps

2020-06-10 Thread GitBox
BasPH commented on pull request #9214: URL: https://github.com/apache/airflow/pull/9214#issuecomment-642174397 How is this change backwards compatible? Does AWS automatically "fallback" in case of a non-secure string? This i

[GitHub] [airflow] turbaszek commented on a change in pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
turbaszek commented on a change in pull request #9215: URL: https://github.com/apache/airflow/pull/9215#discussion_r438315615 ## File path: .github/workflows/ci.yml ## @@ -174,7 +174,7 @@ jobs: timeout-minutes: 80 name: "K8s: ${{matrix.kube-mode}} ${{matrix.python-ver

[GitHub] [airflow] potiuk commented on a change in pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #9215: URL: https://github.com/apache/airflow/pull/9215#discussion_r438314235 ## File path: .github/workflows/ci.yml ## @@ -174,7 +174,7 @@ jobs: timeout-minutes: 80 name: "K8s: ${{matrix.kube-mode}} ${{matrix.python-versio

[GitHub] [airflow] iam432 commented on issue #8609: DAG's parameter access_control is not refreshing in the UI

2020-06-10 Thread GitBox
iam432 commented on issue #8609: URL: https://github.com/apache/airflow/issues/8609#issuecomment-642156970 Thank you for the response, I am able to automate this by creating a DAG and running it with a schedule of every 2 minutes and it solved but it is a workaround. Can you guide

[jira] [Commented] (AIRFLOW-3126) kubernetes executor not allowing new PV

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132538#comment-17132538 ] ASF GitHub Bot commented on AIRFLOW-3126: - brandonwillard commented on a change

[GitHub] [airflow] brandonwillard commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes

2020-06-10 Thread GitBox
brandonwillard commented on a change in pull request #8150: URL: https://github.com/apache/airflow/pull/8150#discussion_r438291967 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -881,6 +881,15 @@ logs_volume_subpath = # A shared volume claim for the logs log

[GitHub] [airflow] l0yu commented on issue #9213: FILTER_BY_ONWER does not work for multi-owner dags

2020-06-10 Thread GitBox
l0yu commented on issue #9213: URL: https://github.com/apache/airflow/issues/9213#issuecomment-642147301 can anyone apply the fix 843a22f to next release like 1.11 ? This is an automated message from the Apache Git Service. T

[GitHub] [airflow] RachaelDS commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-06-10 Thread GitBox
RachaelDS commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r438284379 ## File path: airflow/providers/google/cloud/operators/datafusion.py ## @@ -636,13 +648,15 @@ class CloudDataFusionStartPipelineOperator(BaseOperator):

[GitHub] [airflow] turbaszek opened a new pull request #9215: Fix init containers names on CI

2020-06-10 Thread GitBox
turbaszek opened a new pull request #9215: URL: https://github.com/apache/airflow/pull/9215 ``` --- webserver init container logs - init --- error: container init is not valid for pod airflow-594f8b87d-6ph6k ``` --- Make sure to mark the boxes below before creating

[GitHub] [airflow] crhyatt commented on a change in pull request #9184: Upgrade pendulum to latest major version ~2.0

2020-06-10 Thread GitBox
crhyatt commented on a change in pull request #9184: URL: https://github.com/apache/airflow/pull/9184#discussion_r438265522 ## File path: airflow/models/dag.py ## @@ -412,15 +412,15 @@ def following_schedule(self, dttm): if not self.is_fixed_time_schedule():

[airflow] branch master updated (a26afbf -> fb1c8b8)

2020-06-10 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from a26afbf Make generated job_id more informative in BQ insert_job (#9203) add fb1c8b8 Add test for BQ opera

[airflow] branch master updated (7f02e56 -> a26afbf)

2020-06-10 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 7f02e56 Add metavar to CLI arguments (#9077) add a26afbf Make generated job_id more informative in BQ inse

[GitHub] [airflow] turbaszek merged pull request #9206: Add test for BQ operations using location

2020-06-10 Thread GitBox
turbaszek merged pull request #9206: URL: https://github.com/apache/airflow/pull/9206 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [airflow] turbaszek merged pull request #9203: Make generated job_id more informative in BQ insert_job

2020-06-10 Thread GitBox
turbaszek merged pull request #9203: URL: https://github.com/apache/airflow/pull/9203 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [airflow] jaketf commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-06-10 Thread GitBox
jaketf commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r438261904 ## File path: airflow/providers/google/cloud/operators/datafusion.py ## @@ -636,13 +648,15 @@ class CloudDataFusionStartPipelineOperator(BaseOperator):

[GitHub] [airflow] rjrbt opened a new pull request #9214: flipped boolean to support KMS decryption for ssm ps

2020-06-10 Thread GitBox
rjrbt opened a new pull request #9214: URL: https://github.com/apache/airflow/pull/9214 This is my first PR, so pardon me if I missed any steps, but this proposed change is also _very_ simple. The current implementation of `SystemsManagerParameterStoreBackend` `_get_secret` use

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #9214: flipped boolean to support KMS decryption for ssm ps

2020-06-10 Thread GitBox
boring-cyborg[bot] commented on pull request #9214: URL: https://github.com/apache/airflow/pull/9214#issuecomment-642123308 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contr

[GitHub] [airflow] leahecole commented on pull request #9174: Don't use the term "whitelist" - language matters

2020-06-10 Thread GitBox
leahecole commented on pull request #9174: URL: https://github.com/apache/airflow/pull/9174#issuecomment-642120090 For anyone who is doubting this change or curious as to the history of the words, the U.S. Library of Medicine has [detailed etymology](https://www.ncbi.nlm.nih.gov/pmc/articl

[GitHub] [airflow] blacklistisnotracist removed a comment on pull request #9174: Don't use the term "whitelist" - language matters

2020-06-10 Thread GitBox
blacklistisnotracist removed a comment on pull request #9174: URL: https://github.com/apache/airflow/pull/9174#issuecomment-641921572 @ashb It's ok to make this change, I don't disagree. But pointing out that blacklist has racist connotations is simply wrong. Language matters.

[GitHub] [airflow] kaxil commented on issue #8255: Config value interpolation not working when set as environment variable

2020-06-10 Thread GitBox
kaxil commented on issue #8255: URL: https://github.com/apache/airflow/issues/8255#issuecomment-642105610 +1 to rewriting for 2.0. However, for 1.10.11, I think I will try to fix or add another _hack_ for store_dag_code - let'see ---

[GitHub] [airflow] BasPH commented on issue #8255: Config value interpolation not working when set as environment variable

2020-06-10 Thread GitBox
BasPH commented on issue #8255: URL: https://github.com/apache/airflow/issues/8255#issuecomment-642103854 @ashb WDYT about rewriting the entire module? To me the whole module reads like hack on hack on hack. Putting some thought into it and rewriting it completely seems like the better alt

[GitHub] [airflow] potiuk commented on a change in pull request #9184: Upgrade pendulum to latest major version ~2.0

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #9184: URL: https://github.com/apache/airflow/pull/9184#discussion_r438228371 ## File path: airflow/models/dag.py ## @@ -412,15 +412,15 @@ def following_schedule(self, dttm): if not self.is_fixed_time_schedule():

[GitHub] [airflow] RachaelDS commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-06-10 Thread GitBox
RachaelDS commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r438223116 ## File path: airflow/providers/google/cloud/hooks/datafusion.py ## @@ -382,19 +472,32 @@ def start_pipeline( "v3", "namespace

[GitHub] [airflow] l0yu commented on issue #9213: FILTER_BY_ONWER does not work for multi-owner dags

2020-06-10 Thread GitBox
l0yu commented on issue #9213: URL: https://github.com/apache/airflow/issues/9213#issuecomment-642093845 https://github.com/apache/airflow/commit/843a22f74c01612b1858c69b86d0eb5b1cc89a38 ^^ there was a commit to fix this issue, but somehow missing in 1.10 --

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9213: FILTER_BY_ONWER does not work for multi-owner dags

2020-06-10 Thread GitBox
boring-cyborg[bot] commented on issue #9213: URL: https://github.com/apache/airflow/issues/9213#issuecomment-642090562 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] l0yu opened a new issue #9213: FILTER_BY_ONWER does not work for multi-owner dags

2020-06-10 Thread GitBox
l0yu opened a new issue #9213: URL: https://github.com/apache/airflow/issues/9213 **Apache Airflow version**: 1.10 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): **Environment**: - **Cloud provider or hardware configuration**: - **OS*

[GitHub] [airflow] ashb commented on issue #8255: Config value interpolation not working when set as environment variable

2020-06-10 Thread GitBox
ashb commented on issue #8255: URL: https://github.com/apache/airflow/issues/8255#issuecomment-642086195 Ooh yeah. Tricky one. Tricky tricky one. This is an automated message from the Apache Git Service. To respond to th

[GitHub] [airflow] mik-laj closed issue #9015: Update to Pendulum 2.1.0

2020-06-10 Thread GitBox
mik-laj closed issue #9015: URL: https://github.com/apache/airflow/issues/9015 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] ashb commented on a change in pull request #9184: Upgrade pendulum to latest major version ~2.0

2020-06-10 Thread GitBox
ashb commented on a change in pull request #9184: URL: https://github.com/apache/airflow/pull/9184#discussion_r438199853 ## File path: airflow/models/dag.py ## @@ -412,15 +412,15 @@ def following_schedule(self, dttm): if not self.is_fixed_time_schedule():

[GitHub] [airflow] ashb commented on a change in pull request #9184: Upgrade pendulum to latest major version ~2.0

2020-06-10 Thread GitBox
ashb commented on a change in pull request #9184: URL: https://github.com/apache/airflow/pull/9184#discussion_r438212673 ## File path: airflow/models/dag.py ## @@ -412,15 +412,15 @@ def following_schedule(self, dttm): if not self.is_fixed_time_schedule():

[airflow] branch master updated: Add metavar to CLI arguments (#9077)

2020-06-10 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 7f02e56 Add metavar to CLI arguments (#9

[GitHub] [airflow] mik-laj merged pull request #9077: Add metavar to CLI arguments

2020-06-10 Thread GitBox
mik-laj merged pull request #9077: URL: https://github.com/apache/airflow/pull/9077 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] kaxil commented on pull request #8974: Add VaultHook to hashicorp provider

2020-06-10 Thread GitBox
kaxil commented on pull request #8974: URL: https://github.com/apache/airflow/pull/8974#issuecomment-642081587 Hey for sure I am going to take a look at this again in a few hours :) Apologies it has taken sometime. This is a

[GitHub] [airflow] potiuk commented on a change in pull request #8877: Add S3ToRedshift example dag and system test

2020-06-10 Thread GitBox
potiuk commented on a change in pull request #8877: URL: https://github.com/apache/airflow/pull/8877#discussion_r438204839 ## File path: airflow/providers/amazon/aws/example_dags/example_s3_to_redshift.py ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) u

[airflow] branch master updated: Add S3ToRedshift example dag and system test (#8877)

2020-06-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new a69b031 Add S3ToRedshift example dag and syste

[GitHub] [airflow] potiuk merged pull request #8877: Add S3ToRedshift example dag and system test

2020-06-10 Thread GitBox
potiuk merged pull request #8877: URL: https://github.com/apache/airflow/pull/8877 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

  1   2   >