[GitHub] [airflow] ashb commented on a change in pull request #16681: Introduce RESTARTING state

2021-07-07 Thread GitBox
ashb commented on a change in pull request #16681: URL: https://github.com/apache/airflow/pull/16681#discussion_r665299592 ## File path: airflow/models/dagrun.py ## @@ -487,7 +487,9 @@ def task_instance_scheduling_decisions(self, session: Session = None) -> TISched

[jira] [Commented] (AIRFLOW-4922) If a task crashes, host name is not committed to the database so logs aren't able to be seen in the UI

2021-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376537#comment-17376537 ] ASF GitHub Bot commented on AIRFLOW-4922: - fj-sanchez commented on pull request #6722: URL:

[GitHub] [airflow] fj-sanchez commented on pull request #6722: [AIRFLOW-4922]Fix task get log by Web UI

2021-07-07 Thread GitBox
fj-sanchez commented on pull request #6722: URL: https://github.com/apache/airflow/pull/6722#issuecomment-875574705 We just updated to 2.1.0 and we have started to see the same error. ``` *** Log file does not exist:

[GitHub] [airflow] saurasingh commented on pull request #16735: Adding: Snowflake Role in snowflake provider hook

2021-07-07 Thread GitBox
saurasingh commented on pull request #16735: URL: https://github.com/apache/airflow/pull/16735#issuecomment-875616798 @potiuk @turbaszek Hi Guys, is it possible for you guys to do a quick eyeball on this PR? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #16857: fix(wait_for_airflow_db:prod): you can run this command only iif it exists

2021-07-07 Thread GitBox
boring-cyborg[bot] commented on pull request #16857: URL: https://github.com/apache/airflow/pull/16857#issuecomment-875658892 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] jclegras opened a new pull request #16857: fix(wait_for_airflow_db:prod): you can run this command only iif it exists

2021-07-07 Thread GitBox
jclegras opened a new pull request #16857: URL: https://github.com/apache/airflow/pull/16857 If you are below the 2.X version of airflow, the check failed, the condition must be inverted Reference: https://github.com/apache/airflow/blob/main/UPDATING.md -- This is an automated

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #16858: Bump sshtunnel to current release

2021-07-07 Thread GitBox
boring-cyborg[bot] commented on pull request #16858: URL: https://github.com/apache/airflow/pull/16858#issuecomment-875659109 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] dash-stefan-petrescu opened a new pull request #16858: Bump sshtunnel to current release

2021-07-07 Thread GitBox
dash-stefan-petrescu opened a new pull request #16858: URL: https://github.com/apache/airflow/pull/16858 Bump sshtunnel from 0.2 to 0.4 (current release). Fixes #16699 --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] ashb commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role

2021-07-07 Thread GitBox
ashb commented on issue #16783: URL: https://github.com/apache/airflow/issues/16783#issuecomment-875729318 @jedcunningham Cool -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] ashb opened a new pull request #16860: Support DAGS folder being in different location on scheduler and runners

2021-07-07 Thread GitBox
ashb opened a new pull request #16860: URL: https://github.com/apache/airflow/pull/16860 There has been some vestigial support for this concept in Airflow for a while (all the CLI command already turn the literal `DAGS_FOLDER` in to the real value of the DAGS folder when loading

[airflow] branch main updated (feea380 -> 08f1865)

2021-07-07 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from feea380 Fix impersonation issue with LocalTaskJob (#16852) add 08f1865 Chart: Better comment and example

[GitHub] [airflow] kaxil merged pull request #16859: Chart: Better comment and example for `podTemplate`

2021-07-07 Thread GitBox
kaxil merged pull request #16859: URL: https://github.com/apache/airflow/pull/16859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ashb commented on pull request #6722: [AIRFLOW-4922]Fix task get log by Web UI

2021-07-07 Thread GitBox
ashb commented on pull request #6722: URL: https://github.com/apache/airflow/pull/6722#issuecomment-875580579 99% of the time when this happens, it is because the Celery executor has a problem before the airflow code gets a chance to run. I understand it makes it tricky to diagnose,

[GitHub] [airflow] ashb commented on pull request #16110: Added windows extensions

2021-07-07 Thread GitBox
ashb commented on pull request #16110: URL: https://github.com/apache/airflow/pull/16110#issuecomment-875625692 @casra-developers I've got a WIP stash that fixes that issue, just trying to juggle it, two talks for Airflow Summit and other PR reviews  . I'll open a PR soon. --

[GitHub] [airflow] eladkal commented on pull request #15942: Add KPO pod-template-file jinja template support.

2021-07-07 Thread GitBox
eladkal commented on pull request #15942: URL: https://github.com/apache/airflow/pull/15942#issuecomment-875650569 @theodickson This PR was released in cncf provider 2.0.0 which is a major release (previous was 1.2.0) there can be breaking changes when upgrading a major version. We also

[GitHub] [airflow] github-actions[bot] commented on pull request #16859: Chart: Better comment and example for `podTemplate`

2021-07-07 Thread GitBox
github-actions[bot] commented on pull request #16859: URL: https://github.com/apache/airflow/pull/16859#issuecomment-875693721 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[airflow] branch main updated (2b7c596 -> faffaec)

2021-07-07 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 2b7c596 Add State types for tasks and DAGs (#15285) add faffaec Don't check execution_date in

[GitHub] [airflow] ashb commented on issue #16790: Support manual task annotation.

2021-07-07 Thread GitBox
ashb commented on issue #16790: URL: https://github.com/apache/airflow/issues/16790#issuecomment-875532566 See also https://github.com/apache/airflow/issues/12199 which asks for a new state -- So that you can "acknowledge" the failure without marking it as success. -- This is an

[GitHub] [airflow] ashb commented on issue #16764: TaskGroup dependencies handled inconsistently

2021-07-07 Thread GitBox
ashb commented on issue #16764: URL: https://github.com/apache/airflow/issues/16764#issuecomment-875549625 The workaround for now, is as you said, to move the `start >> taskgroup >> end` to outside of the TG context. -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] warrenstephens edited a comment on issue #15607: airflow throws error: airflow.exceptions.SerializedDagNotFound: DAG 'dns-basic-1min' not found in serialized_dag table

2021-07-07 Thread GitBox
warrenstephens edited a comment on issue #15607: URL: https://github.com/apache/airflow/issues/15607#issuecomment-875567764 @kaxil **I found the issue!** It turns out that I had a minimized window open with the `http://localhost:8080/graph?dag_id=tutorial` in it! This was

[GitHub] [airflow] warrenstephens edited a comment on issue #15607: airflow throws error: airflow.exceptions.SerializedDagNotFound: DAG 'dns-basic-1min' not found in serialized_dag table

2021-07-07 Thread GitBox
warrenstephens edited a comment on issue #15607: URL: https://github.com/apache/airflow/issues/15607#issuecomment-875567764 @kaxil **I found the issue!** It turns out that I had a minimized window open with the `http://localhost:8080/graph?dag_id=tutorial` in it! This was from

[GitHub] [airflow] potiuk commented on issue #15607: airflow throws error: airflow.exceptions.SerializedDagNotFound: DAG 'dns-basic-1min' not found in serialized_dag table

2021-07-07 Thread GitBox
potiuk commented on issue #15607: URL: https://github.com/apache/airflow/issues/15607#issuecomment-875576332 Closing it then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] potiuk closed issue #15607: airflow throws error: airflow.exceptions.SerializedDagNotFound: DAG 'dns-basic-1min' not found in serialized_dag table

2021-07-07 Thread GitBox
potiuk closed issue #15607: URL: https://github.com/apache/airflow/issues/15607 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (AIRFLOW-4922) If a task crashes, host name is not committed to the database so logs aren't able to be seen in the UI

2021-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376591#comment-17376591 ] ASF GitHub Bot commented on AIRFLOW-4922: - abhishektrivedi-hs commented on pull request #6722:

[GitHub] [airflow] abhishektrivedi-hs commented on pull request #6722: [AIRFLOW-4922]Fix task get log by Web UI

2021-07-07 Thread GitBox
abhishektrivedi-hs commented on pull request #6722: URL: https://github.com/apache/airflow/pull/6722#issuecomment-875630370 I was facing the same "Log file does not exist:" issue during initial days of Airflow installation. Later once i understood the end to end design and architecture,

[GitHub] [airflow] yuqian90 commented on a change in pull request #16681: Introduce RESTARTING state

2021-07-07 Thread GitBox
yuqian90 commented on a change in pull request #16681: URL: https://github.com/apache/airflow/pull/16681#discussion_r665419327 ## File path: airflow/models/dagrun.py ## @@ -487,7 +487,9 @@ def task_instance_scheduling_decisions(self, session: Session = None) -> TISched

[GitHub] [airflow] ashb commented on a change in pull request #16860:  Support DAGS folder being in different location on scheduler and runners

2021-07-07 Thread GitBox
ashb commented on a change in pull request #16860: URL: https://github.com/apache/airflow/pull/16860#discussion_r665513573 ## File path: airflow/models/dag.py ## @@ -2379,6 +2413,8 @@ class DagModel(Base): Index('idx_next_dagrun_create_after',

[GitHub] [airflow] ashb commented on a change in pull request #16860:  Support DAGS folder being in different location on scheduler and runners

2021-07-07 Thread GitBox
ashb commented on a change in pull request #16860: URL: https://github.com/apache/airflow/pull/16860#discussion_r665512717 ## File path: airflow/jobs/scheduler_job.py ## @@ -497,18 +497,8 @@ def _enqueue_task_instances_with_queued_state(self, task_instances: List[TI]) ->

[GitHub] [airflow] kaxil closed issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1

2021-07-07 Thread GitBox
kaxil closed issue #14249: URL: https://github.com/apache/airflow/issues/14249 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil merged pull request #16809: Don't check execution_date in refresh_from_db

2021-07-07 Thread GitBox
kaxil merged pull request #16809: URL: https://github.com/apache/airflow/pull/16809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ashb commented on a change in pull request #16852: Fix impersonation issue with LocalTaskJob

2021-07-07 Thread GitBox
ashb commented on a change in pull request #16852: URL: https://github.com/apache/airflow/pull/16852#discussion_r665281668 ## File path: tests/jobs/test_local_task_job.py ## @@ -147,6 +147,54 @@ def test_localtaskjob_heartbeat(self): with

[GitHub] [airflow] ephraimbuddy opened a new pull request #16854: Add 'queued' to DagRunState

2021-07-07 Thread GitBox
ephraimbuddy opened a new pull request #16854: URL: https://github.com/apache/airflow/pull/16854 This change adds 'queued' to DagRunState and improved typing for DagRun state --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] ashb commented on issue #12817: Add default conf to DAG definition to use in scheduled runs

2021-07-07 Thread GitBox
ashb commented on issue #12817: URL: https://github.com/apache/airflow/issues/12817#issuecomment-875678537 cc @msumit -- perhaps we should think about deprecating/merging params and dagrun.conf in the template -- not sure there is real value in having both. -- This is an automated

[GitHub] [airflow] ashb commented on pull request #16569: Unpause DAG on manual trigger

2021-07-07 Thread GitBox
ashb commented on pull request #16569: URL: https://github.com/apache/airflow/pull/16569#issuecomment-875700883 I'm going to choose to treat this as a (UX) Bug Fix so changing the behaviour is fine as it can be easily switched back. -- This is an automated message from the Apache Git

[GitHub] [airflow] ashb commented on pull request #16681: Introduce RESTARTING state

2021-07-07 Thread GitBox
ashb commented on pull request #16681: URL: https://github.com/apache/airflow/pull/16681#issuecomment-875712476 I mostly also thinking somewhere in the published docs tree. Ah https://airflow.apache.org/docs/apache-airflow/stable/concepts/tasks.html#task-instances That

[GitHub] [airflow] jedcunningham opened a new pull request #16861: Better description for `pod_template_file`

2021-07-07 Thread GitBox
jedcunningham opened a new pull request #16861: URL: https://github.com/apache/airflow/pull/16861 In Airflow 2+, `pod_template_file` is the only way to configure workers, so we can remove the "other fields" language. Also drop a link to the `pod_template_file` section for

[GitHub] [airflow] ferruzzi commented on a change in pull request #16766: Add an Amazon EMR on EKS provider package

2021-07-07 Thread GitBox
ferruzzi commented on a change in pull request #16766: URL: https://github.com/apache/airflow/pull/16766#discussion_r665519796 ## File path: airflow/providers/amazon/aws/hooks/emr_containers.py ## @@ -0,0 +1,189 @@ +# Licensed to the Apache Software Foundation (ASF) under one

[jira] [Commented] (AIRFLOW-4922) If a task crashes, host name is not committed to the database so logs aren't able to be seen in the UI

2021-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376549#comment-17376549 ] ASF GitHub Bot commented on AIRFLOW-4922: - ashb commented on pull request #6722: URL:

[GitHub] [airflow] Lyalpha commented on issue #16023: Running tasks marked as 'orphaned' and killed by scheduler

2021-07-07 Thread GitBox
Lyalpha commented on issue #16023: URL: https://github.com/apache/airflow/issues/16023#issuecomment-875614596 Just to add that I'm also seeing this behaviour on 2.1.0, running on bare metal. Have a paused DAG and I am trying to run a task within the DAG manually, but upon the next check

[GitHub] [airflow] ephraimbuddy closed pull request #16852: Fix impersonation issue with LocalTaskJob

2021-07-07 Thread GitBox
ephraimbuddy closed pull request #16852: URL: https://github.com/apache/airflow/pull/16852 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #15607: airflow throws error: airflow.exceptions.SerializedDagNotFound: DAG 'dns-basic-1min' not found in serialized_dag table

2021-07-07 Thread GitBox
potiuk commented on issue #15607: URL: https://github.com/apache/airflow/issues/15607#issuecomment-875647978 > Airflow webserver should probably handle the request without generating an Exception. Why do you think so ? Could you please elaborate? I think showing an exception

[GitHub] [airflow] theodickson commented on pull request #15942: Add KPO pod-template-file jinja template support.

2021-07-07 Thread GitBox
theodickson commented on pull request #15942: URL: https://github.com/apache/airflow/pull/15942#issuecomment-875661060 Sorry I should have been clearer, I realise it was mentioned in the release notes but as a feature not a breaking change. It's not exactly a huge deal and we found the

[GitHub] [airflow] jedcunningham opened a new pull request #16859: Chart: Better comment and example for `podTemplate`

2021-07-07 Thread GitBox
jedcunningham opened a new pull request #16859: URL: https://github.com/apache/airflow/pull/16859 Add better comment and an example for `podTemplate`. Related: #16833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] josh-fell commented on a change in pull request #16732: Update `chain()` and `cross_downstream()` to support XComArgs

2021-07-07 Thread GitBox
josh-fell commented on a change in pull request #16732: URL: https://github.com/apache/airflow/pull/16732#discussion_r665482055 ## File path: airflow/models/baseoperator.py ## @@ -1595,11 +1652,14 @@ def chain(*tasks: Union[BaseOperator, Sequence[BaseOperator]]): def

[GitHub] [airflow] ashb commented on pull request #16857: fix(wait_for_airflow_db:prod): you can run this command only if it exists

2021-07-07 Thread GitBox
ashb commented on pull request #16857: URL: https://github.com/apache/airflow/pull/16857#issuecomment-875710146 The 1.10 series reached end of life last month, so the Dockerfile on main is no longer going to support it. -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] ashb commented on a change in pull request #16681: Introduce RESTARTING state

2021-07-07 Thread GitBox
ashb commented on a change in pull request #16681: URL: https://github.com/apache/airflow/pull/16681#discussion_r665497589 ## File path: airflow/utils/state.py ## @@ -180,6 +184,11 @@ def color_fg(cls, state): A list of states indicating that a task or dag is a success

[GitHub] [airflow] ashb commented on pull request #16814: Automatically create section when migrating config

2021-07-07 Thread GitBox
ashb commented on pull request #16814: URL: https://github.com/apache/airflow/pull/16814#issuecomment-875552930 @kaxil Ohh, I see. So the section exists in the default_config, but when we to the upgrade and try to _write_ the config back , the sections are only checked in the top

[GitHub] [airflow] ashb commented on a change in pull request #16854: Add 'queued' to DagRunState

2021-07-07 Thread GitBox
ashb commented on a change in pull request #16854: URL: https://github.com/apache/airflow/pull/16854#discussion_r665329286 ## File path: airflow/utils/state.py ## @@ -92,6 +93,7 @@ class State: task_states: Tuple[Optional[TaskInstanceState], ...] = (None,) +

[GitHub] [airflow] warrenstephens commented on issue #15607: airflow throws error: airflow.exceptions.SerializedDagNotFound: DAG 'dns-basic-1min' not found in serialized_dag table

2021-07-07 Thread GitBox
warrenstephens commented on issue #15607: URL: https://github.com/apache/airflow/issues/15607#issuecomment-875567764 @kaxil **I found the issue!** It turns out that I had a minimized window open with the `http://localhost:8080/graph?dag_id=tutorial` in it! This was from doing the

[GitHub] [airflow] jedcunningham commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role

2021-07-07 Thread GitBox
jedcunningham commented on issue #16783: URL: https://github.com/apache/airflow/issues/16783#issuecomment-875636662 FAB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] theodickson commented on pull request #15942: Add KPO pod-template-file jinja template support.

2021-07-07 Thread GitBox
theodickson commented on pull request #15942: URL: https://github.com/apache/airflow/pull/15942#issuecomment-875639750 Just pointing out this was a breaking change - if you previously were using values in templated fields which ended in any of these strings, but were _not_ meant to be

[GitHub] [airflow] yuqian90 commented on pull request #16681: Introduce RESTARTING state

2021-07-07 Thread GitBox
yuqian90 commented on pull request #16681: URL: https://github.com/apache/airflow/pull/16681#issuecomment-875648056 > Code and name looks good now, though this is probably worth documenting _somewhere_ so users can know about this behaviour. > > (And you'll have to resolve conflicts

[GitHub] [airflow] andrewgodwin commented on a change in pull request #16854: Add 'queued' to DagRunState

2021-07-07 Thread GitBox
andrewgodwin commented on a change in pull request #16854: URL: https://github.com/apache/airflow/pull/16854#discussion_r665531558 ## File path: airflow/utils/state.py ## @@ -92,6 +93,7 @@ class State: task_states: Tuple[Optional[TaskInstanceState], ...] = (None,) +

[GitHub] [airflow] ashb commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role

2021-07-07 Thread GitBox
ashb commented on issue #16783: URL: https://github.com/apache/airflow/issues/16783#issuecomment-875528146 Is this an Airflow bug, or a FAB bug? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] ashb commented on issue #16802: Ooops

2021-07-07 Thread GitBox
ashb commented on issue #16802: URL: https://github.com/apache/airflow/issues/16802#issuecomment-875530886 What environment are you running in ? You can probably work around this by setting `TZ=UTC` environment variable. Since this is a bug in pendulum (and there's a workaround)

[GitHub] [airflow] ashb closed issue #16802: Ooops

2021-07-07 Thread GitBox
ashb closed issue #16802: URL: https://github.com/apache/airflow/issues/16802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (faffaec -> feea380)

2021-07-07 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from faffaec Don't check execution_date in refresh_from_db (#16809) add feea380 Fix impersonation issue with

[GitHub] [airflow] ashb commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role

2021-07-07 Thread GitBox
ashb commented on issue #16783: URL: https://github.com/apache/airflow/issues/16783#issuecomment-875725650 This is likely a "wont Fix" from airflow's side I'm afraid then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[airflow] branch v2-1-test updated (5b3b7d4 -> a36eca1)

2021-07-07 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from 5b3b7d4 Add changelog updates for 2.1.2 (#16838) new 06f8022 Fix ``CeleryKubernetesExecutor`` (#16700)

[airflow] 01/03: Fix ``CeleryKubernetesExecutor`` (#16700)

2021-07-07 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 06f8022b37f121b17512ac58ef55132b22252b18 Author: Kaxil Naik AuthorDate: Tue Jun 29 23:39:34 2021 +0100 Fix

[airflow] 03/03: Mask value if the key is ``token`` (#16474)

2021-07-07 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit a36eca16b55b5413382c87b1d178bdc5158db16d Author: Robert Saxby AuthorDate: Fri Jul 2 21:21:56 2021 +0200 Mask

[GitHub] [airflow] ashb commented on issue #16764: TaskGroup dependencies handled inconsistently

2021-07-07 Thread GitBox
ashb commented on issue #16764: URL: https://github.com/apache/airflow/issues/16764#issuecomment-875549047 Oh yeah. Okay, so why this happens: TaskGroups don't actually exist as dependencies, so when you do `start >> taskgroup >> end` you are setting the downstream of start

[GitHub] [airflow] warrenstephens commented on issue #15607: airflow throws error: airflow.exceptions.SerializedDagNotFound: DAG 'dns-basic-1min' not found in serialized_dag table

2021-07-07 Thread GitBox
warrenstephens commented on issue #15607: URL: https://github.com/apache/airflow/issues/15607#issuecomment-875578356 Airflow webserver should probably handle the request without generating an Exception. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on pull request #16498: gitpodify Apache Airflow - online development workspace

2021-07-07 Thread GitBox
potiuk commented on pull request #16498: URL: https://github.com/apache/airflow/pull/16498#issuecomment-875626260 Sorry, got some private distractions and was not as available as I would like. I should be able to send my thoughts by next week. Sorry to keep you waiting. -- This is an

[airflow] branch v2-1-test updated: Fix impersonation issue with LocalTaskJob (#16852)

2021-07-07 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-1-test by this push: new 72f0ce4 Fix impersonation issue with

[GitHub] [airflow] jedcunningham commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role

2021-07-07 Thread GitBox
jedcunningham commented on issue #16783: URL: https://github.com/apache/airflow/issues/16783#issuecomment-875727800 I haven't poked around yet, but we may be able to handle the 'no role' scenario more gracefully on our side. Let me take a stab at that before we close this. -- This is

[GitHub] [airflow] ashb commented on a change in pull request #16852: Fix impersonation issue with LocalTaskJob

2021-07-07 Thread GitBox
ashb commented on a change in pull request #16852: URL: https://github.com/apache/airflow/pull/16852#discussion_r665280421 ## File path: tests/jobs/test_local_task_job.py ## @@ -147,6 +147,54 @@ def test_localtaskjob_heartbeat(self): with

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #16852: Fix impersonation issue with LocalTaskJob

2021-07-07 Thread GitBox
ephraimbuddy commented on a change in pull request #16852: URL: https://github.com/apache/airflow/pull/16852#discussion_r665280227 ## File path: tests/jobs/test_local_task_job.py ## @@ -147,6 +147,52 @@ def test_localtaskjob_heartbeat(self): with

[GitHub] [airflow] tfedyanin commented on issue #10544: Kubernetes pod operator: More than one pod running with labels

2021-07-07 Thread GitBox
tfedyanin commented on issue #10544: URL: https://github.com/apache/airflow/issues/10544#issuecomment-875543490 I guess it happens when I use manual retry. There is no label `try_number=1`, that provide uniqueness between runs. So on first manual retry pod is created correctly. And

[GitHub] [airflow] github-actions[bot] commented on pull request #16852: Fix impersonation issue with LocalTaskJob

2021-07-07 Thread GitBox
github-actions[bot] commented on pull request #16852: URL: https://github.com/apache/airflow/pull/16852#issuecomment-875543164 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] ashb commented on pull request #16814: Automatically create section when migrating config

2021-07-07 Thread GitBox
ashb commented on pull request #16814: URL: https://github.com/apache/airflow/pull/16814#issuecomment-875582015 @uranusjr We should probably expand the unit tests to cover this case -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] ashb merged pull request #16852: Fix impersonation issue with LocalTaskJob

2021-07-07 Thread GitBox
ashb merged pull request #16852: URL: https://github.com/apache/airflow/pull/16852 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal edited a comment on pull request #15942: Add KPO pod-template-file jinja template support.

2021-07-07 Thread GitBox
eladkal edited a comment on pull request #15942: URL: https://github.com/apache/airflow/pull/15942#issuecomment-875650569 @theodickson This PR was released in cncf provider 2.0.0 which is a major release (previous was 1.2.0) there can be breaking changes when upgrading a major version. We

[GitHub] [airflow] oscarrobertson commented on pull request #15942: Add KPO pod-template-file jinja template support.

2021-07-07 Thread GitBox
oscarrobertson commented on pull request #15942: URL: https://github.com/apache/airflow/pull/15942#issuecomment-875676096 I agree that this is a breaking change, it does feel like something that could be made clearer in the release notes. I think the code that uses this param to

[GitHub] [airflow] potiuk commented on pull request #16857: fix(wait_for_airflow_db:prod): you can run this command only if it exists

2021-07-07 Thread GitBox
potiuk commented on pull request #16857: URL: https://github.com/apache/airflow/pull/16857#issuecomment-875723974 Agree. We do not support 1.10 any more. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] potiuk closed pull request #16857: fix(wait_for_airflow_db:prod): you can run this command only if it exists

2021-07-07 Thread GitBox
potiuk closed pull request #16857: URL: https://github.com/apache/airflow/pull/16857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ashb edited a comment on pull request #16110: Added windows extensions

2021-07-07 Thread GitBox
ashb edited a comment on pull request #16110: URL: https://github.com/apache/airflow/pull/16110#issuecomment-875625692 @casra-developers I've got a WIP stash that fixes that issue, just trying to juggle it, two talks for Airflow Summit and other PR reviews  . I'll open a PR soon.

[GitHub] [airflow] ferruzzi commented on a change in pull request #16766: Add an Amazon EMR on EKS provider package

2021-07-07 Thread GitBox
ferruzzi commented on a change in pull request #16766: URL: https://github.com/apache/airflow/pull/16766#discussion_r665519796 ## File path: airflow/providers/amazon/aws/hooks/emr_containers.py ## @@ -0,0 +1,189 @@ +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] ferruzzi commented on a change in pull request #16766: Add an Amazon EMR on EKS provider package

2021-07-07 Thread GitBox
ferruzzi commented on a change in pull request #16766: URL: https://github.com/apache/airflow/pull/16766#discussion_r665561297 ## File path: airflow/providers/amazon/aws/operators/emr_containers.py ## @@ -0,0 +1,151 @@ +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] [airflow] jhtimmins opened a new pull request #16862: Sync v2 1 2 stable v2

2021-07-07 Thread GitBox
jhtimmins opened a new pull request #16862: URL: https://github.com/apache/airflow/pull/16862 Sync remaining commits into v2-1-2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] samgans commented on issue #16836: WebUI broke when enable_xcom_pickling Airflow2.1

2021-07-07 Thread GitBox
samgans commented on issue #16836: URL: https://github.com/apache/airflow/issues/16836#issuecomment-875914191 We need to change the `!= None` to `is not None` as the DF cannot evaluate to True or False as a whole. I can make a PR and fix this in close time. Any ideas if it can break

[airflow] branch main updated (b0f7f91 -> 5999cb9)

2021-07-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from b0f7f91 Standardise dataproc location param to region (#16034) add 5999cb9 Adding: Snowflake Role in snowflake

[GitHub] [airflow] potiuk commented on pull request #16735: Adding: Snowflake Role in snowflake provider hook

2021-07-07 Thread GitBox
potiuk commented on pull request #16735: URL: https://github.com/apache/airflow/pull/16735#issuecomment-875928335 > @potiuk How do I merge this ? You can't . But I just did :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] Jorricks edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-07-07 Thread GitBox
Jorricks edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-875932438 > We're only making use of the `default_pool` and tasks are moving from `Queued` to `Running` there but aren't entering the queue despite being in a `Scheduled` state.

[GitHub] [airflow] zachliu commented on issue #16130: Tree View is blocked by DagRuns status when showing all 356 DagRuns

2021-07-07 Thread GitBox
zachliu commented on issue #16130: URL: https://github.com/apache/airflow/issues/16130#issuecomment-875933409 @Tofull is my description of the bug confusing? :thinking: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] Jorricks edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-07-07 Thread GitBox
Jorricks edited a comment on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-875932438 > We're only making use of the `default_pool` and tasks are moving from `Queued` to `Running` there but aren't entering the queue despite being in a `Scheduled` state.

[GitHub] [airflow] psg2 commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-07 Thread GitBox
psg2 commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875968087 Got it, I guess it was a miscommunication issue then, I asked on Slack about this when I noticed that `/dags/{dag_id}` returned a DAG that was inactive while the

[jira] [Commented] (AIRFLOW-4922) If a task crashes, host name is not committed to the database so logs aren't able to be seen in the UI

2021-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376756#comment-17376756 ] ASF GitHub Bot commented on AIRFLOW-4922: - fj-sanchez edited a comment on pull request #6722:

[jira] [Commented] (AIRFLOW-4922) If a task crashes, host name is not committed to the database so logs aren't able to be seen in the UI

2021-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376755#comment-17376755 ] ASF GitHub Bot commented on AIRFLOW-4922: - fj-sanchez commented on pull request #6722: URL:

[GitHub] [airflow] fj-sanchez edited a comment on pull request #6722: [AIRFLOW-4922]Fix task get log by Web UI

2021-07-07 Thread GitBox
fj-sanchez edited a comment on pull request #6722: URL: https://github.com/apache/airflow/pull/6722#issuecomment-875811494 > If any one is able to validate if this is the case that would be helpful. If you see this and you can check all of your worker nodes to see if the log file exists

[GitHub] [airflow] fj-sanchez commented on pull request #6722: [AIRFLOW-4922]Fix task get log by Web UI

2021-07-07 Thread GitBox
fj-sanchez commented on pull request #6722: URL: https://github.com/apache/airflow/pull/6722#issuecomment-875811494 > If any one is able to validate if this is the case that would be helpful. If you see this and you can check all of your worker nodes to see if the log file exists on _any_

[GitHub] [airflow] github-actions[bot] commented on pull request #16735: Adding: Snowflake Role in snowflake provider hook

2021-07-07 Thread GitBox
github-actions[bot] commented on pull request #16735: URL: https://github.com/apache/airflow/pull/16735#issuecomment-875861677 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] potiuk commented on pull request #16735: Adding: Snowflake Role in snowflake provider hook

2021-07-07 Thread GitBox
potiuk commented on pull request #16735: URL: https://github.com/apache/airflow/pull/16735#issuecomment-875860882 Running tests now. Sorry for some delay in review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] jhtimmins opened a new pull request #16865: Use existing cached_property method to skip pylint error.

2021-07-07 Thread GitBox
jhtimmins opened a new pull request #16865: URL: https://github.com/apache/airflow/pull/16865 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] potiuk edited a comment on pull request #16865: Use existing cached_property method to skip pylint error.

2021-07-07 Thread GitBox
potiuk edited a comment on pull request #16865: URL: https://github.com/apache/airflow/pull/16865#issuecomment-875954489 BTW. Those are not my pylint changes - the actual reason for that error is the https://github.com/apache/airflow/pull/15969 which introduced the compat shim (and that's

[GitHub] [airflow] mik-laj commented on a change in pull request #16860:  Support DAGS folder being in different location on scheduler and runners

2021-07-07 Thread GitBox
mik-laj commented on a change in pull request #16860: URL: https://github.com/apache/airflow/pull/16860#discussion_r665734044 ## File path: airflow/models/dag.py ## @@ -734,15 +759,22 @@ def task_group(self) -> "TaskGroup": @property def filepath(self) -> str: -

[GitHub] [airflow] ferruzzi commented on a change in pull request #16766: Add an Amazon EMR on EKS provider package

2021-07-07 Thread GitBox
ferruzzi commented on a change in pull request #16766: URL: https://github.com/apache/airflow/pull/16766#discussion_r665567789 ## File path: airflow/providers/amazon/aws/operators/emr_containers.py ## @@ -0,0 +1,151 @@ +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] [airflow] o-nikolas commented on a change in pull request #16685: reattach_prev_task ECSOperator

2021-07-07 Thread GitBox
o-nikolas commented on a change in pull request #16685: URL: https://github.com/apache/airflow/pull/16685#discussion_r665609681 ## File path: airflow/providers/amazon/aws/operators/ecs.py ## @@ -303,7 +310,6 @@ def _try_reattach_task(self): # Check if the ECS task

[GitHub] [airflow] o-nikolas commented on pull request #16685: reattach_prev_task ECSOperator

2021-07-07 Thread GitBox
o-nikolas commented on pull request #16685: URL: https://github.com/apache/airflow/pull/16685#issuecomment-875829948 > Thanks a lot for the review @ferruzzi @o-nikolas , I made the small changes you mentioned Thanks! I reviewed the latest commit, and it looks like there's one

[GitHub] [airflow] Jorricks commented on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2021-07-07 Thread GitBox
Jorricks commented on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-875932438 > We're only making use of the `default_pool` and tasks are moving from `Queued` to `Running` there but aren't entering the queue despite being in a `Scheduled` state.

  1   2   3   >