[GitHub] [airflow] potiuk commented on a diff in pull request #28502: Migrate DagFileProcessor.manage_slas to Internal API

2023-01-20 Thread GitBox
potiuk commented on code in PR #28502: URL: https://github.com/apache/airflow/pull/28502#discussion_r1082897117 ## airflow/dag_processing/processor.py: ## @@ -365,8 +365,10 @@ def __init__(self, dag_ids: list[str] | None, dag_directory: str, log: logging.L self._dag_di

[GitHub] [airflow] mattppal commented on pull request #28592: Guard not-yet-expanded ti in trigger rule dep

2023-01-20 Thread GitBox
mattppal commented on PR #28592: URL: https://github.com/apache/airflow/pull/28592#issuecomment-1398744685 Hi all, I'm seeing the same issue on `rc2` when passing an XComArg object to the mapped task. Lists appear to work as expected. -- This is an automated message from the Apache Git Se

[GitHub] [airflow] vincbeck commented on a diff in pull request #28502: Migrate DagFileProcessor.manage_slas to Internal API

2023-01-20 Thread GitBox
vincbeck commented on code in PR #28502: URL: https://github.com/apache/airflow/pull/28502#discussion_r1082893036 ## airflow/dag_processing/processor.py: ## @@ -365,8 +365,10 @@ def __init__(self, dag_ids: list[str] | None, dag_directory: str, log: logging.L self._dag_

[GitHub] [airflow] potiuk commented on pull request #29064: Capitalize dag to DAG

2023-01-20 Thread GitBox
potiuk commented on PR #29064: URL: https://github.com/apache/airflow/pull/29064#issuecomment-1398738922 I cannot figure out why 'dag' is not flagged by our spellchecker. It looks like it should, and I tried to experiment a bit but I really cannot find why it has been allowed in the first p

[GitHub] [airflow] Taragolis merged pull request #29063: Migrate Helm tests to `pytest`

2023-01-20 Thread GitBox
Taragolis merged PR #29063: URL: https://github.com/apache/airflow/pull/29063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] Taragolis commented on pull request #29035: Renaming nose compatible methods in flavour of regular pytest naming

2023-01-20 Thread GitBox
Taragolis commented on PR #29035: URL: https://github.com/apache/airflow/pull/29035#issuecomment-1398705272 > > @potiuk, is it the right place? > > Yep. But you can also add "full tests needed" label on a PR to run a complete set of tests. And very easy to forget add this label

[GitHub] [airflow] jedcunningham commented on pull request #28808: Allow setting the name for the base container within K8s Pod Operator

2023-01-20 Thread GitBox
jedcunningham commented on PR #28808: URL: https://github.com/apache/airflow/pull/28808#issuecomment-1398700496 Looking at this with fresh eyes, this works? ``` kpo_task = KubernetesPodOperator(...) kpo_task.BASE_CONTAINER_NAME = "whatever" ``` Is it worth adding this t

[GitHub] [airflow] potiuk commented on pull request #29063: Migrate Helm tests to `pytest`

2023-01-20 Thread GitBox
potiuk commented on PR #29063: URL: https://github.com/apache/airflow/pull/29063#issuecomment-1398695049 (pending tess succeed ofc) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [airflow] bbovenzi opened a new pull request, #29066: Check for run_id url param when linking to graph/gantt views

2023-01-20 Thread GitBox
bbovenzi opened a new pull request, #29066: URL: https://github.com/apache/airflow/pull/29066 Some parts of the app linked over to the graph and gantt views with a `run_id=` param, but we weren't actually using that param to decide which run to show for a user. Now, we will first check

[GitHub] [airflow] potiuk commented on a diff in pull request #28846: Updated app to support configuring the caching hash method for FIPS

2023-01-20 Thread GitBox
potiuk commented on code in PR #28846: URL: https://github.com/apache/airflow/pull/28846#discussion_r1082853296 ## airflow/models/serialized_dag.py: ## @@ -102,7 +102,7 @@ def __init__(self, dag: DAG, processor_subdir: str | None = None): dag_data = SerializedDAG.to_di

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29065: When clearing task instances try to get associated DAGs from database

2023-01-20 Thread GitBox
boring-cyborg[bot] commented on PR #29065: URL: https://github.com/apache/airflow/pull/29065#issuecomment-1398692028 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] sean-rose opened a new pull request, #29065: When clearing task instances try to get associated DAGs from database

2023-01-20 Thread GitBox
sean-rose opened a new pull request, #29065: URL: https://github.com/apache/airflow/pull/29065 This fixes problems when recursively clearing task instances across multiple DAGs: * Task instances in downstream DAGs weren't having their `max_tries` property incremented, which could cause

[GitHub] [airflow] Taragolis opened a new pull request, #29063: Migrate Helm tests to `pytest`

2023-01-20 Thread GitBox
Taragolis opened a new pull request, #29063: URL: https://github.com/apache/airflow/pull/29063 Migrate `kubernetes_tests` to pytest. Also resolve issue that tests use combination of `unittests`/`nose`/`pytest` which found during this PR: - https://github.com/apache/airflow/pull/29

[GitHub] [airflow] BasPH opened a new pull request, #29064: Capitalize dag to DAG

2023-01-20 Thread GitBox
BasPH opened a new pull request, #29064: URL: https://github.com/apache/airflow/pull/29064 DAG is an abbreviation (Directed Acyclic Graph) and should therefore be written capitalized. This PR changes "dags" to "DAGs" wherever it's written for humans (i.e. not code, hyperlinks, etc.).

[GitHub] [airflow] shyft-mike commented on a diff in pull request #28846: Updated app to support configuring the caching hash method for FIPS

2023-01-20 Thread GitBox
shyft-mike commented on code in PR #28846: URL: https://github.com/apache/airflow/pull/28846#discussion_r1082843374 ## airflow/models/serialized_dag.py: ## @@ -102,7 +102,7 @@ def __init__(self, dag: DAG, processor_subdir: str | None = None): dag_data = SerializedDAG.t

[GitHub] [airflow] potiuk commented on pull request #28476: Migrate DagFileProcessorManager._deactivate_stale_dags to Internal API

2023-01-20 Thread GitBox
potiuk commented on PR #28476: URL: https://github.com/apache/airflow/pull/28476#issuecomment-1398678859 > > I guess that one should be updated after discussion in #28502 with `@classmethod` ? > > Correct! I rather wait for #28502 to be merged and then I'll update this PR +1

[GitHub] [airflow] potiuk commented on a diff in pull request #28502: Migrate DagFileProcessor.manage_slas to Internal API

2023-01-20 Thread GitBox
potiuk commented on code in PR #28502: URL: https://github.com/apache/airflow/pull/28502#discussion_r1082833504 ## airflow/dag_processing/processor.py: ## @@ -365,8 +365,10 @@ def __init__(self, dag_ids: list[str] | None, dag_directory: str, log: logging.L self._dag_di

[GitHub] [airflow] shyft-mike commented on a diff in pull request #28846: Updated app to support configuring the caching hash method for FIPS

2023-01-20 Thread GitBox
shyft-mike commented on code in PR #28846: URL: https://github.com/apache/airflow/pull/28846#discussion_r1082832047 ## airflow/www/app.py: ## @@ -131,7 +132,29 @@ def create_app(config=None, testing=False): init_robots(flask_app) +# Configure caching +webserver_

[GitHub] [airflow] potiuk commented on a diff in pull request #28502: Migrate DagFileProcessor.manage_slas to Internal API

2023-01-20 Thread GitBox
potiuk commented on code in PR #28502: URL: https://github.com/apache/airflow/pull/28502#discussion_r1082811184 ## airflow/dag_processing/processor.py: ## @@ -365,8 +365,10 @@ def __init__(self, dag_ids: list[str] | None, dag_directory: str, log: logging.L self._dag_di

[GitHub] [airflow] BasPH opened a new pull request, #29062: Several improvements to the Params doc

2023-01-20 Thread GitBox
BasPH opened a new pull request, #29062: URL: https://github.com/apache/airflow/pull/29062 The PR makes several improvements to https://airflow.apache.org/docs/apache-airflow/stable/concepts/params.html: - Moved task-level params paragraph up to create a more logical structure - Em

[GitHub] [airflow] vincbeck commented on pull request #28476: Migrate DagFileProcessorManager._deactivate_stale_dags to Internal API

2023-01-20 Thread GitBox
vincbeck commented on PR #28476: URL: https://github.com/apache/airflow/pull/28476#issuecomment-1398653005 > I guess that one should be updated after discussion in #28502 with `@classmethod` ? Correct! I rather wait for #28502 to ber merged and then I'll update this PR -- This is a

[GitHub] [airflow] potiuk commented on pull request #28476: Migrate DagFileProcessorManager._deactivate_stale_dags to Internal API

2023-01-20 Thread GitBox
potiuk commented on PR #28476: URL: https://github.com/apache/airflow/pull/28476#issuecomment-1398649440 I guess that should be updated after discussion in https://github.com/apache/airflow/pull/28502 with `@classmethod` ? -- This is an automated message from the Apache Git Service. To re

[GitHub] [airflow] potiuk commented on pull request #28979: Fix rendering parameters in PapermillOperator

2023-01-20 Thread GitBox
potiuk commented on PR #28979: URL: https://github.com/apache/airflow/pull/28979#issuecomment-1398644414 Rebased it as for some reason there was only 1 mergable check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [airflow] potiuk commented on a diff in pull request #28981: Implemented log_containers option to read from all containers in KubernetesPodOperator

2023-01-20 Thread GitBox
potiuk commented on code in PR #28981: URL: https://github.com/apache/airflow/pull/28981#discussion_r1082797194 ## airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py: ## @@ -168,6 +168,9 @@ class KubernetesPodOperator(BaseOperator): :param labels: labels to apply

[GitHub] [airflow] vincbeck commented on a diff in pull request #28502: Migrate DagFileProcessor.manage_slas to Internal API

2023-01-20 Thread GitBox
vincbeck commented on code in PR #28502: URL: https://github.com/apache/airflow/pull/28502#discussion_r1082796111 ## airflow/dag_processing/processor.py: ## @@ -365,8 +365,10 @@ def __init__(self, dag_ids: list[str] | None, dag_directory: str, log: logging.L self._dag_

[GitHub] [airflow] potiuk commented on pull request #28529: Add deferrable mode to DataprocCreateClusterOperator and

2023-01-20 Thread GitBox
potiuk commented on PR #28529: URL: https://github.com/apache/airflow/pull/28529#issuecomment-1398635338 Conflicts :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [airflow] potiuk commented on pull request #28808: Allow setting the name for the base container within K8s Pod Operator

2023-01-20 Thread GitBox
potiuk commented on PR #28808: URL: https://github.com/apache/airflow/pull/28808#issuecomment-1398633574 @jedcunningham @eladkal - any comments or shall we merge it ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] potiuk commented on a diff in pull request #28502: Migrate DagFileProcessor.manage_slas to Internal API

2023-01-20 Thread GitBox
potiuk commented on code in PR #28502: URL: https://github.com/apache/airflow/pull/28502#discussion_r1082777909 ## airflow/dag_processing/processor.py: ## @@ -365,8 +365,10 @@ def __init__(self, dag_ids: list[str] | None, dag_directory: str, log: logging.L self._dag_di

[GitHub] [airflow] potiuk closed issue #28919: Airflow API kerberos authentication error

2023-01-20 Thread GitBox
potiuk closed issue #28919: Airflow API kerberos authentication error URL: https://github.com/apache/airflow/issues/28919 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [airflow] potiuk merged pull request #29054: Fix kerberos authentication for the REST API.

2023-01-20 Thread GitBox
potiuk merged PR #29054: URL: https://github.com/apache/airflow/pull/29054 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on a diff in pull request #29044: Resolve RDS cname

2023-01-20 Thread GitBox
potiuk commented on code in PR #29044: URL: https://github.com/apache/airflow/pull/29044#discussion_r1082740254 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -169,6 +169,33 @@ def get_uri(self) -> str: conn.schema = self.__schema or conn.schema return con

[GitHub] [airflow] potiuk commented on a diff in pull request #29044: Resolve RDS cname

2023-01-20 Thread GitBox
potiuk commented on code in PR #29044: URL: https://github.com/apache/airflow/pull/29044#discussion_r1082740254 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -169,6 +169,33 @@ def get_uri(self) -> str: conn.schema = self.__schema or conn.schema return con

[GitHub] [airflow] potiuk commented on a diff in pull request #29044: Resolve RDS cname

2023-01-20 Thread GitBox
potiuk commented on code in PR #29044: URL: https://github.com/apache/airflow/pull/29044#discussion_r1082731538 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -169,6 +169,33 @@ def get_uri(self) -> str: conn.schema = self.__schema or conn.schema return con

[GitHub] [airflow] potiuk commented on a diff in pull request #29044: Resolve RDS cname

2023-01-20 Thread GitBox
potiuk commented on code in PR #29044: URL: https://github.com/apache/airflow/pull/29044#discussion_r1082731538 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -169,6 +169,33 @@ def get_uri(self) -> str: conn.schema = self.__schema or conn.schema return con

[GitHub] [airflow] potiuk commented on a diff in pull request #29044: Resolve RDS cname

2023-01-20 Thread GitBox
potiuk commented on code in PR #29044: URL: https://github.com/apache/airflow/pull/29044#discussion_r1082731538 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -169,6 +169,33 @@ def get_uri(self) -> str: conn.schema = self.__schema or conn.schema return con

[GitHub] [airflow] VladaZakharova closed pull request #29017: Add deferrable mode to KubernetesPodOperator

2023-01-20 Thread GitBox
VladaZakharova closed pull request #29017: Add deferrable mode to KubernetesPodOperator URL: https://github.com/apache/airflow/pull/29017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] BasPH merged pull request #29027: Annotate and simplify code samples in DAGs doc

2023-01-20 Thread GitBox
BasPH merged PR #29027: URL: https://github.com/apache/airflow/pull/29027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.ap

[GitHub] [airflow] potiuk commented on pull request #29035: Renaming nose compatible methods in flavour of regular pytest naming

2023-01-20 Thread GitBox
potiuk commented on PR #29035: URL: https://github.com/apache/airflow/pull/29035#issuecomment-1398552691 > @potiuk, is it the right place? Yep. But you can also add "full tests needed" label on a PR to run a complete set of tests. -- This is an automated message from the Apache Git

[GitHub] [airflow] snjypl commented on pull request #29012: Fix airflow-webserver cannot create resource pods for k8s exceutor deployed using helm chart

2023-01-20 Thread GitBox
snjypl commented on PR #29012: URL: https://github.com/apache/airflow/pull/29012#issuecomment-1398507760 @arjunanan6 also, you should check `kubectl auth can-i create pods --as system:serviceaccount:airflow:airflowlocal-webserver` -- This is an automated message from the Apache

[GitHub] [airflow] potiuk commented on pull request #29056: Handling error on cluster policy itself

2023-01-20 Thread GitBox
potiuk commented on PR #29056: URL: https://github.com/apache/airflow/pull/29056#issuecomment-1398494098 (it will be obvious from the stack trace that the error was in the policy not in the DAG). -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [airflow] potiuk commented on pull request #29056: Handling error on cluster policy itself

2023-01-20 Thread GitBox
potiuk commented on PR #29056: URL: https://github.com/apache/airflow/pull/29056#issuecomment-1398492336 I thik it's a good idea, but do we really need to add a new exception on that ? Wouldn't just directly hadling ANY Exception (instead of the specific Exceptions we list there) do

[GitHub] [airflow] snjypl commented on pull request #29012: Fix airflow-webserver cannot create resource pods for k8s exceutor deployed using helm chart

2023-01-20 Thread GitBox
snjypl commented on PR #29012: URL: https://github.com/apache/airflow/pull/29012#issuecomment-1398479900 > apiVersion: rbac.authorization.k8s.io/v1 > kind: RoleBinding > metadata: > annotations: > meta.helm.sh/release-name: airflowlocal > meta.helm.sh/release-namespac

[GitHub] [airflow] Taragolis commented on pull request #28953: Updated Telegram Provider to ensure compatbility with >=20.0.0

2023-01-20 Thread GitBox
Taragolis commented on PR #28953: URL: https://github.com/apache/airflow/pull/28953#issuecomment-1398463533 Well... my personal thoughts that there is not a lot of differences for end users between migration to `python-telegram-bot>=20.0.0` and replace by `pyTelegramBotAPI`. I think

[GitHub] [airflow] arjunanan6 commented on pull request #29012: Fix airflow-webserver cannot create resource pods for k8s exceutor deployed using helm chart

2023-01-20 Thread GitBox
arjunanan6 commented on PR #29012: URL: https://github.com/apache/airflow/pull/29012#issuecomment-1398443598 @snjypl Sure, here you go: ``` apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: annotations: meta.helm.sh/release-name: airflowlocal

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #28934: Remove hard-coded executor-database coupling

2023-01-20 Thread GitBox
utkarsharma2 commented on code in PR #28934: URL: https://github.com/apache/airflow/pull/28934#discussion_r1082578543 ## airflow/cli/commands/standalone_command.py: ## @@ -159,14 +159,28 @@ def calculate_env(self): # Make sure we're using a local executor flavour

[GitHub] [airflow] snjypl commented on pull request #29012: Fix airflow-webserver cannot create resource pods for k8s exceutor deployed using helm chart

2023-01-20 Thread GitBox
snjypl commented on PR #29012: URL: https://github.com/apache/airflow/pull/29012#issuecomment-1398439995 @arjunanan6 can you please share the output of `kubectl get rolebinding airflow-pod-launcher-rolebinding -o yaml` -- This is an automated message from the Apache Git Service. To respo

[GitHub] [airflow] Anton-Shutik commented on a diff in pull request #29044: Resolve RDS cname

2023-01-20 Thread GitBox
Anton-Shutik commented on code in PR #29044: URL: https://github.com/apache/airflow/pull/29044#discussion_r1082574233 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -169,6 +169,33 @@ def get_uri(self) -> str: conn.schema = self.__schema or conn.schema retu

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #28934: Remove hard-coded executor-database coupling

2023-01-20 Thread GitBox
utkarsharma2 commented on code in PR #28934: URL: https://github.com/apache/airflow/pull/28934#discussion_r1082565064 ## airflow/cli/commands/standalone_command.py: ## @@ -159,14 +159,28 @@ def calculate_env(self): # Make sure we're using a local executor flavour

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #28934: Remove hard-coded executor-database coupling

2023-01-20 Thread GitBox
utkarsharma2 commented on code in PR #28934: URL: https://github.com/apache/airflow/pull/28934#discussion_r1082563334 ## airflow/sensors/base.py: ## @@ -257,11 +258,14 @@ def _get_next_poke_interval( def prepare_for_execution(self) -> BaseOperator: task = super()

[GitHub] [airflow] VladaZakharova commented on a diff in pull request #29017: Add deferrable mode to KubernetesPodOperator

2023-01-20 Thread GitBox
VladaZakharova commented on code in PR #29017: URL: https://github.com/apache/airflow/pull/29017#discussion_r1082553841 ## airflow/providers/cncf/kubernetes/hooks/kubernetes.py: ## @@ -16,19 +16,28 @@ # under the License. from __future__ import annotations +import contextlib

[GitHub] [airflow] VladaZakharova commented on a diff in pull request #29017: Add deferrable mode to KubernetesPodOperator

2023-01-20 Thread GitBox
VladaZakharova commented on code in PR #29017: URL: https://github.com/apache/airflow/pull/29017#discussion_r1082552556 ## generated/provider_dependencies.json: ## @@ -209,8 +209,10 @@ "cncf.kubernetes": { "deps": [ "apache-airflow>=2.3.0", + "asgiref>=3.5.2"

[GitHub] [airflow] VladaZakharova commented on a diff in pull request #29017: Add deferrable mode to KubernetesPodOperator

2023-01-20 Thread GitBox
VladaZakharova commented on code in PR #29017: URL: https://github.com/apache/airflow/pull/29017#discussion_r1082551668 ## docs/apache-airflow-providers-cncf-kubernetes/operators.rst: ## @@ -1,19 +1,19 @@ .. Licensed to the Apache Software Foundation (ASF) under one -or mo

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29061: Increase length of user identifier columns in `ab_user` and `ab_register_user` tables

2023-01-20 Thread GitBox
boring-cyborg[bot] commented on PR #29061: URL: https://github.com/apache/airflow/pull/29061#issuecomment-1398412860 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] swalkowski opened a new pull request, #29061: Increase length of user identifier columns in `ab_user` and `ab_register_user` tables

2023-01-20 Thread GitBox
swalkowski opened a new pull request, #29061: URL: https://github.com/apache/airflow/pull/29061 To better open Airflow RBAC to external integrations (see below), this change increases the maximum lengths of user identifies columns in `ab_user` and `ab_register_user` tables: - `first_name

[GitHub] [airflow] atrbgithub commented on issue #29026: Status of testing of Apache Airflow 2.5.1rc2

2023-01-20 Thread GitBox
atrbgithub commented on issue #29026: URL: https://github.com/apache/airflow/issues/29026#issuecomment-1398341172 I've span this up locally and can confirm: [Return list of tasks that will be queued (#28066)](https://github.com/apache/airflow/pull/28066) Is fixed in this releas

[GitHub] [airflow] Taragolis commented on pull request #29035: Renaming nose compatible methods in flavour of regular pytest naming

2023-01-20 Thread GitBox
Taragolis commented on PR #29035: URL: https://github.com/apache/airflow/pull/29035#issuecomment-1398319160 > Live is like a box of chocolates, you never know which one you get. You got Helm test this time :) I postpone migrate this tests to pytest as much as possible, so this looks

[GitHub] [airflow] ephraimbuddy opened a new pull request, #29058: Optionally export `airflow db clean` data to CSV files

2023-01-20 Thread GitBox
ephraimbuddy opened a new pull request, #29058: URL: https://github.com/apache/airflow/pull/29058 This adds --export-to-csv and --output-path to db clean command to enable archiving to files on disk. When this option is used, the archival table is automatically dropped since data would n

[GitHub] [airflow] potiuk opened a new pull request, #29057: Fix pre-commit warning for exclude in inclusive-language check

2023-01-20 Thread GitBox
potiuk opened a new pull request, #29057: URL: https://github.com/apache/airflow/pull/29057 The exclude regexp contained /* which was not really what we meant (no harm done but latest pre-commit started to warn about it). --- **^ Add meaningful description above** Re

[GitHub] [airflow] potiuk commented on issue #28919: Airflow API kerberos authentication error

2023-01-20 Thread GitBox
potiuk commented on issue #28919: URL: https://github.com/apache/airflow/issues/28919#issuecomment-1398308134 I made you co-author of the fix @BMFH - see if what I got is good please :) -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] arjunanan6 commented on pull request #29012: Fix airflow-webserver cannot create resource pods for k8s exceutor deployed using helm chart

2023-01-20 Thread GitBox
arjunanan6 commented on PR #29012: URL: https://github.com/apache/airflow/pull/29012#issuecomment-1398288734 Alright, so I attempted this locally where I have no restrictions. As discussed on #28394, scheduled tasks run just fine, but there is an exception when running a task manually:

[GitHub] [airflow] potiuk commented on issue #28919: Airflow API kerberos authentication error

2023-01-20 Thread GitBox
potiuk commented on issue #28919: URL: https://github.com/apache/airflow/issues/28919#issuecomment-1398283300 Ok. Cool Thanks for cofirming :). this sounds cool, providing that the username match :). I will adapt the fix and make some documentation updates to explain it :) -- This is an

[GitHub] [airflow] potiuk commented on pull request #29035: Renaming nose compatible methods in flavour of regular pytest naming

2023-01-20 Thread GitBox
potiuk commented on PR #29035: URL: https://github.com/apache/airflow/pull/29035#issuecomment-1398280798 > Well... I thought that the k8s test failed due to the issues with Github Actions however it also use `setup` methods and we use `pytest.ini` across all tests. > > Let's migrate

[GitHub] [airflow] arjunanan6 commented on pull request #28394: Fix manual task trigger failing for k8s.

2023-01-20 Thread GitBox
arjunanan6 commented on PR #28394: URL: https://github.com/apache/airflow/pull/28394#issuecomment-1398278773 @snjypl A little update, I've tried out this fix in combination with #29012 locally, and still get the same forbidden error. I'll add more details on that PR since it's more relevant

[GitHub] [airflow] okayhooni opened a new pull request, #29056: Handling error on cluster policy itself

2023-01-20 Thread GitBox
okayhooni opened a new pull request, #29056: URL: https://github.com/apache/airflow/pull/29056 We use Airflow cluster policy to ensure some dynamic DAGs and tasks to follow some constraints. But, we realized when there is some error on cluster policy itself(= any errors except intent

[GitHub] [airflow] BMFH commented on a diff in pull request #29054: [Experimental] Treat kerberos identity as email in authentication

2023-01-20 Thread GitBox
BMFH commented on code in PR #29054: URL: https://github.com/apache/airflow/pull/29054#discussion_r1082280492 ## airflow/api/auth/backend/kerberos_auth.py: ## @@ -141,7 +143,7 @@ def decorated(*args, **kwargs): token = "".join(header.split()[1:]) return

[GitHub] [airflow] BMFH commented on issue #28919: Airflow API kerberos authentication error

2023-01-20 Thread GitBox
BMFH commented on issue #28919: URL: https://github.com/apache/airflow/issues/28919#issuecomment-1398123889 @potiuk Thank you for answer! I guessed if we do everything exactly as described in the documentation it would work. But it seems I was wrong. I'm not a developer, I'm just a DevOps

[GitHub] [airflow] Taragolis commented on pull request #29035: Renaming nose compatible methods in flavour of regular pytest naming

2023-01-20 Thread GitBox
Taragolis commented on PR #29035: URL: https://github.com/apache/airflow/pull/29035#issuecomment-1398119466 Well... I thought that the k8s test failed due to the issues with Github Actions however it also use `setup` methods and we use `pytest.ini` across all tests. Let's migrate the

[GitHub] [airflow] csp33 commented on issue #28880: Can't configure Kubernetes and Celery workers in Helm Chart

2023-01-20 Thread GitBox
csp33 commented on issue #28880: URL: https://github.com/apache/airflow/issues/28880#issuecomment-1398093799 I could take this one. How about this proposal? values defined at the "workers" level will be taken by both celery & k8s. ```yaml workers: safeToEvict: false

[GitHub] [airflow] maxnathaniel commented on pull request #28953: Updated Telegram Provider to ensure compatbility with >=20.0.0

2023-01-19 Thread GitBox
maxnathaniel commented on PR #28953: URL: https://github.com/apache/airflow/pull/28953#issuecomment-1398028299 @Taragolis Sorry have been busy past couple of days. Do you think I should go ahead to replace `python-telegram-bot` with [pyTelegramBotAPI](https://github.com/eternnoir/pyTelegram

[GitHub] [airflow] ecerulm commented on a diff in pull request #29016: Fix leak sensitive field via V1EnvVar on exception

2023-01-19 Thread GitBox
ecerulm commented on code in PR #29016: URL: https://github.com/apache/airflow/pull/29016#discussion_r1082180416 ## airflow/utils/log/secrets_masker.py: ## @@ -200,10 +222,18 @@ def _redact(self, item: Redactable, name: str | None, depth: int) -> Redacted: if name

[GitHub] [airflow] phanikumv commented on a diff in pull request #29014: Add deferrable mode to `DbtCloudRunJobOperator`

2023-01-19 Thread GitBox
phanikumv commented on code in PR #29014: URL: https://github.com/apache/airflow/pull/29014#discussion_r1082167660 ## docs/apache-airflow-providers-dbt-cloud/operators.rst: ## @@ -35,10 +35,10 @@ Trigger a dbt Cloud Job Use the :class:`~airflow.providers.dbt.cloud.operators.db

[GitHub] [airflow] potiuk commented on issue #29031: Scheduler pod restarts

2023-01-19 Thread GitBox
potiuk commented on issue #29031: URL: https://github.com/apache/airflow/issues/29031#issuecomment-1397996468 Hard to say. Maybe you have REALLY SLOW network or filesystem mounted in your system? Or maybe your docker engine has limited memory (as it happens on MacOS where we recommend to in

[GitHub] [airflow] Subhashini2610 commented on issue #29031: Scheduler pod restarts

2023-01-19 Thread GitBox
Subhashini2610 commented on issue #29031: URL: https://github.com/apache/airflow/issues/29031#issuecomment-1397974322 @hussein-awala Thanks for the insight. I have only one DAG (hello_world.py) synced, which is also not scheduled for periodic runs. Also, I have checked the pod metrics. The

[GitHub] [airflow] tirkarthi commented on pull request #28256: Include full path to Python files under zip path while clearing import errors.

2023-01-19 Thread GitBox
tirkarthi commented on PR #28256: URL: https://github.com/apache/airflow/pull/28256#issuecomment-1397945755 @potiuk @uranusjr Sorry, I am little confused. Currently `ImportError` model has filename column that stores full path to Python along with zipfile like `/home/karthikeyan/airflow/dag

[GitHub] [airflow] o-nikolas commented on a diff in pull request #29055: [AIP-51] Executors vending CLI commands

2023-01-19 Thread GitBox
o-nikolas commented on code in PR #29055: URL: https://github.com/apache/airflow/pull/29055#discussion_r1082091328 ## airflow/cli/cli_parser.py: ## @@ -17,2165 +17,35 @@ # specific language governing permissions and limitations # under the License. """Command-line interface."

[GitHub] [airflow] schwartzpub commented on issue #29037: Cron schedule and Time Zones leads to incorrect intervals

2023-01-19 Thread GitBox
schwartzpub commented on issue #29037: URL: https://github.com/apache/airflow/issues/29037#issuecomment-1397824977 Last run/next run aside -- what other details are needed that aren't provided above? For reference, the documentation for 2.5 is where the recommendation for `default_timezone

[GitHub] [airflow] potiuk commented on pull request #28693: AIP-44 Migrate DagModel.get_paused_dag_ids to Internal API

2023-01-19 Thread GitBox
potiuk commented on PR #28693: URL: https://github.com/apache/airflow/pull/28693#issuecomment-1397793451 > Is there any action item/comment here I should act on or we can forward with this PR? Rebasing/conflict resolving after we merged #28841 -- This is an automated message from

[GitHub] [airflow] potiuk commented on a diff in pull request #28618: Add deferrable mode to DataprocInstantiateWorkflowTemplateOperator

2023-01-19 Thread GitBox
potiuk commented on code in PR #28618: URL: https://github.com/apache/airflow/pull/28618#discussion_r1082013189 ## airflow/providers/google/cloud/triggers/dataproc.py: ## @@ -84,3 +84,73 @@ async def run(self): raise AirflowException(f"Dataproc job execution

[GitHub] [airflow] potiuk commented on pull request #28956: Move project and license docs down to start with developer-focused docs

2023-01-19 Thread GitBox
potiuk commented on PR #28956: URL: https://github.com/apache/airflow/pull/28956#issuecomment-1397788465 Oh Absolutely. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [airflow] potiuk merged pull request #28956: Move project and license docs down to start with developer-focused docs

2023-01-19 Thread GitBox
potiuk merged PR #28956: URL: https://github.com/apache/airflow/pull/28956 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] hussein-awala commented on issue #29031: Scheduler pod restarts

2023-01-19 Thread GitBox
hussein-awala commented on issue #29031: URL: https://github.com/apache/airflow/issues/29031#issuecomment-1397785578 Since the liveness probe request return `503`, you scheduler could be overloaded, and there are several different causes for this problem: - The request/limit resources for

[GitHub] [airflow] potiuk commented on issue #19781: A way to test email configuration

2023-01-19 Thread GitBox
potiuk commented on issue #19781: URL: https://github.com/apache/airflow/issues/19781#issuecomment-1397782458 > btw should i create a separate ticket on that or will just a PR suffice No need for issues in Airflow. PR is enough in vast majority of canses. -- This is an automated mes

[GitHub] [airflow] potiuk merged pull request #29051: Add a summary table at the end of system test execs

2023-01-19 Thread GitBox
potiuk merged PR #29051: URL: https://github.com/apache/airflow/pull/29051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] o-nikolas opened a new pull request, #29055: Onikolas/aip 51/executor cli vending

2023-01-19 Thread GitBox
o-nikolas opened a new pull request, #29055: URL: https://github.com/apache/airflow/pull/29055 This PR addresses executor coupling **5a** and **5b** in Airflow CLI. See [this issue](https://github.com/apache/airflow/issues/27932) (#27932) for more context. ## Please read before revie

[GitHub] [airflow] potiuk commented on pull request #29051: Add a summary table at the end of system test execs

2023-01-19 Thread GitBox
potiuk commented on PR #29051: URL: https://github.com/apache/airflow/pull/29051#issuecomment-1397779289 > > (and we should wait for it). > > You mean we should not merge this until AIP-52 gives us a way to mark teardown tasks ? Nope. Wait with separating the tables. I think wh

[GitHub] [airflow] potiuk commented on issue #28919: Airflow API kerberos authentication error

2023-01-19 Thread GitBox
potiuk commented on issue #28919: URL: https://github.com/apache/airflow/issues/28919#issuecomment-139007 How do you expect your kerberos user to map to Airflow @BMFH? I believe indeed the current way kerberos user is retrieved is wrong. From what I see. the user that is set by kerberos

[GitHub] [airflow] potiuk opened a new pull request, #29054: [Experimental] Treat kerberos identity as email in authentication

2023-01-19 Thread GitBox
potiuk opened a new pull request, #29054: URL: https://github.com/apache/airflow/pull/29054 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more info

[GitHub] [airflow] vandonr-amz commented on pull request #28816: introduce a method to convert dictionaries to boto-style key-value lists

2023-01-19 Thread GitBox
vandonr-amz commented on PR #28816: URL: https://github.com/apache/airflow/pull/28816#issuecomment-1397776181 @eladkal @uranusjr anything else you'd like changed with that PR ? Otherwise we could merge it 😇 -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [airflow] vandonr-amz opened a new pull request, #29053: introduce base class for EKS sensors

2023-01-19 Thread GitBox
vandonr-amz opened a new pull request, #29053: URL: https://github.com/apache/airflow/pull/29053 most of the EKS sensors do the same thing, this can be grouped in one base class, a bit like what was done here https://github.com/apache/airflow/blob/main/airflow/providers/amazon/aws/sensors/s

[GitHub] [airflow] hussein-awala commented on issue #29002: KubernetesPodOperator xcom push failure

2023-01-19 Thread GitBox
hussein-awala commented on issue #29002: URL: https://github.com/apache/airflow/issues/29002#issuecomment-1397773389 I have the same problem with some of my pod tasks, I just created this [PR](https://github.com/apache/airflow/pull/29052), can you check it please? -- This is an automated

[GitHub] [airflow] hussein-awala opened a new pull request, #29052: Fix KubernetesPodOperator xcom push when `get_logs=False`

2023-01-19 Thread GitBox
hussein-awala opened a new pull request, #29052: URL: https://github.com/apache/airflow/pull/29052 closes: #29002 --- **^ Add meaningful description above** When `get_logs=False`, we should wait the `base` container completion before reading the xcom file, to do tha

[GitHub] [airflow] o-nikolas commented on pull request #28934: Remove hard-coded executor-database coupling

2023-01-19 Thread GitBox
o-nikolas commented on PR #28934: URL: https://github.com/apache/airflow/pull/28934#issuecomment-1397769679 Hey @utkarsharma2! Have you had any time to revisit this one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [airflow] o-nikolas merged pull request #28706: Update provide_bucket_name() decorator to handle new conn_type

2023-01-19 Thread GitBox
o-nikolas merged PR #28706: URL: https://github.com/apache/airflow/pull/28706 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] github-actions[bot] commented on pull request #27156: Add documentation for BigQuery transfer operators

2023-01-19 Thread GitBox
github-actions[bot] commented on PR #27156: URL: https://github.com/apache/airflow/pull/27156#issuecomment-1397768482 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] github-actions[bot] commented on pull request #28056: Move GCP Sheets system tests

2023-01-19 Thread GitBox
github-actions[bot] commented on PR #28056: URL: https://github.com/apache/airflow/pull/28056#issuecomment-1397768424 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] vandonr-amz commented on pull request #29051: Add a summary table at the end of system test execs

2023-01-19 Thread GitBox
vandonr-amz commented on PR #29051: URL: https://github.com/apache/airflow/pull/29051#issuecomment-1397764394 > (and we should wait for it). You mean we should not merge this until AIP-52 gives us a way to mark teardown tasks ? -- This is an automated message from the Apache Git Se

[GitHub] [airflow] o-nikolas merged pull request #29001: uniformize getting hook through cached property in aws sensors

2023-01-19 Thread GitBox
o-nikolas merged PR #29001: URL: https://github.com/apache/airflow/pull/29001 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] potiuk commented on pull request #28902: Chart: Update default git-sync version to 3.6.2

2023-01-19 Thread GitBox
potiuk commented on PR #28902: URL: https://github.com/apache/airflow/pull/28902#issuecomment-1397758005 @aleveille - still working on it ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [airflow] potiuk commented on issue #18010: Airflow scheduler with statsd enabled crashes when dag_id contains unexpected characters

2023-01-19 Thread GitBox
potiuk commented on issue #18010: URL: https://github.com/apache/airflow/issues/18010#issuecomment-1397756161 Should be part of discussion here: https://lists.apache.org/thread/96tco8dfs4mh12kqm1pwjhm5mqr54qbm . I think we should limit ID to ASCII and add extra description. -- This is a

[GitHub] [airflow] potiuk commented on pull request #29051: Add a summary table at the end of system test execs

2023-01-19 Thread GitBox
potiuk commented on PR #29051: URL: https://github.com/apache/airflow/pull/29051#issuecomment-1397754758 > Is the watcher supposed to be the "separator" ? Because in that case, disable_job_queue is part of the teardown and apparently executed before the watcher (I sort by end_date, maybe th

  1   2   3   4   5   6   7   8   9   10   >