Re: [PR] Adding MSGraphOperator in Microsoft Azure provider [airflow]

2024-04-14 Thread via GitHub
dabla commented on PR #38111: URL: https://github.com/apache/airflow/pull/38111#issuecomment-2055439942 Thank you @potiuk :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Feature/pass airflow confi as job param [airflow]

2024-04-14 Thread via GitHub
SubhamSinghal commented on code in PR #39007: URL: https://github.com/apache/airflow/pull/39007#discussion_r1565208571 ## airflow/providers/databricks/operators/databricks.py: ## @@ -313,6 +313,12 @@ def execute(self, context: Context) -> int: if "name" not in

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow [airflow]

2024-04-14 Thread via GitHub
howardyoo commented on PR #37948: URL: https://github.com/apache/airflow/pull/37948#issuecomment-2055263616 @potiuk , I am going to split this PR into part A, and the rest (Part B, C, D..), where part A would have the basic trace capabilities and necessary configuration, and part B will

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow [airflow]

2024-04-14 Thread via GitHub
howardyoo commented on code in PR #37948: URL: https://github.com/apache/airflow/pull/37948#discussion_r1565189532 ## airflow/dag_processing/manager.py: ## @@ -1029,6 +1051,25 @@ def _collect_results_from_processor(self, processor) -> None: )

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow [airflow]

2024-04-14 Thread via GitHub
howardyoo commented on code in PR #37948: URL: https://github.com/apache/airflow/pull/37948#discussion_r1565189095 ## airflow/traces/tracer.py: ## @@ -0,0 +1,280 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

Re: [PR] Remove unnecessary nginx redirect rule from reverse proxy documentation [airflow]

2024-04-14 Thread via GitHub
scottlimmer commented on code in PR #38953: URL: https://github.com/apache/airflow/pull/38953#discussion_r1565188631 ## docs/apache-airflow/howto/run-behind-proxy.rst: ## @@ -62,7 +62,6 @@ Your reverse proxy (ex: nginx) should be configured as follow: server_name

Re: [PR] Redacting the sensitive env variables in env_vars for KPO [airflow]

2024-04-14 Thread via GitHub
dirrao commented on code in PR #39003: URL: https://github.com/apache/airflow/pull/39003#discussion_r1565185894 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -334,6 +335,8 @@ def __init__( self.env_vars = env_vars if pod_runtime_info_envs:

Re: [I] xcom_push of pod_namespace failure can make pod leak [airflow]

2024-04-14 Thread via GitHub
boring-cyborg[bot] commented on issue #39016: URL: https://github.com/apache/airflow/issues/39016#issuecomment-2055195801 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] xcom_push of pod_namespace failure can make pod leak [airflow]

2024-04-14 Thread via GitHub
EugeneChung opened a new issue, #39016: URL: https://github.com/apache/airflow/issues/39016 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.7.3 ### What happened? I'm using

Re: [I] Performance Issues on Initial Page Load [airflow]

2024-04-14 Thread via GitHub
eladkal commented on issue #38776: URL: https://github.com/apache/airflow/issues/38776#issuecomment-2055137088 cc @ephraimbuddy @bbovenzi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] Correcting an example taskflow example [airflow]

2024-04-14 Thread via GitHub
amoghrajesh opened a new pull request, #39015: URL: https://github.com/apache/airflow/pull/39015 Correcting a taskflow DAG example found by someone on slack. After correction, dag runs fine:

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow [airflow]

2024-04-14 Thread via GitHub
howardyoo closed pull request #37948: [AIP-49] OpenTelemetry Traces for Apache Airflow URL: https://github.com/apache/airflow/pull/37948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Feature/pass airflow confi as job param [airflow]

2024-04-14 Thread via GitHub
dirrao commented on code in PR #39007: URL: https://github.com/apache/airflow/pull/39007#discussion_r1565137969 ## airflow/providers/databricks/operators/databricks.py: ## @@ -313,6 +313,12 @@ def execute(self, context: Context) -> int: if "name" not in self.json:

Re: [PR] Feature/pass airflow confi as job param [airflow]

2024-04-14 Thread via GitHub
dirrao commented on code in PR #39007: URL: https://github.com/apache/airflow/pull/39007#discussion_r1565137798 ## airflow/providers/databricks/operators/databricks.py: ## @@ -313,6 +313,12 @@ def execute(self, context: Context) -> int: if "name" not in self.json:

Re: [PR] Add four unit tests for aws/utils [airflow]

2024-04-14 Thread via GitHub
slycyberguy commented on PR #38820: URL: https://github.com/apache/airflow/pull/38820#issuecomment-2054526341 > You will need to remove these lines from the project structure tests: > > https://github.com/apache/airflow/blob/main/tests/always/test_project_structure.py#L85-L86

Re: [PR] Add watchdog for immediately processing changes in the DAGs folder [airflow]

2024-04-14 Thread via GitHub
github-actions[bot] closed pull request #34487: Add watchdog for immediately processing changes in the DAGs folder URL: https://github.com/apache/airflow/pull/34487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-14 Thread via GitHub
zstrathe commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1565032713 ## tests/providers/apache/beam/operators/test_beam.py: ## Review Comment: @e-galan @eladkal I added test_execute_gcs_hook_called_only_with_gs_prefix() to

Re: [PR] check whether AUTH_ROLE_PUBLIC is set in check_authentication [airflow]

2024-04-14 Thread via GitHub
Lee-W closed pull request #39012: check whether AUTH_ROLE_PUBLIC is set in check_authentication URL: https://github.com/apache/airflow/pull/39012 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Revert "check whether AUTH_ROLE_PUBLIC is set in check_authentication… [airflow]

2024-04-14 Thread via GitHub
Lee-W commented on PR #39009: URL: https://github.com/apache/airflow/pull/39009#issuecomment-2054296717 Sure. I just addressed the issue, created a new PR https://github.com/apache/airflow/pull/39012, and added the required label. Thanks for reminding me! -- This is an automated message

Re: [PR] check whether AUTH_ROLE_PUBLIC is set in check_authentication [airflow]

2024-04-14 Thread via GitHub
Lee-W commented on PR #39012: URL: https://github.com/apache/airflow/pull/39012#issuecomment-2054297201 It was originally https://github.com/apache/airflow/pull/38924, but -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-14 Thread via GitHub
zstrathe commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1565032713 ## tests/providers/apache/beam/operators/test_beam.py: ## Review Comment: @e-galan @eladkal I added test_execute_gcs_hook_called_only_with_gs_prefix() to

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-14 Thread via GitHub
zstrathe commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1565032713 ## tests/providers/apache/beam/operators/test_beam.py: ## Review Comment: I added test_execute_gcs_hook_called_only_with_gs_prefix() to the

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-14 Thread via GitHub
zstrathe commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1565030327 ## airflow/providers/apache/beam/operators/beam.py: ## @@ -364,11 +364,13 @@ def execute(self, context: Context): def execute_sync(self, context: Context):

Re: [PR] AWS auth manager documentation: manage environment section [airflow]

2024-04-14 Thread via GitHub
shubham22 commented on code in PR #38932: URL: https://github.com/apache/airflow/pull/38932#discussion_r1565006331 ## docs/apache-airflow-providers-amazon/auth-manager/manage/index.rst: ## @@ -18,3 +18,205 @@ Manage Airflow

Re: [I] on_failure_callback is not called when task is manually marked as failed [airflow]

2024-04-14 Thread via GitHub
FFCMSouza commented on issue #38935: URL: https://github.com/apache/airflow/issues/38935#issuecomment-2054223724 Anyway, I managed to find a way to solve my problem overwriting the `on_kill` method in my spark operator. The only downside is that the `on_kill` method doesn't have access

Re: [I] on_failure_callback is not called when task is manually marked as failed [airflow]

2024-04-14 Thread via GitHub
FFCMSouza commented on issue #38935: URL: https://github.com/apache/airflow/issues/38935#issuecomment-2054217500 > There are no plans to change the behaviour at the moment. Also, I don't know why you would want the callback to run when you intentionally failed a task In my case, I

Re: [PR] Deprecation of AutoML services: Add deprecation warnings and raise exceptions for already deprecated ones [airflow]

2024-04-14 Thread via GitHub
molcay commented on code in PR #38673: URL: https://github.com/apache/airflow/pull/38673#discussion_r1564955547 ## airflow/providers/google/cloud/hooks/vertex_ai/prediction_service.py: ## @@ -0,0 +1,85 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

Re: [PR] Deprecation of AutoML services: Add deprecation warnings and raise exceptions for already deprecated ones [airflow]

2024-04-14 Thread via GitHub
molcay commented on code in PR #38673: URL: https://github.com/apache/airflow/pull/38673#discussion_r1564955395 ## tests/providers/google/cloud/operators/test_automl.py: ## @@ -21,11 +21,16 @@ from unittest import mock import pytest -from google.api_core.gapic_v1.method

Re: [PR] Revert "check whether AUTH_ROLE_PUBLIC is set in check_authentication… [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #39009: URL: https://github.com/apache/airflow/pull/39009#issuecomment-205423 The PR to run FAB tests together with API changes is merged now @Lee-W -> so you should be able to redo the PR and see the tess failing in your PR. The "too selective" case should be

Re: [PR] Fix trigger_kwargs encryption/decryption on db migration [airflow]

2024-04-14 Thread via GitHub
ephraimbuddy commented on PR #38876: URL: https://github.com/apache/airflow/pull/38876#issuecomment-2054195117 Hi @hussein-awala , can you add more information to the commit message, I'm finding it difficult to understand what you worked on. -- This is an automated message from the

Re: [I] Status of testing Providers that were prepared on April 10, 2024 [airflow]

2024-04-14 Thread via GitHub
molcay commented on issue #38904: URL: https://github.com/apache/airflow/issues/38904#issuecomment-2054191696 The changes in #37787 works as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Test swagger UI removeavl connexion3 [airflow]

2024-04-14 Thread via GitHub
potiuk opened a new pull request, #39011: URL: https://github.com/apache/airflow/pull/39011 --- **^ Add meaningful description above** Read the **[Pull Request

(airflow-site) branch gh-pages updated (2a59c887ef -> 50c14ff30f)

2024-04-14 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard 2a59c887ef Rewritten history to remove past gh-pages deployments new 50c14ff30f Rewritten history to

Re: [PR] Provide a UUID to trace log when RPC internal server error... WDYT? [airflow]

2024-04-14 Thread via GitHub
dstandish commented on PR #38989: URL: https://github.com/apache/airflow/pull/38989#issuecomment-2054172152 > No. It's good to provide UUID in this case. We should never present traceback on the client side. Like EVER. My comment was not about the "customers" but users of the API - I hope

Re: [PR] Provide a UUID to trace log when RPC internal server error... WDYT? [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #38989: URL: https://github.com/apache/airflow/pull/38989#issuecomment-2054171002 No. It's good to provide UUID in this case. We should never present traceback on the client side. Like EVER. My comment was not about the "customers" but users of the API - I hope it

Re: [PR] Provide a UUID to trace log when RPC internal server error... WDYT? [airflow]

2024-04-14 Thread via GitHub
dstandish commented on PR #38989: URL: https://github.com/apache/airflow/pull/38989#issuecomment-2054170002 @potiuk If we don’t need to suppress the server side traceback then there’s no need for this here. I.e. if no security concern. In that case it is better to simply include the server

Re: [PR] Provide a UUID to trace log when RPC internal server error... WDYT? [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #38989: URL: https://github.com/apache/airflow/pull/38989#issuecomment-2054169430 No. It's fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Provide a UUID to trace log when RPC internal server error... WDYT? [airflow]

2024-04-14 Thread via GitHub
dstandish commented on PR #38989: URL: https://github.com/apache/airflow/pull/38989#issuecomment-2054169060 @potiuk i think it would be helpful eg for a customer service team or cluster admin type person more than the individual user. User just reports trace back and the admin or csr puts

Re: [PR] Provide a UUID to trace log when RPC internal server error... WDYT? [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #38989: URL: https://github.com/apache/airflow/pull/38989#issuecomment-2054168158 Not sure how our users would react to it and find it, as there is little information what to do with the UUID, but yes it might be helpful to tell users what to do if they see it. --

(airflow) branch main updated: Activate RUF015 that checks for unnecessary iterable allocation for first element (#38949)

2024-04-14 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6520653c43 Activate RUF015 that checks for

Re: [PR] Activate RUF015 that checks for unnecessary iterable allocation for first element [airflow]

2024-04-14 Thread via GitHub
hussein-awala merged PR #38949: URL: https://github.com/apache/airflow/pull/38949 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow-site) branch main updated: Add Power BI plugin Github link in ecosystem (#1006)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 9303c309c8 Add Power BI plugin Github link in

Re: [PR] Add Power BI plugin GitHub link in ecosystem. [airflow-site]

2024-04-14 Thread via GitHub
potiuk merged PR #1006: URL: https://github.com/apache/airflow-site/pull/1006 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Adding MSGraphOperator in Microsoft Azure provider [airflow]

2024-04-14 Thread via GitHub
potiuk merged PR #38111: URL: https://github.com/apache/airflow/pull/38111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Adding MSGraphOperator in Microsoft Azure provider (#38111)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1c9a6609f3 Adding MSGraphOperator in Microsoft

Re: [PR] Adding MSGraphOperator in Microsoft Azure provider [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #38111: URL: https://github.com/apache/airflow/pull/38111#issuecomment-2054164536 The docker example error workarounded in main. Merging -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] on_failure_callback is not called when task is manually marked as failed [airflow]

2024-04-14 Thread via GitHub
ephraimbuddy commented on issue #38935: URL: https://github.com/apache/airflow/issues/38935#issuecomment-2054154690 > Yes, I saw that. So there is no intention to change this behavior? To me this doesn't make sense, marking a task as failed should trigger the failure callback. If that's

Re: [PR] Fix `update-common-sql-api-stubs` pre-commit check [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #38915: URL: https://github.com/apache/airflow/pull/38915#issuecomment-2054154487 I don't think we disagree here. The attempt so far in the DBAPiHook and common.sql was to make a "common" interface out of "not-that common" DBAPI + SQL - and then make it

Re: [PR] Use default_factory instead of calling the method in dataclass [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #38948: URL: https://github.com/apache/airflow/pull/38948#issuecomment-2054149367 With cache it's good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(airflow) branch main updated: Trigger FAB provider tests on API change (#39010)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ac1f74402f Trigger FAB provider tests on API

Re: [PR] Trigger FAB provider tests on API change [airflow]

2024-04-14 Thread via GitHub
potiuk merged PR #39010: URL: https://github.com/apache/airflow/pull/39010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Use default_factory instead of calling the method in dataclass [airflow]

2024-04-14 Thread via GitHub
hussein-awala commented on PR #38948: URL: https://github.com/apache/airflow/pull/38948#issuecomment-2054123990 > AFAIK it is exact the opposite, default values assigned exactly once, as well as module imports exactly once, if do not count manual reload through importlib or clear

(airflow) branch improve-warning-system updated (f81e7493ff -> 07fd91785e)

2024-04-14 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch improve-warning-system in repository https://gitbox.apache.org/repos/asf/airflow.git discard f81e7493ff Change capture warnings output format to the JSON add 28bd256829 Resolve internal

[PR] Trigger FAB provider tests on API change [airflow]

2024-04-14 Thread via GitHub
potiuk opened a new pull request, #39010: URL: https://github.com/apache/airflow/pull/39010 Follow up after #38924 which was not triggered when API changed --- **^ Add meaningful description above** Read the **[Pull Request

(airflow) branch main updated: Revert "check whether AUTH_ROLE_PUBLIC is set in check_authentication (#38924)" (#39009)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7fc2169a41 Revert "check whether AUTH_ROLE_PUBLIC

Re: [PR] Revert "check whether AUTH_ROLE_PUBLIC is set in check_authentication… [airflow]

2024-04-14 Thread via GitHub
potiuk merged PR #39009: URL: https://github.com/apache/airflow/pull/39009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Revert "check whether AUTH_ROLE_PUBLIC is set in check_authentication… [airflow]

2024-04-14 Thread via GitHub
potiuk closed pull request #39009: Revert "check whether AUTH_ROLE_PUBLIC is set in check_authentication… URL: https://github.com/apache/airflow/pull/39009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Revert "check whether AUTH_ROLE_PUBLIC is set in check_authentication… [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #39009: URL: https://github.com/apache/airflow/pull/39009#issuecomment-2054088961 Hey @Lee-W - this PR has a side effect on failing a number of tests in ``` tests/providers/fab/auth_manager/api_endpoints/test_role_and_permission_endpoint.py ``` By

[PR] Revert "check whether AUTH_ROLE_PUBLIC is set in check_authentication… [airflow]

2024-04-14 Thread via GitHub
potiuk opened a new pull request, #39009: URL: https://github.com/apache/airflow/pull/39009 … (#38924)" This reverts commit 7b608250468740954c6b0af7a5f7f23dfa52b473. --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] fix: use `sqlalchemy_url` property in `get_uri` for postgresql provider [airflow]

2024-04-14 Thread via GitHub
rawwar commented on code in PR #38831: URL: https://github.com/apache/airflow/pull/38831#discussion_r1564753091 ## airflow/providers/postgres/hooks/postgres.py: ## @@ -113,6 +114,18 @@ def schema(self): def schema(self, value): self.database = value +

[PR] Add support to search by dag_display_name. [airflow]

2024-04-14 Thread via GitHub
tirkarthi opened a new pull request, #39008: URL: https://github.com/apache/airflow/pull/39008 closes: #38990 related: #38990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Allow searching dag by display name [airflow]

2024-04-14 Thread via GitHub
tirkarthi commented on issue #38990: URL: https://github.com/apache/airflow/issues/38990#issuecomment-2054071274 I am working on this and can submit a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Feature/pass airflow confi as job param [airflow]

2024-04-14 Thread via GitHub
SubhamSinghal opened a new pull request, #39007: URL: https://github.com/apache/airflow/pull/39007 Fixes issue: #39002 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Partially enable `B028` rule: no-explicit-stacklevel [airflow]

2024-04-14 Thread via GitHub
potiuk merged PR #39000: URL: https://github.com/apache/airflow/pull/39000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Partially enable `B028` rule: no-explicit-stacklevel (#39000)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new aa73bca2b5 Partially enable `B028` rule:

Re: [I] Add colour to logs [airflow]

2024-04-14 Thread via GitHub
tirkarthi commented on issue #37443: URL: https://github.com/apache/airflow/issues/37443#issuecomment-2054060070 I have opened https://github.com/apache/airflow/pull/39006 with screenshots as one possible approach. Using custom handler would also mean the log endpoint will emit logs with

Re: [PR] Add `inherit_cache` attribute for `CreateTableAs` custom SA Clause [airflow]

2024-04-14 Thread via GitHub
potiuk merged PR #38985: URL: https://github.com/apache/airflow/pull/38985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Add `inherit_cache` attribute for `CreateTableAs` custom SA Clause (#38985)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new bdb8f749ee Add `inherit_cache` attribute for

(airflow) branch main updated: Use `model_validate` instead of `parse_obj` for de-serialize Pydantic V2 model (#38999)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 30f56020f7 Use `model_validate` instead of

Re: [PR] Use `model_validate` instead of `parse_obj` for de-serialize Pydantic V2 model [airflow]

2024-04-14 Thread via GitHub
potiuk merged PR #38999: URL: https://github.com/apache/airflow/pull/38999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Add color to log lines in UI for error and warnings based on keywords [airflow]

2024-04-14 Thread via GitHub
tirkarthi opened a new pull request, #39006: URL: https://github.com/apache/airflow/pull/39006 closes: #37443 related: #37443 We have been using this feature in Airflow 2.3.4 in the legacy log page by applying regex against lines to apply color for them. It received positive

Re: [PR] Ensure we can serialize DatasetEventAccessor(s) properly [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #38993: URL: https://github.com/apache/airflow/pull/38993#issuecomment-2054058758 Needs tests/static fix but looks good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

(airflow) branch main updated (70e17f6e00 -> 710d28f341)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 70e17f6e00 Mark `extending/add-build-essential-extend/Dockerfile` docker example test as XFAIL (#38978) add

Re: [PR] Update testing_packages.rst [airflow]

2024-04-14 Thread via GitHub
potiuk merged PR #38996: URL: https://github.com/apache/airflow/pull/38996 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Update testing_packages.rst [airflow]

2024-04-14 Thread via GitHub
potiuk commented on PR #38996: URL: https://github.com/apache/airflow/pull/38996#issuecomment-2054058137 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow) branch main updated: Mark `extending/add-build-essential-extend/Dockerfile` docker example test as XFAIL (#38978)

2024-04-14 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 70e17f6e00 Mark

Re: [PR] Mark `extending/add-build-essential-extend/Dockerfile` docker example test as XFAIL [airflow]

2024-04-14 Thread via GitHub
potiuk merged PR #38978: URL: https://github.com/apache/airflow/pull/38978 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Status of testing Providers that were prepared on April 13, 2024 [airflow]

2024-04-14 Thread via GitHub
SubhamSinghal commented on issue #38997: URL: https://github.com/apache/airflow/issues/38997#issuecomment-2054043046 Testes #38702, working as expected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Fix bug in LivyOperator when its trigger times out [airflow]

2024-04-14 Thread via GitHub
eladkal merged PR #38916: URL: https://github.com/apache/airflow/pull/38916 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] livy provider can not kill the spark job when the trigger is timeout. [airflow]

2024-04-14 Thread via GitHub
eladkal closed issue #37898: livy provider can not kill the spark job when the trigger is timeout. URL: https://github.com/apache/airflow/issues/37898 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

(airflow) branch main updated: Fix bug in LivyOperator when its trigger times out (#38916)

2024-04-14 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new bf5ab8fc46 Fix bug in LivyOperator when its

Re: [I] Connection to Postgres failed due to special characters in a password [airflow]

2024-04-14 Thread via GitHub
rawwar commented on issue #38187: URL: https://github.com/apache/airflow/issues/38187#issuecomment-2054020442 PR to fix this issue: Here's the PR: https://github.com/apache/airflow/pull/38831 @DEworkerDE , I've already tested the fix and it's working. Can you please test it as well?

Re: [PR] [WIP] Pinecone provider support for `pinecone-client`>=3 [airflow]

2024-04-14 Thread via GitHub
rawwar commented on PR #37307: URL: https://github.com/apache/airflow/pull/37307#issuecomment-2054005270 I am not sure why a few tests are failing. But, I'm looking into it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

(airflow) branch main updated: Add in Trove classifiers Python 3.12 support (#39004)

2024-04-14 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c2f96ff38d Add in Trove classifiers Python 3.12

Re: [PR] Add in Trove classifiers Python 3.12 support [airflow]

2024-04-14 Thread via GitHub
Taragolis merged PR #39004: URL: https://github.com/apache/airflow/pull/39004 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] update Python 3.12 supports to pyproject.toml [airflow]

2024-04-14 Thread via GitHub
Lee-W opened a new pull request, #39004: URL: https://github.com/apache/airflow/pull/39004 --- **^ Add meaningful description above** Read the **[Pull Request

(airflow) branch main updated: check whether AUTH_ROLE_PUBLIC is set in check_authentication (#38924)

2024-04-14 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7b60825046 check whether AUTH_ROLE_PUBLIC is

Re: [I] 401 UNAUTHORIZED when using AUTH_ROLE_PUBLIC = "Admin" [airflow]

2024-04-14 Thread via GitHub
Taragolis closed issue #38900: 401 UNAUTHORIZED when using AUTH_ROLE_PUBLIC = "Admin" URL: https://github.com/apache/airflow/issues/38900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] check whether AUTH_ROLE_PUBLIC is set in check_authentication [airflow]

2024-04-14 Thread via GitHub
Taragolis merged PR #38924: URL: https://github.com/apache/airflow/pull/38924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: AWS auth manager documentation: manage environment section (#38932)

2024-04-14 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3eac9778f6 AWS auth manager documentation:

Re: [PR] AWS auth manager documentation: manage environment section [airflow]

2024-04-14 Thread via GitHub
Taragolis merged PR #38932: URL: https://github.com/apache/airflow/pull/38932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Fix wrong path in `providers/aws/utils/__init__.py` (#38987)

2024-04-14 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2f39cdec3c Fix wrong path in

Re: [PR] Fix wrong path in `providers/aws/utils/__init__.py` [airflow]

2024-04-14 Thread via GitHub
Taragolis merged PR #38987: URL: https://github.com/apache/airflow/pull/38987 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Add `sqlalchemy_url` property to `DbApiHook` class (#38871)

2024-04-14 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 08f4b923ab Add `sqlalchemy_url` property to

Re: [PR] Add `sqlalchemy_url` property to `DbApiHook` class [airflow]

2024-04-14 Thread via GitHub
Taragolis merged PR #38871: URL: https://github.com/apache/airflow/pull/38871 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch constraints-main updated: Updating constraints. Github run id:8679127029

2024-04-14 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new fc8e0e124f Updating

Re: [PR] Redacting the sensitive env variables in env_vars for KPO [airflow]

2024-04-14 Thread via GitHub
amoghrajesh commented on PR #39003: URL: https://github.com/apache/airflow/pull/39003#issuecomment-2053980771 @hussein-awala yea some test coverage would be nice. Just wanted to understand how the portion of interacting with a DB can be tested here. I was trying something along these

Re: [PR] Create CloudComposerRunAirflowCLICommandOperator operator [airflow]

2024-04-14 Thread via GitHub
eladkal commented on code in PR #38965: URL: https://github.com/apache/airflow/pull/38965#discussion_r1564583403 ## airflow/providers/google/cloud/triggers/cloud_composer.py: ## @@ -78,3 +78,71 @@ async def run(self): "operation_done": operation.done,

Re: [PR] Allow Task Group Ids to be passed as branches in BranchMixIn [airflow]

2024-04-14 Thread via GitHub
eladkal commented on PR #38883: URL: https://github.com/apache/airflow/pull/38883#issuecomment-2053978424 Tests are failing. Can you fix them? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Ability to add custom facet in OpenLineage events [airflow]

2024-04-14 Thread via GitHub
eladkal commented on code in PR #38982: URL: https://github.com/apache/airflow/pull/38982#discussion_r1564573821 ## airflow/providers/openlineage/provider.yaml: ## @@ -101,6 +101,15 @@ config: example: full.path.to.ExtractorClass;full.path.to.AnotherExtractorClass

  1   2   >