Re: [I] Connection to Postgres failed due to special characters in a password [airflow]

2024-04-08 Thread via GitHub
rawwar commented on issue #38187: URL: https://github.com/apache/airflow/issues/38187#issuecomment-2043038768 @Taragolis , based on your comment [here](https://github.com/apache/airflow/issues/38195#issuecomment-2042644747) > As far as i remember, we recently discuss this case in slac

[I] Change the UI color for ExternalSensor tasks [airflow]

2024-04-08 Thread via GitHub
Timelessprod opened a new issue, #38845: URL: https://github.com/apache/airflow/issues/38845 ### Description With the recent update of the graph display in DAGs, there's a visibility issue regarding external task sensors. Indeed those tasks have a `#19647e` tile color while the statu

Re: [I] Change the UI color for ExternalSensor tasks [airflow]

2024-04-08 Thread via GitHub
boring-cyborg[bot] commented on issue #38845: URL: https://github.com/apache/airflow/issues/38845#issuecomment-2043032243 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] White INFO log is hard to read [airflow]

2024-04-08 Thread via GitHub
romanzdk opened a new issue, #38844: URL: https://github.com/apache/airflow/issues/38844 ### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? 2.9.0 ### What happened? White INFO log is hard to read ![ima

Re: [PR] Update dagrun.py to fix #38816 [airflow]

2024-04-08 Thread via GitHub
boring-cyborg[bot] commented on PR #38843: URL: https://github.com/apache/airflow/pull/38843#issuecomment-2043013963 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

[PR] Update dagrun.py to fix #38816 [airflow]

2024-04-08 Thread via GitHub
MyLong opened a new pull request, #38843: URL: https://github.com/apache/airflow/pull/38843 'DagRunNote' object has no attribute dag_id\dagrun_id\run_id\map_index so replace those attributes to existing attributes dag_run_id\content -- This is an automated message from the Apache Git Se

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1556009587 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def create_chat

Re: [PR] fix: try002 for provider google [airflow]

2024-04-08 Thread via GitHub
dondaum commented on code in PR #38803: URL: https://github.com/apache/airflow/pull/38803#discussion_r1556001809 ## airflow/providers/google/cloud/hooks/dataflow.py: ## @@ -252,7 +252,7 @@ def _get_current_jobs(self) -> list[dict]: self._job_id = jobs[0]["id"]

Re: [I] Applying queue with SparkSubmitOperator is ignored from 2.6.0 onwards [airflow]

2024-04-08 Thread via GitHub
pateash commented on issue #38461: URL: https://github.com/apache/airflow/issues/38461#issuecomment-2042992467 Checking this, can any one please assign this to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Expose AWS IAM missing param in Hashicorp secret [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on code in PR #38536: URL: https://github.com/apache/airflow/pull/38536#discussion_r1555990209 ## airflow/providers/hashicorp/provider.yaml: ## @@ -55,6 +55,7 @@ versions: dependencies: - apache-airflow>=2.6.0 - hvac>=1.1.0 + - apache-airflow-providers-am

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-08 Thread via GitHub
zstrathe commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1555982325 ## airflow/providers/apache/beam/operators/beam.py: ## @@ -364,11 +364,13 @@ def execute(self, context: Context): def execute_sync(self, context: Context):

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-08 Thread via GitHub
zstrathe commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1555982325 ## airflow/providers/apache/beam/operators/beam.py: ## @@ -364,11 +364,13 @@ def execute(self, context: Context): def execute_sync(self, context: Context):

Re: [PR] Enhance Secret Name Selection Based on gitSync Image Version [airflow]

2024-04-08 Thread via GitHub
boring-cyborg[bot] commented on PR #38841: URL: https://github.com/apache/airflow/pull/38841#issuecomment-2042967880 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

[PR] Enhance Secret Name Selection Based on gitSync Image Version [airflow]

2024-04-08 Thread via GitHub
Bagautdino opened a new pull request, #38841: URL: https://github.com/apache/airflow/pull/38841 ### Enhance Secret Name Selection Based on `gitSync` Image Version This pull request introduces a conditional logic enhancement within the Helm chart templates to dynamically select the app

Re: [PR] Chart: Default Airflow version to 2.9.0 [airflow]

2024-04-08 Thread via GitHub
jedcunningham commented on code in PR #38839: URL: https://github.com/apache/airflow/pull/38839#discussion_r1555951689 ## chart/newsfragments/38478.significant.rst: ## Review Comment: This should be renamed to 38839.significant.rst. -- This is an automated message from

Re: [PR] Update ACL during job reset [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on code in PR #38741: URL: https://github.com/apache/airflow/pull/38741#discussion_r1555946331 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -655,6 +656,16 @@ def get_repo_by_path(self, path: str) -> str | None: return None +def upda

Re: [PR] Update ACL during job reset [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on PR #38741: URL: https://github.com/apache/airflow/pull/38741#issuecomment-2042912496 Thanks @potiuk for help out! @SubhamSinghal There's one static check failure. We might need your help to fix it -- This is an automated message from the Apache Git Service. To re

Re: [I] `TriggerDagRunOperator` not working as expected in defer with `wait_for_completion` [airflow]

2024-04-08 Thread via GitHub
raphaelauv commented on issue #38353: URL: https://github.com/apache/airflow/issues/38353#issuecomment-2042883562 bug is still present in 2.9.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] Update version added field in config.yml [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy opened a new pull request, #38840: URL: https://github.com/apache/airflow/pull/38840 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] Update ACL during job reset [airflow]

2024-04-08 Thread via GitHub
potiuk commented on PR #38741: URL: https://github.com/apache/airflow/pull/38741#issuecomment-2042844353 Rebased it -> your PR was 62 commits behind -> next time you can do it yourself too (see our contributing docs and search for rebase) https://github.com/apache/airflow/blob/main/contrib

[PR] Chart: Default Airflow version to 2.9.0 [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy opened a new pull request, #38839: URL: https://github.com/apache/airflow/pull/38839 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] Add four unit tests for aws/utils [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38820: URL: https://github.com/apache/airflow/pull/38820#discussion_r1555891937 ## tests/providers/amazon/aws/utils/test_sagemaker.py: ## Review Comment: I had the same impression that most of the tests produced / generated here are useless.

Re: [PR] Add four unit tests for aws/utils [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38820: URL: https://github.com/apache/airflow/pull/38820#discussion_r1555893343 ## tests/providers/amazon/aws/utils/test_sagemaker.py: ## Review Comment: Potential Value added vs. time needed to review/classify those tests do not seem to be

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
nathadfield commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555881195 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def creat

Re: [PR] Expose AWS IAM missing param in Hashicorp secret [airflow]

2024-04-08 Thread via GitHub
ReadytoRocc commented on code in PR #38536: URL: https://github.com/apache/airflow/pull/38536#discussion_r1555877716 ## airflow/providers/hashicorp/_internal_client/vault_client.py: ## @@ -318,15 +322,23 @@ def _auth_azure(self, _client: hvac.Client) -> None: )

[PR] Airflow 2.9.0 has been released [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy opened a new pull request, #38837: URL: https://github.com/apache/airflow/pull/38837 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[I] DB migrate throws na error on encrypt_trigger_kwargs [airflow]

2024-04-08 Thread via GitHub
romanzdk opened a new issue, #38836: URL: https://github.com/apache/airflow/issues/38836 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.0 ### What happened? `airflow db migr

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
nathadfield commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555871738 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def creat

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
nathadfield commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555871738 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def creat

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
nathadfield commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555871245 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def creat

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
nathadfield commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555854814 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def creat

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
nathadfield commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555850833 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def creat

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
utkarsharma2 commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555852061 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def crea

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
nathadfield commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555850833 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def creat

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
utkarsharma2 commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1555846716 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def crea

Re: [PR] Resolve core unit test internal warning [airflow]

2024-04-08 Thread via GitHub
Owen-CH-Leung commented on code in PR #38835: URL: https://github.com/apache/airflow/pull/38835#discussion_r1555842384 ## tests/deprecations_ignore.yml: ## @@ -70,9 +70,6 @@ # Core -- tests/core/test_core.py::TestCore::test_dag_params_and_task_params -- tests/core/test_core

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-08 Thread via GitHub
e-galan commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1555830278 ## airflow/providers/apache/beam/operators/beam.py: ## @@ -364,11 +364,13 @@ def execute(self, context: Context): def execute_sync(self, context: Context):

(airflow-site) branch gh-pages updated (d51b51096d -> 12ec5916dd)

2024-04-08 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard d51b51096d Rewritten history to remove past gh-pages deployments new 12ec5916dd Rewritten history to

Re: [PR] Resolve core unit test internal warning [airflow]

2024-04-08 Thread via GitHub
Taragolis commented on code in PR #38835: URL: https://github.com/apache/airflow/pull/38835#discussion_r1555814562 ## tests/deprecations_ignore.yml: ## @@ -70,9 +70,6 @@ # Core -- tests/core/test_core.py::TestCore::test_dag_params_and_task_params -- tests/core/test_core.py:

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-08 Thread via GitHub
e-galan commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1555810282 ## airflow/providers/apache/beam/operators/beam.py: ## @@ -364,11 +364,13 @@ def execute(self, context: Context): def execute_sync(self, context: Context):

(airflow-site) branch gh-pages updated (ecb04021c3 -> d51b51096d)

2024-04-08 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard ecb04021c3 Rewritten history to remove past gh-pages deployments new d51b51096d Rewritten history to

[PR] Resolve core unit test internal warning [airflow]

2024-04-08 Thread via GitHub
Owen-CH-Leung opened a new pull request, #38835: URL: https://github.com/apache/airflow/pull/38835 Part of the fixes for https://github.com/apache/airflow/issues/38642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[I] cli | airflow dags unpause - fail cause not founding dags [airflow]

2024-04-08 Thread via GitHub
raphaelauv opened a new issue, #38834: URL: https://github.com/apache/airflow/issues/38834 ### Apache Airflow version 2.9.0b2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? in 2.8.4 running `airflow dags unpaus

Re: [PR] create aws provider operators for glue session [airflow]

2024-04-08 Thread via GitHub
Taragolis commented on code in PR #38830: URL: https://github.com/apache/airflow/pull/38830#discussion_r1555704350 ## airflow/providers/amazon/aws/hooks/glue_session.py: ## @@ -0,0 +1,265 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [I] Build correct SQLAlchemy URI in hooks based on `DbApiHook` [airflow]

2024-04-08 Thread via GitHub
Taragolis commented on issue #38195: URL: https://github.com/apache/airflow/issues/38195#issuecomment-2042644747 As far as i remember, we recently discuss this case in slack. I guess better introduce `sa_uri` property (or some similar naming) which returns [`sqlalchemy.engine.URL`](https://

[PR] fix: Build correct SqlAlchemy URI [airflow]

2024-04-08 Thread via GitHub
rawwar opened a new pull request, #38831: URL: https://github.com/apache/airflow/pull/38831

(airflow-site) branch main updated: Blog post for Airflow 2.9 (#997)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 71683317fc Blog post for Airflow 2.9

Re: [PR] Blog post for Airflow 2.9 [airflow-site]

2024-04-08 Thread via GitHub
ephraimbuddy merged PR #997: URL: https://github.com/apache/airflow-site/pull/997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow-site) branch 2.9.0-blog-post deleted (was f50181478a)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch 2.9.0-blog-post in repository https://gitbox.apache.org/repos/asf/airflow-site.git was f50181478a Blog post for Airflow 2.9 The revisions that were on this branch are still contain

Re: [PR] Deferrable mode for Custom Training Job operators [airflow]

2024-04-08 Thread via GitHub
e-galan commented on code in PR #38584: URL: https://github.com/apache/airflow/pull/38584#discussion_r1555716328 ## airflow/providers/google/cloud/operators/vertex_ai/custom_job.py: ## @@ -539,6 +564,94 @@ def on_kill(self) -> None: if self.hook: self.hook.

Re: [I] Automate links to configuration of providers [airflow]

2024-04-08 Thread via GitHub
poorvirohidekar commented on issue #34306: URL: https://github.com/apache/airflow/issues/34306#issuecomment-2042585648 @potiuk - Understood. Thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

(airflow-site) branch announce-290 deleted (was 19edb3bd2c)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch announce-290 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 19edb3bd2c Announce Apache Airflow 2.9.0 release The revisions that were on this branch are stil

Re: [I] Status of testing of Apache Airflow Helm Chart 1.13.1rc1 [airflow]

2024-04-08 Thread via GitHub
mewa commented on issue #38382: URL: https://github.com/apache/airflow/issues/38382#issuecomment-2042577764 Looks good :+1: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

(airflow-site) branch main updated: Announce Apache Airflow 2.9.0 release (#999)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 13e2e43e44 Announce Apache Airflow 2.

Re: [PR] Announce Apache Airflow 2.9.0 release [airflow-site]

2024-04-08 Thread via GitHub
ephraimbuddy merged PR #999: URL: https://github.com/apache/airflow-site/pull/999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated (e77c1358eb -> 87fc581262)

2024-04-08 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from e77c1358eb Add FAB provider to Airflow extras in the image (#38824) add 87fc581262 Remove false negative SparkI

Re: [PR] Remove SparkInK8s internal deprecations [airflow]

2024-04-08 Thread via GitHub
Taragolis merged PR #38777: URL: https://github.com/apache/airflow/pull/38777 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[PR] Announce Apache Airflow 2.9.0 release [airflow-site]

2024-04-08 Thread via GitHub
ephraimbuddy opened a new pull request, #999: URL: https://github.com/apache/airflow-site/pull/999 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

(airflow-site) 01/01: Announce Apache Airflow 2.9.0 release

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch announce-290 in repository https://gitbox.apache.org/repos/asf/airflow-site.git commit 19edb3bd2cce4a9632cc2f4a0cf55d746ecff1c0 Author: Ephraim Anierobi AuthorDate: Mon Apr 8 12:48:32 20

(airflow-site) branch announce-290 created (now 19edb3bd2c)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch announce-290 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 19edb3bd2c Announce Apache Airflow 2.9.0 release This branch includes the following new commits:

[PR] create glue session provider operators [airflow]

2024-04-08 Thread via GitHub
mathiaHT opened a new pull request, #38830: URL: https://github.com/apache/airflow/pull/38830 --- GlueCreateSessionOperators and GlueDeleteSessionOperators to handle glue session in a dag When using dbt-glue, we need to handle when and how the sessio

Re: [PR] Update ACL during job reset [airflow]

2024-04-08 Thread via GitHub
subham611 commented on PR #38741: URL: https://github.com/apache/airflow/pull/38741#issuecomment-2042504136 @Lee-W How to fix `Wait for CI images` error. Need help here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Paginate Airflow task logs [airflow]

2024-04-08 Thread via GitHub
ashb commented on code in PR #38807: URL: https://github.com/apache/airflow/pull/38807#discussion_r1555658225 ## airflow/providers/amazon/aws/log/s3_task_handler.py: ## @@ -178,7 +180,13 @@ def s3_read(self, remote_log_location: str, return_error: bool = False) -> str:

Re: [PR] Always use the executemany method when inserting rows in DbApiHook as it's way much faster [airflow]

2024-04-08 Thread via GitHub
dabla commented on code in PR #38715: URL: https://github.com/apache/airflow/pull/38715#discussion_r1555654733 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -531,8 +539,6 @@ def insert_rows( target_fields=None, commit_every=1000, replace=False, -

Re: [PR] Add lineage_job_namespace and lineage_job_name OpenLineage macros [airflow]

2024-04-08 Thread via GitHub
dolfinus commented on code in PR #38829: URL: https://github.com/apache/airflow/pull/38829#discussion_r1555644919 ## airflow/providers/openlineage/plugins/macros.py: ## @@ -26,22 +26,44 @@ from airflow.models import TaskInstance +def lineage_job_namespace(): +""" +

Re: [PR] Add lineage_job_namespace and lineage_job_name OpenLineage macros [airflow]

2024-04-08 Thread via GitHub
kacpermuda commented on code in PR #38829: URL: https://github.com/apache/airflow/pull/38829#discussion_r1555641741 ## airflow/providers/openlineage/plugins/macros.py: ## @@ -26,22 +26,44 @@ from airflow.models import TaskInstance +def lineage_job_namespace(): +"""

Re: [PR] Add documentation for Apache Airflow 2.9.0 [airflow-site]

2024-04-08 Thread via GitHub
ephraimbuddy merged PR #998: URL: https://github.com/apache/airflow-site/pull/998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [PR] [OpenLineage] Add macros for namespace and job_name [airflow]

2024-04-08 Thread via GitHub
kacpermuda closed pull request #38828: [OpenLineage] Add macros for namespace and job_name URL: https://github.com/apache/airflow/pull/38828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] [OpenLineage] Add macros for namespace and job_name [airflow]

2024-04-08 Thread via GitHub
dolfinus commented on PR #38828: URL: https://github.com/apache/airflow/pull/38828#issuecomment-2042461264 I've just created https://github.com/apache/airflow/pull/38829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] Add lineage_job_namespace and lineage_job_name OpenLineage macros [airflow]

2024-04-08 Thread via GitHub
dolfinus opened a new pull request, #38829: URL: https://github.com/apache/airflow/pull/38829 Add new `lineage_job_namespace()` and `lineage_job_name(task_instance)` macros to OpenLineage provider. Same as https://github.com/OpenLineage/OpenLineage/pull/2582. Also upd

[PR] [OpenLineage] Add macros for namespace and job_name [airflow]

2024-04-08 Thread via GitHub
kacpermuda opened a new pull request, #38828: URL: https://github.com/apache/airflow/pull/38828 New macros are being added by @dolfinus to the OL package in this [PR](https://github.com/OpenLineage/OpenLineage/pull/2582). This PR implements the same feature to the openlineage pr

[PR] Add documentation for Apache Airflow 2.9.0 [airflow-site]

2024-04-08 Thread via GitHub
ephraimbuddy opened a new pull request, #998: URL: https://github.com/apache/airflow-site/pull/998 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

(airflow-site) branch 2.9.0-docs created (now 74b4a47604)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch 2.9.0-docs in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 74b4a47604 Add documentation for Apache Airflow 2.9.0 This branch includes the following new commi

Re: [PR] Improve editable airflow installation by adding preinstalled deps [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy commented on code in PR #38764: URL: https://github.com/apache/airflow/pull/38764#discussion_r170656 ## hatch_build.py: ## @@ -577,29 +594,48 @@ def get_provider_requirement(provider_spec: str) -> str: return f"apache-airflow-providers-{provider_spec.re

Re: [I] DAGs are able to see historical dataset events when created new / disabled - enabled / deleted - recreated. [airflow]

2024-04-08 Thread via GitHub
boring-cyborg[bot] commented on issue #38826: URL: https://github.com/apache/airflow/issues/38826#issuecomment-2042361504 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] DAGs are able to see historical dataset events when created new / disabled - enabled / deleted - recreated. [airflow]

2024-04-08 Thread via GitHub
tosheer opened a new issue, #38826: URL: https://github.com/apache/airflow/issues/38826 ### Apache Airflow version 2.9.0b2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? * If a new dataset triggreed DAG is created for

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-08 Thread via GitHub
e-galan commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r1555475918 ## airflow/providers/google/cloud/triggers/dataflow.py: ## @@ -142,3 +152,496 @@ def _get_async_hook(self) -> AsyncDataflowHook: impersonation_chain=self

Re: [I] Old, unused tags cannot be removed easily [airflow]

2024-04-08 Thread via GitHub
ITJamie commented on issue #31184: URL: https://github.com/apache/airflow/issues/31184#issuecomment-2042359143 an idea for this would be to have an additional column on the dag_tag table. a last_seen date field. when the scheduler / importer runs it could update the dag_tag table w

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-08 Thread via GitHub
e-galan commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r167629 ## airflow/providers/google/cloud/sensors/dataflow.py: ## @@ -115,10 +124,52 @@ def poke(self, context: Context) -> bool: return False +def execute(s

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
nathadfield commented on PR #38736: URL: https://github.com/apache/airflow/pull/38736#issuecomment-2042348699 I'd love a review on this when someone gets an opportunity. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Always use the executemany method when inserting rows in DbApiHook as it's way much faster [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38715: URL: https://github.com/apache/airflow/pull/38715#discussion_r156847 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -531,8 +539,6 @@ def insert_rows( target_fields=None, commit_every=1000, replace=False,

Re: [PR] Starts execution directly from triggerer without going to worker [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on PR #38674: URL: https://github.com/apache/airflow/pull/38674#issuecomment-2042328464 > I wonder if we could somehow infer `starts_execution_from_triggerer` from the combination of other arguments, instead of setting the flag explicitly. For this specific case, I wou

(airflow) branch v2-9-test updated: Update Airflow version in Dockerfile

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-9-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-9-test by this push: new f8365f2293 Update Airflow versio

(airflow) branch v2-9-test updated: Add FAB provider to Airflow extras in the image (#38824)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-9-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-9-test by this push: new 225fe51310 Add FAB provider to A

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-08 Thread via GitHub
Taragolis commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r132965 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -91,3 +98,155 @@ def execute(self, context: Context) -> dict[str, str | int]: self.log.info("Be

Re: [PR] Add four unit tests for aws/utils [airflow]

2024-04-08 Thread via GitHub
hussein-awala commented on code in PR #38820: URL: https://github.com/apache/airflow/pull/38820#discussion_r129961 ## tests/providers/amazon/aws/utils/test_rds.py: ## Review Comment: This test is useless ## tests/providers/amazon/aws/utils/test_sagemaker.py:

Re: [PR] Add FAB provider to Airflow extras in the image [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy merged PR #38824: URL: https://github.com/apache/airflow/pull/38824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

(airflow) branch main updated: Add FAB provider to Airflow extras in the image (#38824)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e77c1358eb Add FAB provider to Airflow ext

Re: [PR] Fix triggerer race condition in HA setting [airflow]

2024-04-08 Thread via GitHub
uranusjr commented on code in PR #38666: URL: https://github.com/apache/airflow/pull/38666#discussion_r124367 ## airflow/jobs/triggerer_job_runner.py: ## @@ -672,6 +672,17 @@ def update_triggers(self, requested_trigger_ids: set[int]): self.failed_triggers.ap

Re: [PR] Fix triggerer race condition in HA setting [airflow]

2024-04-08 Thread via GitHub
uranusjr commented on code in PR #38666: URL: https://github.com/apache/airflow/pull/38666#discussion_r123455 ## tests/jobs/test_triggerer_job.py: ## @@ -309,6 +309,140 @@ def test_update_trigger_with_triggerer_argument_change( assert "got an unexpected keyword argu

(airflow) branch main updated (c81449372e -> f50cc6180c)

2024-04-08 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from c81449372e Ignore subdag operator warnings during the (re)initialise DB in tests (#38823) add f50cc6180c Workarou

Re: [PR] Workaround occasional failures of docker system prune in public runners [airflow]

2024-04-08 Thread via GitHub
potiuk merged PR #38825: URL: https://github.com/apache/airflow/pull/38825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (b687f59008 -> c81449372e)

2024-04-08 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from b687f59008 fix: try002 for provider microsoft azure (#38805) add c81449372e Ignore subdag operator warnings dur

Re: [PR] Ignore subdag operator warnings during the (re)initialise DB in tests [airflow]

2024-04-08 Thread via GitHub
Taragolis merged PR #38823: URL: https://github.com/apache/airflow/pull/38823 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Make _get_ti compatible with RPC [airflow]

2024-04-08 Thread via GitHub
uranusjr commented on code in PR #38570: URL: https://github.com/apache/airflow/pull/38570#discussion_r119108 ## airflow/serialization/pydantic/dag_run.py: ## @@ -78,7 +78,7 @@ def get_task_instances( def get_task_instance( self, task_id: str, -

[PR] Workaround occasional failures of docker system prune in public runners [airflow]

2024-04-08 Thread via GitHub
potiuk opened a new pull request, #38825: URL: https://github.com/apache/airflow/pull/38825 Occasionally `docker syetem prune` fails on Public docker runners with file resize error: truncate /var/lib/docker/buildkit/containerdmeta.db: read-only file system This happens after do

[PR] Add FAB provider to Airflow extras in the image [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy opened a new pull request, #38824: URL: https://github.com/apache/airflow/pull/38824 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

(airflow) branch constraints-2-9 updated (4642ff83ce -> b86d9678fb)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch constraints-2-9 in repository https://gitbox.apache.org/repos/asf/airflow.git omit 4642ff83ce Upgrade constraints to rc1 This update removed existing revisions from the reference, l

(airflow) annotated tag 2.9.0 updated (50f22ffe91 -> 5473f1ad31)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to annotated tag 2.9.0 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag 2.9.0 was modified! *** from 50f22ffe91 (commit) to 5473f1ad31 (tag) tagging 50f22

Re: [PR] Always use the executemany method when inserting rows in DbApiHook as it's way much faster [airflow]

2024-04-08 Thread via GitHub
uranusjr commented on code in PR #38715: URL: https://github.com/apache/airflow/pull/38715#discussion_r1555493833 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -531,8 +539,6 @@ def insert_rows( target_fields=None, commit_every=1000, replace=False

<    1   2   3   >