Re: [PR] Enhance Secret Name Selection Based on gitSync Image Version [airflow]

2024-04-30 Thread via GitHub
romsharon98 commented on code in PR #38841: URL: https://github.com/apache/airflow/pull/38841#discussion_r1585904243 ## chart/templates/_helpers.yaml: ## @@ -223,27 +223,28 @@ If release name contains chart name it will be used as a full name. - name:

Re: [PR] Handling exception getting logs when pods finish success [airflow]

2024-04-30 Thread via GitHub
amoghrajesh commented on code in PR #39296: URL: https://github.com/apache/airflow/pull/39296#discussion_r1585874551 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -766,9 +767,18 @@ def _clean(self, event: dict[str, Any]): # Skip await_pod_completion when

Re: [PR] Handling exception getting logs when pods finish success [airflow]

2024-04-30 Thread via GitHub
amoghrajesh commented on code in PR #39296: URL: https://github.com/apache/airflow/pull/39296#discussion_r1583091919 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -766,9 +767,15 @@ def _clean(self, event: dict[str, Any]): # Skip await_pod_completion when

Re: [PR] fix: configuration drift with triggerer pvc [airflow]

2024-04-30 Thread via GitHub
vyas-n commented on PR #39180: URL: https://github.com/apache/airflow/pull/39180#issuecomment-2087954219 @romsharon98, My bad, I did some digging and found that ArgoCD does have an option to fix this now called "Server-side Diff"

Re: [PR] Refresh properties on KubernetesPodOperator when k8s fails due to token expiration [airflow]

2024-04-30 Thread via GitHub
amoghrajesh commented on code in PR #39325: URL: https://github.com/apache/airflow/pull/39325#discussion_r1585856967 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -647,6 +646,20 @@ def execute_sync(self, context: Context): if self.do_xcom_push:

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on PR #39245: URL: https://github.com/apache/airflow/pull/39245#issuecomment-2087900203 Assuming the CI goes green, I'm done. @vincbeck - you may want to have a peek at the changes since you approved, or not. @Taragolis - I ended up doing the generic base

Re: [PR] Add support for role arn for aws creds in storage transfer job operators [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on PR #38911: URL: https://github.com/apache/airflow/pull/38911#issuecomment-2087893762 Congrats! Thanks for your contribution and your patience! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

(airflow) branch main updated (f6fb4ccf2d -> e3e6aa9b3e)

2024-04-30 Thread ferruzzi
This is an automated email from the ASF dual-hosted git repository. ferruzzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from f6fb4ccf2d docs: fix environment variable names when section name has dot in it (#39312) add e3e6aa9b3e Add

Re: [PR] Add support for role arn for aws creds in storage transfer job operators [airflow]

2024-04-30 Thread via GitHub
boring-cyborg[bot] commented on PR #38911: URL: https://github.com/apache/airflow/pull/38911#issuecomment-2087893372 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

Re: [PR] Add support for role arn for aws creds in storage transfer job operators [airflow]

2024-04-30 Thread via GitHub
ferruzzi merged PR #38911: URL: https://github.com/apache/airflow/pull/38911 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] CloudDataTransferServiceS3ToGCSOperator does not support identify federation [airflow]

2024-04-30 Thread via GitHub
ferruzzi closed issue #35271: CloudDataTransferServiceS3ToGCSOperator does not support identify federation URL: https://github.com/apache/airflow/issues/35271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] In provider docs, the environment variables in Configuration tab are shown incorrectly [airflow]

2024-04-30 Thread via GitHub
jedcunningham closed issue #39311: In provider docs, the environment variables in Configuration tab are shown incorrectly URL: https://github.com/apache/airflow/issues/39311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

(airflow) branch main updated: docs: fix environment variable names when section name has dot in it (#39312)

2024-04-30 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f6fb4ccf2d docs: fix environment variable

Re: [PR] docs: fix environment variable names when section name has dot in it [airflow]

2024-04-30 Thread via GitHub
jedcunningham merged PR #39312: URL: https://github.com/apache/airflow/pull/39312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] ECS Executor: Set tasks to RUNNING state once active [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39212: URL: https://github.com/apache/airflow/pull/39212#discussion_r1585813910 ## airflow/providers/celery/executors/celery_executor.py: ## @@ -368,8 +368,10 @@ def update_all_task_states(self) -> None: if state:

Re: [PR] Return stacktrace from the DAG file in `test_should_not_do_database_queries` [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on PR #39331: URL: https://github.com/apache/airflow/pull/39331#issuecomment-2087890786 Sorry I missed this one, thanks for that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585794448 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -0,0 +1,519 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585748958 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -211,3 +262,149 @@ def execute(self, context: Context) -> Any: ) else:

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585748958 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -211,3 +262,149 @@ def execute(self, context: Context) -> Any: ) else:

Re: [I] Catchup still occurs for the current day even though catchup is set to false. [airflow]

2024-04-30 Thread via GitHub
contentfree commented on issue #31937: URL: https://github.com/apache/airflow/issues/31937#issuecomment-2087783003 This behavior, while possibly working "as designed", is certainly a violation of the Principle of Least Surprise. If I have a DAG that I disabled some time ago and it has a

Re: [PR] Integrate the run coverage command to the CI [airflow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #37152: URL: https://github.com/apache/airflow/pull/37152#issuecomment-2087752395 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] Raising Exception if dag is not in active state while triggering dag [airflow]

2024-04-30 Thread via GitHub
github-actions[bot] closed pull request #37836: Raising Exception if dag is not in active state while triggering dag URL: https://github.com/apache/airflow/pull/37836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Fix GCSObjectExistenceSensor xcom push [airflow]

2024-04-30 Thread via GitHub
github-actions[bot] closed pull request #37949: Fix GCSObjectExistenceSensor xcom push URL: https://github.com/apache/airflow/pull/37949 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
syedahsn commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585669891 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -211,3 +262,149 @@ def execute(self, context: Context) -> Any: ) else:

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585647311 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -0,0 +1,519 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review

Re: [I] Create new Callback Types [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on issue #39338: URL: https://github.com/apache/airflow/issues/39338#issuecomment-2087687186 TaskGroup it is a UI / Namespace feature, and do not have own states, like a DAG or TaskInstance have, that mean without [significant

[I] Create new Callback Types [airflow]

2024-04-30 Thread via GitHub
GuilhermePessoni opened a new issue, #39338: URL: https://github.com/apache/airflow/issues/39338 ### Description In versions 2.7.3 or lower, in the process of sending callbacks, they are sent in each taskgroup, and not by dag's. The proposal is to create a new callback flow where

Re: [PR] Set parallelism log messages to warning level for better visiblity [airflow]

2024-04-30 Thread via GitHub
RNHTTR commented on code in PR #39298: URL: https://github.com/apache/airflow/pull/39298#discussion_r1585619655 ## airflow/executors/base_executor.py: ## @@ -221,7 +221,10 @@ def heartbeat(self) -> None: self.log.debug("%s running task instances", num_running_tasks)

Re: [PR] Set parallelism log messages to warning level for better visiblity [airflow]

2024-04-30 Thread via GitHub
RNHTTR commented on PR #39298: URL: https://github.com/apache/airflow/pull/39298#issuecomment-2087625988 > > This is only true for users whose Airflow environment is somewhat optimized. There are a lot of configs to consider (parallelism, worker concurrency, pools) in order to limit task

Re: [I] KubernetesPodOperator callback example from Doc doesn't work [airflow]

2024-04-30 Thread via GitHub
RNHTTR commented on issue #39291: URL: https://github.com/apache/airflow/issues/39291#issuecomment-2087623879 This is reproducible via the Astro CLI with the code in the original post. One odd thing about the docs example is the `exit 1` in the command -- I don't think exiting with

[PR] Run ruff on migrations [airflow]

2024-04-30 Thread via GitHub
jedcunningham opened a new pull request, #39337: URL: https://github.com/apache/airflow/pull/39337 Turns out we weren't running ruff against our migrations. This meant that the `airflow_version` variable had mixed quotes, so the precommit that populates the

Re: [PR] Add support for role arn for aws creds in storage transfer job operators [airflow]

2024-04-30 Thread via GitHub
wlinamchurch commented on PR #38911: URL: https://github.com/apache/airflow/pull/38911#issuecomment-2087581555 Thanks @Taragolis! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Add support for role arn for aws creds in storage transfer job operators [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on PR #38911: URL: https://github.com/apache/airflow/pull/38911#issuecomment-2087554516 @wlinamchurch You still need one more approval, perhaps drop a message in the slack #contributors channel asking for another set of eyes. -- This is an automated message from the

Re: [PR] ECS Executor: Set tasks to RUNNING state once active [airflow]

2024-04-30 Thread via GitHub
vincbeck commented on code in PR #39212: URL: https://github.com/apache/airflow/pull/39212#discussion_r1585601202 ## airflow/providers/celery/executors/celery_executor.py: ## @@ -368,8 +368,10 @@ def update_all_task_states(self) -> None: if state:

Re: [I] Logs from elastic not visible in Grid menu [airflow]

2024-04-30 Thread via GitHub
RNHTTR commented on issue #39323: URL: https://github.com/apache/airflow/issues/39323#issuecomment-2087551208 It's also worth checking your browser's dev tools to see what happens when loading the page. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Move core system tests outside of providers system tests [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on PR #39335: URL: https://github.com/apache/airflow/pull/39335#issuecomment-2087527201 I'm not sure this one fix link to the sources. Because these directory not included into the build this kind of sources in `apache-airflow` documentation. I could have a look

Re: [PR] ECS Executor: Set tasks to RUNNING state once active [airflow]

2024-04-30 Thread via GitHub
o-nikolas commented on code in PR #39212: URL: https://github.com/apache/airflow/pull/39212#discussion_r1585593615 ## airflow/providers/celery/executors/celery_executor.py: ## @@ -368,8 +368,10 @@ def update_all_task_states(self) -> None: if state:

Re: [PR] Add support for role arn for aws creds in storage transfer job operators [airflow]

2024-04-30 Thread via GitHub
wlinamchurch commented on PR #38911: URL: https://github.com/apache/airflow/pull/38911#issuecomment-2087524469 Thanks @ferruzzi! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Move core system tests outside of providers system tests [airflow]

2024-04-30 Thread via GitHub
RNHTTR commented on PR #39335: URL: https://github.com/apache/airflow/pull/39335#issuecomment-2087480935 Will this also fix the associated "[source]" link that's currently broken? ![image](https://github.com/apache/airflow/assets/25823361/1cf007a0-8fa3-4e5e-9853-08e57c57ec40)

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585546006 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -211,3 +262,149 @@ def execute(self, context: Context) -> Any: ) else:

(airflow) branch constraints-main updated: Updating constraints. Github run id:8901290226

2024-04-30 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 64af6634c2 Updating

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585546006 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -211,3 +262,149 @@ def execute(self, context: Context) -> Any: ) else:

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585542063 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -68,7 +70,56 @@ def __init__( super().__init__(**kwargs) self.deferrable = deferrable

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585541983 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -211,3 +262,149 @@ def execute(self, context: Context) -> Any: ) else:

[PR] Remove try_number shenanigans [airflow]

2024-04-30 Thread via GitHub
dstandish opened a new pull request, #39336: URL: https://github.com/apache/airflow/pull/39336 Previously, there was a lot of bad stuff happening around try_number. We incremented it when task started running. And because of that, we had this logic to return "_try_number + 1" when task

Re: [PR] Modify KPO to log container log periodically [airflow]

2024-04-30 Thread via GitHub
jonathonbattista commented on PR #37279: URL: https://github.com/apache/airflow/pull/37279#issuecomment-2087353608 Should this already be compatible with the `GKEStartPodOperator` out of the box or does it also need updated? I am on `apache-airflow-providers-cncf-kubernetes v8.1.1`

Re: [PR] Feature/iceberg provider [airflow]

2024-04-30 Thread via GitHub
Fokko commented on code in PR #39155: URL: https://github.com/apache/airflow/pull/39155#discussion_r1585534358 ## airflow/providers/apache/iceberg/hooks/iceberg.py: ## @@ -23,30 +23,30 @@ from airflow.hooks.base import BaseHook -DEFAULT_TABULAR_URL =

Re: [PR] Fix automatic termination issue in EmrOperator by ensuring `waiter_max_attempts` is set for deferrable triggers [airflow]

2024-04-30 Thread via GitHub
o-nikolas commented on code in PR #38658: URL: https://github.com/apache/airflow/pull/38658#discussion_r1585509531 ## tests/providers/amazon/aws/operators/test_emr_containers.py: ## @@ -144,6 +144,20 @@ def test_operator_defer(self, mock_submit_job, mock_check_query_status):

Re: [PR] fix: configuration drift with triggerer pvc [airflow]

2024-04-30 Thread via GitHub
romsharon98 commented on PR #39180: URL: https://github.com/apache/airflow/pull/39180#issuecomment-2087214450 The changes are correct but maybe the change should come in ArgoCD? because volumeClaimTemplate create a PersistentVolumeClaim implicitly without a need to mention it. -- This

Re: [PR] Fix dag task scheduled and queued duration metrics [airflow]

2024-04-30 Thread via GitHub
o-nikolas commented on PR #37936: URL: https://github.com/apache/airflow/pull/37936#issuecomment-2087212321 > Developers should use Stats.timer() context manager as much as they can, but there are scenarios like the one above when Stats.timing() is needed. Then developers should pass

Re: [PR] Added logging_config,snapstart,ephemeral_storage parameters to aws lambdacreatefunctionoperator [airflow]

2024-04-30 Thread via GitHub
boring-cyborg[bot] commented on PR #39300: URL: https://github.com/apache/airflow/pull/39300#issuecomment-2087171242 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

(airflow) branch main updated (92ffab6004 -> 91490fb054)

2024-04-30 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 92ffab6004 Remove unused index `idx_last_scheduling_decision` on `dag_run` table (#39275) add 91490fb054 Added

Re: [PR] Added logging_config,snapstart,ephemeral_storage parameters to aws lambdacreatefunctionoperator [airflow]

2024-04-30 Thread via GitHub
o-nikolas merged PR #39300: URL: https://github.com/apache/airflow/pull/39300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix Pod Number Exception in the sync mode if reattach_on_restart parameter is False [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on code in PR #39329: URL: https://github.com/apache/airflow/pull/39329#discussion_r1585482895 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -105,8 +105,8 @@ class PodEventType(Enum): NORMAL = "Normal" -class

[PR] Move core system tests outside of providers system tests [airflow]

2024-04-30 Thread via GitHub
Taragolis opened a new pull request, #39335: URL: https://github.com/apache/airflow/pull/39335 This tests for Core, not providers, so better store it separatly. I'm not sure but maybe we should move it into the `airflow/example_dags` --- **^ Add meaningful

Re: [PR] Remove unused index `idx_last_scheduling_decision` on `dag_run` table [airflow]

2024-04-30 Thread via GitHub
jedcunningham merged PR #39275: URL: https://github.com/apache/airflow/pull/39275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Remove unused index `idx_last_scheduling_decision` on `dag_run` table (#39275)

2024-04-30 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 92ffab6004 Remove unused index

Re: [PR] Set parallelism log messages to warning level for better visiblity [airflow]

2024-04-30 Thread via GitHub
hussein-awala commented on PR #39298: URL: https://github.com/apache/airflow/pull/39298#issuecomment-2086872578 > This is only true for users whose Airflow environment is somewhat optimized. There are a lot of configs to consider (parallelism, worker concurrency, pools) in order to limit

Re: [PR] Remove unused index `idx_last_scheduling_decision` on `dag_run` table [airflow]

2024-04-30 Thread via GitHub
jedcunningham commented on code in PR #39275: URL: https://github.com/apache/airflow/pull/39275#discussion_r1585438399 ## airflow/utils/db.py: ## @@ -91,6 +91,8 @@ "2.8.0": "10b52ebd31f7", "2.8.1": "88344c1d9134", "2.9.0": "1949afb29106", +"2.9.1":

Re: [PR] Remove unused index `idx_last_scheduling_decision` on `dag_run` table [airflow]

2024-04-30 Thread via GitHub
jedcunningham commented on code in PR #39275: URL: https://github.com/apache/airflow/pull/39275#discussion_r1585436788 ## airflow/utils/db.py: ## @@ -91,6 +91,8 @@ "2.8.0": "10b52ebd31f7", "2.8.1": "88344c1d9134", "2.9.0": "1949afb29106", +"2.9.1":

Re: [PR] Fix Pod Number Exception in the sync mode if reattach_on_restart parameter is False [airflow]

2024-04-30 Thread via GitHub
e-galan commented on code in PR #39329: URL: https://github.com/apache/airflow/pull/39329#discussion_r1585372878 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -105,8 +105,8 @@ class PodEventType(Enum): NORMAL = "Normal" -class

Re: [PR] Add `job_id` parameter to `BigQueryGetDataOperator` [airflow]

2024-04-30 Thread via GitHub
shahar1 commented on code in PR #39315: URL: https://github.com/apache/airflow/pull/39315#discussion_r1585358900 ## airflow/providers/google/cloud/hooks/bigquery.py: ## @@ -2390,6 +2397,50 @@ def var_print(var_name): return project_id, dataset_id, table_id +

[PR] Reroute AutoML operator links to Google Translation links [airflow]

2024-04-30 Thread via GitHub
e-galan opened a new pull request, #39334: URL: https://github.com/apache/airflow/pull/39334 Updates AutoML operators with new links. Due to its future deprecation AutoML's URLs have been divided between Google Translation and Vertex AI services. This PR updates the

Re: [PR] Reroute AutoML operator links to Google Translation links [airflow]

2024-04-30 Thread via GitHub
e-galan closed pull request #39334: Reroute AutoML operator links to Google Translation links URL: https://github.com/apache/airflow/pull/39334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] `KubernetesPodOperator` return the manifest in `dry_run()` [airflow]

2024-04-30 Thread via GitHub
romsharon98 commented on PR #37186: URL: https://github.com/apache/airflow/pull/37186#issuecomment-2086625730 because dry_run is a function inherited from BaseOperator we cannot change it return type. any suggestions? -- This is an automated message from the Apache Git Service. To

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
syedahsn commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585346316 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -211,3 +262,149 @@ def execute(self, context: Context) -> Any: ) else:

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585350222 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -68,7 +70,56 @@ def __init__( super().__init__(**kwargs) self.deferrable = deferrable

Re: [PR] Add more proper typing to `StreamLogWriter` [airflow]

2024-04-30 Thread via GitHub
Grub4K commented on code in PR #37347: URL: https://github.com/apache/airflow/pull/37347#discussion_r1585337313 ## airflow/utils/log/logging_mixin.py: ## @@ -193,12 +202,15 @@ def write(self, message): :param message: message to log """ -if not

Re: [PR] Add more proper typing to `StreamLogWriter` [airflow]

2024-04-30 Thread via GitHub
Grub4K commented on code in PR #37347: URL: https://github.com/apache/airflow/pull/37347#discussion_r1585335537 ## airflow/utils/log/logging_mixin.py: ## @@ -147,18 +147,27 @@ def supports_external_link(self) -> bool: # We have to ignore typing errors here because Python I/O

Re: [PR] Add more proper typing to `StreamLogWriter` [airflow]

2024-04-30 Thread via GitHub
Grub4K commented on code in PR #37347: URL: https://github.com/apache/airflow/pull/37347#discussion_r1585279677 ## airflow/utils/log/logging_mixin.py: ## @@ -147,18 +147,27 @@ def supports_external_link(self) -> bool: # We have to ignore typing errors here because Python I/O

Re: [PR] Resolve deprecations in Python sensors/operators/decorators [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on code in PR #39241: URL: https://github.com/apache/airflow/pull/39241#discussion_r1585320865 ## tests/deprecations_ignore.yml: ## @@ -349,23 +313,8 @@ -

(airflow) branch main updated (2872d37017 -> 3204d6492a)

2024-04-30 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 2872d37017 Enable explicitly `pycodestyle` rules (#39322) add 3204d6492a Return stacktrace from the DAG file

Re: [PR] Return stacktrace from the DAG file in `test_should_not_do_database_queries` [airflow]

2024-04-30 Thread via GitHub
Taragolis merged PR #39331: URL: https://github.com/apache/airflow/pull/39331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Do not provide deprecated `execution_date` in `@apply_lineage` (#39327)

2024-04-30 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 10e34d348e Do not provide deprecated

Re: [PR] Do not provide deprecated `execution_date` in `@apply_lineage` [airflow]

2024-04-30 Thread via GitHub
Taragolis merged PR #39327: URL: https://github.com/apache/airflow/pull/39327 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Feature/iceberg provider [airflow]

2024-04-30 Thread via GitHub
romsharon98 commented on code in PR #39155: URL: https://github.com/apache/airflow/pull/39155#discussion_r1585268609 ## docs/apache-airflow-providers-tabular/redirects.txt: ## @@ -0,0 +1,3 @@ +connections.rst ../apache-airflow-providers-apache-iceberg/connections.rst

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585263688 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -0,0 +1,519 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Fix Pod Number Exception in the sync mode if reattach_on_restart parameter is False [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on code in PR #39329: URL: https://github.com/apache/airflow/pull/39329#discussion_r1585257901 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -105,8 +105,8 @@ class PodEventType(Enum): NORMAL = "Normal" -class

Re: [I] Logs from elastic not visible in Grid menu [airflow]

2024-04-30 Thread via GitHub
ivanp221 commented on issue #39323: URL: https://github.com/apache/airflow/issues/39323#issuecomment-2086146937 will test and report thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585236367 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -0,0 +1,519 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
ferruzzi commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585192344 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -0,0 +1,519 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1585162253 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -68,7 +70,56 @@ def __init__( super().__init__(**kwargs) self.deferrable = deferrable

Re: [I] FTPHook doesn't not allow to change port [airflow]

2024-04-30 Thread via GitHub
Lee-W commented on issue #39308: URL: https://github.com/apache/airflow/issues/39308#issuecomment-2085866954 Sure! Just assigned to you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[I] DAG disappearing from Airflow in case of standalone DAG processor [airflow]

2024-04-30 Thread via GitHub
kosteev opened a new issue, #39332: URL: https://github.com/apache/airflow/issues/39332 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.7.3 ### What happened? If DAG processor

Re: [PR] ECS Executor: Set tasks to RUNNING state once active [airflow]

2024-04-30 Thread via GitHub
o-nikolas commented on code in PR #39212: URL: https://github.com/apache/airflow/pull/39212#discussion_r1585076323 ## airflow/providers/celery/executors/celery_executor.py: ## @@ -368,8 +368,10 @@ def update_all_task_states(self) -> None: if state:

Re: [PR] fix: sqa deprecations for airflow tests [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on PR #39299: URL: https://github.com/apache/airflow/pull/39299#issuecomment-2085701895 > Fixed but still raise a warning

Re: [PR] Enable explicitly `pycodestyle` rules [airflow]

2024-04-30 Thread via GitHub
uranusjr merged PR #39322: URL: https://github.com/apache/airflow/pull/39322 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated (8a33c337bb -> 2872d37017)

2024-04-30 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8a33c337bb XFAIL `TestSigtermOnRunner.test_process_sigterm_works_with_retries` if no expected messages in logs

(airflow) branch main updated (d4bdffc45c -> 8a33c337bb)

2024-04-30 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d4bdffc45c migrate to dbt v3 api for project endpoints (#39214) add 8a33c337bb XFAIL

Re: [PR] XFAIL `TestSigtermOnRunner.test_process_sigterm_works_with_retries` if no expected messages in logs [airflow]

2024-04-30 Thread via GitHub
Taragolis merged PR #39053: URL: https://github.com/apache/airflow/pull/39053 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add `job_id` parameter to `BigQueryGetDataOperator` [airflow]

2024-04-30 Thread via GitHub
Lee-W commented on code in PR #39315: URL: https://github.com/apache/airflow/pull/39315#discussion_r1584995218 ## airflow/providers/google/cloud/hooks/bigquery.py: ## @@ -2390,6 +2397,50 @@ def var_print(var_name): return project_id, dataset_id, table_id +

Re: [PR] Return stacktrace from the DAG file in `test_should_not_do_database_queries` [airflow]

2024-04-30 Thread via GitHub
Taragolis closed pull request #39331: Return stacktrace from the DAG file in `test_should_not_do_database_queries` URL: https://github.com/apache/airflow/pull/39331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] Return stacktrace from the DAG file in `test_should_not_do_database_queries` [airflow]

2024-04-30 Thread via GitHub
Taragolis opened a new pull request, #39331: URL: https://github.com/apache/airflow/pull/39331 Right now stacktrace is hardcoded to the 5, this one give to ability redefine default values, as well as filter it starting from the specific value This one should help to

Re: [PR] Fix TaskHandlerWithCustomFormatter now adds prefix only once [airflow]

2024-04-30 Thread via GitHub
Lee-W commented on PR #38502: URL: https://github.com/apache/airflow/pull/38502#issuecomment-2085548139 Just approved the CI. If everything works fine, I'm planning on merging this early tomorrow. Please let me know if anyone wants to take a deep look -- This is an automated message from

Re: [PR] Amazon Bedrock - Knowledge Bases and Data Sources [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on code in PR #39245: URL: https://github.com/apache/airflow/pull/39245#discussion_r1584955631 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -0,0 +1,519 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Fix TaskHandlerWithCustomFormatter now adds prefix only once [airflow]

2024-04-30 Thread via GitHub
TiDeane commented on PR #38502: URL: https://github.com/apache/airflow/pull/38502#issuecomment-2085408806 Hello, I just rebased my fork, could I get this bug fix merged? @Lee-W -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] OpenAI Files & Vector Store Hooks [airflow]

2024-04-30 Thread via GitHub
nathadfield commented on PR #39248: URL: https://github.com/apache/airflow/pull/39248#issuecomment-2085352469 Any chance of a merge before the next round of providers are cut? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] openlineage: use `ProcessPoolExecutor` over `ThreadPoolExecutor`. [airflow]

2024-04-30 Thread via GitHub
JDarDagran commented on code in PR #39235: URL: https://github.com/apache/airflow/pull/39235#discussion_r1584819869 ## airflow/providers/openlineage/plugins/listener.py: ## @@ -250,7 +250,7 @@ def on_failure(): @property def executor(self): if not

Re: [PR] openlineage: use `ProcessPoolExecutor` over `ThreadPoolExecutor`. [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on code in PR #39235: URL: https://github.com/apache/airflow/pull/39235#discussion_r1584818061 ## airflow/providers/openlineage/plugins/listener.py: ## @@ -250,7 +250,7 @@ def on_failure(): @property def executor(self): if not

Re: [I] Missing termination argument in DockerOperator [airflow]

2024-04-30 Thread via GitHub
Taragolis commented on issue #39319: URL: https://github.com/apache/airflow/issues/39319#issuecomment-2085313755 > Indeed, this fix the layout, but we lose this nice functionality... If you find a correct way to handle [PEP-657](https://peps.python.org/pep-0657/) in runtime in

  1   2   >