FrancisLfg opened a new issue, #55678: URL: https://github.com/apache/airflow/issues/55678
### Apache Airflow Provider(s) amazon ### Versions of Apache Airflow Providers apache-airflow-providers-amazon==9.11.0 apache-airflow-providers-common-sql==1.27.0 ### Apache Airflow version 2.10.5 ### Operating System Debian GNU/Linux 12 (bookworm) ### Deployment Other 3rd-party Helm chart ### Deployment details _No response_ ### What happened The `SQLValueCheckOperator` doesn't work as expected with the `AthenaSQLHook`, and throws an error: `TypeError: AwsGenericHook.__init__() got an unexpected keyword argument 's3_staging_dir'` The `s3_staging_dir`/`work_group` is actually a mandatory parameter for [Athena Connection](https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/athena.html). But since the [addition](https://github.com/kandharvishnu/airflow/blob/0b8b1544fa4aa5d2725330604232e2e64237b647/providers/common/sql/src/airflow/providers/common/sql/operators/sql.py#L169) of `extra_dejson` in `hook_params` change from this [PR](https://github.com/apache/airflow/pull/49282), the hook constructor is failing. It is not able to manage mandatory `hook_params`. ### What you think should happen instead The `AthenaSQLHook` should manage properly the `hook_params` since they are passed automatically through this [constructor](https://github.com/apache/airflow/blob/bf4d89012ffcc32146930a783eee34f7ea79e6f9/airflow-core/src/airflow/models/connection.py#L414). ### How to reproduce ```python def test_athena_hook_fail(): """Test to reproduce the Athena hook issue with s3_staging_dir parameter.""" from airflow.models.connection import Connection # Mock Athena connection with s3_staging_dir in extra athena_conn = Connection( conn_id="athena_conn", conn_type="athena", description="Connection to a Athena API", schema="athena_sql_schema1", extra={"s3_staging_dir": "s3://mybucket/athena/", "region_name": "eu-west-1"}, ) with patch("airflow.hooks.base.BaseHook.get_connection", return_value=athena_conn): # This should reproduce the TypeError: AwsGenericHook.__init__() got an unexpected keyword argument 's3_staging_dir' operator = SQLValueCheckOperator( task_id="value_check", sql="SELECT TRUE", pass_value=True, conn_id="athena_conn" ) context = {"ds": "2024-01-01", "execution_date": None} operator.execute(context) ``` ### Anything else `hook_params` lacks documentation, so I am unsure how to use it. Should it be managed in the hook constructor ? If yes, we need to change the signature ? Also, is it really needed to feed `hook_params` with `extra_dejson` as it is done [here](https://github.com/kandharvishnu/airflow/blob/0b8b1544fa4aa5d2725330604232e2e64237b647/providers/common/sql/src/airflow/providers/common/sql/operators/sql.py#L169) ? ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
