flolas commented on PR #41511: URL: https://github.com/apache/airflow/pull/41511#issuecomment-2293269207
> Can you clarify what's changed since [#34583 (comment)](https://github.com/apache/airflow/issues/34583#issuecomment-1762259858)? Okay, you got me, lol! I think I made my point in [this comment](https://github.com/apache/airflow/issues/34583#issuecomment-1732598175). Why did I opt for AthenaHook here? Unfortunately, DB API 2.0 does not specify a standard for async/awaitable query execution. I was considering #30451, but I found out that every library implements different async strategies, which would be really hard to maintain in Airflow (also see https://github.com/apache/airflow/issues/30451#issuecomment-1496043958). I think the decision between Athena SQL Hook and Athena Hook, and why we need to support both, is covered in the [docs](https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/operators/athena/index.html): ``` Amazon Athena (API): Choose this option if you need to execute a single statement without bringing back the results in Airflow. Amazon Athena SQL (DB API Connection): Opt for this if you need to execute multiple queries in the same operator, and it’s essential to retrieve and process query results directly in Airflow, such as for sensing values or further data manipulation. ``` Perhaps another option is to support using both connections (Athena SQL Hook and Athena Hook) for AthenaOperator. Note: I haven’t worked on tests or docs yet, I’ll proceed with those once we’re happy with the implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
