eladkal opened a new issue, #34823: URL: https://github.com/apache/airflow/issues/34823
### Body The problem: `AthenaOperator` store query result in S3 and push to xcom the query_id. To read the result one must find the path in S3 of the query output (using query_id), download the output file, open it and then ask questions on it. This is very very very frusturating. My query for example is `SELECT(1) FROM...`. I simply want to count records and compare it to something. I want the record number to be pushed to Xcom. This enables variety of options such as Branching, ShortCircuit and other We probably can not (don't want) to just add `push_query_result` flag to `AthenaOperator` from what I see the API of the hook doesn't allow this easily. We should see if we can overcome this somehow. (maybe a new hook with [`PyAthena`](https://pypi.org/project/PyAthena/)? by doing so Athena should be compatible with all our current SQL operators.) This is a big blocker from utalizing Athena efficiently. The Task: (at least how I see it) Create `AthenaSqlHook` which is based on `PyAthena`. Hook should inherit from `DbApiHook` ### Committer - [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
