flolas opened a new issue, #39310: URL: https://github.com/apache/airflow/issues/39310
### Description Add support for deferrable executions to the SQLExecuteOperator. Currently, the DBApi Hook does not support an asynchronous cursor (additionally, PEP does not currently specify an async cursor). Perhaps every hook should implement an async method for running asynchronous commands, and only implement an abstract method in the dbapi hook? Given this, the SQLExecuteOperator should expose something that allows behavior to be changed, perhaps when `deferrable=True` and raise an exception when the async method is not implemented in the particular connection hook? I am planning to work on a proposal PR for the SQLExecuteOperator plus an Async AWS Athena Connection, but I would like to hear some thoughts from the community. Related discussions from Python: [Asynchronous DBAPI Discussion](https://discuss.python.org/t/asynchronous-dbapi/2206/23?page=2) and [PEP-0249](https://peps.python.org/pep-0249/) ### Use case/motivation When you use the SQLExecuteOperator with multiple statements, there is an 'idle' time while waiting for query results. Perhaps by using an asynchronous cursor, we could delegate the waiting to the trigger process and free up workers. ### Related issues _No response_ ### Are you willing to submit a PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
