moomindani opened a new pull request, #74406: URL: https://github.com/apache/airflow/pull/74406
On Airflow 3.0.x every deferrable Databricks operator fails in the triggerer with `RuntimeError: You cannot use AsyncToSync in the same thread as an async event loop`, because the hook resolves its connection with the sync `get_connection` from the trigger's event loop. The hook now fills its `databricks_conn` cache through `get_async_connection(..., hook=self)` before the first async API call, so the existing sync helpers read the cached connection. This takes over #71840 (the original author is credited as co-author); the async twins of the sync helpers from that PR are not needed with this approach. `common-compat` is raised to `>=1.17.0` because the `hook=` argument of `get_async_connection` first shipped in that release. Verified against a real Databricks workspace (deferrable `DatabricksRunNowOperator`, DB-backed connection, official images): 3.0.6 fails with provider 7.21.0 and succeeds with this change; 3.1.0 and 3.2.2 also succeed with this change. Not covered here: reading `conn.extra_dejson` from async code can still raise `DeadlockImminentError` when another trigger's `asend` is in flight. The fix is `get_async_extra_dejson` (#74147), which is not in a released common-compat yet. closes: #71525 supersedes: #71840 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
