anish-heidi commented on issue #71525: URL: https://github.com/apache/airflow/issues/71525#issuecomment-5275819329
Update: this is already resolved in Airflow core >= 3.2.1 (possibly earlier, haven't bisected precisely) - just not in a way that's obvious from the databricks provider alone, which is why I initially thought a databricks-specific fix was still needed. Checked the source at the `3.2.1` tag directly: - `task-sdk/src/airflow/sdk/execution_time/secrets/execution_api.py`'s `get_connection()` now catches the `RuntimeError` from `async_to_sync` and falls back to `greenback.await_(self.aget_connection(conn_id))` when called inside a greenback portal. - `airflow-core/src/airflow/jobs/triggerer_job_runner.py` confirms every trigger runs inside such a portal (`await greenback.ensure_portal()`). This is the general core-level fix from #55799 (merged 2025-09-18, landed in 3.1.0), regressed by #56602 (2025-10-14), and re-fixed by #57154 (2025-10-23). It patches the problem one layer below any individual hook/provider, in Airflow's own connection-fetch machinery - so `DatabricksHook.databricks_conn`'s synchronous `cached_property` pattern (and presumably any other provider hook with the same pattern) works transparently once core is upgraded past the #57154 fix, without needing #55568 or any other databricks-specific patch. For anyone hitting this: the fix is "upgrade Airflow core to >= 3.2.x", not "wait for a provider patch." Leaving this open since 3.0.x/3.1.0-3.1.x users without the #57154 fix would still hit it, but wanted to close the loop on my own findings here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
