moomindani commented on PR #63775: URL: https://github.com/apache/airflow/pull/63775#issuecomment-4131609854
**Real environment validation:** Tested the latest changes against a real Databricks workspace using `DatabricksSubmitRunOperator` with `deferrable=True`. **Triggerer E2E test (fixed code, API server + scheduler + triggerer):** - Ran local `airflow api-server`, `airflow scheduler`, and `airflow triggerer` with DB-registered Databricks connection - Operator submitted the job and transitioned to DEFERRED - **Triggerer picked up the deferred task, polled the Databricks job every 30 seconds (PENDING → RUNNING → TERMINATED), and fired the completion event successfully** - DAG run completed with SUCCESS — no `AsyncToSync` error **What I could NOT validate locally:** - The actual `AsyncToSync` error reproduction with unfixed code — this requires the triggerer's subprocess IPC (`TriggerCommsDecoder.send()` → `async_to_sync()`) which is not triggered when connections are resolved via `MetastoreBackend` in server context. Env var connections also bypass `ExecutionAPISecretsBackend`. - Azure AD / managed identity / federated K8s auth paths — only token auth tested against real Databricks; other paths covered by unit tests. **Code review notes:** The fix looks good to me — all async methods now use `conn = await self.a_databricks_conn()` (via `get_async_connection()`) instead of sync `self.databricks_conn` (which would hit `get_connection()` → `async_to_sync()` in the triggerer). The `__aenter__` pre-warms the cache so even residual sync accesses return the cached value. This follows the pattern recommended by Airflow's own greenback fallback: *"You should not use sync calls here -- use `await aget_connection` instead"*. The incidental fixes (`async for` retry, `aiohttp.ClientResponseError`, `a_cancel_sql_statement` in SQL trigger timeout) are also correct and covered by unit tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
