dabla opened a new pull request, #76528: URL: https://github.com/apache/airflow/pull/76528
`SSHHookAsync._get_conn` parses the connection's extra through `conn.extra_dejson`. That property masks the extra's secrets with a synchronous call to the supervisor, so in an async task that has another call in flight on the same event loop (for example items run concurrently), opening an SSH connection raises `DeadlockImminentError`. The hook already fetches the connection with `get_async_connection`; this reads the extra the same way, with common.compat's `get_async_extra_dejson`, and hands it to `_parse_extras`. The async hook also never set a compression preference, so asyncssh's own default applies, which offers `none` before `[email protected]`. `SSHHook` compresses by default and lets the `compress` extra turn it off; `SSHHookAsync` now does the same, offering `[email protected]`, `zlib`, `none` (paramiko's order for `SSHHook`) unless `compress` is false. The connection docs already describe `compress` for the SSH connection as a whole. For bulk transfers over SSH this is a measurable difference: on our result files zlib made downloads 5-57% faster, depending on the file. Notes for reviewers: - `_parse_extras` now takes the extra as a parameter instead of reading `conn.extra_dejson` itself. It is private, and `_get_conn` is its only caller in the repository. - The common.compat floor stays as it is, with `# use next version` (`get_async_extra_dejson` came in 1.21.0). - Behaviour change: connections made by `SSHHookAsync`, including the `SSHRemoteJobTrigger`'s, now negotiate zlib when the server offers it, as `SSHHook` connections always have. `"compress": "false"` in the extra restores the previous behaviour. - `SFTPHookAsync._parse_extras` reads `conn.extra_dejson` the same way; I left it for a separate PR. Tests: `test_get_conn_reads_extra_asynchronously` checks that `_get_conn` never touches `extra_dejson` and uses the asynchronously read extra; `test_get_conn_compresses_like_sync_hook` covers the default and the `compress` extra as string and boolean. The `_parse_extras` tests now pass the extra explicitly (through a small helper where they build a real `Connection`); the two new tests fail without the change. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Opus 5.5 Generated-by: Claude Opus 5.5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
