dabla opened a new pull request, #74149: URL: https://github.com/apache/airflow/pull/74149
> [!IMPORTANT] > **Depends on #74147, which needs to be merged first.** That PR adds `get_async_extra_dejson()` to the common.compat provider, which this hook needs to read the connection's extra asynchronously on every supported Airflow version. Until it is merged this PR stays a draft and uses a local `_aextra_dejson()` fallback marked with a TODO. Once #74147 is merged, this branch is rebased, the fallback is replaced by `get_async_extra_dejson()`, and the common-compat requirement is bumped to the release that ships it. `OpenAIHook` only has a synchronous path: `get_conn()` resolves the connection with `get_connection()` and reads `extra_dejson`, whose secret masking sends `MaskSecret` to the supervisor synchronously. Inside an async task with another async SDK call in flight (e.g. items running concurrently), both raise `DeadlockImminentError`, so the hook can't be used from async code. This adds an async path that mirrors the sync one: - `aget_conn()` returns an `AsyncOpenAI` client configured exactly like `get_conn()`'s client (`openai_client_kwargs`, `base_url`, `api_key`, `auth_type` `api_key` / `workload_identity`). The connection comes from `get_async_connection()` and its extra from `Connection.aextra_dejson()` (#71890, Airflow 3.3.2+), which masks secrets asynchronously; on older Airflow versions the raw extra is deserialized with `json.loads`, as #72130 does for the ADF/Synapse async hooks (TODO to drop that fallback once the provider requires 3.3.2+). - The client settings move into one `_client_kwargs()` helper shared by `get_conn()` and `aget_conn()`, so the two paths can't drift. - `acreate_embeddings()` is the async counterpart of `create_embeddings()`, same overloads and return shape (one vector for a single input, one per item for a batch), on a client created once per hook like the sync `conn`. Further async counterparts (chat completions, responses, ...) can follow the same pattern. Tests: the existing `get_conn` tests pass unchanged; new tests cover `aget_conn` (API key in password/extra, client kwargs, workload identity, invalid `auth_type`, and a connection without `aextra_dejson()` as on Airflow < 3.3.2) with a connection whose `extra_dejson` raises and a sync `get_connection` that raises, and `acreate_embeddings` for text, token and batch input. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Opus 5.5 Generated-by: Claude Opus 5.5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
