kaxil opened a new pull request, #69069:
URL: https://github.com/apache/airflow/pull/69069
Add Workload Identity authentication to the OpenAI provider, so connections
can authenticate with short-lived identity tokens instead of a long-lived API
key. The mechanism is selected with a new ``auth_type`` key in the connection
``extra``; it defaults to ``api_key``, so existing connections are unchanged.
> Stacked on #69068 (requires ``openai>=2.37.0``). Review the latest commit;
the first commit is #69068 and will drop out once it merges.
## Why first-class wiring instead of the existing passthrough
The connection already forwards ``openai_client_kwargs`` to the OpenAI
client, but Workload Identity needs a token-provider *callable*, which can't be
expressed in a JSON connection ``extra``. So ``get_conn`` builds the provider
from declarative config keys instead.
## Usage
Set ``auth_type`` to ``workload_identity`` and choose a token source with
``workload_identity_provider``:
- ``kubernetes`` -- service account token from ``token_file_path`` (defaults
to the in-cluster path)
- ``azure`` -- Azure managed identity (optional ``resource``, ``client_id``,
``object_id``, ``msi_res_id``, ``api_version``)
- ``gcp`` -- GCP ID token for ``audience``
- ``custom`` -- import ``token_provider`` (a dotted path to a ``Callable[[],
str]``); ``token_type`` is ``jwt`` (default) or ``id``
``identity_provider_id`` and ``service_account_id`` are required;
``refresh_buffer_seconds`` is optional.
Example (Kubernetes pod):
```json
{
"auth_type": "workload_identity",
"workload_identity_provider": "kubernetes",
"identity_provider_id": "idp-123",
"service_account_id": "sa-456"
}
```
## Notes
- The API-key path is unchanged. ``api_key`` is popped from
``openai_client_kwargs`` for every path so it is never forwarded alongside
``workload_identity`` (the client rejects both being set).
- The ``custom`` source imports and calls the named callable in the process
running the hook, so point it only at trusted code. Connection ``extra`` is an
operator/admin surface, consistent with how other providers resolve callables
from config.
- The selector is named ``workload_identity_provider`` to mirror OpenAI's
own "workload identity provider" terminology.
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR,
please
change below checkbox to `[X]` followed by the name of the tool, uncomment
the "Generated-by".
-->
- [ ] Yes (please specify the tool below)
<!--
Generated-by: [Tool Name] following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
