Baptiste,

Thank you for writing this up and sharing.
This looks very interesting and the team has discussed it in various small
groups.

Just fyi, there is also a provider (akeyless) that may be of interest for
this use case.

I am flagging this for Kaxil, as he brought up this concept earlier.

Best regards,
Vikram


On Mon, Nov 16, 2026 at 10:54 PM Baptiste Gouhoury <
[email protected]> wrote:

> Hi all,
>
> I'd like to discuss an idea for letting Airflow act as an OpenID
> Connect issuer for its own tasks.
> Before starting any implementation, I'd like to get some feedback on
> the proposed design.
>
> The idea is to let a running task request a short-lived ID token for
> an audience of its choice, then use it with any service supporting
> OIDC federation
> (AWS AssumeRoleWithWebIdentity, GCP Workload Identity Federation,
> Azure federated credentials, Vault JWT auth, Kubernetes, internal
> APIs, etc.).
>
> This would allow tasks to authenticate without storing long-lived
> credentials in Airflow Connections.
> GitHub Actions, GitLab CI and Kubernetes service account tokens
> already follow a similar approach.
>
> Here's what I have in mind:
>
> 1/ The API server exposes two public endpoints under a configured
> issuer URL: /.well-known/openid-configuration and
> /.well-known/jwks.json.
> These endpoints expose public keys and OpenID issuer metadata.
>
> 2/ A new Execution API endpoint issues the token. The request is
> authenticated using the task's execution token and the existing
> ti:self check.
> The worker only provides the audience. The API server derives the
> subject and other claims (dag_id, task_id, run_id, bundle_name,
> team_name, etc.) from the task instance in the database before signing
> the token. Task code cannot choose its own identity.
>
> 3/ The Task SDK exposes a get_id_token(audience=...) method, routed
> through the supervisor like other Execution API calls.
>
> Providers can then use the token with their existing authentication
> mechanisms:
> Amazon: assume_role_with_web_identity_federation
> Google: SubjectTokenSupplier
> Microsoft Azure: ClientAssertionCredential
> HashiCorp: JWT auth type
>
> Most of the building blocks are already there: JWTGenerator and the
> JWKS helpers in api_fastapi/auth/tokens.py, task-instance-scoped
> execution tokens, and the supervisor message channel.
>
> There are a few things I'd like to get the community's input on:
>
> a) Signing key
>
> I'd suggest using a dedicated asymmetric key configured in a new
> [workload_identity] section, separate from jwt_private_key_path in
> [api_auth].
> Many deployments use jwt_secret (HS512), which cannot be published as
> a JWKS key. Having a separate key would also ensure that workload
> identity tokens cannot be used to authenticate against Airflow's own
> APIs. Internal audiences would be rejected as well.
>
> b) Subject format
>
> I'd suggest a Dag-level default subject format:
> bundle:<bundle>:dag:<dag_id> with a configurable template.
> Azure requires an exact match on sub and doesn't support wildcards.
> With a per-task subject, users would need to create one federated
> credential per task.
> Including the bundle (and team) makes sense to me because these values
> come from deployment configuration, while dag_id is chosen by the Dag
> author.
>
> c) Audiences
>
> We should allow any audience except Airflow's internal ones.
>
> d) Security model
>
> The identity would essentially mean "code running in this Dag, from
> this bundle".
> This is consistent with the current security model, where Dag authors
> are trusted to run code on workers. Still, I think this should be
> explicitly documented in security model docs.
> Tokens would be short-lived (5 minutes by default), masked in logs,
> and issuance would be audited.
> Do you see any security concerns or conflicts with the existing model,
> particularly in multi-team deployments?
>
> e) Process
>
> I think this would require an AIP. It introduces a public endpoint, a
> new Execution API route, and a new Task SDK API.
>
> I am new here so I'm looking forward to hearing your thoughts!
> I'd be happy to work on the implementation if there's interest in the
> proposal.
>
> Thanks,
> Baptiste
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>

Reply via email to