Baptiste, Thank you for writing this up and sharing. This looks very interesting and the team has discussed it in various small groups.
Just fyi, there is also a provider (akeyless) that may be of interest for this use case. I am flagging this for Kaxil, as he brought up this concept earlier. Best regards, Vikram On Mon, Nov 16, 2026 at 10:54 PM Baptiste Gouhoury < [email protected]> wrote: > Hi all, > > I'd like to discuss an idea for letting Airflow act as an OpenID > Connect issuer for its own tasks. > Before starting any implementation, I'd like to get some feedback on > the proposed design. > > The idea is to let a running task request a short-lived ID token for > an audience of its choice, then use it with any service supporting > OIDC federation > (AWS AssumeRoleWithWebIdentity, GCP Workload Identity Federation, > Azure federated credentials, Vault JWT auth, Kubernetes, internal > APIs, etc.). > > This would allow tasks to authenticate without storing long-lived > credentials in Airflow Connections. > GitHub Actions, GitLab CI and Kubernetes service account tokens > already follow a similar approach. > > Here's what I have in mind: > > 1/ The API server exposes two public endpoints under a configured > issuer URL: /.well-known/openid-configuration and > /.well-known/jwks.json. > These endpoints expose public keys and OpenID issuer metadata. > > 2/ A new Execution API endpoint issues the token. The request is > authenticated using the task's execution token and the existing > ti:self check. > The worker only provides the audience. The API server derives the > subject and other claims (dag_id, task_id, run_id, bundle_name, > team_name, etc.) from the task instance in the database before signing > the token. Task code cannot choose its own identity. > > 3/ The Task SDK exposes a get_id_token(audience=...) method, routed > through the supervisor like other Execution API calls. > > Providers can then use the token with their existing authentication > mechanisms: > Amazon: assume_role_with_web_identity_federation > Google: SubjectTokenSupplier > Microsoft Azure: ClientAssertionCredential > HashiCorp: JWT auth type > > Most of the building blocks are already there: JWTGenerator and the > JWKS helpers in api_fastapi/auth/tokens.py, task-instance-scoped > execution tokens, and the supervisor message channel. > > There are a few things I'd like to get the community's input on: > > a) Signing key > > I'd suggest using a dedicated asymmetric key configured in a new > [workload_identity] section, separate from jwt_private_key_path in > [api_auth]. > Many deployments use jwt_secret (HS512), which cannot be published as > a JWKS key. Having a separate key would also ensure that workload > identity tokens cannot be used to authenticate against Airflow's own > APIs. Internal audiences would be rejected as well. > > b) Subject format > > I'd suggest a Dag-level default subject format: > bundle:<bundle>:dag:<dag_id> with a configurable template. > Azure requires an exact match on sub and doesn't support wildcards. > With a per-task subject, users would need to create one federated > credential per task. > Including the bundle (and team) makes sense to me because these values > come from deployment configuration, while dag_id is chosen by the Dag > author. > > c) Audiences > > We should allow any audience except Airflow's internal ones. > > d) Security model > > The identity would essentially mean "code running in this Dag, from > this bundle". > This is consistent with the current security model, where Dag authors > are trusted to run code on workers. Still, I think this should be > explicitly documented in security model docs. > Tokens would be short-lived (5 minutes by default), masked in logs, > and issuance would be audited. > Do you see any security concerns or conflicts with the existing model, > particularly in multi-team deployments? > > e) Process > > I think this would require an AIP. It introduces a public endpoint, a > new Execution API route, and a new Task SDK API. > > I am new here so I'm looking forward to hearing your thoughts! > I'd be happy to work on the implementation if there's interest in the > proposal. > > Thanks, > Baptiste > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
