allenc97 opened a new pull request, #69509:
URL: https://github.com/apache/airflow/pull/69509
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
Extends the Databricks provider hook with OIDC token federation for Airflow
running with AWS credentials, parallel to the existing Kubernetes path
(#61458). The hook mints an AWS-signed OIDC JWT via AWS STS
`GetWebIdentityToken` and exchanges it for a Databricks service-principal token
using the same RFC 8693 exchange as `federated_k8s` — only the JWT source
differs. Like the Kubernetes path it stores no secrets in the connection;
identity comes from the caller's assumed IAM role. Set `login=federated_aws`
and provide `client_id` and `aws_conn_id` in the connection extra. See the
Databricks [AWS IAM workload identity federation
guide](https://docs.databricks.com/aws/en/dev-tools/auth/provider-aws-iam).
* related: #54291
### Integration testing
Verified end-to-end against a real workspace with this connection (no
secrets stored) and DAG:
```json
{
"conn_type": "databricks",
"host": "<workspace>.cloud.databricks.com",
"login": "federated_aws",
"extra": {
"client_id": "<databricks-sp-client-id>",
"aws_conn_id": "<aws_conn_id>"
}
}
```
```python
from __future__ import annotations
from airflow.providers.databricks.hooks.databricks import DatabricksHook
from airflow.sdk import dag
from airflow.sdk import task
@dag(
dag_id="test_databricks_federated_aws",
schedule=None,
catchup=False,
description="Integration test: authenticate to Databricks via AWS IAM
federation and confirm the SP identity.",
)
def test_databricks_federated_aws():
@task
def test_connection():
hook = DatabricksHook(databricks_conn_id="databricks_federated_aws")
print(hook.test_connection())
print(hook._do_api_call(("GET", "/2.0/preview/scim/v2/Me")))
test_connection()
test_databricks_federated_aws()
```
Requires outbound web identity federation enabled on the AWS account, the
role granted `sts:GetWebIdentityToken`, and a Databricks service-principal
federation policy whose `subject` is the role ARN and `audiences` is
`["databricks"]`.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Opus 4.8) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
