pbetkier opened a new issue #17500: URL: https://github.com/apache/airflow/issues/17500
# Short I'm pretty sure the authentication flow for GCP in the `airflow.providers.hashicorp.secrets.vault.VaultBackend` is incorrect and not usable. The Vault API used [`/auth/gcp/config`](https://www.vaultproject.io/api/auth/gcp#configure) has a different purpose, [`/auth/gcp/login`](https://www.vaultproject.io/api/auth/gcp#login) should be used instead. I've seen the `/auth/<type>/config` API being used in a couple other auth types, e.g. RADIUS and Azure, so the problem is probably broader than just GCP. It's a bold statement as it means nobody has really used `VaultBackend` with these auth types. I may be wrong, but I would be surprised. # Details I've analysed Airflow's `VaultBackend` code while researching how to integrate Vault with GCP for a different project. Here's the discrepancy I found between the current implementation and the Vault's design. ## Current implementation GCP auth is implemented in [vault_client.py#L277](https://github.com/apache/airflow/blob/main/airflow/providers/hashicorp/_internal_client/vault_client.py#L277). The implementation delegates to [official Vault client's method](https://github.com/hvac/hvac/blob/develop/hvac/api/auth_methods/gcp.py#L22) that calls `/auth/gcp/config` while providing the Airflow's GCP credentials to the call. This is however an **administrative API** to configure Vault's capability to verify JWT tokens via GCP API. The credentials expected are credentials for Vault to use when communicating with GCP IAM API, **not application's credentials**. This is explained in [/auth/gcp/config API docs](https://www.vaultproject.io/api/auth/gcp#configure). The API doesn't respond with any data, so from the point of view of `VaultBackend` this call is basically just a no-op. [/auth/gcp/config](https://www.vaultproject.io/api/auth/gcp#configure) call requires `X-Vault-Token` header with a token that has proper permissions. I'm guessing either this auth method was not tested at all or some administrative (root?) Vault token was passed (by mistake?) during manual tests? I haven't seen any integration tests for the auth methods, I understand they would be difficult/costly. ## Expected implementation The proper API to use is [/auth/gcp/login](https://www.vaultproject.io/api/auth/gcp#login). It requires a signed JWT token issued by [IAM's signJwt](https://cloud.google.com/iam/docs/reference/credentials/rest/v1/projects.serviceAccounts/signJwt) based on application's GCP credentials. This is described in the [Vault docs on GCP auth](https://www.vaultproject.io/docs/auth/gcp#iam-login) and on Google blog along with an implementation example [Authenticating to HashiCorp Vault using Google Cloud IAM](https://opensource.googleblog.com/2017/08/hashicorp-vault-and-google-cloud-iam.html). --- Am I missing something? What do you think about this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
