adobe-ddavidso opened a new pull request, #60953:
URL: https://github.com/apache/airflow/pull/60953
Add AppRole token caching support to HashiCorp Vault secrets backend
## Description
This PR adds file-based token caching for AppRole authentication to the
HashiCorp Vault secrets backend. When enabled via the `cache_approle_token`
configuration parameter, the backend will cache and reuse the authentication
token across all Airflow processes (scheduler, workers, webserver) until it
expires, significantly reducing authentication overhead and improving
performance in high-frequency secret retrieval scenarios.
### Key Features
- **File-based caching**: Tokens are cached in the system's temporary
directory with a unique filename based on Vault URL and role_id
- **Cross-process sharing**: Cache is shared across all Airflow processes
through file-based storage
- **Thread-safe and process-safe**: Uses `fcntl.flock` for advisory file
locking to prevent race conditions
- **Automatic invalidation**: Cache is automatically cleared when:
- Token expires (with 60-second safety buffer)
- Authentication fails
- Process restarts with an expired cached token
- **Backward compatible**: Disabled by default, opt-in via configuration
### Implementation Details
**Modified Files:**
-
`providers/hashicorp/src/airflow/providers/hashicorp/_internal_client/vault_client.py`:
Core caching implementation (+141 lines)
- `_get_cache_file_path()`: Generates deterministic cache file path
- `_read_cached_token()`: Reads token from cache with file locking
- `_write_cached_token()`: Writes token to cache with file locking
- `_clear_cached_token()`: Removes expired cache file
- `providers/hashicorp/src/airflow/providers/hashicorp/secrets/vault.py`:
Exposed `cache_approle_token` parameter in `VaultBackend` (+5 lines)
- `providers/hashicorp/src/airflow/providers/hashicorp/hooks/vault.py`:
Exposed `cache_approle_token` parameter in `VaultHook` (+5 lines)
**Tests:**
-
`providers/hashicorp/tests/unit/hashicorp/_internal_client/test_vault_client.py`:
Added comprehensive unit tests (+224 lines)
- Test token caching functionality
- Test cache expiry handling
- Test cache clearing on authentication failure
- Test file locking behavior
**Documentation:**
- `providers/hashicorp/docs/secrets-backends/hashicorp-vault.rst`: Added
"AppRole Token Caching" section with usage examples (+26 lines)
### Configuration Example
```ini
[secrets]
backend = airflow.providers.hashicorp.secrets.vault.VaultBackend
backend_kwargs = {
"connections_path": "connections",
"variables_path": "variables",
"mount_point": "airflow",
"url": "http://127.0.0.1:8200";,
"auth_type": "approle",
"role_id": "your-role-id",
"secret_id": "your-secret-id",
"cache_approle_token": true
}
```
### Testing
All changes are covered by comprehensive unit tests that verify:
- Token is cached after successful authentication
- Cached token is reused until expiry
- Cache is automatically cleared on expiry or failure
- File locking works correctly for concurrent access
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Cursor AI following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
