krutoileshii opened a new issue, #10309: URL: https://github.com/apache/seatunnel/issues/10309
### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. ### Description #### Background Many organizations already rely on centralized secret management systems (key vaults) such as **Azure Key Vault**, **AWS Secrets Manager**, **HashiCorp Vault**, or **GCP Secret Manager** to securely store and rotate credentials. Native support for these systems would significantly improve SeaTunnel’s security posture and enterprise adoption. #### Proposed Feature Introduce **native support for external key vaults** in SeaTunnel configuration resolution, starting with **Azure Key Vault**, and designed to be extensible for other providers. ##### Key Capabilities 1. **Secret Reference Syntax in Configs** Allow configuration values to reference secrets stored in a key vault, for example: ```hocon password = ${keyvault:azure:my-key-vault/db-password} ``` 2. **Pluggable Vault Provider Architecture** * Define a generic `SecretProvider` interface. * Implement initial providers: * Azure Key Vault * (Future) AWS Secrets Manager * (Future) HashiCorp Vault * (Future) GCP Secret Manager 3. **Authentication Support** For Azure Key Vault: * Managed Identity * Service Principal (Client ID / Secret) * Default Azure Credential chain 4. **Runtime Secret Resolution** * Secrets resolved at runtime before job execution. * No secrets persisted in logs, metrics, or checkpoints. 5. **Backward Compatibility** * Existing configs continue to work unchanged. * Vault integration is opt-in. #### Example Use Case ```hocon env { execution.parallelism = 2 } sink { Jdbc { url = "jdbc:sqlserver://mydb.database.windows.net:1433" user = "seatunnel_user" password = ${keyvault:azure:prod-kv/sql-password} } } ``` #### Benefits * Improved security and compliance (SOC 2, ISO 27001, HIPAA, etc.) * Centralized secret management and rotation * Reduced operational risk * Stronger enterprise and cloud-native adoption of SeaTunnel #### Additional Considerations * Caching and TTL for secret lookups * Failure behavior (e.g., fail-fast if secret is unavailable) * Clear documentation and examples * Masking secrets in logs and error messages #### Open Questions * Should secret resolution occur at parse time or execution time? * Should secrets be refreshable for long-running jobs? * Preferred syntax and naming convention ### Usage Scenario _No response_ ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
