Akanksha-kedia opened a new pull request, #17651: URL: https://github.com/apache/pinot/pull/17651
This commit implements comprehensive Vault integration for secure token management across all Pinot components (Controller, Broker, Server, Minion). Key Features: - Vault-based authentication with automatic token refresh - AuthProviderFactory for centralized auth provider creation - Comprehensive error handling and logging - Thread-safe token caching with configurable TTL - Support for multiple Vault authentication methods (AppRole, Token) Components Added: - VaultAuth: Core Vault authentication logic - VaultConfig: Configuration management for Vault settings - VaultResponse: Response parsing and validation - VaultStartupManager: Singleton initialization manager - VaultTokenAuthProvider: Auth provider implementation - VaultTokenCache: Thread-safe token caching - VaultUtil: Utility methods for Vault operations - AuthProviderFactory: Factory pattern for auth providers Enhancements: - Updated AuthProviderUtils with Vault support - Enhanced StaticTokenAuthProvider with null checks - Integrated Vault initialization in all component starters - Added HttpSegmentFetcher auth support Testing: - AuthProviderFactoryTest: 390 lines of comprehensive tests - VaultTokenAuthProviderTest: 214 lines of provider tests - VaultUtilTest: 69 lines of utility tests Security: - Fixed security vulnerabilities - Removed debug/development tags - Proper secret handling and token lifecycle management PR Title Distribute Service Tokens Across Pinot Components with Static and Vault‑Based Auth Providers ✅ PR Description This change introduces a clear and explicit service‑token distribution model for Apache Pinot components (Controller, Broker, Server, Minion) and documents it visually and configurationally. 🔐 Service Token Distribution All Pinot components now support service‑level authentication tokens that are independent of user authentication mechanisms (Basic, ZK, LDAP). For simplicity and clarity: Admin credentials are reused as service tokens In production deployments, service credentials must be separated from admin users ✅ Supported Auth Provider Modes (Final) The following parameter is introduced and treated as authoritative plain text: auth.provider.type = static | null | vault This parameter is not optional, not experimental, and not open for discussion. 🧩 Behavior by Mode 1️⃣ auth.provider.type=static Service tokens are configured manually in component config files No external dependency Example: # Enable the controller to fetch segments using a service token controller.segment.fetcher.auth.token=Basic YWRtaW46dmVyeXNlY3JldA Basic + base64encode(admin:verysecret) ✅ Tokens must NOT be surrounded by quotes ✅ Restart affected components for changes to take effect 2️⃣ auth.provider.type=vault Controller, Broker, Server, and Minion: Authenticate to Vault at startup Fetch credentials (username/password) Generate a Basic Auth service token Cache the token in memory No Vault calls at runtime Example configuration: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
