Jackie-Jiang opened a new pull request, #16661: URL: https://github.com/apache/pinot/pull/16661
## Summary This PR fixes the broker request ID generator to prevent the generation of duplicate request IDs across different brokers or broker restarts. ## Problem The previous implementation used the broker ID hash code as a mask for generating request IDs. This approach had several issues: - Brokers with similar IDs could generate the same hash code, leading to duplicate request IDs - Broker restarts would regenerate the same sequence of request IDs - Hash collisions could occur between different broker instances ## Solution - Replaced deterministic hash-based approach with random-based approach - Centralized request ID generator creation in BaseBrokerStarter - Updated all broker request handlers to use shared generator instance - Improved documentation to reflect the new random-based approach ## Changes ### Core Changes - BrokerRequestIdGenerator: Changed from hash-based to random-based ID generation - BaseBrokerStarter: Creates a single shared BrokerRequestIdGenerator instance - All Request Handlers: Updated constructors to accept the shared generator ### Files Modified - BaseBrokerStarter.java - Creates and injects shared request ID generator - BaseBrokerRequestHandler.java - Updated constructor to accept generator - BaseSingleStageBrokerRequestHandler.java - Updated constructor signature - BrokerRequestIdGenerator.java - Changed from hash-based to random-based generation - GrpcBrokerRequestHandler.java - Updated constructor - MultiStageBrokerRequestHandler.java - Updated constructor - SingleConnectionBrokerRequestHandler.java - Updated constructor - TimeSeriesRequestHandler.java - Updated constructor - Test files updated to use new constructor signatures ## Technical Details ### ID Generation Strategy - Base: Random integer multiplied by 1,000,000,000 - Counter: Auto-incrementing counter for uniqueness - Result: Unique 19-digit request ID with high probability of uniqueness ### Benefits 1. Eliminates hash collisions between brokers 2. Handles broker restarts gracefully 3. Maintains performance with atomic counter 4. Preserves uniqueness across distributed system ## Testing - Code compiles successfully - All existing tests updated and passing - Manual testing with multiple broker instances - Verified no duplicate IDs generated across restarts This change improves the reliability of request tracking in both single-stage and multi-stage query engines. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
