Jackie-Jiang opened a new pull request, #16625: URL: https://github.com/apache/pinot/pull/16625
## Problem Currently, when a query is cancelled, there is a race condition where new operator chains for that query might still get scheduled after the cancellation request. This can lead to unnecessary resource usage and unexpected behavior. ## Solution This PR introduces a synchronization mechanism to prevent operator chains from being scheduled for cancelled queries: 1. **Added per-query read-write locks**: Implemented a lock array (1024 locks) using request ID hashing to synchronize scheduling and cancellation operations 2. **Added cancelled query cache**: Tracks cancelled queries with configurable size and expiration time 3. **Modified registration flow**: Before scheduling an operator chain, acquire a read lock and check if the query has been cancelled 4. **Modified cancellation flow**: Acquire a write lock and mark the query as cancelled in the cache ## Configuration Added new configuration options: - `pinot.server.query.cancelled.cache.size` (default: 1000) - Maximum number of cancelled queries to cache - `pinot.server.query.cancelled.cache.ms` (default: 60000) - Cache expiration time in milliseconds ## Files Changed - `OpChainSchedulerService.java`: Core logic for synchronized scheduling and cancellation - `CommonConstants.java`: Added new configuration constants ## Testing The fix ensures that operator chains are not scheduled for queries that have already been cancelled, preventing the race condition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
