Jackie-Jiang opened a new pull request #4764: Fix the bug of releasing the segment when there are still threads working on it URL: https://github.com/apache/incubator-pinot/pull/4764 Issue: When we execute a query, the main thread holds the segment reference, and avoid the segment getting released during the query execution. When the query timed out, the main thread will return the exception and release the segment reference without waiting for all the worker threads finish the work. This can cause severe issue when segment get released due to segment deletion/refresh while worker threads are still accessing it, even JVM crash. Solution: Use a Phaser for the query execution to ensure when the main thread returns the result, all the worker threads are not accessing the segment (not scheduled or done). To shorten the period of waiting, check the interrupt flag when the operator starts processing the next block, and interrupt the execution if the thread is interrupted. Applied the solution to both planning phase and execution phase, and also added tests to check the behavior.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
