gianm commented on PR #12513: URL: https://github.com/apache/druid/pull/12513#issuecomment-1126802203
Tagging this "bug" too since it also fixes a bug with ConcurrentGrouper. There was a race condition as part of the switch from hash-based to thread-local buffer selection: it's possible that thread A is still aggregating into grouper B (selected via hash) while thread B is simultaneously trying to aggregate into it (selected via thread-locality). The fix is to always synchronize on the grouper, regardless of how it was selected. I expect that this fix will be fine performance-wise, since contention will be nil once things switch over to thread-local selection. I noticed this when running ConcurrentGrouperTest in an infinite loop. The tests that involved spilling with `mergeThreadLocal: false` failed on one out of every few thousand iterations. The bug isn't triggered when `mergeThreadLocal: true`, because the switch doesn't happen. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
