zhang-arvin opened a new pull request, #20090: URL: https://github.com/apache/druid/pull/20090
### Description Fixes apache/druid#18040 - Runtime Exception when executing a query twice in a short duration. #### Root Cause The `ParallelCombiner` and `StreamingMergeSortedGrouper` are designed for concurrent read/write from different threads. However, `SketchBufferAggregatorHelper` used `IdentityHashMap` and `Int2ObjectOpenHashMap`, which are not thread-safe. Concurrent access to these maps by: - The **writing thread** (via `aggregate()` → `getOrCreateUnion()`) - The **reading thread** (via `get()`) could corrupt the internal map structure, causing `ArrayIndexOutOfBoundsException` during sketches aggregation (e.g., `Index 180 out of bounds for length 129`). #### Fix Replaced `IdentityHashMap` with `ConcurrentHashMap` and `Int2ObjectOpenHashMap` with `ConcurrentHashMap` to ensure thread safety when used with `ParallelCombiner`. #### Changes - `extensions-core/datasketches/.../SketchBufferAggregatorHelper.java`: Replace non-thread-safe maps with `ConcurrentHashMap` #### Key Features/Changes - Thread-safe access to Union cache and memory cache in `SketchBufferAggregatorHelper` - Uses `ConcurrentHashMap.computeIfAbsent` for atomic map initialization #### Verification The existing tests for `ParallelCombiner` and `StreamingMergeSortedGrouper` cover the concurrent read/write pattern. The fix ensures that Datasketches-based aggregators work correctly under parallel combining. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
