gianm commented on PR #12498: URL: https://github.com/apache/druid/pull/12498#issuecomment-1225044484
Two test runs failed with the below segfault in `KllDoublesSketchAggregatorTest` ([first time](https://app.travis-ci.com/github/apache/druid/jobs/580645782)) and `KllFloatsSketchAggregatorTest` ([second time](https://app.travis-ci.com/github/apache/druid/jobs/580645784)). Judging by the fact it's `BaseWritableMemoryImpl.getNativeOrderedLong`, it could be an out-of-bound access or a use-after-free. I tried running the test case in a loop in my IDE, and unfortunately wasn't able to repro this. In looking at the code, I suspect it's due to a use-after-free due to returning an object from `KllSketchBuildBufferAggregatorHelper.get` that references the underlying buffer. That's used to implement `BufferAggregator.get` and `VectorAggregator.get`, which by contract aren't permitted to return objects that reference the underlying buffer. (It may be freed, or reused, while those get'ed objects still exist.) Similar code in other sketch helpers creates on-heap copies (hll uses `copy()`, quantiles uses `compact()`). Could we do the same thing here? ``` # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f69d737e924, pid=21102, tid=21103 # # JRE version: OpenJDK Runtime Environment (11.0.2+9) (build 11.0.2+9) # Java VM: OpenJDK 64-Bit Server VM (11.0.2+9, mixed mode, tiered, compressed oops, g1 gc, linux-amd64) # Problematic frame: # J 14439 c1 org.apache.datasketches.memory.internal.BaseWritableMemoryImpl.getNativeOrderedLong(J)J (38 bytes) @ 0x00007f69d737e924 [0x00007f69d737e7a0+0x0000000000000184] # # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # An error report file with more information is saved as: # /home/travis/build/apache/druid/extensions-core/datasketches/hs_err_pid21102.log Could not load hsdis-amd64.so; library not loadable; PrintAssembly is disabled # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
