abhishekrb19 opened a new pull request, #17056: URL: https://github.com/apache/druid/pull/17056
A few CI jobs have been consistently failing on master due to OOMs. Specifically, some tests in the `Unit & Integration tests CI / unit tests (jdk17, sql-compat=true) / other_modules_test / other modules test (push)` are failing -- see error below. Looking at the heap dump, it appears that there are global references being retained which sort of point to Mockito. One candidate is the use of Mockito spy objects [here](https://github.com/apache/druid/blob/master/extensions-core/multi-stage-query/src/test/java/org/apache/druid/msq/test/MSQTestBase.java#L531) that somehow have global references and don't get garbage collected. It's not evident yet. The heap dump artifact can be found in any of the recent failed jobs. In the mean time, increase the JVM size from 2GB to 2.5GB to see if it helps alleviate the ongoing test failures. ``` Warning: Tests run: 158, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 46.51 s -- in org.apache.druid.msq.test.CalciteArraysQueryMSQTest [INFO] Running org.apache.druid.msq.test.CalciteSelectJoinQueryMSQTest$SortMergeTest Warning: Tests run: 586, Failures: 0, Errors: 0, Skipped: 85, Time elapsed: 131.9 s -- in org.apache.druid.msq.test.CalciteSelectJoinQueryMSQTest$SortMergeTest [INFO] Running org.apache.druid.msq.test.CalciteSelectQueryMSQTest java.lang.OutOfMemoryError: Java heap space Dumping heap to /home/runner/work/druid/druid/target/java_pid22365.hprof ... Heap dump file created [3049991269 bytes in 9.662 secs] # # java.lang.OutOfMemoryError: Java heap space # -XX:OnOutOfMemoryError="chmod 644 /home/runner/work/druid/druid/target/*.hprof" # Executing /bin/sh -c "chmod 644 /home/runner/work/druid/druid/target/*.hprof"... Terminating due to java.lang.OutOfMemoryError: Java heap space ``` A few failed jobs: 1. https://github.com/apache/druid/actions/runs/10834038415/job/30068115361 2. https://github.com/apache/druid/actions/runs/10824552673/job/30035411027 3. https://github.com/apache/druid/actions/runs/10835792545/job/30073489146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
