wForget opened a new issue, #10937: URL: https://github.com/apache/incubator-gluten/issues/10937
### Backend VL (Velox) ### Bug description When I tested DynamicOffHeapSizingMemory on jdk 1.8, totalHeapMemory cannot be released, but we should suspend asyncOnHeapMemoryShrink. However, from the log, the behavior of `DynamicOffHeapSizingMemoryTarget.ASYNC_GC_SUSPEND` is not as expected, and asyncOnHeapMemoryShrink is not suspended. ``` 25/10/24 10:21:19 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1943664488, Total Off-heap: 671088640, Used On-Heap: 132384920, Executor memory: 3113877504. 25/10/24 10:21:19 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1943785912, Total Off-heap: 671088640, Used On-Heap: 132263496, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:19 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:19 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:19 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1943662736, Total Off-heap: 671088640, Used On-Heap: 132386672, Executor memory: 3113877504. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953996232, Total Off-heap: 671088640, Used On-Heap: 122053176, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1943661368, Total Off-heap: 671088640, Used On-Heap: 132388040, Executor memory: 3113877504. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953994736, Total Off-heap: 671088640, Used On-Heap: 122054672, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1943659880, Total Off-heap: 671088640, Used On-Heap: 132389528, Executor memory: 3113877504. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953991088, Total Off-heap: 671088640, Used On-Heap: 122058320, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942971872, Total Off-heap: 671088640, Used On-Heap: 133077536, Executor memory: 3113877504. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953995552, Total Off-heap: 671088640, Used On-Heap: 122053856, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942976136, Total Off-heap: 662700032, Used On-Heap: 133073272, Executor memory: 3113877504. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953992888, Total Off-heap: 662700032, Used On-Heap: 122056520, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942973648, Total Off-heap: 671088640, Used On-Heap: 133075760, Executor memory: 3113877504. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953990056, Total Off-heap: 671088640, Used On-Heap: 122059352, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:20 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:20 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942970832, Total Off-heap: 671088640, Used On-Heap: 133078576, Executor memory: 3113877504. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953996648, Total Off-heap: 671088640, Used On-Heap: 122052760, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942977416, Total Off-heap: 671088640, Used On-Heap: 133071992, Executor memory: 3113877504. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953992824, Total Off-heap: 671088640, Used On-Heap: 122056584, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942973576, Total Off-heap: 671088640, Used On-Heap: 133075832, Executor memory: 3113877504. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953983456, Total Off-heap: 671088640, Used On-Heap: 122065952, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942964216, Total Off-heap: 671088640, Used On-Heap: 133085192, Executor memory: 3113877504. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953989136, Total Off-heap: 671088640, Used On-Heap: 122060272, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942969896, Total Off-heap: 671088640, Used On-Heap: 133079512, Executor memory: 3113877504. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953995920, Total Off-heap: 671088640, Used On-Heap: 122053488, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:21 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:21 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942976688, Total Off-heap: 671088640, Used On-Heap: 133072720, Executor memory: 3113877504. 25/10/24 10:21:22 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1943781568, Total Off-heap: 671088640, Used On-Heap: 132267840, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:22 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:22 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:22 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942974016, Total Off-heap: 671088640, Used On-Heap: 133075392, Executor memory: 3113877504. 25/10/24 10:21:22 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953990944, Total Off-heap: 671088640, Used On-Heap: 122058464, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:22 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:22 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:22 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942971712, Total Off-heap: 662700032, Used On-Heap: 133077696, Executor memory: 3113877504. 25/10/24 10:21:22 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953988272, Total Off-heap: 662700032, Used On-Heap: 122061136, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:22 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. 25/10/24 10:21:22 INFO DynamicOffHeapSizingMemoryTarget: Updated VM flags: MaxHeapFreeRatio from 30 to 5. 25/10/24 10:21:22 WARN DynamicOffHeapSizingMemoryTarget: Starting async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1942826024, Total Off-heap: 671088640, Used On-Heap: 133223384, Executor memory: 3113877504. 25/10/24 10:21:22 WARN DynamicOffHeapSizingMemoryTarget: Finished async full gc to shrink JVM memory: Total On-heap: 2076049408, Free On-heap: 1953988272, Total Off-heap: 671088640, Used On-Heap: 122061136, Executor memory: 3113877504, [GC Retry times: 0]. 25/10/24 10:21:22 INFO DynamicOffHeapSizingMemoryTarget: Reverted VM flags back. ...... ``` ### Gluten version main branch ### Spark version Spark-3.5.x ### Spark configurations _No response_ ### System information _No response_ ### Relevant logs ```bash ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
