zhztheplayer opened a new pull request, #9115: URL: https://github.com/apache/incubator-gluten/pull/9115
1. Pass an allocation listener that reserves memory from `GlobalOffHeapmemory` from Java to C++ for Velox's global memory manager 2. Reduce the default overhead memory ratio from 0.3 * off-heap-size to 0.15 * off-heap-size After the patch is applied, the global memory usage that doesn't belong to any of the Velox tasks will be counted into Spark's **off-heap storage memory** allocation rather than into overhead memory allocation. Hence, user could reduce their overhead memory settings and enlarge off-heap correspondingly. Future TODOs: 1. leak-detection 2. evict / spill from global off-heap memory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
