LuciferYang commented on code in PR #13094:
URL: https://github.com/apache/gluten/pull/13094#discussion_r4178341401
##########
gluten-core/src/main/scala/org/apache/spark/memory/SparkMemoryUtil.scala:
##########
@@ -50,9 +52,24 @@ object SparkMemoryUtil {
// We assume storage memory can be fully transferred to execution memory so
far
def getCurrentAvailableOffHeapMemory: Long = {
- val mm = SparkEnv.get.memoryManager
- val smp = smpField.get(mm).asInstanceOf[StorageMemoryPool]
- val emp = empField.get(mm).asInstanceOf[ExecutionMemoryPool]
+ val env = SparkEnv.get
+ val mm = env.memoryManager
+ // With dynamic off-heap sizing enabled, Gluten's global reservations are
+ // charged to the ON-heap pools (see GlobalOffHeapMemoryTarget), so the
+ // available figure must be read from the same pools to stay meaningful;
+ // reading the off-heap pools would ignore every reservation.
+ val dynamicSizingEnabled =
Review Comment:
Good catch, done in 7fbcac365. `getCurrentAvailableOffHeapMemory` now reads
the flag through `GlutenCoreConfig.get.dynamicOffHeapSizingEnabled`, the same
call `GlobalOffHeapMemoryTarget.mode` uses, so the metric and the reservations
always pick the same pool family.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
