LuciferYang opened a new pull request, #12622: URL: https://github.com/apache/gluten/pull/12622
### What changes were proposed in this pull request? `GlutenDriverPlugin.setPredefinedConfigs` reads the off-heap size in its non-dynamic-sizing branch with the single-arg `conf.getSizeAsBytes(SPARK_OFFHEAP_SIZE_KEY)`. `checkOffHeapSettings` returns early for untracked memory mode (`spark.gluten.memory.untracked=true`) and skips the requirement that the size be set, so in that mode the key can be absent. `SparkConf.get(String)` throws `NoSuchElementException` on a missing key, so driver plugin init crashes instead of proceeding under untracked semantics. The same single-arg read exists on the Velox driver-start path: `VeloxListenerApi.onDriverStart` also calls `conf.getSizeAsBytes(SPARK_OFFHEAP_SIZE_KEY)`, and it runs after `setPredefinedConfigs` (via `init` calling `Component.onDriverStart`). The dynamic-sizing branch already shields that reader by doing `conf.set(SPARK_OFFHEAP_SIZE_KEY, "0")`; untracked mode did not, so it was the one unshielded path. This change normalizes the key in the untracked branch: when `spark.memory.offHeap.size` is absent, set it to `0`, the same way the dynamic-sizing branch does. One change covers both readers. Normal mode is unaffected because `checkOffHeapSettings` already guarantees the key is set there. ### How was this patch tested? Added `GlutenDriverPluginSuite` with two tests: untracked mode without an off-heap size no longer throws and yields a 0 off-heap budget, and the normal path reads a configured `512m`. The untracked test also asserts `spark.memory.offHeap.size` is readable afterward, which is what the downstream `VeloxListenerApi` path needs. The first test fails on the current code (it throws `NoSuchElementException`) and passes after the fix. Closes #12621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
