This is an automated email from the ASF dual-hosted git repository.
yuanzhou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new f643e06ff9 [GLUTEN-9344][VL] Document dynamic offheap sizing feature
(#9391)
f643e06ff9 is described below
commit f643e06ff9f0494de378a2733894f23e3e382930
Author: Yuan <[email protected]>
AuthorDate: Sat Aug 23 19:12:33 2025 +0100
[GLUTEN-9344][VL] Document dynamic offheap sizing feature (#9391)
this patch adds documentation for dynamic off-heap sizing feature
fixes: #9344
---------
Signed-off-by: Yuan <[email protected]>
---
docs/developers/VeloxDynamicSizingOffheap.md | 33 ++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/docs/developers/VeloxDynamicSizingOffheap.md
b/docs/developers/VeloxDynamicSizingOffheap.md
new file mode 100644
index 0000000000..e6ff3b399d
--- /dev/null
+++ b/docs/developers/VeloxDynamicSizingOffheap.md
@@ -0,0 +1,33 @@
+---
+layout: page
+title: VeloxDynamicSizingOffheap
+nav_order: 16
+parent: Developer Overview
+---
+
+## Dynamic Off-heap Sizing
+Gluten requires setting both on-heap and off-heap memory sizes, which
initializes different memory layouts. Improper configuration of these settings
can lead to lower performance.
+
+To fix this issue, dynamic off-heap sizing is an experimental feature designed
to simplify this process. When enabled, off-heap settings are ignored, and
Velox uses the on-heap size as the memory size.
+## Detail implementations
+To understand the details, it's essential to learn the basics of JVM memory
management. There are many resources discussing JVM internals:
+- https://exia.dev/blog/2019-12-10/JVM-Memory-Model/
+-
https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/garbage_collect.html
+- https://www.scaler.com/topics/memory-management-in-java/
+-
https://developers.redhat.com/articles/2021/09/09/how-jvm-uses-and-allocates-memory#
+
+In general, the feature works as follows:
+
+- Spark first attempts to allocate memory based on the on-heap size. Note that
the maximum memory size is controlled by `spark.executor.memory`.
+- When Velox tries to allocate memory, Gluten attempts to allocate from system
memory and records this in the memory allocator.
+- If there is sufficient memory, allocations proceed normally.
+- If memory is insufficient, Spark performs garbage collection (GC) to free
on-heap memory, allowing Velox to allocate memory.
+- If memory remains insufficient after GC, Spark reports an out-of-memory
(OOM) issue.
+
+We then enforce a total memory quota, calculated as the sum of committed and
in-use memory in the Java heap (using `Runtime.getRuntime().totalMemory()`)
plus tracked off-heap memory in `TreeMemoryConsumer`. If an allocation exceeds
this total committed memory, the allocation fails and triggers an OOM.
+
+With this change, the "quota check" is performed when Gluten receives an
memory allocation request. In practice, this means the Java codebase can
oversubscribe memory within the on-heap quota, even if off-heap usage is
sufficient to fail the allocation.
+
+## Limitations
+
+This feature is in the preliminary stages of development and will be improved
in future updates.
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
