This is an automated email from the ASF dual-hosted git repository.
zabetak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new c126422a91b HIVE-27695: HIVE-26828: Intermittent OOM when running
TestMiniTezCliDriver (Stamatis Zampetakis reviewed by Ayush Saxena)
c126422a91b is described below
commit c126422a91be695c75ec4a750638a0aa4d1ba6cd
Author: Stamatis Zampetakis <[email protected]>
AuthorDate: Wed Oct 11 12:17:23 2023 +0200
HIVE-27695: HIVE-26828: Intermittent OOM when running TestMiniTezCliDriver
(Stamatis Zampetakis reviewed by Ayush Saxena)
java.lang.OutOfMemoryError: GC overhead limit exceeded is thrown by the Tez
Application Master (AM) cause the current heap size (128MB) is not enough to
accommodate the needs of multiple Tez containers running.
Each running container requires roughly 10MB of memory in the AM. The AM
accumulates/manipulates multiple configuration objects (some of them retaining
as much as 1MB of heap) per container.
The heap gradually becomes full and GC is spending a lot of CPU time to
clean things up without really making much progress since containers are reused
and heap cannot shrink.
There are multiple solutions to the problem but the easiest and most
effective is to increase the heap size for the AM. At this point in time, 512MB
is a good value.
The hybridgrace_hashjoin_2.q test which was failing to due OOM can now be
re-enabled.
Close apache/hive#4792
---
data/conf/tez/tez-site.xml | 2 +-
ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q | 1 -
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/data/conf/tez/tez-site.xml b/data/conf/tez/tez-site.xml
index ba4df319a55..88adb6a57e8 100644
--- a/data/conf/tez/tez-site.xml
+++ b/data/conf/tez/tez-site.xml
@@ -1,7 +1,7 @@
<configuration>
<property>
<name>tez.am.resource.memory.mb</name>
- <value>128</value>
+ <value>512</value>
</property>
<property>
<name>tez.task.resource.memory.mb</name>
diff --git a/ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q
b/ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q
index 93b8c13a49f..6ed771ba685 100644
--- a/ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q
+++ b/ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q
@@ -1,7 +1,6 @@
--! qt:dataset:srcpart
--! qt:dataset:src1
--! qt:dataset:src
---! qt:disabled:HIVE-26820 Disable hybridgrace_hashjoin_2.q flaky test
set hive.mapred.mode=nonstrict;
set hive.explain.user=false;
-- Hybrid Grace Hash Join
