ConfX created HIVE-27520:
----------------------------
Summary: Out of Memory when set hive.hashtable.initialCapacity to
a large number
Key: HIVE-27520
URL: https://issues.apache.org/jira/browse/HIVE-27520
Project: Hive
Issue Type: Bug
Reporter: ConfX
Attachments: reproduce.sh
h2. What happened:
When setting {{hive.hashtable.initialCapacity}} to a large number, Hive throws
an out-of-memory exception due to inappropriate checking and handling.
*There is no checking or error-handling logic for this parameter at all.*
h2. StackTrace:
{noformat}
java.lang.IllegalArgumentException: Illegal initial capacity: -923553553
at java.base/java.util.HashMap.<init>(HashMap.java:445)
at
org.apache.hadoop.hive.ql.exec.persistence.HashMapWrapper.<init>(HashMapWrapper.java:89)
at
org.apache.hadoop.hive.ql.exec.persistence.HashMapWrapper.<init>(HashMapWrapper.java:81)
at
org.apache.hadoop.hive.ql.exec.vector.mapjoin.MapJoinTestConfig.createMapJoin(MapJoinTestConfig.java:837)
at
org.apache.hadoop.hive.ql.exec.vector.mapjoin.MapJoinTestConfig.createMapJoinImplementation(MapJoinTestConfig.java:988)
at
org.apache.hadoop.hive.ql.exec.vector.mapjoin.MapJoinTestConfig.createMapJoinImplementation(MapJoinTestConfig.java:971)
{noformat}
h2. Reproduce:
(1) Set {{hive.hashtable.initialCapacity}} to a large value, e.g., 1223962862
(2) Run a simple test that exercises this parameter, e.g.
{{org.apache.hadoop.hive.ql.exec.vector.mapjoin.TestMapJoinOperator#testMultiKey3}}
For an easy reproduction, run the reproduce.sh in the attachment.
We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
