About the time I send this I figure something out but I'm wondering if there is
a better way. Setting "set tez.grouping.split-count=1;" makes the error go away.
On 6/7/19, 7:09 PM, "Shawn Weeks" <swe...@weeksconsulting.us> wrote:
Hi, I maintain a few docker containers for testing Hive and I’m running
into an issue with the release of Hive 3.1 distributed with HDP 3.1.0. I’m
assuming it’s something to do with running Tez and Hive in local mode but all
I’ve found on Jira are a few unresolved tickets.
Does anyone know of a config change that will get around this or should I
submit a Jira?
https://github.com/shawnweeks/hive_docker/tree/HDP-3.1.0.0-78
Thanks
Shawn
Error: Error while processing statement: FAILED: Execution Error, return
code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
vertexName=Map 1, vertexId=vertex_1559952000236_0001_12_00, diagnostics=[Vertex
vertex_1559952000236_0001_12_00 [Map 1] killed/failed due
to:ROOT_INPUT_INIT_FAILURE, Vertex Input: test2 initializer failed,
vertex=vertex_1559952000236_0001_12_00 [Map 1],
java.lang.IllegalArgumentException: Illegal Capacity: -1547
at java.util.ArrayList.<init>(ArrayList.java:157)
at
org.apache.tez.mapreduce.grouper.TezSplitGrouper.getGroupedSplits(TezSplitGrouper.java:189)
at
org.apache.hadoop.mapred.split.TezMapredSplitsGrouper.getGroupedSplits(TezMapredSplitsGrouper.java:81)
at
org.apache.hadoop.hive.ql.exec.tez.SplitGrouper.group(SplitGrouper.java:91)
at
org.apache.hadoop.hive.ql.exec.tez.SplitGrouper.generateGroupedSplits(SplitGrouper.java:193)
at
org.apache.hadoop.hive.ql.exec.tez.SplitGrouper.generateGroupedSplits(SplitGrouper.java:160)
at
org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:287)
at
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
at
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
at
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
at
com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
at
com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
at
com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
