Rajesh Balamohan created TEZ-864:
------------------------------------
Summary: PipelinedSorter throws BufferOverflow exception
Key: TEZ-864
URL: https://issues.apache.org/jira/browse/TEZ-864
Project: Apache Tez
Issue Type: Bug
Affects Versions: 0.3.0
Environment: Hadoop 2.3.0, Hive 0.13, Tez 0.3.0
Reporter: Rajesh Balamohan
When running the following query, BufferOverflowException is thrown at times.
>>
SELECT SUBSTR(sourceIP, 1, 10), SUM(adRevenue) FROM uservisits GROUP BY
SUBSTR(sourceIP, 1, 10)
>>
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.nio.BufferOverflowException
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:287)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.flush(VectorGroupByOperator.java:320)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:249)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:129)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
... 9 more
Caused by: java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:513)
at java.nio.ByteBufferAsIntBufferL.put(ByteBufferAsIntBufferL.java:122)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.collect(PipelinedSorter.java:237)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.write(PipelinedSorter.java:183)
at
org.apache.tez.runtime.library.output.OnFileSortedOutput$1.write(OnFileSortedOutput.java:96)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor$KVOutputCollector.collect(TezProcessor.java:170)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.collect(ReduceSinkOperator.java:364)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:270)
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
