Rajesh Balamohan created TEZ-3265:
-------------------------------------
Summary: Add preconditions check in SortSpan when available mem is
less than metasize
Key: TEZ-3265
URL: https://issues.apache.org/jira/browse/TEZ-3265
Project: Apache Tez
Issue Type: Bug
Reporter: Rajesh Balamohan
Priority: Minor
{noformat}
2016-05-21 09:01:48,523 [INFO] [TezChild] |impl.PipelinedSorter|: Reducer 3:
reserved.remaining()=14680064, reserved.metasize=16777216
...
...
...
...
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.position(Buffer.java:244)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter$SortSpan.<init>(PipelinedSorter.java:737)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.sort(PipelinedSorter.java:255)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.collect(PipelinedSorter.java:310)
at
org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter.write(PipelinedSorter.java:283)
at
org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput$1.write(OrderedPartitionedKVOutput.java:164)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor$TezKVOutputCollector.collect(TezProcessor.java:198)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.collect(ReduceSinkOperator.java:542)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.process(ReduceSinkOperator.java:386)
... 45 more
{noformat}
It would be good to have a Preconditions check in SortSpan instead of throwing
exception from Buffer.java.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
