[
https://issues.apache.org/jira/browse/TEZ-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510615#comment-14510615
]
Gopal V commented on TEZ-2348:
------------------------------
[~sseth]: Admittedly, I'd like it to throw exceptions for all case of bad usage
- not just this.
The getCurrentKey() etc, is what makes this different from the regular
iterators - I'd really like to be able to trip up any user who calls next()
twice without checking, because the key/values might return stale objects.
Nobody should be doing it, but if anyone is using the readers badly, they
should find out quickly & without involving a support case - the Exception has
to be part of the contract for all readers for user-safety.
That thought is coming purely from a "debug a hang on 350 nodes" issue where
the Tez team has to debug a user usage issue, so that detecting this sort of
problem becomes a self-service task for the programmer working on
Hive/Pig/Cascading etc instead of involving the Tez internals folks.
> EOF exception during UnorderedKVReader.next()
> ---------------------------------------------
>
> Key: TEZ-2348
> URL: https://issues.apache.org/jira/browse/TEZ-2348
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.5.2
> Reporter: Jason Dere
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2348.1.patch, TEZ-2348.2.patch, TEZ-2348.3.patch,
> _tez_session_dir.tgz
>
>
> {noformat}
> Caused by: java.lang.RuntimeException: java.io.IOException: Reached EOF.
> Completed reading 516605
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:278)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:184)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
> ... 13 more
> Caused by: java.io.IOException: Reached EOF. Completed reading 516605
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile.checkState(IFile.java:817)
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.positionToNextRecord(IFile.java:698)
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.readRawKey(IFile.java:731)
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.nextRawKey(IFile.java:727)
> at
> org.apache.tez.runtime.library.common.readers.UnorderedKVReader.readNextFromCurrentReader(UnorderedKVReader.java:151)
> at
> org.apache.tez.runtime.library.common.readers.UnorderedKVReader.next(UnorderedKVReader.java:112)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$KeyValuesFromKeyValue.next(ReduceRecordSource.java:439)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:232)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
