abstractdog commented on code in PR #6955:
URL: https://github.com/apache/hive/pull/6955#discussion_r4599339897
##########
ql/src/java/org/apache/hadoop/hive/ql/io/RecordReaderWrapper.java:
##########
@@ -72,14 +71,9 @@ static RecordReader create(InputFormat inputFormat,
HiveInputFormat.HiveInputSpl
int headerCount = Utilities.getHeaderCount(tableDesc);
int footerCount = Utilities.getFooterCount(tableDesc, jobConf);
- RecordReader innerReader = null;
- try {
- innerReader = inputFormat.getRecordReader(split.getInputSplit(), jobConf,
reporter);
- } catch (InterruptedIOException iioe) {
- // If reading from the underlying record reader is interrupted, return a
no-op record reader
- LOG.info("Interrupted while getting the input reader for {}",
split.getInputSplit());
- return new ZeroRowsInputFormat().getRecordReader(split.getInputSplit(),
jobConf, reporter);
- }
+ // An interrupt propagates: HiveInputFormat turns it into an empty split
only once the limit is
+ // reached, and otherwise the task must fail rather than finish with this
split's rows missing.
+ RecordReader innerReader =
inputFormat.getRecordReader(split.getInputSplit(), jobConf, reporter);
Review Comment:
removing this is safe: `HiveInputFormat.getRecordReader` is the only caller
of `RecordReaderWrapper.create`, so every InterruptedIOException now reaches
the HIVE-25751 handler, which uses `JavaUtils.isInterrupt`, so what can happen
is:
- Limit reached and the error is an interrupt (including
InterruptedIOException): NullRowsRecordReader, the case the qtests needed. Same
outcome as before.
- Limit not reached: creating the reader fails. For a task that's being
killed this changes nothing about results, only that it fails with an exception
instead of finishing with zero rows.
- SocketTimeoutException: always fails the task, as an I/O error should.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]