[
https://issues.apache.org/jira/browse/HIVE-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marta Kuczora updated HIVE-24530:
---------------------------------
Description:
During testing a NPE occurred in the FileSinkOperator.closeRecordwriters method.
After investigating, turned out there was an underlaying IOException during
executing the FileSinkOperator.process method. It got caught by the following
code part:
{noformat}
} catch (IOException e) {
closeWriters(true);
throw new HiveException(e);
} catch (SerDeException e) {
closeWriters(true);
throw new HiveException(e);
}
{noformat}
First the closeWriters method was called:
{noformat}
private void closeWriters(boolean abort) throws HiveException {
fpaths.closeWriters(true);
closeRecordwriters(true);
}
private void closeRecordwriters(boolean abort) {
for (RecordWriter writer : rowOutWriters) {
try {
LOG.info("Closing {} on exception", writer);
writer.close(abort);
} catch (IOException e) {
LOG.error("Error closing rowOutWriter" + writer, e);
}
}
{noformat}
If the writers had got closed successfully, a HiveException would have been
thrown with the original IOException.
But when the IOException occurred the writers in the rowOutWriters were not yet
initialised, so a NPE occurred. This was very misleading as the NPE was not the
real issue, but the original IOException was hidden.
> Potential NPE in FileSinkOperator.closeRecordwriters method
> -----------------------------------------------------------
>
> Key: HIVE-24530
> URL: https://issues.apache.org/jira/browse/HIVE-24530
> Project: Hive
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: Marta Kuczora
> Assignee: Marta Kuczora
> Priority: Major
>
> During testing a NPE occurred in the FileSinkOperator.closeRecordwriters
> method.
> After investigating, turned out there was an underlaying IOException during
> executing the FileSinkOperator.process method. It got caught by the following
> code part:
> {noformat}
> } catch (IOException e) {
> closeWriters(true);
> throw new HiveException(e);
> } catch (SerDeException e) {
> closeWriters(true);
> throw new HiveException(e);
> }
> {noformat}
> First the closeWriters method was called:
> {noformat}
> private void closeWriters(boolean abort) throws HiveException {
> fpaths.closeWriters(true);
> closeRecordwriters(true);
> }
> private void closeRecordwriters(boolean abort) {
> for (RecordWriter writer : rowOutWriters) {
> try {
> LOG.info("Closing {} on exception", writer);
> writer.close(abort);
> } catch (IOException e) {
> LOG.error("Error closing rowOutWriter" + writer, e);
> }
> }
> {noformat}
> If the writers had got closed successfully, a HiveException would have been
> thrown with the original IOException.
> But when the IOException occurred the writers in the rowOutWriters were not
> yet initialised, so a NPE occurred. This was very misleading as the NPE was
> not the real issue, but the original IOException was hidden.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
