[
https://issues.apache.org/jira/browse/MAPREDUCE-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409561#comment-16409561
]
Jason Lowe commented on MAPREDUCE-7068:
---------------------------------------
Is this reported against 2.7 or something else? IOUtils.cleanupWithLogger
doesn't exist in 2.7.
I'm also not seeing how the finally block is smashing an exception with its
own, since cleanupWithLogger looks like this:
{code}
public static void cleanupWithLogger(Logger logger,
java.io.Closeable... closeables) {
for (java.io.Closeable c : closeables) {
if (c != null) {
try {
c.close();
} catch (Throwable e) {
if (logger != null) {
logger.debug("Exception in closing {}", c, e);
}
}
}
}
}
{code}
and closeQuietly similarly suppresses all Exceptions.
> Fix Reduce Exception was overwrited by ReduceTask
> -------------------------------------------------
>
> Key: MAPREDUCE-7068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
> Affects Versions: 2.7.1
> Environment: CentOS 7
> Hadoop-2.7.1
> Hive-1.2.1
> Reporter: tartarus
> Priority: Major
> Attachments: MAPREDUCE_7068.patch
>
>
>
> {code:java}
> try {
> //increment processed counter only if skipping feature is enabled
> boolean incrProcCount = SkipBadRecords.getReducerMaxSkipGroups(job)>0 &&
> SkipBadRecords.getAutoIncrReducerProcCount(job);
>
> ReduceValuesIterator<INKEY,INVALUE> values = isSkipping() ?
> new SkippingReduceValuesIterator<INKEY,INVALUE>(rIter,
> comparator, keyClass, valueClass,
> job, reporter, umbilical) :
> new ReduceValuesIterator<INKEY,INVALUE>(rIter,
> comparator, keyClass, valueClass,
> job, reporter);
> values.informReduceProgress();
> while (values.more()) {
> reduceInputKeyCounter.increment(1);
> reducer.reduce(values.getKey(), values, collector, reporter);
> if(incrProcCount) {
> reporter.incrCounter(SkipBadRecords.COUNTER_GROUP,
> SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS, 1);
> }
> values.nextKey();
> values.informReduceProgress();
> }
> reducer.close();
> reducer = null;
>
> out.close(reporter);
> out = null;
> } finally {
> IOUtils.cleanupWithLogger(LOG, reducer);
> closeQuietly(out, reporter);
> }
> }
> {code}
> if {color:#d04437}reducer.close();{color} throw Exception ,
> {color:#d04437}reducer = null;{color} will not run, then
> {color:#d04437}IOUtils.cleanupWithLogger(LOG, reducer); {color}
>
> will throw Exception and overwrite the Exception of reducer.close();
> so we should catch it and print log to help targeting issues
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
