[
https://issues.apache.org/jira/browse/MAPREDUCE-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409685#comment-16409685
]
Jason Lowe commented on MAPREDUCE-7068:
---------------------------------------
Errors are much more severe than Exceptions. If we're getting something like
OOMError then that's arguably more important than whatever exception occurred
in the try block. If we're worred about lack of logging of the error, in
practice the YarnUncaughtExceptionHandler is going to catch this anyway
(installed by YarnChild when the task runs).
> Fix Reduce Exception was overwrited by ReduceTask
> -------------------------------------------------
>
> Key: MAPREDUCE-7068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
> Affects Versions: 2.7.1
> Environment: CentOS 7
> Hadoop-2.7.1
> Hive-1.2.1
> Reporter: tartarus
> Priority: Major
> Attachments: MAPREDUCE_7068.patch
>
>
>
> {code:java}
> try {
> //increment processed counter only if skipping feature is enabled
> boolean incrProcCount = SkipBadRecords.getReducerMaxSkipGroups(job)>0 &&
> SkipBadRecords.getAutoIncrReducerProcCount(job);
>
> ReduceValuesIterator<INKEY,INVALUE> values = isSkipping() ?
> new SkippingReduceValuesIterator<INKEY,INVALUE>(rIter,
> comparator, keyClass, valueClass,
> job, reporter, umbilical) :
> new ReduceValuesIterator<INKEY,INVALUE>(rIter,
> comparator, keyClass, valueClass,
> job, reporter);
> values.informReduceProgress();
> while (values.more()) {
> reduceInputKeyCounter.increment(1);
> reducer.reduce(values.getKey(), values, collector, reporter);
> if(incrProcCount) {
> reporter.incrCounter(SkipBadRecords.COUNTER_GROUP,
> SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS, 1);
> }
> values.nextKey();
> values.informReduceProgress();
> }
> reducer.close();
> reducer = null;
>
> out.close(reporter);
> out = null;
> } finally {
> IOUtils.cleanupWithLogger(LOG, reducer);
> closeQuietly(out, reporter);
> }
> }
> {code}
> if {color:#d04437}reducer.close();{color} throw Exception ,
> {color:#d04437}reducer = null;{color} will not run, then
> {color:#d04437}IOUtils.cleanupWithLogger(LOG, reducer); {color}
>
> will throw Exception and overwrite the Exception of reducer.close();
> so we should catch it and print log to help targeting issues
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
