Mohammad Kamrul Islam created MAPREDUCE-5812:
------------------------------------------------
Summary: Make task context available to
OutputCommitter.isRecoverySupported()
Key: MAPREDUCE-5812
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5812
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.3.0
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Background
==========
The system like Hive provides its version of OutputCommitter. The custom
implementation of isRecoverySupported() requires task context. From
taskContext:getConfiguration(), hive checks if hive-defined specific property
is set or not. Based on the property value, it returns true or false. However,
in the current OutputCommitter:isRecoverySupported(), there is no way of
getting task config. As a result, user can't turn on/off the MRAM recovery
feature.
Proposed resolution:
===============
1. Pass Task Context into isRecoverySupported() method.
Pros: Easy and clean
Cons: Possible backward compatibility issue due to aPI changes. (Is it true?)
2. Call outputCommitter.setupTask(taskContext) from MRAM: The new
OutputCommitter will store the context in the class level variable and use it
from isRecoverySupported()
Props: No API changes. No backward compatibility issue. This call can be made
from MRAppMaster.getOutputCommitter() method for old API case.
Cons: Might not be very clean solution due to class level variable.
Please give your comments.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
