[
https://issues.apache.org/jira/browse/MAPREDUCE-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706018#comment-14706018
]
Chris Douglas commented on MAPREDUCE-6434:
------------------------------------------
Thanks for updating the patch, [~augustorsouza]. Could you check this change?
{noformat}
- committer = new FileOutputCommitter(output, context);
+ try {
+ if (context.getConfiguration().getBoolean(MRJobConfig.TASK_PREEMPTION,
+ false)
+ && context.getReducerClass()
+ .isAnnotationPresent(Checkpointable.class)) {
+ committer = new PartialFileOutputCommitter(output, context);
+ } else {
+ committer = new FileOutputCommitter(output, context);
+ }
+ } catch (ClassNotFoundException c) {
+ throw new RuntimeException(
+ "Internal error: reducer class is not defined ", c);
+ }
{noformat}
Since preemption in MAPREDUCE-5269 only supports reduce tasks, even if
preemption is enabled for map-only jobs, the reduce class can be undefined.
> Add support for PartialFileOutputCommiter when checkpointing is an option
> during preemption
> -------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6434
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6434
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Augusto Souza
> Assignee: Augusto Souza
> Attachments: MAPREDUCE-6434.001.patch, MAPREDUCE-6434.002.patch,
> MAPREDUCE-6434.003.patch, MAPREDUCE-6434.004.patch, MAPREDUCE-6434.005.patch
>
>
> Finish up some renaming work related to the annotation @Preemptable (it
> should be @Checkpointable now) and help in the splitting of patch in
> MAPREDUCE-5269 that is too large for being reviewed or accepted by Jenkins CI
> scripts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
