[
https://issues.apache.org/jira/browse/MAPREDUCE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768236#action_12768236
]
Iyappan Srinivasan commented on MAPREDUCE-947:
----------------------------------------------
+1 from QA for mr-947-y20-new.patch.
1) When mapreduce.fileoutputcomitter.marksuccessfuljobs parm is set to false
a) Successful jobs passes normally without creating a _SUCCESS file in the
output directory.
b) Failed jobs fails normally without creating any file in the output directory.
2) When mapreduce.fileoutputcomitter.marksuccessfuljobs is set to true, then
a) Successful jobs passes and creates _SUCCESS under output directory.
b) Failed jobs, which are running midway, does not create _ SUCCESS when killed
c) Failed jobs, when doing their cleanup does not create _SUCCESS file when
killed.
d) Failed jobs, when doing setup, does not create any output directory and so
does not create any file under it.
3) After JT restart, and When mapreduce.fileoutputcomitter.marksuccessfuljobs
is set to true, then
a) Successful jobs passes and creates _ SUCCESS under output directory.
b) Failed jobs, which are running midway, does not create _ SUCCESS when killed
c) Failed jobs, when doing setup, does not create any output directory and so
does not create any file under it.
4) Run 100 jobs and kill/fail some jobs in the middle. The killed/failed ones
should fail properly and no string displayed. The successful jobs should
display success.
5) Run randomwriter and sort. Both should work properly.
> OutputCommitter should have an abortJob method
> ----------------------------------------------
>
> Key: MAPREDUCE-947
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-947
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.21.0
> Reporter: Owen O'Malley
> Assignee: Amar Kamat
> Fix For: 0.22.0
>
> Attachments: mapred-948-v1.12-branch-0.20-internal.patch,
> mapred-948-v1.12.patch, mapred-948-v1.13-branch-0.20-internal.patch,
> mapred-948-v1.2.patch, mapred-948-v1.3.patch, mapred-948-v1.4.patch,
> mapred-948-v1.7.patch, mapred-948-v2.1-branch-0.20.patch,
> mapred-948-v2.3-branch-0.20.patch, mapred-948-v2.3.patch,
> mapred-948-v3.1.patch, mapred-948-v3.2.patch, mapred-948-v3.4.patch,
> mr-947-trunk-new.patch, mr-947-trunk-new.patch, mr-947-trunk.patch,
> mr-947-trunk.patch, mr-947-trunk.patch, mr-947-y20-new.patch, mr-947-y20.patch
>
>
> The OutputCommitter needs an abortJob method to clean up from failed jobs.
> Currently there is no way to distinguish between failed or succeeded jobs,
> making it impossible to write output promotion code.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
