[
https://issues.apache.org/jira/browse/MAPREDUCE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13674365#comment-13674365
]
Jason Lowe commented on MAPREDUCE-5267:
---------------------------------------
Please don't hesitate to contribute because a unit test is the problem. A
patch that fixes the problem without a unit test is better than no patch at
all, and often someone can come along and either add the unit tests themselves
or provide suggestions on how to test.
In this particular case, we might be able to use a mock/spied FileContext to
achieve what we want. We can't easily inject this new FileContext given the
way HistoryFileManager is written, but a minor change can make it so. For
example we can refactor the code that calls
FileContext.getFileContext(Configuration) to call a protected method like
getFileContext(Configuration) that simply delegates to
FileContext.getFileContext(Configuration). Now we have a method we can
override in the unit test to force the HistoryFileManager to pick up a
mocked/spied FileContext that can do what we want.
> History server should be more robust when cleaning old jobs
> -----------------------------------------------------------
>
> Key: MAPREDUCE-5267
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5267
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver
> Affects Versions: 0.23.7, 2.0.4-alpha
> Reporter: Jason Lowe
>
> Ran across a situation where an admin user had accidentally created a
> directory in one of the date directories under /mapred/history/done/ that was
> not readable by the historyserver user. That effectively prevented the
> history server from cleaning any jobs from that date forward, as it hit an
> IOException trying to scan the directory and that aborted the entire clean
> process.
> The history server should localize IOException handling to the directory/file
> being processed and move on to the next entry in the list rather than
> aborting the entire cleaning process.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
