[
https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695864#comment-13695864
]
Sandy Ryza commented on MAPREDUCE-5351:
---------------------------------------
The motivation for the changes in JIP is that we don't want to close the
FileSystem and then immediately reopen it from the CleanupQueue code. Just
closing the FS instance (not doing closeAllForUGI) should be fine.
I'll upload a patch today or tomorrow.
> JobTracker memory leak caused by CleanupQueue reopening FileSystem
> ------------------------------------------------------------------
>
> Key: MAPREDUCE-5351
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 1.1.2
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Priority: Critical
> Attachments: MAPREDUCE-5351.patch
>
>
> When a job is completed, closeAllForUGI is called to close all the cached
> FileSystems in the FileSystem cache. However, the CleanupQueue may run after
> this occurs and call FileSystem.get() to delete the staging directory, adding
> a FileSystem to the cache that will never be closed.
> People on the user-list have reported this causing their JobTrackers to OOME
> every two weeks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
