[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800082#comment-13800082
]
Chris Nauroth commented on MAPREDUCE-5508:
------------------------------------------
Hello, [~viswaj]. I'm sorry to hear that this bug is causing trouble for you.
It is not committed for 1.2.1. It would be included in the 1.3.0 release.
I recommend against doing what you described: trying to obtain a compiled jar
for a patched 1.2.1 from an individual dev. The problem with this approach is
that you wouldn't know for sure what was included in that jar. This would open
your deployment to security risk, because whoever compiled that jar could have
snuck in a backdoor. Even if you decide to trust a particular Apache dev for
this, there is still the risk of a man-in-the-middle interfering with your
download.
The Apache release process actually provides everything you need to obtain the
official 1.2.1 source, apply additional patches, and then build from source
yourself. Official Apache releases come with the signature of the release
manager and a checksum that you can use to verify the contents. Patches are
publicly posted in jira, Subversion, and mirrored to git. Commits to the
repositories are authenticated, so you know that only Apache committers have
committed the patches. Every committed patch requires agreement from at least
two people: the contributor and at least one other committer.
You can find a mirror for downloading a distro from here:
http://www.apache.org/dyn/closer.cgi/hadoop/common/
For example, here is a direct link to 1.2.1 at one of the mirrors:
http://www.interior-dsgn.com/apache/hadoop/common/hadoop-1.2.1/
After downloading the source, there are multiple patches related to this memory
leak that you'll want to apply, from MAPREDUCE-5351 and MAPREDUCE-5508:
https://issues.apache.org/jira/secure/attachment/12590105/MAPREDUCE-5351-2.patch
https://issues.apache.org/jira/secure/attachment/12590672/MAPREDUCE-5351-addendum-1.patch
https://issues.apache.org/jira/secure/attachment/12604722/MAPREDUCE-5508.3.patch
Then, running "ant jar" ought to be sufficient to build and create the jar that
you need. By following this process, you'll get a much more trustworthy jar
than something compiled and provided by an individual Apache developer.
> JobTracker memory leak caused by unreleased FileSystem objects in
> JobInProgress#cleanupJob
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5508
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5508
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 1-win, 1.2.1
> Reporter: Xi Fang
> Assignee: Xi Fang
> Priority: Critical
> Fix For: 1-win, 1.3.0
>
> Attachments: MAPREDUCE-5508.1.patch, MAPREDUCE-5508.2.patch,
> MAPREDUCE-5508.3.patch, MAPREDUCE-5508.patch
>
>
> MAPREDUCE-5351 fixed a memory leak problem but introducing another filesystem
> object (see "tempDirFs") that is not properly released.
> {code} JobInProgress#cleanupJob()
> void cleanupJob() {
> ...
> tempDirFs = jobTempDirPath.getFileSystem(conf);
> CleanupQueue.getInstance().addToQueue(
> new PathDeletionContext(jobTempDirPath, conf, userUGI, jobId));
> ...
> if (tempDirFs != fs) {
> try {
> fs.close();
> } catch (IOException ie) {
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
