[
https://issues.apache.org/jira/browse/MAPREDUCE-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012436#comment-16012436
]
Jason Lowe commented on MAPREDUCE-6889:
---------------------------------------
Seems like a reasonable approach. It would be good to have a unit test that
verifies closing the job closes the cluster (or at least the client protocol
provider). Also should cluster be set to null after we call close on it?
Also this change alone isn't sufficient to fix the problem described in the
JIRA summary. Callers would have to be updated to invoke the new close method
at the appropriate time, so the headline is a bit misleading if we marked this
as fixed.
> FileSystem leak when Job#submit() used when ATS1.5 enabled
> ----------------------------------------------------------
>
> Key: MAPREDUCE-6889
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6889
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Attachments: MAPREDUCE-6889.0001.patch
>
>
> ATS1.5 uses FileSystemTimelineWriter which creates FS object on every writer
> initialization. If writer is not closed, then there is possibility of OOM see
> YARN-5438 fixes closing FS object.
> TimelineClient is used by YarnClient. So all the user who uses YarnClient
> with ATS1.5 need to stop service properly. Otherwise there is big chance of
> FS object leak.
> Of course MR uses YARN client submit job. If MR do not stop YarnClient then
> there is FS object leak.
> JobClient provides a API to stop all these service using *JobClient#close*.
> But many MR clients uses *Job* object to submit a job. But do not stop
> started services by default. So, Job class should provide a API to close the
> services very similar to JobClient#close.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
