[
https://issues.apache.org/jira/browse/MAPREDUCE-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012632#comment-16012632
]
Jason Lowe commented on MAPREDUCE-6889:
---------------------------------------
I don't know the full history behind the newInstance call. Some software
archaeology shows the newInstance change was added during the review rework
process for YARN-4696, but even before that the code was explicitly setting the
conf key to disable caching for the filesystem. The disabling of the
filesystem cache was done partway through the review on YARN-4234 but I could
not find a discussion that explained why that addition to the patch was
necessary. Pinging [~xgong] and [~gtCarrera9] to see if they know the logic
behind avoiding the fs cache in FileSystemTimelineWriter.
> Provide an API Job#close to shutdown MR client services.
> --------------------------------------------------------
>
> Key: MAPREDUCE-6889
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6889
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Attachments: MAPREDUCE-6889.0001.patch
>
>
> ATS1.5 uses FileSystemTimelineWriter which creates FS object on every writer
> initialization. If writer is not closed, then there is possibility of OOM see
> YARN-5438 fixes closing FS object.
> TimelineClient is used by YarnClient. So all the user who uses YarnClient
> with ATS1.5 need to stop service properly. Otherwise there is big chance of
> FS object leak.
> Of course MR uses YARN client submit job. If MR do not stop YarnClient then
> there is FS object leak.
> JobClient provides a API to stop all these service using *JobClient#close*.
> But many MR clients uses *Job* object to submit a job. But do not stop
> started services by default.
> So, provide a API *Job#close* to shutdown MR client services. This API can be
> utilized by caller explicitly to stop service which avoids FS leak.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
