[
https://issues.apache.org/jira/browse/YARN-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396150#comment-15396150
]
Rohith Sharma K S commented on YARN-5438:
-----------------------------------------
bq. Since the filesystem cache will implicitly link what looks like two
separate creations of a filesystem to a single instance, closing one will break
any subsequent use of the other.
If the user creates file system object using api {{FileSystem#newInstance}}
with in the JVM then always new *FS* object is given. For every *newInstance*
api call, object created using the combination of URI, Conf and *UniqueKey*. If
FS object is created using {{FS#get}} then this api search from cache. This API
always creates object with combination of URI and CONF only. So mainly it
matters how the FS object is being created.
Basically closing one instance which is created using
{{FileSystem#newInstance}} should not affect other FS object which is created
using {{FS#get}}. And also note that if two FS objects are created using
{{FS#get}} then closing one will definitely affect other FS object.
> TimelineClientImpl leaking FileSystem Instances causing Long running services
> like HiverServer2 daemon going OOM
> ----------------------------------------------------------------------------------------------------------------
>
> Key: YARN-5438
> URL: https://issues.apache.org/jira/browse/YARN-5438
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
> Affects Versions: 2.8.0, 2.7.3
> Reporter: Karam Singh
> Assignee: Rohith Sharma K S
> Attachments: YARN-5438.0.patch
>
>
> TimelineClientImpl leaking FileSystem Instances causing Long running services
> like HiverServer2 daemon going OOM
> In org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl,
> FileSystem.newInstance is invoked and is not closed. Causing over time
> Filesystem instances getting accumulated in long runninh Client (like
> Hiveserver2), finally causing them to OOM
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
