[
https://issues.apache.org/jira/browse/YARN-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-5341:
-----------------------------
Summary: DefaultMetricsSystem leaks the source name when a source
unregisters (was: Nodemanager leaks ContainerResource names in UniqueNames for
each container)
This looks like a problem in DefaultMetricsSystem. There are two UniqueNames
instances in that class, mBeanNames and sourceNames. When a ContainerMetrics
source unregisters the names are getting cleaned out properly from mBeanNames
but not from sourceNames. Each source will register a new name in sourceNames,
but the only way to clear them out is by shutting down the DefaultMetricsSystem
instance. There needs to be a way to remove elements from sourceNames when a
source unregisters.
I updated the summary based on this analysis, and I'll move the JIRA to the
HADOOP project since that's where the fix is needed.
> DefaultMetricsSystem leaks the source name when a source unregisters
> --------------------------------------------------------------------
>
> Key: YARN-5341
> URL: https://issues.apache.org/jira/browse/YARN-5341
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.7.2
> Reporter: Jason Lowe
> Priority: Critical
>
> Ran across a nodemanager that was spending most of its time in GC. Upon
> examination of the heap most of the memory was going to the map of names in
> org.apache.hadoop.metrics2.lib.UniqueNames. In this case the map had almost
> 2 million entries. Looking at a few of the map showed entries like
> "ContainerResource_container_e01_1459548490386_8560138_01_002020",
> "ContainerResource_container_e01_1459548490386_2378745_01_000410", etc.
> Looks like the ContainerMetrics for each container will cause a unique name
> to be registered with UniqueNames and the name will never be unregistered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
