[
https://issues.apache.org/jira/browse/HIVE-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15321265#comment-15321265
]
Naveen Gangam commented on HIVE-13749:
--------------------------------------
[~thejas] After disabling the compactor.Initiator and the Compactor threads,
(because this customer is not using the fix from HIVE-13151), there appear to
be no more leaks.
However, there are still about 400 instances of Configuration objects in memory
(about 80MB of retained objects, 12% in this case), about 11 of them from
static initializers in *Writable classes and the remaining of them stashed in
thread locals, 1 per thread. So HMS roughly has 390 threads, each has 1
instance of Configuration set in its threadlocals. These references should be
re-set when the thread gets re-assigned but they would be retained until this
occurs. Would it make sense to do this cleanup sooner. Something like this
{code}
try {
ms.shutdown();
} finally {
threadLocalConf.remove();
threadLocalMS.remove();
{code}
As always, thank you for your input in advance.
> Memory leak in Hive Metastore
> -----------------------------
>
> Key: HIVE-13749
> URL: https://issues.apache.org/jira/browse/HIVE-13749
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 1.1.0
> Reporter: Naveen Gangam
> Assignee: Naveen Gangam
> Attachments: HIVE-13749.patch, Top_Consumers7.html
>
>
> Looking a heap dump of 10GB, a large number of Configuration objects(> 66k
> instances) are being retained. These objects along with its retained set is
> occupying about 95% of the heap space. This leads to HMS crashes every few
> days.
> I will attach an exported snapshot from the eclipse MAT.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
