[
https://issues.apache.org/jira/browse/HIVE-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359603#comment-15359603
]
Naveen Gangam commented on HIVE-13749:
--------------------------------------
Have a new heap dump from a customer that has been running for an extended
period of time, with the following properties removed.
{code}
<property><name>hive.compactor.initiator.on</name><value>true</value></property>
<property><name>hive.compactor.worker.threads</name><value>1</value></property>
{code}
The leak from the above compactor thread was later addressed in HIVE-13151 that
the customer was not running with. So I had them remove these properties.
HMS has been stable, no OOM crashes yet. There are about 214 instances of
Configuration occupying 66MB of heap space, roughly 18% of the total active
heap. Each of this instance was from an individual thread holding on to the
object in the threadlocals. I think we can save this unnecessary reference and
do better by cleaning up the threadlocal before the thread is returning to the
pool.
I have uploaded a new patch that does just that. Let me know of any thoughts.
Thanks
> Memory leak in Hive Metastore
> -----------------------------
>
> Key: HIVE-13749
> URL: https://issues.apache.org/jira/browse/HIVE-13749
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 1.1.0
> Reporter: Naveen Gangam
> Assignee: Naveen Gangam
> Attachments: HIVE-13749.1.patch, HIVE-13749.patch, Top_Consumers7.html
>
>
> Looking a heap dump of 10GB, a large number of Configuration objects(> 66k
> instances) are being retained. These objects along with its retained set is
> occupying about 95% of the heap space. This leads to HMS crashes every few
> days.
> I will attach an exported snapshot from the eclipse MAT.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
