[
https://issues.apache.org/jira/browse/HIVE-24711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277860#comment-17277860
]
LinZhongwei commented on HIVE-24711:
------------------------------------
yes. Following messages are from hivemetastore.log. But in my hdp version
hive, I can not find metastore.housekeeping.threads.on in the config dir and
ambari web . I use 'grep -R -i "metastore.housekeeping.threads.on"'.
Because when I turned off 'compactor.Initiator' , PartitionDiscoveryTask logs
disappeared. I think when I turned off compactor.Initiator,
'metastore.housekeeping.threads.on' was also turned off. I will try to set it
on hive, and restart hive metastore.
2021-02-03T16:56:21,709 ERROR [PartitionDiscoveryTask-2]:
metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(197)) -
MetaException(message:java.security.AccessControlException: Permission denied:
user=hive, access=WRITE,
inode="/apps/edl_cn/staging/edl_cn.PAYMENT_EVENT_DELTA_incremental/etl_run_id=20200625005959":gp_etl_edl_batch:gp_etl_edl_batch:drwxr-xr-x
> hive metastore memory leak
> --------------------------
>
> Key: HIVE-24711
> URL: https://issues.apache.org/jira/browse/HIVE-24711
> Project: Hive
> Issue Type: Bug
> Components: Hive, Metastore
> Affects Versions: 3.1.0
> Reporter: LinZhongwei
> Priority: Major
>
> hdp version:3.1.5.31-1
> hive version:3.1.0.3.1.5.31-1
> hadoop version:3.1.1.3.1.5.31-1
> We find that the hive metastore has memory leak if we set
> compactor.initiator.on to true.
> If we disable the configuration, the memory leak disappear.
> How can we resolve this problem?
> Even if we set the heap size of hive metastore to 40 GB, after 1 month the
> hive metastore service will be down with outofmemory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
