Sahana Bhat created HIVE-24406:
----------------------------------
Summary: HiveConf creation in HiveMetastore's getMS() call adds
~100ms latency
Key: HIVE-24406
URL: https://issues.apache.org/jira/browse/HIVE-24406
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 2.3.4
Reporter: Sahana Bhat
Attachments: Screenshot 2020-11-17 at 5.17.50 PM.png
The changes to HMSHandler's getMSForConf introduced in the commit
[https://github.com/apache/hive/commit/9a47cf9f92d1c8a4e72890e3dfe2d9567f12bfb5]
makes getMSForConf and newRawStoreForConf static and hence adds an additional
HiveConf object creation step for every metastore connection created.
In a client like Presto, which creates a new metastore connection for every
query, the object creation rate shoots up leading to higher heap usage, more
frequent garbage collection, higher garbage collection times and increased
latency in the service. We noticed a constant ~100ms increase in latency of all
metastore calls during migration of HMS from 1x to 2x for Presto.
PFA the latency difference of a get_table call between 1x and 2x.
!Screenshot 2020-11-17 at 5.17.50 PM.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
