[
https://issues.apache.org/jira/browse/HIVE-25074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17335262#comment-17335262
]
Miklos Szurap commented on HIVE-25074:
--------------------------------------
With PERFLOG enabled from a Production environment:
{noformat}
$ cat $(ls -1trx hadoop-cmf-hive-HIVEMETASTORE-*.log.out*) | grep -P "/PERFLOG
method=flushCache" | wc -l
119734
$ cat $(ls -1trx hadoop-cmf-hive-HIVEMETASTORE-*.log.out*) | grep -P "/PERFLOG"
| wc -l
1196686
{noformat}
and the number of 1+ second executions for the flushCache were ~23% during the
observed high load situation:
{noformat}
$ cat $(ls -1trx hadoop-cmf-hive-HIVEMETASTORE-*.log.out*) | grep -P "/PERFLOG
method=flushCache.*duration=[0-9]{4}" | wc -l
28648
{noformat}
> Remove Metastore flushCache usage
> ---------------------------------
>
> Key: HIVE-25074
> URL: https://issues.apache.org/jira/browse/HIVE-25074
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, Standalone Metastore
> Affects Versions: 4.0.0
> Reporter: Miklos Szurap
> Assignee: Zoltan Chovan
> Priority: Major
>
> The "flushCache" in HiveMetaStore with the ObjectStore implementation is
> currently a NOOP:
> {code:java}
> public void flushCache() {
> // NOP as there's no caching
> } {code}
> The HBaseStore (HBaseReadWrite) had some logic in it, however it has been
> removed in HIVE-17234.
> As I see the calls are going like this:
> HiveMetaStoreClient.flushCache() -> CachedStore.flushCache() ->
> ObjectStore.flushCache()
> There are significant amount of calls (about 10% of all calls) made from the
> client to the server - to do nothing. We could spare the call to the server
> completely, including getting a DB connection which can take 1+ seconds under
> high load scenarios slowing down Hive queries unnecessarily.
> Can we:
> # Deprecate the RawStore.flushCache (if there are other implementations)
> # Deprecate the HiveMetaStoreClient.flushCache()
> # Do the NOOP on the client side in HiveMetaStoreClient.flushCache() (while
> it is not removed in a next version)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
