[
https://issues.apache.org/jira/browse/HIVE-30215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-30215:
----------------------------------
Labels: pull-request-available (was: )
> HS2 HMS cache: every partition and statistics lookup of a non-transactional
> table costs an extra get_table call
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-30215
> URL: https://issues.apache.org/jira/browse/HIVE-30215
> Project: Hive
> Issue Type: Improvement
> Reporter: Denys Kuzmenko
> Priority: Major
> Labels: pull-request-available
>
> HiveMetaStoreClientWithLocalCache keys its entries on a
> TableWatermark(validWriteIdList, tableId), and isValid() requires a non-null
> write-id list. A non-transactional table therefore can never be cached. Six
> call sites (listPartitionsByExpr, getPartitionsSpecByExpr,
> getAggrColStatsFor, getPartitionsByNames, listPartitionNames,
> getTableColumnStatistics) fetch the table id first and test validity
> afterwards. This client sits inside the query-level cache
> (Hive.createMetaStoreClient), so each such getTable is a metastore round-trip
> made only to find out that nothing can be cached.
>
>
>
>
>
> On a TPC-DS 1 TB run on external Parquet tables, 9,356 of 16,111 get_table
> calls came immediately before one of these sites. That is about 0.23 s of
> compile per query, ~23 s per 99-query
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
