[
https://issues.apache.org/jira/browse/HIVE-21787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16853438#comment-16853438
]
Sam An edited comment on HIVE-21787 at 5/31/19 9:31 PM:
--------------------------------------------------------
I made the 4th patch upload, and hopefully the final one.
Tested the TPC-DS with LRU cache with 30 queries, on average (variation is not
very much), explain plan runs for 6 min 27 secs without cache, 6 min 7 seconds
with cache and cache update, 6 mins 2 - 7 seconds without cache update. So with
cache is definitely better for the queries I run. Disabling cache update does
not show for the query set. Maybe if we include more queries and more
partitions might show some difference.
was (Author: samuelan):
I made the 4th patch upload, and hopefully the final one.
Tested the TPC-DS with LRU cache with 30 queries, explain plan runs for 6 min
27 secs without cache, 6 min 7 seconds with cache and cache update, 6 mins 2 -
7 seconds without cache update. So with cache is definitely better for the
queries I run. Disabling cache update does not show for the query set. Maybe if
we include more queries and more partitions might show some difference.
> Metastore table cache enhancement
> ---------------------------------
>
> Key: HIVE-21787
> URL: https://issues.apache.org/jira/browse/HIVE-21787
> Project: Hive
> Issue Type: New Feature
> Components: Standalone Metastore
> Reporter: Sam An
> Assignee: Sam An
> Priority: Major
> Attachments: HIVE-21787.1.patch, HIVE-21787.2.patch,
> HIVE-21787.3.patch, HIVE-21787.4.patch
>
>
> Metastore currently uses black/white list to specify patterns of tables to
> load into the cache. Cache is loaded in one shot "prewarm", and updated by a
> background thread. This is not a very efficient design.
> In this feature, we try to enhance the cache for Tables with LRU to improve
> cache utilization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
