Hello Bharath Vissapragada, Tianyi Wang, Vuk Ercegovac,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/11208
to review the following change.
Change subject: WIP: LRU caching of partitions in impalad
......................................................................
WIP: LRU caching of partitions in impalad
This changes the CatalogdMetaProvider to use a Guava-based LRU cache.
The eviction strategy is currently time-based (10 minutes), and it only
performs caching of partition information and table column statistics.
It does not cache the table entries themselves, which means that we
don't need to do any invalidation propagation via the statestore quite
yet. Instead, every query will do an initial fetch of the table metadata
in order to know the current version number. That version number is then
used as part of the cache key for all further metadata, so when the
version number changes, all of the prior cache entries become
"unreachable" and effectively evicted.
This is a WIP - needs a fair bit of cleanup, but basic tests seem to
pass and I can verify cache hits by enabling trace logging.
Change-Id: I9a57521ad851da605604a1e7c48d3d6627da5df5
---
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
3 files changed, 283 insertions(+), 35 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/11208/1
--
To view, visit http://gerrit.cloudera.org:8080/11208
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9a57521ad851da605604a1e7c48d3d6627da5df5
Gerrit-Change-Number: 11208
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Bharath Vissapragada <[email protected]>
Gerrit-Reviewer: Tianyi Wang <[email protected]>
Gerrit-Reviewer: Vuk Ercegovac <[email protected]>
