Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23944
Change subject: IMPALA-14724: Add support for using soft references in CatalogdMetaProvider ...................................................................... IMPALA-14724: Add support for using soft references in CatalogdMetaProvider This adds the --local_catalog_cache_use_soft_references boolean startup flag. If set to true (the default is false), CatalogdMetaProvider configures the Guava Cache to use soft references for values. Soft references can be garbage collected when the system is under memory pressure. Effectively, the JVM garbage collector can shrink the cache to keep the system running. There are some conditions where this could make the system more resilient: - A large query could come in and use excess memory for compilation - The cache itself could underestimate the memory usage of the metadata it is caching. That could lead it to oversubscribe the JVM memory. Allowing the cache memory to be garbage collected allows the system to adapt. The downside is that non-cache JVM memory use can crowd out the cache and cause more cache misses and traffic to catalogd. Change-Id: I10efb6743d2f9e5b8cc47cdcc9c5893b005b9aa5 --- M be/src/runtime/exec-env.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java 4 files changed, 22 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/23944/1 -- To view, visit http://gerrit.cloudera.org:8080/23944 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I10efb6743d2f9e5b8cc47cdcc9c5893b005b9aa5 Gerrit-Change-Number: 23944 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <[email protected]>
