[
https://issues.apache.org/jira/browse/IMPALA-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664468#comment-16664468
]
Vuk Ercegovac commented on IMPALA-6354:
---
https://issues.apache.org/jira/browse/IMPALA-7448 addresses many of these
issues and does so by also keeping track of accesses from each coordinator
(catalog only operations don't track when tables are used at the coordinators).
[~twang], have a look to see if this can simplify things and take into account
coordinator cached accesses.
Assigning it your way and lowering the priority.
> Consider using Guava LoadingCache to cache metadata objects opposed to a
> ConcurrentHashMap
> --
>
> Key: IMPALA-6354
> URL: https://issues.apache.org/jira/browse/IMPALA-6354
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
>Reporter: Mostafa Mokhtar
>Assignee: Vuk Ercegovac
>Priority: Critical
> Labels: catalog, catalog-server, scalability
>
> Look into replacing the ConcurrentHashMap(s) used in the Catalog service to
> cache metadata with
> [LoadingCache|https://google.github.io/guava/releases/19.0/api/docs/com/google/common/cache/LoadingCache.html].
> Caching metadata using a LoadingCache will allow:
> * Better coherency by adding expireAfter and refreshAfter clause to the cache
> * Weak reference eviction to respond to garbage collection
> * Putting a cap on maximum number of caches entires to avoid OOMs and
> excessive GC
> * Assigning different weights for each entry (For more efficient eviction)
> https://github.com/google/guava/wiki/CachesExplained
> https://google.github.io/guava/releases/19.0/api/docs/com/google/common/cache/CacheBuilder.html
> https://google.github.io/guava/releases/19.0/api/docs/com/google/common/cache/LoadingCache.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
