[
https://issues.apache.org/jira/browse/IMPALA-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866898#comment-16866898
]
Todd Lipcon commented on IMPALA-6876:
-------------------------------------
Looked at this a few minutes today. There are a few more sketchy things about
this class:
- the comparators for PQ entries reach into Table classes and get mutable
fields like access count, which means that the PriorityQueue's heap invariant
might be transiently broken. For example, while inserting table A, it will get
compared against some existing heap entry B. Entry B might be concurrently
mutated, and not yet updated in the heap, which means that it could be
misplaced in the priority queue. Depending on the implementation of the PQ,
breaking the invariant could cause either wrong results or an exception to be
thrown.
- the "alwaysEvict" policy doesn't make much sense -- the original review says
"I just wanted a cheap way to make sure that frequently accessed tables from
the past didn't prevent newly accessed tables from ever being inserted in the
cache. A more elaborate schemes could be time-based rank reduction or something
like that." However, what it really accomplishes is that you get 24 MFU tables
plus one MRU table tacked onto the end. ie it doesn't allow multiple
recently-accessed tables to get into the cache, it just gets _one_ tacked onto
the end.
> Entries in CatalogUsageMonitor are not cleared after invalidation
> -----------------------------------------------------------------
>
> Key: IMPALA-6876
> URL: https://issues.apache.org/jira/browse/IMPALA-6876
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.0, Impala 2.12.0
> Reporter: Dimitris Tsirogiannis
> Priority: Major
> Labels: memory-leak, ramp-up
>
> The CatalogUsageMonitor in the catalog maintains a small cache of references
> to tables that: a) are accessed frequently in the catalog and b) have the
> highest memory requirements. These entries are not cleared upon server or
> table invalidation, thus preventing the GC from collecting the memory of
> these tables. We should make sure that the CatalogUsageMonitor does not
> maintain entries of tables that have been invalidated or deleted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
