Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11224 )
Change subject: IMPALA-7448: Invalidate recently unused tables from catalogd ...................................................................... Patch Set 3: > Patch Set 3: > > This is also an essential feature for the old catalog (2.x branch). Yep, this is important for both "v2" and "v1". I thought in general the 2.x branch is not getting new features, though. > Haven't dug deeper, but I have a question that can we add a whitelist for > this eviction? It's expensive to load metadata of some huge tables, so it'd > be great if they can always being cached. Otherwise the first unlucky query > accessing these table will be stuck for a long time. To be simple, we can let > the whitelist configurable only in the startup options. Unfortunately those huge tables also are the ones that take up the most memory, so there is the most benefit for evicting :) I guess a user-configurable way to prevent eviction would be useful but maybe we can add that as a follow-up instead of increasing the scope of this patch? One option might be to add a new table property, so the auto-invalidator skips tables when it sees something like "impala.pin_metadata" = "true" -- To view, visit http://gerrit.cloudera.org:8080/11224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib549717abefcffb14d9a3814ee8cf0de8bd49e89 Gerrit-Change-Number: 11224 Gerrit-PatchSet: 3 Gerrit-Owner: Tianyi Wang <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Tianyi Wang <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]> Gerrit-Comment-Date: Fri, 31 Aug 2018 17:55:57 +0000 Gerrit-HasComments: No
