Peter Ebert created IMPALA-10841:
------------------------------------
Summary: Pin Specific Directories in the Remote Data Cache
Key: IMPALA-10841
URL: https://issues.apache.org/jira/browse/IMPALA-10841
Project: IMPALA
Issue Type: New Feature
Affects Versions: Impala 3.4.0
Reporter: Peter Ebert
For remote reads, a warm cache is important for best performance. A feature to
force a directory into the cache that pins it, regardless of the LRU or LIRS
algorithm, to ensure SLAs can be met would be helpful.
The reason I say a directory is that this would give us the flexibility to
select a specific db, table, and/or partition(s) to keep in the cache.
Ideally this would fire at startup, there may be some overlap here with other
jiras to warm the cache, so another option could be a flag to specify whether
it could later be flushed out of the cache (by LRU/LIRS) or not, as pinning has
the advantage for high SLA workloads that may not be read frequently enough to
keep them in the cache.
Would have to think about scenarios where the directory is bigger than the
cache and how to handle/warn about that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
