[
https://issues.apache.org/jira/browse/HIVE-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18938:
------------------------------------
Issue Type: Sub-task (was: Bug)
Parent: HIVE-18939
> LLAP: cache priorities (soft pin, exclusions) configuration
> -----------------------------------------------------------
>
> Key: HIVE-18938
> URL: https://issues.apache.org/jira/browse/HIVE-18938
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sergey Shelukhin
> Priority: Major
>
> We want persistent configuration and not an ad-hoc pin API because the latter
> is both unreliable (e.g. node might go down and restart, so someone has to
> keep track of global state anyway - might as well have it done explicitly),
> and also dangerous because errant pin calls can fill cache with garbage and
> make it unusable.
> So, the idea is that we can allow to
> 1) Give some tables or partitions (tags, see HIVE-16992) a priority boost at
> cache time.
> 2) Exclude some tables or partitions from being cached.
> Ideally we also want this changeable without LLAP restart (and managed by
> HS2). So the choice here is either make this a config setting, like many
> Hadoop settings that have user/group/etc lists, and implement
> ZK-based/HS2-based configuration changes without restart, that we want to do
> anyway at some point;
> or make this a separate thing, probably managed and stored as policies in
> metastore (kind of like workload management) and update via ZK.
> It's possible to go with config settings that do require restart for the
> first cut.
> Separately, we can add a workload management policy that would disable cache
> for certain query pools (i.e. users/groups/apps).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
