[ https://issues.apache.org/jira/browse/HIVE-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sergey Shelukhin updated HIVE-18938: ------------------------------------ Issue Type: Sub-task (was: Bug) Parent: HIVE-18939 > LLAP: cache priorities (soft pin, exclusions) configuration > ----------------------------------------------------------- > > Key: HIVE-18938 > URL: https://issues.apache.org/jira/browse/HIVE-18938 > Project: Hive > Issue Type: Sub-task > Reporter: Sergey Shelukhin > Priority: Major > > We want persistent configuration and not an ad-hoc pin API because the latter > is both unreliable (e.g. node might go down and restart, so someone has to > keep track of global state anyway - might as well have it done explicitly), > and also dangerous because errant pin calls can fill cache with garbage and > make it unusable. > So, the idea is that we can allow to > 1) Give some tables or partitions (tags, see HIVE-16992) a priority boost at > cache time. > 2) Exclude some tables or partitions from being cached. > Ideally we also want this changeable without LLAP restart (and managed by > HS2). So the choice here is either make this a config setting, like many > Hadoop settings that have user/group/etc lists, and implement > ZK-based/HS2-based configuration changes without restart, that we want to do > anyway at some point; > or make this a separate thing, probably managed and stored as policies in > metastore (kind of like workload management) and update via ZK. > It's possible to go with config settings that do require restart for the > first cut. > Separately, we can add a workload management policy that would disable cache > for certain query pools (i.e. users/groups/apps). -- This message was sent by Atlassian JIRA (v7.6.3#76005)