[
https://issues.apache.org/jira/browse/HIVE-26428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-26428 started by Ádám Szita.
-----------------------------------------
> Limit usage of LLAP BPWrapper to threads of IO threadpools
> ----------------------------------------------------------
>
> Key: HIVE-26428
> URL: https://issues.apache.org/jira/browse/HIVE-26428
> Project: Hive
> Issue Type: Improvement
> Reporter: Ádám Szita
> Assignee: Ádám Szita
> Priority: Major
>
> BPWrapper is used in LRFU cache eviction policy to decrease the time spent
> waiting for lock on the heap. This is done by adding a buffer as threadlocal
> and accumulating CacheableBuffer instances there before trying to acquire a
> lock. This works well when we have threads from pools such as IO-Elevator
> threads or OrcEncode threads.
> For ephemeral threads there's no advantage of doing this as the buffers in
> threadlocals may never reach the heap or list structures of LRFU, thereby
> also making evictions less efficient. This can happen e.g. LLAPCacheAwareFS
> is used with Parquet, where we're using the Tez threads for both execution
> and IO.
> We should disable BPWrappers for such cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
