[
https://issues.apache.org/jira/browse/HIVE-26428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ádám Szita resolved HIVE-26428.
-------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Merged to master, thanks for the review [~lpinter]
> Limit usage of LLAP BPWrapper to threads of IO threadpools
> ----------------------------------------------------------
>
> Key: HIVE-26428
> URL: https://issues.apache.org/jira/browse/HIVE-26428
> Project: Hive
> Issue Type: Improvement
> Reporter: Ádám Szita
> Assignee: Ádám Szita
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> BPWrapper is used in LRFU cache eviction policy to decrease the time spent
> waiting for lock on the heap. This is done by adding a buffer as threadlocal
> and accumulating CacheableBuffer instances there before trying to acquire a
> lock. This works well when we have threads from pools such as IO-Elevator
> threads or OrcEncode threads.
> For ephemeral threads there's no advantage of doing this as the buffers in
> threadlocals may never reach the heap or list structures of LRFU, thereby
> also making evictions less efficient. This can happen e.g. LLAPCacheAwareFS
> is used with Parquet, where we're using the Tez threads for both execution
> and IO.
> We should disable BPWrappers for such cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
