[
https://issues.apache.org/jira/browse/HBASE-24436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117435#comment-17117435
]
Anoop Sam John edited comment on HBASE-24436 at 5/27/20, 6:39 AM:
------------------------------------------------------------------
Just moving the discussion from PR comments to here.
If I understand the jira correctly, what you are trying to solve is below case.
One region with say 2 stores. Store1 having much more files than other. Say the
config for the #threads in open pool is 10. Now it will create 2 pools for each
store with 5 threads each. The Store2 will get finished soon. But store1 will
take much longer. So if it was a shared pool of 10 threads the overall time for
opening both stores would have been lesser. my understanding correct?
Also this make more sense when hbase.hstore.open.and.close.threads.max is also
configured to be 1+ By default we open stores serially (only 1 thread).. Then
also this change will give a larger thread pool for the store file open.
was (Author: anoop.hbase):
Just moving the discussion from PR comments to here.
If I understand the jira correctly, what you are trying to solve is below case.
One region with say 2 stores. Store1 having much more files than other. Say the
config for the #threads in open pool is 10. Now it will create 2 pools for each
store with 5 threads each. The Store2 will get finished soon. But store1 will
take much longer. So if it was a shared pool of 10 threads the overall time for
opening both stores would have been lesser. my understanding correct?
> The store file open and close thread pool should be shared at the region level
> ------------------------------------------------------------------------------
>
> Key: HBASE-24436
> URL: https://issues.apache.org/jira/browse/HBASE-24436
> Project: HBase
> Issue Type: Improvement
> Reporter: Junhong Xu
> Assignee: Junhong Xu
> Priority: Minor
>
> For now, we provide threads per column family evenly in general, but there
> are some cases that some column families have much more store files than
> others( maybe that's the life, right? ). So in that case, some Stores have
> beed done quickly while others are struggling.We should share the thread pool
> at the region level in case of data skew.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
