[
https://issues.apache.org/jira/browse/HBASE-24436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136687#comment-17136687
]
Junhong Xu edited comment on HBASE-24436 at 6/16/20, 2:18 PM:
--------------------------------------------------------------
I test about 4 or 5 times, and the result is similar, here is the last one.
||HBase branch ||RS || Row count || Region || CF || Store || Store file ||
| branch-2.2 |3| 322663930 | 61 | 10 | 622 | 4,131 |
There are 3 RS， s1 for HBase-24436 with 16 threads, s2 for branch-2.2 with 16
threads, s3 for HBASE-24436 with 4 threads. There are 10 cf, the C of which is
the heavy one, with about 80+ store files on average, others have one or a
little more. In almost all cases, the store file opening dominates the region
assigning.
Here is the case for 95-percentile
||Open/RS||s1(HBASE-24436 with 16 threads) || s2 (branch-2.2 with 16 threads)
|| s3(HBASE-24436 with 4 threads )
|Open single storefile | 72 ms| 42 ms | 42 ms|
| Open single region | 722 ms | 3491 ms | 1172 ms |
was (Author: joseph295):
I test about 4 or 5 times, and the result is similar, here is the last one.
||HBase branch ||RS || Row count || Region || CF || Store || Store file ||
| branch-2.2 |3| 322663930 | 61 | 10 | 622 | 4,131 |
There are 3 RS， s1 for HBase-24436 with 16 threads, s2 for branch-2.2 with 16
threads, s3 for HBASE-24436 with 4 threads. There are 10 cf, the C of which is
the heavy one, with about 80+ store files on average, others have one or a
little more. In almost all cases, the store file opening dominates the region
assigning.
Here is the case for 95-percentile
||Open/RS||s1(HBASE-24436 with 16 threads) || s2 (branch-2.2 with 16 threads)
|| s3(HBASE-24436 with 4 threads )
|Open single storefile | 72 ms| 42 ms | 42 ms|
| Open region | 722 ms | 3491 ms | 1172 ms |
> The store file open and close thread pool should be shared at the region level
> ------------------------------------------------------------------------------
>
> Key: HBASE-24436
> URL: https://issues.apache.org/jira/browse/HBASE-24436
> Project: HBase
> Issue Type: Improvement
> Reporter: Junhong Xu
> Assignee: Junhong Xu
> Priority: Minor
>
> For now, we provide threads per column family evenly in general, but there
> are some cases that some column families have much more store files than
> others( maybe that's the life, right? ). So in that case, some Stores have
> beed done quickly while others are struggling.We should share the thread pool
> at the region level in case of data skew.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
