[
https://issues.apache.org/jira/browse/HBASE-11263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14013948#comment-14013948
]
Andrew Purtell commented on HBASE-11263:
----------------------------------------
Do we need storeFileThreadPoolCounter? Looks like this is being used
effectively as a boolean flag when you could achieve the same by testing if
storeFileThreadPool is null or not in an exclusive section? Save some heap per
HRegion.
What happens if you use one thread pool for opening and closing store files per
RegionServer? If we have thousands of HRegions and they all have thread pools,
surely we are not able to fully schedule that degree of parallelism?
> Share the open/close store file thread pool for all store in a region
> ---------------------------------------------------------------------
>
> Key: HBASE-11263
> URL: https://issues.apache.org/jira/browse/HBASE-11263
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.99.0
> Reporter: Liu Shaohui
> Assignee: Liu Shaohui
> Priority: Minor
> Attachments: HBASE-11263-trunk-v1.diff
>
>
> Currently, the open/close store file thread pool is divided equally to all
> stores of a region.
> {code}
> protected ThreadPoolExecutor getStoreFileOpenAndCloseThreadPool(
> final String threadNamePrefix) {
> int numStores = Math.max(1, this.htableDescriptor.getFamilies().size());
> int maxThreads = Math.max(1,
> conf.getInt(HConstants.HSTORE_OPEN_AND_CLOSE_THREADS_MAX,
> HConstants.DEFAULT_HSTORE_OPEN_AND_CLOSE_THREADS_MAX)
> / numStores);
> return getOpenAndCloseThreadPool(maxThreads, threadNamePrefix);
> }
> {code}
> This is not very optimal in following scenarios:
> # The data of some column families are very large and there are many hfiles
> in those stores, and others may be very small and in-memory column families.
> # Usually we preserve some column families for later needs. The thread pool
> for these column families are wasted。
> The simple way is to share a big thread pool for all stores to open/close
> hfiles.
> Suggestions are welcomed.
> Thanks.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
