[
https://issues.apache.org/jira/browse/HBASE-21286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645162#comment-16645162
]
Ted Yu commented on HBASE-21286:
--------------------------------
Where is the ExecutorService shutdown ?
{code}
432 ExecutorService executor = Executors.newFixedThreadPool(nrThreads);
{code}
If you can share some data on performance improvement, that would be nice.
> Parallelize computeHDFSBlocksDistribution when getting splits of a
> HBaseSnapshot
> --------------------------------------------------------------------------------
>
> Key: HBASE-21286
> URL: https://issues.apache.org/jira/browse/HBASE-21286
> Project: HBase
> Issue Type: Improvement
> Components: snapshots
> Affects Versions: 1.4.0
> Reporter: Lavinia-Stefania Sirbu
> Priority: Minor
> Attachments: HBASE-21286.branch-1.4.001.patch
>
>
> Even if this step is called computeHDFSBlocksDistribution, this is executed
> no matter the file system of the snapshot. For example, we have observed an
> important slowness when we have a snapshot in s3 (~26k regions, 5column
> families, 2 files per column family) the getsplits time is ~40min due to the
> calls in s3 for listing the files to get the best locations.
> Parallelizing this operation can reduce the overall setup time. The thread
> pool should be configurable and a good choice could be
> "hbase.snapshot.thread.pool.max" that is also used in RestoreSnapshotHelper.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
