[
https://issues.apache.org/jira/browse/HBASE-20786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522941#comment-16522941
]
stack commented on HBASE-20786:
-------------------------------
bq. Does all the creation happen on master?
Yes.
bq. Can it be farmed out to region servers?
RSs are participating in region deploy. The creation of regions is bottlenecked
on back and forth w/ NN. M has an executor running so can do a bunch of creates
against the NN at a time. Could farm this out but RS will just be asking NN.
bq. Is this a result of all splits needing to go through master becoming a
bottleneck?
M is running the process so is likely bottleneck.
But, we've not spent time on this. Lets dig in on where we are bottlenecking.
Let me add some more notes.
> Table create with thousands of regions takes too long
> -----------------------------------------------------
>
> Key: HBASE-20786
> URL: https://issues.apache.org/jira/browse/HBASE-20786
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
> Reporter: stack
> Priority: Major
>
> Internal testing has create of a table with 33k regions taking 18 minutes.
> Let me provide more info below. We have an executor with default ten threads
> handling the creation of the regions in HDFS which helps distribute out the
> load but its not enough. This cluster had >600 servers. Let me add detail.
> Need to spend some time on speeding up create/assigns. Made this an umbrella
> issue so can pick off pieces of the problem as subtasks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
