[
https://issues.apache.org/jira/browse/HBASE-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584318#action_12584318
]
stack commented on HBASE-555:
-----------------------------
Going to commit this thing. Just tested it over on the Lars cluster (2100
regions on 20 servers). The two fixes in this patch made it so on restart,
there were no more "should not have opened region's" -- all came up smoothly
and regions are spread pretty evenly (looks better than what it used to be but
I only did one restart).
> Only one Worker in HRS; on startup, if assigned tens of regions, havoc of
> reassignments because open processing is done in series
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-555
> URL: https://issues.apache.org/jira/browse/HBASE-555
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.16.0, 0.2.0, 0.1.0
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Attachments: 555-0.1-v2.patch, 555-0.1.patch
>
>
> On the Lars clusters, he's up into the thousands of regions. Starting this
> cluster, there is a load of churn in the master log as we assign regions,
> they report their opening and then after the hbase.hbasemaster.maxregionopen
> of one minute elapses, we assign the region elsewhere.
> Problem seems to be the fact that we only run a single Worker thread in our
> regionserver; means that region opens are processed in series.
> For example, the below shows when a master assigned a region and then the
> regionserver side log when it got around to opening it:
> {code}
> 2008-03-29 04:48:51,638 INFO org.apache.hadoop.hbase.HMaster: assigning
> region pdc-docs,US20060158177_20060720,1205765009844 to server
> 192.168.105.19:60020
> ..
> 2008-03-29 04:50:58,124 INFO org.apache.hadoop.hbase.HRegionServer:
> MSG_REGION_OPEN : pdc-docs,US20060158177_20060720,1205765009844
> {code}
> There is > 2 minutes between the two loggings (I checked clocks on this
> cluster and they are synced).
> Looking in the regionserver log, its just filled with logging on the opening
> of regions. The region opens are running pretty fast at about a second each
> but there are hundreds of regions to open in this case so its easy to go over
> our default of 60 seconds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
