[
https://issues.apache.org/jira/browse/HBASE-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875656#action_12875656
]
stack commented on HBASE-1111:
------------------------------
Small ideas:
+ We could index the WAL so that recovering, we could skip old edits the faster
(could skip reading the file altogether if we also had metadata with last seqno
in it and it was < than current regions maximum)
+ Read recovery log once, instead of once per family as we do now. Seems like
we always replayed at Store level even though there is a noop play
reconstruction log in HRegion (It was added by hbase-677; transactional hbase
needed it).
> [performance] Crash recovery takes way too long
> -----------------------------------------------
>
> Key: HBASE-1111
> URL: https://issues.apache.org/jira/browse/HBASE-1111
> Project: HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.21.0
>
>
> Watching hbase recover from crashes, its taking way too long:
> 1. Must wait first on lease to expire (if server is rebooted, it should
> cancel the old servers' lease but make sure the lease expiration code runs)
> 2. Master splits logs. This is single-threaded. At least a maximum of 64
> logs but seems to run slow anyways.
> 3. Assign out the regions that were on dead-server (minutes or even tens of
> minutes could have elapsed at this stage)
> 4. Wait on the regionservers to open. If small cluster, because
> regionservers open regions in series, could take a long time opening a bunch
> of issues. Meantime the regions are not available, clients will likely
> timeout.
> 5. To make things worse, I've seen load-balancer cut in to 'help out' telling
> regionserver close some of its regions though its busy opening a bunch.
> Andrew Purtell notes that HBASE-1110 will change a bunch of the above.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
