[
https://issues.apache.org/jira/browse/HBASE-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12658237#action_12658237
]
Andrew Purtell commented on HBASE-1050:
---------------------------------------
I have the config mods in place and yet have been getting lethal DFS errors
(corrupt or missing blocks in mapfile data or index files) once my tables get
above 1000 regions or so and I restart after a clean shutdown. I have 23 HRS on
23 data nodes and a table with equiprobable inserts. I suspect after a night of
heavy writing and a shutdown, more than 50% of regions want to split upon
restart, so every HRS is suddenly running splits. HBASE-1062 helps.
Missing block in the root region sounds like something different though. Should
not be much/any writing there.
> Allow regions to split around scanners
> --------------------------------------
>
> Key: HBASE-1050
> URL: https://issues.apache.org/jira/browse/HBASE-1050
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: client, regionserver
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Blocker
> Fix For: 0.20.0
>
>
> We have a number of scanners iterating over a table that also sees a lot of
> constant write activity. If the scans are too frequent we will suppress
> splitting. At a lull then a number of splits happen all at once, occasionally
> overwhelming DFS and causing file corruption.
> I wonder how much work it would be to split regions around scanners. Rather
> than wait for scanner leases to expire, suspend/block the scanner, split the
> table, and then negotiate with the client to continue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
