Andrew:
I guess you don't see the following (from line 1356 in Store.checkSplit())
in your log:
if (LOG.isDebugEnabled()) {
LOG.debug("cannot split because midkey is the same as first or "
+
"last row");
}
Was it possible that the largest StoreFile grew (symmetrically) on both
sides of the midkey (which happened to be the start row) ?
On Mon, Dec 6, 2010 at 3:16 PM, Andrew Purtell <[email protected]> wrote:
> I'm catching a flight to Narita, then another to Los Angeles today, and
> will look at this when I get home.
>
> Just so you don't think I dropped this on jira and ran off. :-)
>
> I also have that OOME issue to look at also, and I wonder if this split
> problem is related.
>
> - Andy
>
> -----Original Message-----
> From: "stack (JIRA)" <[email protected]>
> Date: Mon, 6 Dec 2010 14:01:27
> To: <[email protected]>
> Subject: [jira] Commented: (HBASE-3312) Region fails to split
>
>
> [
> https://issues.apache.org/jira/browse/HBASE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968365#action_12968365]
>
> stack commented on HBASE-3312:
> ------------------------------
>
> This is the formatter for the message you see:
>
> {code}
> LOG.info("Split row is not inside region key range or is equal to " +
> "startkey: " + Bytes.toString(this.splitrow));
> {code}
>
> So, when it logs:
>
> {code}
> Split row is not inside region key range or is equal to startkey:
> DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF
> {code}
>
> ... we are actually printing out the proposed 'splitrow' which indeed is
> same as the start row.
>
> Why do we keep picking the same splitrow over and over though region is
> large? Must be something dumb we're doing w/ mutliple families or so?
>
> Good on you Andrew.
>
>
>
> > Region fails to split
> > ---------------------
> >
> > Key: HBASE-3312
> > URL: https://issues.apache.org/jira/browse/HBASE-3312
> > Project: HBase
> > Issue Type: Bug
> > Reporter: Andrew Purtell
> > Priority: Critical
> > Fix For: 0.90.0
> >
> >
> > "SplitTransaction: Split row is not inside region key range or is equal
> to startkey".
> > I stopped writers after realizing one region of the table was growing
> unbounded:
> > {noformat}
> > webtable,DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF,1291557809928.
> > deccb20bcbf8e634008cf093105c4fc5.
> > stores=3, storefiles=10, storefileSizeMB=6753,
> memstoreSizeMB=2,
> > storefileIndexSizeMB=2
> > {noformat}
> > In the regionserver log, every compaction of this region fails to split
> with the following message:
> > {noformat}
> > 2010-12-05 09:04:50,156 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> > Split row is not inside region key range or is equal to startkey:
> > DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF
> > {noformat}
> > However there are many rows in the region:
> > {noformat}
> > 10/12/05 09:33:33 DEBUG client.HTable$ClientScanner: Advancing internal
> scanner
> > to startKey at 'DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF'
> > [...]
> > Current count: 258000, row: DE424FBDBD15FF3B3E9D0C3DB149ECD29B0F615B
> > Current count: 259000, row: DF27251479D6C91B27AA9B1561070A53011A6D1E
> > 10/12/05 09:33:36 DEBUG client.HTable$ClientScanner: Finished with region
> REGION =>
> > {NAME => 'webtable,DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF,
> > 1291557809928.deccb20bcbf8e634008cf093105c4fc5.',
> > STARTKEY => 'DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF',
> > ENDKEY => 'DF76CF458433DB5D0CB2C50042452B296E3721A7',
> > ENCODED => deccb20bcbf8e634008cf093105c4fc5, TABLE => {{NAME =>
> 'webtable',
> > FAMILIES => [{NAME => 'content', BLOOMFILTER => 'NONE', REPLICATION_SCOPE
> => '0',
> > VERSIONS => '2147483647', COMPRESSION => 'LZO', TTL => '2147483647',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'},
> > {NAME => 'info', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
> > VERSIONS => '2147483647', COMPRESSION => 'LZO', TTL => '2147483647',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'},
> > {NAME => 'url', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
> > VERSIONS => '1', COMPRESSION => 'LZO', TTL => '2147483647',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}}
> > 10/12/05 09:33:36 DEBUG client.HTable$ClientScanner: Advancing internal
> scanner
> > to startKey at 'DF76CF458433DB5D0CB2C50042452B296E3721A7'
> > {noformat}
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
