[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267612#comment-14267612 ] Cosmin Lehene commented on HBASE-5532: -- [~terry_zhang], [~apurtell] this may not apply anymore. The code has changed a bit, and it seems we check for null readers. I'm not sure about the locking semantics. The logic has moved to StoreFile.java and there's related logic in StripeStoreFileManager isMajorCompaction() is in HStore.java get NPE during MajorCompactionChecker -- Key: HBASE-5532 URL: https://issues.apache.org/jira/browse/HBASE-5532 Project: HBase Issue Type: Bug Components: regionserver Reporter: terry zhang Labels: delete Attachments: HBASE-5532-v2.patch, HBASE-5532.patch We found error log (NullPointerException) below on our online cluster: 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) at org.apache.hadoop.hbase.Chore.run(Chore.java:66) After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13224761#comment-13224761 ] Nicolas Spiegelberg commented on HBASE-5532: Sounds like this is a race condition, so this patch wouldn't be sufficient. We need some way to lock the reader during this query. get NPE during MajorCompactionChecker -- Key: HBASE-5532 URL: https://issues.apache.org/jira/browse/HBASE-5532 Project: HBase Issue Type: Bug Components: regionserver Reporter: terry zhang Attachments: HBASE-5532.patch We found error log (NullPointerException) below on our online cluster: 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) at org.apache.hadoop.hbase.Chore.run(Chore.java:66) After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[
https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13224955#comment-13224955
]
terry zhang commented on HBASE-5532:
yes,Nicolas. This is a race condition and we can use lock to avoid this issue.
And now in the function toDetermines if Store should be split also didn't use
lock to protect.
{code:title=Store.java|borderStyle=solid}
/**
* Determines if Store should be split
* @return byte[] if store should be split, null otherwise.
*/
public byte[] getSplitPoint() {
this.lock.readLock().lock();
try {
// sanity checks
if (this.storefiles.isEmpty()) {
return null;
}
StoreFile.Reader r = sf.getReader(); * = check if the reader is null*
if (r == null) {
LOG.warn(Storefile + sf + Reader is null);
continue;
}
long size = r.length();
if (size maxSize) {
// This is the largest one so far
maxSize = size;
largestSf = sf;
}
}
StoreFile.Reader r = largestSf.getReader(); *= check if the reader is
null*
if (r == null) {
LOG.warn(Storefile + largestSf + Reader is null);
return null;
}
// Get first, last, and mid keys. Midkey is the key that starts block
// in middle of hfile. Has column and timestamp. Need to return just
// the row we want to split on as midkey.
byte [] midkey = r.midkey();
.
return null;
}
{code}
get NPE during MajorCompactionChecker
--
Key: HBASE-5532
URL: https://issues.apache.org/jira/browse/HBASE-5532
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: terry zhang
Attachments: HBASE-5532.patch
We found error log (NullPointerException) below on our online cluster:
2012-03-05 00:17:09,592 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker:
Caught exception
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
at
org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
at
org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
at
org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
After Check the code we found although it already check whether store files
has null reader at the begin of the function(isMajorCompaction), but it still
has some possibility the reader is closed before it return(eg mini
compaction). So we need to check store file reader before we use it to avoid
this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223291#comment-13223291 ] Zhihong Yu commented on HBASE-5532: --- Which version of HBase were you using ? get NPE during MajorCompactionChecker -- Key: HBASE-5532 URL: https://issues.apache.org/jira/browse/HBASE-5532 Project: HBase Issue Type: Bug Components: regionserver Reporter: terry zhang Attachments: HBASE-5532.patch We found error log (NullPointerException) below on our online cluster: 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) at org.apache.hadoop.hbase.Chore.run(Chore.java:66) After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223909#comment-13223909 ] terry zhang commented on HBASE-5532: Hi, Ted. Our is base on 0.90.2 (including many patches from 90.3 till 90.5). get NPE during MajorCompactionChecker -- Key: HBASE-5532 URL: https://issues.apache.org/jira/browse/HBASE-5532 Project: HBase Issue Type: Bug Components: regionserver Reporter: terry zhang Attachments: HBASE-5532.patch We found error log (NullPointerException) below on our online cluster: 2012-03-05 00:17:09,592 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) at org.apache.hadoop.hbase.Chore.run(Chore.java:66) After Check the code we found although it already check whether store files has null reader at the begin of the function(isMajorCompaction), but it still has some possibility the reader is closed before it return(eg mini compaction). So we need to check store file reader before we use it to avoid this NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
