[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-5578: -- Status: Open (was: Patch Available) Unmarking patch available, it's a year old. NPE when regionserver reported server load, caused rs stop. --- Key: HBASE-5578 URL: https://issues.apache.org/jira/browse/HBASE-5578 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0 Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 Reporter: Storm Lee Priority: Critical Fix For: 0.92.3 Attachments: 5589.txt The regeionserver log: 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server data3,60020,1331286604591: Unhandled exception: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) at org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) at org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) at java.lang.Thread.run(Thread.java:662) 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [] 2012-03-11 11:55:37,808 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, blockCacheHitCount=87713, blockCacheMissCount=22144560, blockCacheEvictedCount=122, blockCacheHitRatio=0%, blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 2012-03-11 11:55:37,992 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5578: - Fix Version/s: (was: 0.92.2) 0.92.3 NPE when regionserver reported server load, caused rs stop. --- Key: HBASE-5578 URL: https://issues.apache.org/jira/browse/HBASE-5578 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0 Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 Reporter: Storm Lee Priority: Critical Fix For: 0.92.3 Attachments: 5589.txt The regeionserver log: 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server data3,60020,1331286604591: Unhandled exception: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) at org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) at org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) at java.lang.Thread.run(Thread.java:662) 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [] 2012-03-11 11:55:37,808 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, blockCacheHitCount=87713, blockCacheMissCount=22144560, blockCacheEvictedCount=122, blockCacheHitRatio=0%, blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 2012-03-11 11:55:37,992 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5578: - Attachment: 5589.txt How about this. Goes through Store and checks all Reader instances for null before using. We were doing this in half the cases already. Converts the NPE into a null warning. Means we don't crash. Puts off having to spend time on why the Reader is null at particular junctures. Should go into 0.94? NPE when regionserver reported server load, caused rs stop. --- Key: HBASE-5578 URL: https://issues.apache.org/jira/browse/HBASE-5578 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0 Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 Reporter: Storm Lee Priority: Critical Fix For: 0.92.2 Attachments: 5589.txt The regeionserver log: 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server data3,60020,1331286604591: Unhandled exception: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) at org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) at org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) at java.lang.Thread.run(Thread.java:662) 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [] 2012-03-11 11:55:37,808 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, blockCacheHitCount=87713, blockCacheMissCount=22144560, blockCacheEvictedCount=122, blockCacheHitRatio=0%, blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 2012-03-11 11:55:37,992 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5578: - Status: Patch Available (was: Open) NPE when regionserver reported server load, caused rs stop. --- Key: HBASE-5578 URL: https://issues.apache.org/jira/browse/HBASE-5578 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0 Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 Reporter: Storm Lee Priority: Critical Attachments: 5589.txt The regeionserver log: 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server data3,60020,1331286604591: Unhandled exception: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) at org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) at org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) at java.lang.Thread.run(Thread.java:662) 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [] 2012-03-11 11:55:37,808 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, blockCacheHitCount=87713, blockCacheMissCount=22144560, blockCacheEvictedCount=122, blockCacheHitRatio=0%, blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 2012-03-11 11:55:37,992 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5578: - Fix Version/s: 0.92.2 NPE when regionserver reported server load, caused rs stop. --- Key: HBASE-5578 URL: https://issues.apache.org/jira/browse/HBASE-5578 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0 Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 Reporter: Storm Lee Priority: Critical Fix For: 0.92.2 Attachments: 5589.txt The regeionserver log: 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server data3,60020,1331286604591: Unhandled exception: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) at org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) at org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) at java.lang.Thread.run(Thread.java:662) 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [] 2012-03-11 11:55:37,808 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, blockCacheHitCount=87713, blockCacheMissCount=22144560, blockCacheEvictedCount=122, blockCacheHitRatio=0%, blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 2012-03-11 11:55:37,992 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.
[ https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5578: - Priority: Critical (was: Major) Thats pretty nasty. Can you reproduce easily Storm? Is it happening all the time? Is this fresh start up or something? You have no content under /hbase? NPE when regionserver reported server load, caused rs stop. --- Key: HBASE-5578 URL: https://issues.apache.org/jira/browse/HBASE-5578 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0 Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0 Reporter: Storm Lee Priority: Critical The regeionserver log: 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server data3,60020,1331286604591: Unhandled exception: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788) at org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994) at org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800) at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678) at java.lang.Thread.run(Thread.java:662) 2012-03-11 11:55:37,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [] 2012-03-11 11:55:37,808 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, blockCacheHitCount=87713, blockCacheMissCount=22144560, blockCacheEvictedCount=122, blockCacheHitRatio=0%, blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100 2012-03-11 11:55:37,992 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled exception: null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
