[
https://issues.apache.org/jira/browse/HBASE-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662587#action_12662587
]
stack commented on HBASE-1114:
------------------------------
Hmm... the code:
{code}
869 for (FileStatus fstatus:fs.listStatus(path)) {
870 len += fstatus.getLen();
871 }
{code}
.. is hiding fact that listStatus returns FileStatus [], not an Iterator, and
that it can return null.
Then, the NPE ran into checkOOME. Because the NPE was not an OOME, we tried
all combinations of the search for OOME the last of which is looking for
'java.lang.OutOfMemoryError' in the Throwable's message only in this case,
going by stack traces above, it looks like message could be null.
Patch is small:
{code}
drwxr-xr-x+ 74 stack stack 2516 Jan 9 16:22 .
Index: src/java/org/apache/hadoop/hbase/regionserver/HStore.java
===================================================================
--- src/java/org/apache/hadoop/hbase/regionserver/HStore.java (revision
732490)
+++ src/java/org/apache/hadoop/hbase/regionserver/HStore.java (working copy)
@@ -866,8 +866,10 @@
return null;
}
int len = 0;
- for (FileStatus fstatus:fs.listStatus(path)) {
- len += fstatus.getLen();
+ // listStatus can come back null.
+ FileStatus [] fss = this.fs.listStatus(path);
+ for (int ii = 0; fss != null && i < fss.length; ii++) {
+ len += fss[ii].getLen();
}
fileSizes[i] = len;
totalSize += len;
Index: src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
===================================================================
--- src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
(revision 732490)
+++ src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
(working copy)
@@ -695,8 +695,9 @@
private boolean checkOOME(final Throwable e) {
boolean stop = false;
if (e instanceof OutOfMemoryError ||
- (e.getCause()!= null && e.getCause() instanceof OutOfMemoryError) ||
- e.getMessage().contains("java.lang.OutOfMemoryError")) {
+ (e.getCause() != null && e.getCause() instanceof OutOfMemoryError) ||
+ (e.getMessage() != null &&
+ e.getMessage().contains("java.lang.OutOfMemoryError"))) {
LOG.fatal("OutOfMemoryError, aborting.", e);
abort();
stop = true;
{code}
> Weird NPEs compacting
> ---------------------
>
> Key: HBASE-1114
> URL: https://issues.apache.org/jira/browse/HBASE-1114
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
>
> Over on apurtell cluster, hbase TRUNK on hadoop 0.18.0 branch, see below.
> They are weird because first would seem to be an empty fileystem data member
> and the second's line number is start of the method and even then, we check
> for null everywhere first before doing stuff in here (because of Erik Holstad
> experience in past).
> {code}
> 2009-01-09 16:28:21,262 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region
> content,c84bbfc94b2143e41ba119d159be2958,1231518442461
> 2009-01-09 16:28:21,265 ERROR
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed
> for region content,c84bbfc94b2143e41ba119d159be2958,1231518442461
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:869)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:709)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:666)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:105)
> 2009-01-09 16:28:45,896 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.lang.NullPointerException
> 2009-01-09 16:28:45,896 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 2 on 60020, call next(3999959915225939603, 30) from
> XX.XX.XX.33:36665: error: java.io.IOException: java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOOME(HRegionServer.java:696)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:664)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:648)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1579)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894)
> 2009-01-09 16:28:47,947 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.lang.NullPointerException
> 2009-01-09 16:28:47,948 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 9 on 60020, call next(3999959915225939603, 30) from
> XX.XX.XX.33:36665: error: java.io.IOException: java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOOME(HRegionServer.java:696)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:664)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:648)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1579)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
