Hi, everyone,
I often get an NPE during the YCSB load process if timeseries is set and I'm
loading lots of records. Here's the syntax I'm using:
YCSB Client 0.1
Command line: -load -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada
-p columnfamily=family -p recordcount=10000000 -threads 32 -s -p
measurementtype=timeseries -p timeseries.granularity=5000
100 million records load successfully with 32 threads if timeseries is not set.
But when I set timeseries, I often get NPEs with stacks that move around
depending on the number of records and the timeseries granularity.
For example:
1) 10 million records gets an NPE when timeseries is enabled with a granularity
of 5000 ms (but 4000 ms loads successfully):
java.lang.NullPointerException
at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:916)
at
org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:109)
at
org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:101)
at java.util.TreeMap$NavigableSubMap.checkUpperBound(TreeMap.java:2666)
at java.util.TreeMap$NavigableSubMap.isInRange(TreeMap.java:2688)
at java.util.TreeMap$NavigableSubMap.get(TreeMap.java:2502)
at
org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:96)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:826)
2) 100 million records gets an NPE with 4000 ms (note the changed stack):
java.lang.NullPointerException
at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:916)
at
org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:109)
at
org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:101)
at java.util.TreeMap$NavigableSubMap.cmp(TreeMap.java:2748)
at
java.util.TreeMap$NavigableSubMap.findLowerEntryImpl(TreeMap.java:2881)
at java.util.TreeMap$NavigableSubMap.findLowerEntry(TreeMap.java:3000)
at java.util.TreeMap$NavigableSubMap.smallerEntry(TreeMap.java:2994)
at
java.util.TreeMap$AscendingSubMapIterator.getBoundaryNode(TreeMap.java:281)
at java.util.TreeMap$AbstractSubMapIterator.<init>(TreeMap.java:186)
at java.util.TreeMap$AscendingSubMapKeySet.iterator(TreeMap.java:1968)
at java.util.TreeMap$NavigableSubMap.isEmpty(TreeMap.java:2477)
at
org.apache.hadoop.hbase.util.SoftValueSortedMap.isEmpty(SoftValueSortedMap.java:151)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:826)
3) 100 million records gets an NPE with 3000 ms (and the stack changes):
java.lang.NullPointerException
at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:916)
at
org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:109)
at
org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:101)
at java.util.TreeMap.cmp(TreeMap.java:4514)
at java.util.TreeMap.get(TreeMap.java:4386)
at
org.apache.hadoop.hbase.util.SoftValueSortedMap.get(SoftValueSortedMap.java:96)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getCachedLocation(HConnectionManager.java:830)
I found HBASE-3878, but it looks like that was fixed in 0.90.4.
I also found HBASE-5088, which is marked fixed in 0.90.6 -- does anyone
recognize if this problem fits that profile?
many thanks,
-jean
