[
https://issues.apache.org/jira/browse/HBASE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614498#action_12614498
]
Andrew Purtell commented on HBASE-750:
--------------------------------------
Regarding the performance test itself, these were my findings:
I created a test table with a single column family and a single row. The test
added 1000 cells at a time, using unique qualified column family addresses,
into the single column family in that single row. The test attempted to add
eventually 1M cells to the family/row and hence 1M unique qualified column
family members. During the test both the time required for insert of 1000 cells
in batch, the time required to read back one cell from the row, and the time
required to read back all cells from the row were measured.
I executed the test using a test cluster running Hadoop 0.17.1 and HBase trunk
(0.2.0-dev) revision 675659. The test client and regionserver ran on different
hosts linked by a Gigabit Ethernet network.
The time required to insert 1000 cells remained within a narrow range over the
duration of the test, ranging from 6 to 85 milliseconds.
The time required to retrieve a single cell from the column/row remained within
a narrower range over the duration of the test, ranging from 1 to 42
milliseconds.
The time required to read back all values in the column/row increased in
essentially a linear manner over the duration of the test, rising from 18
milliseconds to 6,064 milliseconds from 1,000 cells to 865,000 cells. See
attached chart. I don't find this surprising and I do not think this behavior
indicates a bug. Maybe the slope and possibly also some of the variability can
be decreased through profile driven changes, but there will always be this kind
of linear increase as cells are added to a column family/row in this manner.
> NPE caused by StoreFileScanner.updateReaders
> --------------------------------------------
>
> Key: HBASE-750
> URL: https://issues.apache.org/jira/browse/HBASE-750
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.2.0
> Environment: CentOS 5.1 x86_64 / 64-bit JDK build 1.6.0_03-b05
> Reporter: Andrew Purtell
> Attachments: chart.png, columntest.java,
> HRegionServer-sjdc-atr-test-4.log, output.txt
>
>
> Running a test to determine performance during inserts of many 100,000s of
> cells into a single column family in a single row, the region server involved
> went down after taking a NPE:
> 2008-07-17 18:12:18,051 FATAL org.apache.hadoop.hbase.regionserver.Flusher:
> Replay of hlog required. Forcing server restart
> org.apache.hadoop.hbase.DroppedSnapshotException
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
> at
> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
> at org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
> Caused by: java.lang.NullPointerException
> at java.lang.String.<init>(String.java:516)
> at org.apache.hadoop.hbase.util.Bytes.toString(Bytes.java:71)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.updateReaders(StoreFileScanner.java:374)
> at
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:797)
> at
> org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:784)
> at
> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:755)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
> ... 3 more
> Any ideas about this one?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
