[
https://issues.apache.org/jira/browse/HBASE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217728#comment-13217728
]
stack commented on HBASE-5074:
------------------------------
I see these in the logs when I run the patch; its a little odd because it says
not using PureJavaCrc32 but will use CRC32 but then prints out stacktrace
anyways:
{code}
2012-02-27 23:34:20,911 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open
region: TestTable,0000150828,1330380684339.ebb37d5d0e2c1f4a8b111830a46e7cbc.
2012-02-27 23:34:20,914 INFO org.apache.hadoop.hbase.regionserver.Store: time
to purge deletes set to 0ms in store null
2012-02-27 23:34:20,930 INFO org.apache.hadoop.hbase.util.ChecksumType:
org.apache.hadoop.util.PureJavaCrc32 not available.
2012-02-27 23:34:20,930 INFO org.apache.hadoop.hbase.util.ChecksumType:
Checksum using java.util.zip.CRC32
2012-02-27 23:34:20,931 WARN org.apache.hadoop.hbase.util.ChecksumType:
org.apache.hadoop.util.PureJavaCrc32C not available.
java.io.IOException: java.lang.ClassNotFoundException:
org.apache.hadoop.util.PureJavaCrc32C
at
org.apache.hadoop.hbase.util.ChecksumFactory.newConstructor(ChecksumFactory.java:65)
at
org.apache.hadoop.hbase.util.ChecksumType$3.initialize(ChecksumType.java:113)
at
org.apache.hadoop.hbase.util.ChecksumType.<init>(ChecksumType.java:148)
at
org.apache.hadoop.hbase.util.ChecksumType.<init>(ChecksumType.java:37)
at
org.apache.hadoop.hbase.util.ChecksumType$3.<init>(ChecksumType.java:100)
at
org.apache.hadoop.hbase.util.ChecksumType.<clinit>(ChecksumType.java:100)
at org.apache.hadoop.hbase.io.hfile.HFile.<clinit>(HFile.java:163)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1252)
at
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:516)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:606)
at org.apache.hadoop.hbase.regionserver.Store$1.call(Store.java:375)
at org.apache.hadoop.hbase.regionserver.Store$1.call(Store.java:370)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.util.PureJavaCrc32C
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.hbase.util.ChecksumFactory.getClassByName(ChecksumFactory.java:97)
at
org.apache.hadoop.hbase.util.ChecksumFactory.newConstructor(ChecksumFactory.java:60)
... 19 more
{code}
I'm not sure on whats happening. It would seem we're using default CRC32 but
then I'm not sure how I get the above exception reading code.
Also, not sure if I have this facility turned on. Its on by default but I don't
see anything in logs saying its on (and I don't have metrics on this cluster,
nor do I have a good handle on before and after regards whether this feature
makes a difference).
I caught this in a heap dump:
{code}
"IPC Server handler 0 on 7003" daemon prio=10 tid=0x00007f4a1410c800 nid=0x24b2
runnable [0x00007f4a20487000]
java.lang.Thread.State: RUNNABLE
at java.util.zip.CRC32.updateBytes(Native Method)
at java.util.zip.CRC32.update(CRC32.java:45)
at org.apache.hadoop.util.DataChecksum.update(DataChecksum.java:223)
at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:240)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
- locked <0x00000006fc68e9d8> (a
org.apache.hadoop.hdfs.BlockReaderLocal)
at
org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1457)
- locked <0x00000006fc68e9d8> (a
org.apache.hadoop.hdfs.BlockReaderLocal)
at
org.apache.hadoop.hdfs.BlockReaderLocal.read(BlockReaderLocal.java:326)
- locked <0x00000006fc68e9d8> (a
org.apache.hadoop.hdfs.BlockReaderLocal)
at
org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
at
org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1760)
at
org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2330)
at
org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2397)
at
org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1333)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1769)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1633)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:328)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:213)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:462)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:482)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:333)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:291)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:256)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:518)
- locked <0x00000006fc67cd70> (a
org.apache.hadoop.hbase.regionserver.StoreScanner)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:401)
- locked <0x00000006fc67cd70> (a
org.apache.hadoop.hbase.regionserver.StoreScanner)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:127)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3388)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3344)
- locked <0x00000006fc67cc50> (a
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3361)
- locked <0x00000006fc67cc50> (a
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4145)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4035)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1957)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1344)
{code}
Maybe its not on? Thanks Dhruba.
> support checksums in HBase block cache
> --------------------------------------
>
> Key: HBASE-5074
> URL: https://issues.apache.org/jira/browse/HBASE-5074
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: D1521.1.patch, D1521.1.patch, D1521.10.patch,
> D1521.10.patch, D1521.10.patch, D1521.10.patch, D1521.10.patch,
> D1521.2.patch, D1521.2.patch, D1521.3.patch, D1521.3.patch, D1521.4.patch,
> D1521.4.patch, D1521.5.patch, D1521.5.patch, D1521.6.patch, D1521.6.patch,
> D1521.7.patch, D1521.7.patch, D1521.8.patch, D1521.8.patch, D1521.9.patch,
> D1521.9.patch
>
>
> The current implementation of HDFS stores the data in one block file and the
> metadata(checksum) in another block file. This means that every read into the
> HBase block cache actually consumes two disk iops, one to the datafile and
> one to the checksum file. This is a major problem for scaling HBase, because
> HBase is usually bottlenecked on the number of random disk iops that the
> storage-hardware offers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
