[
https://issues.apache.org/jira/browse/HBASE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855452#comment-16855452
]
Zheng Hu commented on HBASE-22532:
----------------------------------
There are some LEAF_INDEX block too:
{code}
$ cat out.log | grep 'onDiskSizeWithoutHeader=1' | grep -v BLOOM_CHUNK
[blockType=LEAF_INDEX, fileOffset=206262863, headerSize=33,
onDiskSizeWithoutHeader=131134, uncompressedSizeWithoutHeader=131098,
prevBlockOffset=-1, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131131,
getOnDiskSizeWithHeader=131167, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131167, cap= 131200]],
dataBeginsWith=\x00\x00\x0C,\x00\x00\x00\x00\x00\x00\x005\x00\x00\x00Z\x00\x00\x00\x7F\x00\x00\x00\xA5\x00\x00\x00\xCC\x00\x00\x00\xF3,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65697]
[blockType=LEAF_INDEX, fileOffset=412656269, headerSize=33,
onDiskSizeWithoutHeader=131138, uncompressedSizeWithoutHeader=131102,
prevBlockOffset=206262863, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131135,
getOnDiskSizeWithHeader=131171, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131171, cap= 131204]],
dataBeginsWith=\x00\x00\x0C,\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x99\x00\x00\x00\xBF\x00\x00\x00\xE5,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65690]
[blockType=LEAF_INDEX, fileOffset=619246481, headerSize=33,
onDiskSizeWithoutHeader=131133, uncompressedSizeWithoutHeader=131097,
prevBlockOffset=412656269, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131130,
getOnDiskSizeWithHeader=131166, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131166, cap= 131199]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00K\x00\x00\x00q\x00\x00\x00\x97\x00\x00\x00\xBD\x00\x00\x00\xE2,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65689]
[blockType=LEAF_INDEX, fileOffset=825640330, headerSize=33,
onDiskSizeWithoutHeader=131147, uncompressedSizeWithoutHeader=131111,
prevBlockOffset=619246481, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131144,
getOnDiskSizeWithHeader=131180, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131180, cap= 131213]],
dataBeginsWith=\x00\x00\x0C,\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00s\x00\x00\x00\x9A\x00\x00\x00\xC1\x00\x00\x00\xE6,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65680]
[blockType=LEAF_INDEX, fileOffset=1032296868, headerSize=33,
onDiskSizeWithoutHeader=131147, uncompressedSizeWithoutHeader=131111,
prevBlockOffset=825640330, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131144,
getOnDiskSizeWithHeader=131180, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131180, cap= 131213]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x99\x00\x00\x00\xBF\x00\x00\x00\xE5,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65692]
[blockType=LEAF_INDEX, fileOffset=1238756507, headerSize=33,
onDiskSizeWithoutHeader=131140, uncompressedSizeWithoutHeader=131104,
prevBlockOffset=1032296868, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131137,
getOnDiskSizeWithHeader=131173, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131173, cap= 131206]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00r\x00\x00\x00\x99\x00\x00\x00\xBF\x00\x00\x00\xE5,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65679]
[blockType=LEAF_INDEX, fileOffset=1445215922, headerSize=33,
onDiskSizeWithoutHeader=131141, uncompressedSizeWithoutHeader=131105,
prevBlockOffset=1238756507, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131138,
getOnDiskSizeWithHeader=131174, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131174, cap= 131207]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00r\x00\x00\x00\x97\x00\x00\x00\xBE\x00\x00\x00\xE4,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65697]
[blockType=LEAF_INDEX, fileOffset=1651675367, headerSize=33,
onDiskSizeWithoutHeader=131108, uncompressedSizeWithoutHeader=131072,
prevBlockOffset=1445215922, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131105,
getOnDiskSizeWithHeader=131141, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131141, cap= 131174]],
dataBeginsWith=\x00\x00\x0C+\x00\x00\x00\x00\x00\x00\x00%\x00\x00\x00J\x00\x00\x00q\x00\x00\x00\x97\x00\x00\x00\xBD\x00\x00\x00\xE3,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65695]
[blockType=LEAF_INDEX, fileOffset=1858069262, headerSize=33,
onDiskSizeWithoutHeader=131137, uncompressedSizeWithoutHeader=131101,
prevBlockOffset=1651675367, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131134,
getOnDiskSizeWithHeader=131170, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131170, cap= 131203]],
dataBeginsWith=\x00\x00\x0C,\x00\x00\x00\x00\x00\x00\x00%\x00\x00\x00L\x00\x00\x00r\x00\x00\x00\x98\x00\x00\x00\xBF\x00\x00\x00\xE6,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65689]
[blockType=LEAF_INDEX, fileOffset=2064725841, headerSize=33,
onDiskSizeWithoutHeader=131112, uncompressedSizeWithoutHeader=131076,
prevBlockOffset=1858069262, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131109,
getOnDiskSizeWithHeader=131145, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131145, cap= 131178]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x9A\x00\x00\x00\xC0\x00\x00\x00\xE5,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65689]
[blockType=LEAF_INDEX, fileOffset=2271185364, headerSize=33,
onDiskSizeWithoutHeader=131130, uncompressedSizeWithoutHeader=131094,
prevBlockOffset=2064725841, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131127,
getOnDiskSizeWithHeader=131163, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131163, cap= 131196]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00(\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x99\x00\x00\x00\xBE\x00\x00\x00\xE5,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65692]
[blockType=LEAF_INDEX, fileOffset=2477644797, headerSize=33,
onDiskSizeWithoutHeader=131121, uncompressedSizeWithoutHeader=131085,
prevBlockOffset=2271185364, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131118,
getOnDiskSizeWithHeader=131154, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131154, cap= 131187]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00%\x00\x00\x00K\x00\x00\x00o\x00\x00\x00\x96\x00\x00\x00\xBD\x00\x00\x00\xE3,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65691]
[blockType=LEAF_INDEX, fileOffset=2684170132, headerSize=33,
onDiskSizeWithoutHeader=131121, uncompressedSizeWithoutHeader=131085,
prevBlockOffset=2477644797, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131118,
getOnDiskSizeWithHeader=131154, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131154, cap= 131187]],
dataBeginsWith=\x00\x00\x0C,\x00\x00\x00\x00\x00\x00\x00$\x00\x00\x00I\x00\x00\x00n\x00\x00\x00\x94\x00\x00\x00\xB9\x00\x00\x00\xDF,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65694]
[blockType=LEAF_INDEX, fileOffset=2890694829, headerSize=33,
onDiskSizeWithoutHeader=131147, uncompressedSizeWithoutHeader=131111,
prevBlockOffset=2684170132, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131144,
getOnDiskSizeWithHeader=131180, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131180, cap= 131213]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00K\x00\x00\x00q\x00\x00\x00\x97\x00\x00\x00\xBC\x00\x00\x00\xE3,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65692]
[blockType=LEAF_INDEX, fileOffset=3097220485, headerSize=33,
onDiskSizeWithoutHeader=131141, uncompressedSizeWithoutHeader=131105,
prevBlockOffset=2890694829, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131138,
getOnDiskSizeWithHeader=131174, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131174, cap= 131207]],
dataBeginsWith=\x00\x00\x0C,\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x99\x00\x00\x00\xBE\x00\x00\x00\xE3,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65701]
[blockType=LEAF_INDEX, fileOffset=3303744549, headerSize=33,
onDiskSizeWithoutHeader=131141, uncompressedSizeWithoutHeader=131105,
prevBlockOffset=3097220485, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131138,
getOnDiskSizeWithHeader=131174, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131174, cap= 131207]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x99\x00\x00\x00\xBE\x00\x00\x00\xE4,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65692]
[blockType=LEAF_INDEX, fileOffset=3510204562, headerSize=33,
onDiskSizeWithoutHeader=131112, uncompressedSizeWithoutHeader=131076,
prevBlockOffset=3303744549, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131109,
getOnDiskSizeWithHeader=131145, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131145, cap= 131178]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x98\x00\x00\x00\xBE\x00\x00\x00\xE5,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65705]
[blockType=LEAF_INDEX, fileOffset=3716794841, headerSize=33,
onDiskSizeWithoutHeader=131126, uncompressedSizeWithoutHeader=131090,
prevBlockOffset=3510204562, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131123,
getOnDiskSizeWithHeader=131159, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131159, cap= 131192]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x99\x00\x00\x00\xBF\x00\x00\x00\xE5,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65680]
[blockType=LEAF_INDEX, fileOffset=3923255347, headerSize=33,
onDiskSizeWithoutHeader=131130, uncompressedSizeWithoutHeader=131094,
prevBlockOffset=3716794841, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131127,
getOnDiskSizeWithHeader=131163, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131163, cap= 131196]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00M\x00\x00\x00t\x00\x00\x00\x9A\x00\x00\x00\xC0\x00\x00\x00\xE5,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65692]
[blockType=LEAF_INDEX, fileOffset=4129845654, headerSize=33,
onDiskSizeWithoutHeader=131137, uncompressedSizeWithoutHeader=131101,
prevBlockOffset=3923255347, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131134,
getOnDiskSizeWithHeader=131170, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131170, cap= 131203]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00r\x00\x00\x00\x98\x00\x00\x00\xBE\x00\x00\x00\xE4,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65687]
[blockType=LEAF_INDEX, fileOffset=4336370991, headerSize=33,
onDiskSizeWithoutHeader=131140, uncompressedSizeWithoutHeader=131104,
prevBlockOffset=4129845654, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131137,
getOnDiskSizeWithHeader=131173, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131173, cap= 131206]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00M\x00\x00\x00t\x00\x00\x00\x9B\x00\x00\x00\xC0\x00\x00\x00\xE6,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65702]
[blockType=LEAF_INDEX, fileOffset=4542897095, headerSize=33,
onDiskSizeWithoutHeader=131125, uncompressedSizeWithoutHeader=131089,
prevBlockOffset=4336370991, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131122,
getOnDiskSizeWithHeader=131158, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131158, cap= 131191]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00%\x00\x00\x00J\x00\x00\x00p\x00\x00\x00\x95\x00\x00\x00\xBB\x00\x00\x00\xE0,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65693]
[blockType=LEAF_INDEX, fileOffset=4749421918, headerSize=33,
onDiskSizeWithoutHeader=131149, uncompressedSizeWithoutHeader=131113,
prevBlockOffset=4542897095, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131146,
getOnDiskSizeWithHeader=131182, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131182, cap= 131215]],
dataBeginsWith=\x00\x00\x0C,\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00r\x00\x00\x00\x99\x00\x00\x00\xBE\x00\x00\x00\xE4,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65696]
[blockType=LEAF_INDEX, fileOffset=4955815833, headerSize=33,
onDiskSizeWithoutHeader=131131, uncompressedSizeWithoutHeader=131095,
prevBlockOffset=4749421918, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131128,
getOnDiskSizeWithHeader=131164, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131164, cap= 131197]],
dataBeginsWith=\x00\x00\x0C,\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x9A\x00\x00\x00\xC0\x00\x00\x00\xE6,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65689]
[blockType=LEAF_INDEX, fileOffset=5162471614, headerSize=33,
onDiskSizeWithoutHeader=131144, uncompressedSizeWithoutHeader=131108,
prevBlockOffset=4955815833, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131141,
getOnDiskSizeWithHeader=131177, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131177, cap= 131210]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00r\x00\x00\x00\x98\x00\x00\x00\xBD\x00\x00\x00\xE3,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65688]
[blockType=LEAF_INDEX, fileOffset=5368997417, headerSize=33,
onDiskSizeWithoutHeader=131145, uncompressedSizeWithoutHeader=131109,
prevBlockOffset=5162471614, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131142,
getOnDiskSizeWithHeader=131178, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131178, cap= 131211]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00%\x00\x00\x00J\x00\x00\x00p\x00\x00\x00\x97\x00\x00\x00\xBE\x00\x00\x00\xE4,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65683]
[blockType=LEAF_INDEX, fileOffset=5575522268, headerSize=33,
onDiskSizeWithoutHeader=131136, uncompressedSizeWithoutHeader=131100,
prevBlockOffset=5368997417, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131133,
getOnDiskSizeWithHeader=131169, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131169, cap= 131202]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00'\x00\x00\x00M\x00\x00\x00u\x00\x00\x00\x9C\x00\x00\x00\xC2\x00\x00\x00\xE8,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65697]
[blockType=LEAF_INDEX, fileOffset=5782178180, headerSize=33,
onDiskSizeWithoutHeader=131146, uncompressedSizeWithoutHeader=131110,
prevBlockOffset=5575522268, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131143,
getOnDiskSizeWithHeader=131179, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131179, cap= 131212]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00r\x00\x00\x00\x9A\x00\x00\x00\xC0\x00\x00\x00\xE8,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65686]
[blockType=LEAF_INDEX, fileOffset=5988703298, headerSize=33,
onDiskSizeWithoutHeader=131142, uncompressedSizeWithoutHeader=131106,
prevBlockOffset=5782178180, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131139,
getOnDiskSizeWithHeader=131175, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131175, cap= 131208]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00q\x00\x00\x00\x97\x00\x00\x00\xBD\x00\x00\x00\xE3,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65674]
[blockType=LEAF_INDEX, fileOffset=6195293786, headerSize=33,
onDiskSizeWithoutHeader=131113, uncompressedSizeWithoutHeader=131077,
prevBlockOffset=5988703298, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131110,
getOnDiskSizeWithHeader=131146, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131146, cap= 131179]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x99\x00\x00\x00\xBE\x00\x00\x00\xE6,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65686]
[blockType=LEAF_INDEX, fileOffset=6401819131, headerSize=33,
onDiskSizeWithoutHeader=131145, uncompressedSizeWithoutHeader=131109,
prevBlockOffset=6195293786, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131142,
getOnDiskSizeWithHeader=131178, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131178, cap= 131211]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00J\x00\x00\x00o\x00\x00\x00\x95\x00\x00\x00\xBA\x00\x00\x00\xE0,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65690]
[blockType=LEAF_INDEX, fileOffset=6608278620, headerSize=33,
onDiskSizeWithoutHeader=131117, uncompressedSizeWithoutHeader=131081,
prevBlockOffset=6401819131, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131114,
getOnDiskSizeWithHeader=131150, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131150, cap= 131183]],
dataBeginsWith=\x00\x00\x0C-\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00L\x00\x00\x00r\x00\x00\x00\x98\x00\x00\x00\xBE\x00\x00\x00\xE3,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65684]
[blockType=LEAF_INDEX, fileOffset=6814934722, headerSize=33,
onDiskSizeWithoutHeader=131127, uncompressedSizeWithoutHeader=131091,
prevBlockOffset=6608278620, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131124,
getOnDiskSizeWithHeader=131160, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131160, cap= 131193]],
dataBeginsWith=\x00\x00\x0C.\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00M\x00\x00\x00s\x00\x00\x00\x98\x00\x00\x00\xBD\x00\x00\x00\xE2,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65689]
[blockType=LEAF_INDEX, fileOffset=7021526031, headerSize=33,
onDiskSizeWithoutHeader=131149, uncompressedSizeWithoutHeader=131113,
prevBlockOffset=6814934722, isUseHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, onDiskDataSizeWithHeader=131146,
getOnDiskSizeWithHeader=131182, totalChecksumBytes=36, isUnpacked=true,
buf=[SingleByteBuff[pos=0, lim=131182, cap= 131215]],
dataBeginsWith=\x00\x00\x0C/\x00\x00\x00\x00\x00\x00\x00&\x00\x00\x00K\x00\x00\x00q\x00\x00\x00\x97\x00\x00\x00\xBC\x00\x00\x00\xE1,
fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
includesTags=false, compressAlgo=NONE, compressTags=false,
cryptoContext=[cipher=NONE keyHash=NONE],
name=123a8968c0d641038c678117c3948bd6], nextBlockOnDiskSize=65700]
{code}
> There's still too much cpu wasting on validating checksum even if
> buffer.size=65KB
> ----------------------------------------------------------------------------------
>
> Key: HBASE-22532
> URL: https://issues.apache.org/jira/browse/HBASE-22532
> Project: HBase
> Issue Type: Sub-task
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Major
> Attachments: async-prof-pid-27827-cpu-3.svg,
> async-prof-pid-64695-cpu-1.svg
>
>
> After disabled the block cache, and with the following config:
> {code}
> # Disable the block cache
> hfile.block.cache.size=0
> hbase.ipc.server.allocator.buffer.size=66560
> hbase.ipc.server.reservoir.minimal.allocating.size=0
> {code}
> The ByteBuff for block should be expected to be a SingleByteBuff, which will
> use the hadoop native lib to validate the checksum, while in the cpu flame
> graph
> [async-prof-pid-27827-cpu-3.svg|https://issues.apache.org/jira/secure/attachment/12970683/async-prof-pid-27827-cpu-3.svg],
> we can still see that about 32% CPU wasted on PureJavaCrc32#update, which
> means it's not using the faster hadoop native lib.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
