When I am reading the Hfile using the './hbase
org.apache.hadoop.hbase.io.hfile.Hfile' tool (the HFile which is compressed
with SNAPPY & encrypted with AES)
I am getting the below error "Problem reading HFile Trailer by using HFile tool"
Is there any problem in the usage of the below command, or is it a bug.
------------------------------------------------------------------------------------------
Version details: Hadoop 2.4.1, HBase 0.98.3
Configuration Details: (enabled Hfile and Wal encryption AES, as below)
hfile.format.version=3
hbase.crypto.keyprovider=org.apache.hadoop.hbase.io.crypto.KeyStoreKeyProvider
hbase.crypto.keyprovider.parameters=
jceks:///opt/shankar1/kdc_keytab/hbase.jks?password=shankar@234
We created a table with SNAPPY compression
> Create 't3', {NAME => 'cf1', COMPRESSION => 'SNAPPY'}
> Put 't3','r1','cf1:a','1000'
> Flush 't3'
shankar1@host1:~/DataSight/hbase/bin> ./hbase
org.apache.hadoop.hbase.io.hfile.HFile -v -f
hdfs://host1:65110/hbase/data/default/t3/337d2996bed579340a702feaa3d3f165/cf1/5817635667d7457989b6d0b0be25dbc4
2014-08-01 19:18:28,368 INFO [main] Configuration.deprecation:
hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2014-08-01 19:18:28,504 INFO [main] util.ChecksumType: Checksum using
org.apache.hadoop.util.PureJavaCrc32
2014-08-01 19:18:28,506 INFO [main] util.ChecksumType: Checksum can use
org.apache.hadoop.util.PureJavaCrc32C
2014-08-01 19:18:28,739 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2014-08-01 19:18:29,082 INFO [main] hdfs.DFSClient: Set
dfs.client.block.write.replace-datanode-on-failure.replication to 0
2014-08-01 19:18:29,406 INFO [main] Configuration.deprecation: fs.default.name
is deprecated. Instead, use fs.defaultFS
Scanning ->
hdfs://host1:65110/hbase/data/default/t3/337d2996bed579340a702feaa3d3f165/cf1/5817635667d7457989b6d0b0be25dbc4
2014-08-01 19:18:29,409 INFO [main] hdfs.DFSClient: Set
dfs.client.block.write.replace-datanode-on-failure.replication to 0
INFO: Watching file:/opt/shankar1/DataSight/hbase/conf/log4j.properties for
changes with interval : 60000
2014-08-01 19:18:29,779 ERROR [main] hfile.HFilePrettyPrinter: Error reading
hdfs://host1:65110/hbase/data/default/t3/337d2996bed579340a702feaa3d3f165/cf1/5817635667d7457989b6d0b0be25dbc4
org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
Trailer from file
hdfs://host1:65110/hbase/data/default/t3/337d2996bed579340a702feaa3d3f165/cf1/5817635667d7457989b6d0b0be25dbc4
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:552)
at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:595)
at
org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.processFile(HFilePrettyPrinter.java:217)
at
org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.run(HFilePrettyPrinter.java:196)
at org.apache.hadoop.hbase.io.hfile.HFile.main(HFile.java:873)
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native
Method)
at
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
at
org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:190)
at
org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:176)
at
org.apache.hadoop.hbase.io.compress.Compression$Algorithm.getDecompressor(Compression.java:336)
at
org.apache.hadoop.hbase.io.compress.Compression.decompress(Compression.java:433)
at
org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultDecodingContext.prepareDecoding(HFileBlockDefaultDecodingContext.java:91)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1522)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1314)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlock(HFileBlock.java:1151)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlockWithBlockType(HFileBlock.java:1159)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.<init>(HFileReaderV2.java:146)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV3.<init>(HFileReaderV3.java:72)
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:542)
... 4 more
Scanned kv count -> 0
shankar1@host1:~/DataSight/hbase/bin>
Thanks
-Shankar
[X]
This e-mail and its attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed above.
Any use of the information contained herein in any way (including, but not
limited to, total or partial disclosure, reproduction, or dissemination) by
persons other than the intended recipient(s) is prohibited. If you receive this
e-mail in error, please notify the sender by phone or email immediately and
delete it!
[X]
