Add hadoop\lib\native to the HBASE CLASSPATH The $HADOOP_HOME\lib\native contains the snappy libs
Thumbs Up ! KASHIF -----Original Message----- From: Hanish Bansal [mailto:[email protected]] Sent: 11 July 2014 16:08 To: [email protected] Subject: Re: Snappy compression not working with HBase 0.98.3 We are using hbase 0.98.3 with hadoop 2.4.0. Run a compression test using tool, getting below error: [root@IMPETUS-I0141 hbase-0.98.3-hadoop2]# bin/hbase org.apache.hadoop.hbase.util.CompressionTest file:///tmp/test.txt snappy 2014-07-11 16:05:10,572 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available 2014-07-11 16:05:11,006 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2014-07-11 16:05:11,241 INFO [main] util.ChecksumType: Checksum using org.apache.hadoop.util.PureJavaCrc32 2014-07-11 16:05:11,242 INFO [main] util.ChecksumType: Checksum can use org.apache.hadoop.util.PureJavaCrc32C Exception in thread "main" java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:62) at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:131) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:147) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:162) at org.apache.hadoop.hbase.io.compress.Compression$Algorithm.getCompressor(Compression.java:310) at org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultEncodingContext.<init>(HFileBlockDefaultEncodingContext.java:92) at org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:690) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:117) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:109) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:97) at org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:393) at org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:118) at org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:148) On Fri, Jul 11, 2014 at 10:21 AM, Hanish Bansal < [email protected]> wrote: > Hi All, > > Recently i have upgraded HBase environment from 0.94 to 0.98.3. Now > trying to use snappy compression with it. > > I have installed snappy library as per guide mentioned in > https://hbase.apache.org/book/snappy.compression.html > > When i am creating a table with snappy compression enabled, i am > getting below error: > > > hbase(main):001:0> create 'test', {NAME=>'cf1', COMPRESSION=>'SNAPPY'} > 2014-07-08 20:06:33,265 WARN [main] util.NativeCodeLoader: Unable to > load native-hadoop library for your platform... using builtin-java > classes where applicable > > ERROR: java.io.IOException: Compression algorithm 'snappy' previously > failed test. > at > org.apache.hadoop.hbase.util.CompressionTest.testCompression(Compressi > onTest.java:85) > at > org.apache.hadoop.hbase.master.HMaster.checkCompression(HMaster.java:1 > 774) > at > org.apache.hadoop.hbase.master.HMaster.checkCompression(HMaster.java:1 > 767) at > org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1749) > at > org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1784) > at > org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$ > 2.callBlockingMethod(MasterProtos.java:40470) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98) > at > org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.ja > va:73) at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471 > ) at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j > ava:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. > java:615) at java.lang.Thread.run(Thread.java:722) > > > Please let me know if anyone aware of this issue. > > -- > *Thanks & Regards* > *Hanish Bansal* > -- *Thanks & Regards* *Hanish Bansal*
