The change happened in 0.94.5 Please see HBASE-5458
Cheers On Sun, Jun 9, 2013 at 5:35 AM, Levy Meny <[email protected]> wrote: > Hi, > Anyone knows why org.apache.hadoop.hbase.io.hfile.Compression has changed > in 0.94.2 to use the SystemClassLoader to load the snappy class, instead of > the ContextClassLoader in previous versions (e.g. in 0.92.1)? > > private CompressionCodec buildCodec(Configuration conf) { > try { > Class<?> externalCodec = > ClassLoader.getSystemClassLoader().loadClass("org.apache.hadoop.io.compress.SnappyCodec"); > return (CompressionCodec) > ReflectionUtils.newInstance(externalCodec,conf); > > (btw you will notice that ContextClassLoader is still used for loading > e.g. Lz4Codec) > > The error I got: > > 2013-05-31 00:01:25,704 [ERROR] [BulkImportManager-2-thread-1] > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles > (LoadIncrementalHFiles.java:343) - Unexpected execution exception during > splitting > java.util.concurrent.ExecutionException: java.lang.RuntimeException: > java.lang.ClassNotFoundException: org.apache.hadoop.io.compress.SnappyCodec > at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source) > at java.util.concurrent.FutureTask.get(Unknown Source) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:333) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:232) > ... > Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.hadoop.io.compress.SnappyCodec > at > org.apache.hadoop.hbase.io.hfile.Compression$Algorithm$4.buildCodec(Compression.java:207) > at > org.apache.hadoop.hbase.io.hfile.Compression$Algorithm$4.getCodec(Compression.java:192) > at > org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:302) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:745) > at > org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:134) > at > org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:125) > at > org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:105) > at > org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:394) > at > org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:1003) > at > org.apache.hadoop.hbase.regionserver.StoreFile$Writer.<init>(StoreFile.java:948) > at > org.apache.hadoop.hbase.regionserver.StoreFile$WriterBuilder.build(StoreFile.java:851) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.copyHFileHalf(LoadIncrementalHFiles.java:541) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.splitStoreFile(LoadIncrementalHFiles.java:514) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.splitStoreFile(LoadIncrementalHFiles.java:375) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:439) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321) > at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) > at java.util.concurrent.FutureTask.run(Unknown Source) > ... 3 more > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.io.compress.SnappyCodec > at java.net.URLClassLoader$1.run(Unknown Source) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(Unknown Source) > at java.lang.ClassLoader.loadClass(Unknown Source) > at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) > at java.lang.ClassLoader.loadClass(Unknown Source) > at > org.apache.hadoop.hbase.io.hfile.Compression$Algorithm$4.buildCodec(Compression.java:201) > ... 21 more > > This change make it impossible for me to bulk upload files into HBase from > my app when my app runs as a Tomcat web application since Tomcat does not > allow setting the System class loader and all HBase and Hadoop jars are in > my WEB-INF/lib and are not available to the System class loader.. > > Thanks, > > Meny Levy > R&D > > T +972-3-7663350 > M +972-52-8543350 > [email protected] <mailto:[email protected]> > www.comverse.com<http://www.comverse.com/> > > > ________________________________ > "This e-mail message may contain confidential, commercial or privileged > information that constitutes proprietary information of Comverse Technology > or its subsidiaries. If you are not the intended recipient of this message, > you are hereby notified that any review, use or distribution of this > information is absolutely prohibited and we request that you delete all > copies and contact us by e-mailing to: [email protected]. Thank You." >
