Thanks for your reply. WritableByteArrayComparable is in the same place in HBase 0.93. I also registered my HBase jar from within Pig i.e register /opt/hbase/hbase-0.93.jar and that command returned without error. I also edited the line in the pig startup script that specifies the path to HBase. Is there some other config file that controls how Pig finds HBase jar? Is there a default location Pig searches for jars that I could copy my jar into?
Cheers, Royston On 2 Feb 2012, at 23:38, Dmitriy Ryaboy <[email protected]> wrote: > "Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.filter.WritableByteArrayComparable" indicates that > you don't have HBase on your classpath, or that the version of HBase you > are testing against moved this class someplace else. We've tested against > the 0.90 series, but not 0.92+... did they refactor something? > > D > > On Thu, Feb 2, 2012 at 12:43 PM, Royston Sellman < > [email protected]> wrote: > >> Hi, >> >> >> >> I'm trying to use Pig 0.9.2 with HBase 0.93 (i.e. the latest from HBase >> trunk) and following the tutorial. >> >> >> >> This line loads the sample file from HDFS successfully: >> >> raw = LOAD 'test/excite-small.log' USING PigStorage('\t') AS (user, time, >> query); >> >> >> >> This line seems to work: >> >> T= FOREACH raw GENERATE CONCAT(CONCAT(user, '\u0000'), time), query; >> >> >> >> Because when I do: >> >> DUMP T; >> >> I get the response I expected. >> >> >> >> But if I then do: >> >> STORE T INTO 'excite' USING >> org.apache.pig.backend.hadoop.hbase.HBaseStorage('colfam1:query'); >> >> I get (in the log): >> >> Pig Stack Trace >> >> --------------- >> >> ERROR 2998: Unhandled internal error. >> org/apache/hadoop/hbase/filter/WritableByteArrayComparable >> >> >> >> java.lang.NoClassDefFoundError: >> org/apache/hadoop/hbase/filter/WritableByteArrayComparable >> >> at java.lang.Class.forName0(Native Method) >> >> at java.lang.Class.forName(Class.java:247) >> >> at >> org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:428) >> >> at >> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:458) >> >> at >> >> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder >> .java:723) >> >> at >> >> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.ja >> va:712) >> >> at >> >> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator. >> java:4340) >> >> at >> >> org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator >> .java:5956) >> >> at >> >> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.ja >> va:1122) >> >> at >> >> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGene >> rator.java:683) >> >> at >> >> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.ja >> va:483) >> >> at >> >> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:3 >> 69) >> >> at >> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:171) >> >> at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609) >> >> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582) >> >> at org.apache.pig.PigServer.registerQuery(PigServer.java:584) >> >> at >> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942) >> >> at >> >> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser. >> java:386) >> >> at >> >> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188 >> ) >> >> at >> >> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164 >> ) >> >> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) >> >> at org.apache.pig.Main.run(Main.java:495) >> >> at org.apache.pig.Main.main(Main.java:111) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> at >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39 >> ) >> >> at >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl >> .java:25) >> >> at java.lang.reflect.Method.invoke(Method.java:597) >> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >> >> Caused by: java.lang.ClassNotFoundException: >> org.apache.hadoop.hbase.filter.WritableByteArrayComparable >> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> >> at java.security.AccessController.doPrivileged(Native Method) >> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> >> ... 28 more >> >> >> >> Grateful for any help with this. >> >> >> >> Thanks, >> >> Royston >> >>
