I moved forward, but I am still not able to do anything.
Here's what I am doing:
================================
grunt> register /usr/local/hadoop/pigtmp/pig-0.6.0/lib/ElephantBird.jar
grunt> register /usr/local/hadoop/pigtmp/pig-0.6.0/lib/hbase-0.20.0.jar
grunt> register /usr/local/hadoop/pigtmp/pig-0.6.0/lib/hbase-0.20.0-test.jar
grunt> register
/usr/local/hadoop/pigtmp/pig-0.6.0/lib/zookeeper-hbase-1329.jar
grunt> a = load 'hbase://silk1' USING
com.twitter.elephantbird.pig.load.HBaseLoader('f1:destination_port') AS
(destination_port);
2010-06-30 13:44:02,583 [main] INFO
com.twitter.elephantbird.pig.load.HBaseLoader - no-arg constructor
2010-06-30 13:44:02,602 [main] INFO
com.twitter.elephantbird.pig.load.HBaseLoader - no-arg constructor
grunt> dump a;
2010-06-30 13:44:05,321 [main] INFO
com.twitter.elephantbird.pig.load.HBaseLoader - no-arg constructor
2010-06-30 13:44:05,337 [main] INFO
com.twitter.elephantbird.pig.load.HBaseLoader - no-arg constructor
2010-06-30 13:44:05,452 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 1
2010-06-30 13:44:05,452 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 1
2010-06-30 13:44:08,361 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2010-06-30 13:44:08,394 [Thread-12] WARN org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2010-06-30 13:44:08,597 [Thread-12] INFO
com.twitter.elephantbird.pig.load.HBaseLoader - no-arg constructor
2010-06-30 13:44:08,597 [Thread-12] INFO
com.twitter.elephantbird.pig.load.HBaseLoader - tablename: hbase://silk1
2010-06-30 13:44:08,651 [Thread-12] ERROR
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper - no clientPort found in
zoo.cfg
2010-06-30 13:44:08,652 [Thread-12] ERROR
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper - no clientPort found in
zoo.cfg
2010-06-30 13:44:08,652 [Thread-12] ERROR
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper - no clientPort found in
zoo.cfg
2010-06-30 13:44:08,653 [Thread-12] ERROR
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper - no clientPort found in
zoo.cfg
2010-06-30 13:44:09,390 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Cannot get jobid for this job
2010-06-30 13:44:09,391 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2010-06-30 13:44:09,391 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map reduce job(s) failed!
2010-06-30 13:44:09,402 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed to produce result in:
"hdfs://fchadoop01:54310/tmp/temp642740681/tmp668990886"
2010-06-30 13:44:09,403 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed!
2010-06-30 13:44:09,405 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 2997: Unable to recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to
create input slice for: hbase://silk1
Details at logfile: /usr/local/hadoop/pigtmp/pig-0.6.0/pig_1277919819066.log
grunt>
================================
Here's the log file
================================
Backend error message during job submission
-------------------------------------------
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to
create input slice for: hbase://silk1
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:269)
at
org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:810)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:781)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Could not read quorum servers from zoo.cfg
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.<init>(ZooKeeperWrapper.java:81)
at
org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher.getZooKeeperWrapper(HConnectionManager.java:199)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getZooKeeperWrapper(HConnectionManager.java:878)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:894)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:573)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:555)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:686)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:582)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:555)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:686)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:586)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:549)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:125)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:103)
at com.twitter.elephantbird.pig.load.HBaseLoader.ensureTable(Unknown
Source)
at com.twitter.elephantbird.pig.load.HBaseLoader.validate(Unknown
Source)
at
org.apache.pig.impl.io.ValidatingInputFileSpec.validate(ValidatingInputFileSpec.java:59)
at
org.apache.pig.impl.io.ValidatingInputFileSpec.<init>(ValidatingInputFileSpec.java:44)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:240)
... 7 more
Pig Stack Trace
---------------
ERROR 2997: Unable to recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to
create input slice for: hbase://silk1
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias a
at org.apache.pig.PigServer.openIterator(PigServer.java:482)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:539)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75)
at org.apache.pig.Main.main(Main.java:352)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2997:
Unable to recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to
create input slice for: hbase://si\
lk1
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:176)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:253)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:249)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:781)
at org.apache.pig.PigServer.store(PigServer.java:529)
at org.apache.pig.PigServer.openIterator(PigServer.java:465)
... 6 more
================================
I really appreciate all your help.
On Tue, Jun 29, 2010 at 2:51 PM, Dmitriy Ryaboy <[email protected]> wrote:
> you need to register the jar before using it:
>
> register /path/to/elephant-bird/eb.jar
>
> On Tue, Jun 29, 2010 at 11:47 AM, Pavel Gutin <[email protected]>
> wrote:
>
> > I have been trying it with 0.7.
> > I switched to 0.6 but I get the following error
> >
> > Could not resolve com.twitter.elephantbird.pig.load.HBaseLoader using
> > imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> >
> > Where do I need to add the JAR in my config in order to get pig to see
> it?
> >
> >
> > On Tue, Jun 29, 2010 at 12:43 PM, Dmitriy Ryaboy <[email protected]>
> > wrote:
> >
> > > What version of pig are you running?
> > > Elephant Bird works with Pig 0.6, not 0.7
> > >
> > > There is another version on the Jira that works with 0.7 but has a few
> > > drawbacks (the big one being that it expects everything to be strings)
> > > -D
> > >
> > > On Tue, Jun 29, 2010 at 8:10 AM, Pavel Gutin <[email protected]>
> > wrote:
> > >
> > > > Thank you for trying to help me out. Here's the error that's in my
> log
> > > file
> > > >
> > > > ERROR 2998: Unhandled internal error. org/apache/pig/Slicer
> > > >
> > > > java.lang.NoClassDefFoundError: org/apache/pig/Slicer
> > > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > > at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
> > > > at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
> > > > at
> > > >
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> > > > at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> > > > at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> > > > at java.security.AccessController.doPrivileged(Native Method)
> > > > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > > > at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:296)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> > > > at java.lang.Class.forName0(Native Method)
> > > > at java.lang.Class.forName(Class.java:247)
> > > > at
> > > > org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:422)
> > > > at
> > > >
> > >
> >
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:452)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.NonEvalFuncSpec(QueryParser.java:5087)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.LoadClause(QueryParser.java:1434)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1245)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:911)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:700)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
> > > > at
> > org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1164)
> > > > at
> > > org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1114)
> > > > at org.apache.pig.PigServer.registerQuery(PigServer.java:425)
> > > > at
> > > >
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:737)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:324)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138)
> > > > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75)
> > > > at org.apache.pig.Main.main(Main.java:357)
> > > > Caused by: java.lang.ClassNotFoundException: org.apache.pig.Slicer
> > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > > > at java.security.AccessController.doPrivileged(Native Method)
> > > > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > > > at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> > > > ... 32 more
> > > >
> > > >
> > > > On Mon, Jun 28, 2010 at 7:56 PM, Dmitriy Ryaboy <[email protected]>
> > > > wrote:
> > > >
> > > > > Pavel,
> > > > > What does the log say?
> > > > >
> > > > > I am guessing you need to a) make sure that all the hbase config
> > stuff
> > > is
> > > > > on
> > > > > the classpath and b) load 'hbase://silk1' (no host:port)
> > > > >
> > > > > -D
> > > > >
> > > > > On Mon, Jun 28, 2010 at 10:37 AM, Pavel Gutin <
> [email protected]>
> > > > > wrote:
> > > > >
> > > > > > My apologies, i pasted the wrong line. I was testing to see if
> pig
> > > was
> > > > > able
> > > > > > to locate the JAR by misspelling the name on purpose
> > > > > >
> > > > > > Here's the correct error.
> > > > > >
> > > > > > grunt> a = load 'hbase://localhost:60000/silk1' USING
> > > > > >
> > com.twitter.elephantbird.pig.load.HBaseLoader('f1:destination_port')
> > > AS
> > > > > > (destination_port);
> > > > > > 2010-06-28 13:19:01,288 [main] ERROR
> > org.apache.pig.tools.grunt.Grunt
> > > -
> > > > > > ERROR 2998: Unhandled internal error. org/apache/pig/Slicer
> > > > > > Details at logfile:
> > > > > > /usr/local/hadoop/pigtmp/pig-0.7.0/pig_1277744862785.log
> > > > > > grunt>
> > > > > >
> > > > > >
> > > > > > On Mon, Jun 28, 2010 at 1:34 PM, Pavel Gutin <
> [email protected]
> > >
> > > > > wrote:
> > > > > >
> > > > > > > This seems like it might work for me. I downloaded it, compiled
> > it,
> > > > and
> > > > > > > added the JAR to PIG_CLASSPATH
> > > > > > >
> > > > > > > However, when i try to run the following command, i get an
> error
> > > > > > >
> > > > > > > grunt> a = load 'hbase://myTable' USING
> > > > > > > co.twitter.elephantbird.pig.load.HBaseLoader('f1:col1') AS
> > (col1);
> > > > > > > 2010-06-28 13:13:59,607 [main] ERROR
> > > org.apache.pig.tools.grunt.Grunt
> > > > -
> > > > > > > ERROR 1070: Could not resolve
> > > > > > co.twitter.elephantbird.pig.load.HBaseLoader
> > > > > > > using imports: [, org.apache.pig.builtin.,
> > > > > org.apache.pig.impl.builtin.]
> > > > > > > Details at logfile:
> > > > > > > /usr/local/hadoop/pigtmp/pig-0.7.0/pig_1277744862785.log
> > > > > > > grunt>
> > > > > > >
> > > > > > > I have a feeling I am not referencing the table the right way.
> > > > > > >
> > > > > > > On Mon, Jun 28, 2010 at 11:43 AM, Dmitriy Ryaboy <
> > > [email protected]
> > > > > > >wrote:
> > > > > > >
> > > > > > >> There's an HBase LoadFunc that works with 0.6 in
> Elephant-Bird.
> > > > > > >> http://github.com/kevinweil/elephant-bird
> > > > > > >>
> > > > > > >> There are slides here that show usage:
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://squarecog.wordpress.com/2010/05/20/pig-hbase-hadoop-and-twitter-hug-talk-slides/
> > > > > > >>
> > > > > > >> -D
> > > > > > >>
> > > > > > >> On Mon, Jun 28, 2010 at 7:59 AM, Pavel Gutin <
> > > [email protected]>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > I am trying to get Pig to query my HBase table, but I cannot
> > > find
> > > > > any
> > > > > > >> > examples on the web. Can anyone provide me with a simple
> > > example?
> > > > > > >> >
> > > > > > >> > The best I could find so far, was a little blurb on the
> > > following
> > > > > page
> > > > > > >> >
> > > > http://wiki.apache.org/pig/PigMultiQueryPerformanceSpecificationbut
> > > > > > >> that
> > > > > > >> > didn't help much.
> > > > > > >> >
> > > > > > >> > Thanks in advance.
> > > > > > >> >
> > > > > > >> > - Pavel
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
