I don't think the hadoop conf directory is on the pig classpath. You need to make sure that the following things are on the classpath (the easiest way to do that is to set PIG_CLASSPATH env var): the hadoop conf directory all the jars in the hadoop lib directory guava jar hbase jar zookeeper jar
-D On Tue, Mar 15, 2011 at 4:13 PM, sulabh choudhury <[email protected]> wrote: > The logs do not provide any useful info. > > I am trying to use the new jar from branch-0.8 now. > > Now it pops other errors > Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor > core-site.xml was found in the classpath) > but since I am using Cloudera dist which does not have hadoop-site.xml and > core-site.xml instead it has mapred-site.xml, hdfs-site.xml,core-site.xml > and I have already specified the path to my Hadoop conf folder. > > > > On Tue, Mar 15, 2011 at 3:11 PM, Dmitriy Ryaboy <[email protected]>wrote: > >> Anything interesting in Map task logs? >> >> As for your previous question -- just check out the branch-0.8, run "ant >> jar-withouthadoop" >> and use the the resulting jar. >> >> >> On Tue, Mar 15, 2011 at 11:26 AM, sulabh choudhury <[email protected]>wrote: >> >>> Dmitriy... >>> I just found out that after 20 mins I get throws errors >>> >>> 2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO >>> org.apache.zookeeper.ClientCnxn - Server connection successful >>> 2011-03-15 11:16:39,186 [Thread-4] INFO >>> org.apache.hadoop.mapred.JobClient - Cleaning up the staging area >>> hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033 >>> 2011-03-15 11:16:39,188 [main] INFO >>> >>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >>> - job null has failed! Stop running all dependent jobs >>> 2011-03-15 11:16:39,194 [main] INFO >>> >>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >>> - 100% complete >>> 2011-03-15 11:16:39,203 [main] ERROR >>> org.apache.pig.tools.pigstats.PigStats - ERROR 2997: Unable to recreate >>> exception from backend error: >>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out >>> trying to locate root region >>> 2011-03-15 11:16:39,204 [main] ERROR >>> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! >>> >>> Are the previous errors related or I am making heads-up and start looking >>> into different direction now ? >>> >>> On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury <[email protected]>wrote: >>> >>>> I am a little confused here.... >>>> Can you elaborate a little more on how to achieve the above ( Pig jar >>>> from the pig-0.8 branch) and also then how to use the same to make it work >>>> with Hbase 0.90 ? >>>> >>>> On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy <[email protected]>wrote: >>>> >>>>> The cloudera Pig 0.8 does not work with HBase 0.90 if I recall >>>>> correctly. >>>>> You can compile a Pig jar from the pig-0.8 branch, and it should work >>>>> with 90+ >>>>> >>>>> D >>>>> >>>>> >>>>> On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury >>>>> <[email protected]>wrote: >>>>> >>>>>> I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4) >>>>>> distribution. >>>>>> Yes it does not crash it just hangs after the >>>>>> "org.apache.zookeeper.ClientCnxn - Server connection successful" >>>>>> >>>>>> When I apply the latest patch >>>>>> (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>) >>>>>> all the Hunks failed... >>>>>> >>>>>> Also I see that >>>>>> Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - >>>>>> Attempting connection to server localhost/127.0.0.1:2181 >>>>>> So that means that it is trying to connect to the correct (2181) >>>>>> zookeeper port... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes >>>>>>> out pretty fast. >>>>>>> Sulabh, are you sure that your hbase conf is on the Pig path? >>>>>>> >>>>>>> D >>>>>>> >>>>>>> >>>>>>> On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> What I'm saying is that if your version of HBase is 0.89.0 or >>>>>>>> greater, >>>>>>>> you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0) >>>>>>>> and rebuild. >>>>>>>> >>>>>>>> I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you >>>>>>>> can try applying the patch to that version, but I'd expect you'd run >>>>>>>> into a lot of conflicts. You'd be better off to just upgrade Pig to >>>>>>>> 0.8.0 and apply the patch, or just build Pig from the trunk. >>>>>>>> >>>>>>>> On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury < >>>>>>>> [email protected]> wrote: >>>>>>>> > Thanks Bill for the response. >>>>>>>> > Just to be sure for HBase >=0.89 only Pig-1680 would work and no >>>>>>>> other >>>>>>>> > version, even 0.70 ? >>>>>>>> > >>>>>>>> > On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham < >>>>>>>> [email protected]> wrote: >>>>>>>> >> >>>>>>>> >> What version of Pig and what version of HBase? If HBase >= 0.89.0 >>>>>>>> >> you'll need PIG-1680: >>>>>>>> >> >>>>>>>> >> https://issues.apache.org/jira/browse/PIG-1680 >>>>>>>> >> >>>>>>>> >> On Mon, Mar 14, 2011 at 5:52 PM, SULABH <[email protected]> >>>>>>>> wrote: >>>>>>>> >> > Hi, >>>>>>>> >> > >>>>>>>> >> > I am trying to extract data stored in HBase using Pig. >>>>>>>> >> > I got Pig to work on Hadoop and then also ran a sample M/R job >>>>>>>> and got >>>>>>>> >> > correct results. >>>>>>>> >> > Now when I try to access HBase using the HbaseStorage the >>>>>>>> compiler just >>>>>>>> >> > hangs after >>>>>>>> >> > 2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO >>>>>>>> >> > org.apache.zookeeper.ClientCnxn - Server connection successful >>>>>>>> >> > Even after waiting for several minutes it just remains there. >>>>>>>> There is >>>>>>>> >> > no >>>>>>>> >> > information in the log files too. >>>>>>>> >> > >>>>>>>> >> > Please help! >>>>>>>> >> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> -- >>>>>> Thanks and Regards, >>>>>> Sulabh Choudhury >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> SC >>>> >>>> >>> >>> >>> -- >>> >>> SC >>> >> >> > > > -- > > >
