Looks like it: http://archive.cloudera.com/cdh/3/pig-0.8.0+20.3.CHANGES.txt
I am assuming the issue was PIG-1680 and it shows that your change was rolled in to the update. Thanks a bunch. I'll try it out. daniel On Tue, Apr 12, 2011 at 6:42 PM, Dmitriy Ryaboy <[email protected]> wrote: > Daniel, > Please upgrade your pig version to the latest in the 0.8 branch. The 0.8 > release is not compatible with 0.20+ versions of hbase; we bumped up the > support in 0.8.1, which is nearing release. Cloudera's latest CDH3 GA might > have these patches (it was just released today) but CDH3B4 didn't. > > D > > > On Tue, Apr 12, 2011 at 3:38 PM, Daniel Eklund <[email protected]> wrote: > >> Interesting. My exact stacktrace is: >> >> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed >> out >> trying to locate root region >> at >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280) >> at >> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:944) >> at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:961) >> at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170) >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880) >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:396) >> at >> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) >> at >> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833) >> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807) >> at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) >> at >> >> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) >> at >> org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) >> at java.lang.Thread.run(Thread.java:662) >> Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: >> Timed >> out trying to locate root region >> at >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:983) >> at >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:625) >> at >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:601) >> at >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:670) >> at >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:630) >> >> >> So, I go to >> >> https://repository.cloudera.com/content/repositories/releases/org/apache/hbase/hbase/0.90.1-CDH3B4/hbase-0.90.1-CDH3B4-sources.jar >> to look at HConnectionManager and see that there's no locateRootRegion() >> method there. >> >> So, it looks like while I am running an HBase 0.90, the pig libs show me >> in >> /usr/lib/pig/lib >> hbase-0.20.6.jar zookeeper-hbase-1329.jar >> >> I am not quite sure about the cloudera versus apache versioning schemes >> going on here. >> >> >> >> On Tue, Apr 12, 2011 at 6:35 PM, Bill Graham <[email protected]> >> wrote: >> >> > Can you include more of your stack trace? I'm not sure of the >> > specifics of what is stored where in ZK, but it seems you're timing >> > out just trying to connect to ZK. Are you seeing any exceptions on the >> > TT nodes, or just on the client? >> > >> > >> > On Tue, Apr 12, 2011 at 3:24 PM, Daniel Eklund <[email protected]> >> wrote: >> > > Bill, I have done all that both you and Jameson have suggested and >> still >> > > get the same error. >> > > >> > > I can telnet into the zookeeper. I have also used the zkClient.sh and >> > can >> > > look at /hbase/rs to see the regionservers. >> > > Should I be able to see anything at /hbase/root-region-server? >> > > >> > > thanks, >> > > daniel >> > > >> > > >> > > On Tue, Apr 12, 2011 at 11:58 AM, Bill Graham <[email protected]> >> > wrote: >> > > >> > >> Yes, Pig's HBaseStorage using the HBase client to read/write directly >> > >> to HBase from within a MR job, but chains to other Pig-generated MR >> > >> jobs as needed to transform. >> > >> >> > >> Daniel, check that you have defined HBASE_CONF_DIR properly, or that >> > >> you have hbase-site.xml in your classpath. Then try to telnet to the >> > >> defined zookeeper host from the machine where the exception is being >> > >> generated. There is some communication from Pig to HBase/ZK from the >> > >> node that the client runs on before the MR jobs start on the cluster >> > >> FYI. >> > >> >> > >> >> > >> On Tue, Apr 12, 2011 at 8:40 AM, Jameson Lopp <[email protected]> >> > wrote: >> > >> > I'm by no means an expert, but I think it's the latter. My >> rudimentary >> > >> > understanding is that pig uses HBaseStorage to load the data from >> > hbase >> > >> and >> > >> > passes the input splits along to hadoop/MR. Feel free to correct me >> if >> > >> I'm >> > >> > wrong. >> > >> > -- >> > >> > Jameson Lopp >> > >> > Software Engineer >> > >> > Bronto Software, Inc. >> > >> > >> > >> > On 04/12/2011 10:50 AM, Daniel Eklund wrote: >> > >> >> >> > >> >> As a follow-up to my own question, which accurately describes the >> > >> >> component >> > >> >> call-stack of the pig script I included in my post? >> > >> >> >> > >> >> pig -> mapreduce/hadoop -> Hbase >> > >> >> pig -> Hbase -> mapreduce/hadoop >> > >> >> >> > >> >> >> > >> >> >> > >> >> On Tue, Apr 12, 2011 at 9:53 AM, Daniel Eklund<[email protected] >> > >> > >> wrote: >> > >> >> >> > >> >>> This question might be better diagnosed as an Hbase issue, but >> since >> > >> it's >> > >> >>> ultimately a Pig script I want to use, I figure someone on this >> > group >> > >> >>> could >> > >> >>> help me out. I tried asking the IRC channel, but I think it was >> in a >> > >> >>> lull. >> > >> >>> >> > >> >>> My scenario: I want to use Pig to call an HBase store. >> > >> >>> My installs: Apache Pig version 0.8.0-CDH3B4 --- hbase >> version: >> > >> >>> hbase-0.90.1-CDH3B4. >> > >> >>> My sample script: >> > >> >>> >> > >> >>> ----------- >> > >> >>> A = load 'passwd' using PigStorage(':'); >> > >> >>> rawDocs = LOAD 'hbase://daniel_product' >> > >> >>> USING >> > >> >>> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage('base:testCol1'); >> > >> >>> vals = foreach rawDocs generate $0 as val; >> > >> >>> dump vals; >> > >> >>> store vals into 'daniel.out'; >> > >> >>> ----------- >> > >> >>> >> > >> >>> I am consistently getting a >> > >> >>> Failed Jobs: >> > >> >>> JobId Alias Feature Message Outputs >> > >> >>> N/A rawDocs,vals MAP_ONLY Message: >> > >> >>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: >> > Timed >> > >> >>> out >> > >> >>> trying to locate root region >> > >> >>> at >> > >> >>> >> > >> >>> >> > >> >> > >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280) >> > >> >>> >> > >> >>> >> > >> >>> Googling shows me similar issues: >> > >> >>> >> > >> >>> >> > >> >>> >> > >> >> > >> http://search-hadoop.com/m/RPLkD1bmY4l&subj=Re+Cannot+connect+HBase+to+Pig >> > >> >>> >> > >> >>> My current understanding is that somewhere in the interaction >> > between >> > >> >>> Pig, >> > >> >>> Hadoop, HBase, and Zookeper, there is a configuration file that >> > needs >> > >> to >> > >> >>> be >> > >> >>> included in a classpath or a configuration directory somewhere. >> I >> > have >> > >> >>> tried various combinations of making hadoop aware of Hbase and >> > >> >>> vice-versa. >> > >> >>> I have tried ZK running on its own, and also managed by HBase. >> > >> >>> >> > >> >>> Can someone explain the dependencies here? Any insight as to >> what I >> > am >> > >> >>> missing? What would your diagnosis of the above message be? >> > >> >>> >> > >> >>> thanks, >> > >> >>> daniel >> > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> > >> >> >> > >> > >> > >> >> > > >> > >> > >
