I had this issue and had to add the HBase conf dir to HADOOP_CLASSPATH in conf/hadoop-env.sh on each of the nodes in the cluster so they could find Zookeeper.
On Sun, Apr 24, 2011 at 1:04 PM, Dmitriy Ryaboy <[email protected]> wrote: > I suspect the problem here is that you don't have your hbase config > directory on Pig's classpath. You can add it by modifying the PIG_CLASSPATH > environment variable. > > D > > On Sun, Apr 24, 2011 at 12:41 PM, sulabh choudhury <[email protected]>wrote: > >> Have you tried loading like :- >> >> *my_data = LOAD 'hbase://test' using >> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:a', '-caching 100') as >> *(a:bytearray)*;dump my_data;* >> or >> *my_data = LOAD 'hbase://test' using >> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:a', '-caching 100') as >> (a);dump my_data;* >> >> I am just trying to see if it works if you supply a different datatype , >> bytearray instead of chararray, or without any datatype at all. >> >> On Sun, Apr 24, 2011 at 10:40 AM, byambajargal <[email protected] >> >wrote: >> >> > Hello guys >> > >> > I am running cloudere distribution cdh3u0 on my cluster and i am trying >> to >> > connect pig with Hbase. I have 11 nodes on my cluster so i have >> configured >> > one machine as HBaseMaster and rest are Hbase Region Servers. >> > On my Master manshine: >> > >> > 1. Output of the command $*sudo jps is :* >> > >> > 6907 HMaster >> > 14108 Jps >> > 31470 QuorumPeerMain >> > 22240 JobTracker >> > 21602 SecondaryNameNode >> > 21155 NameNode >> > >> > 2. Output of the command*$status 'detailed'* >> > >> > -bash-3.2$ hbase shell >> > HBase Shell; enter 'help<RETURN>' for list of supported commands. >> > Type "exit<RETURN>" to leave the HBase Shell >> > Version 0.90.1-cdh3u0, r, Fri Mar 25 16:10:51 PDT 2011 >> > >> > hbase(main):001:0> *status 'detailed'* >> > version 0.90.1-cdh3u0 >> > 0 regionsInTransition >> > 10 live servers >> > haisen2.ux.uis.no:60020 1303287066114 >> > requests=0, regions=1, usedHeap=34, maxHeap=987 >> > -ROOT-,,0 >> > stores=1, storefiles=1, storefileSizeMB=0, memstoreSizeMB=0, >> > storefi >> > leIndexSizeMB=0 >> > haisen10.ux.uis.no:60020 1303287733515 >> > requests=0, regions=1, usedHeap=35, maxHeap=987 >> > task1,,1303326469811.418e9f137f5b1d33604419ef6a8acb6e. >> > stores=1, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0, >> > storefi >> > leIndexSizeMB=0 >> > haisen4.ux.uis.no:60020 1303287334273 >> > requests=0, regions=0, usedHeap=33, maxHeap=987 >> > haisen1.ux.uis.no:60020 1303643944066 >> > requests=0, regions=0, usedHeap=35, maxHeap=987 >> > haisen7.ux.uis.no:60020 1303287560071 >> > requests=0, regions=1, usedHeap=32, maxHeap=987 >> > test,,1303644907708.63ec7397e09d3335c8471aa90bcc5549. >> > stores=1, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0, >> > storefi >> > leIndexSizeMB=0 >> > haisen9.ux.uis.no:60020 1303287672340 >> > requests=0, regions=1, usedHeap=24, maxHeap=987 >> > .META.,,1 >> > stores=1, storefiles=1, storefileSizeMB=0, memstoreSizeMB=0, >> > storefi >> > leIndexSizeMB=0 >> > haisen5.ux.uis.no:60020 1303287407405 >> > requests=0, regions=0, usedHeap=28, maxHeap=987 >> > haisen6.ux.uis.no:60020 1303287480842 >> > requests=0, regions=0, usedHeap=23, maxHeap=987 >> > haisen3.ux.uis.no:60020 1303287189985 >> > requests=0, regions=0, usedHeap=27, maxHeap=987 >> > haisen8.ux.uis.no:60020 1303287626373 >> > requests=0, regions=0, usedHeap=30, maxHeap=987 >> > 0 dead servers >> > >> > >> > I can put data into HBase and pull it out just fine by hbase shell >> command >> > and i have a problem when i load data from hbase by using the following >> Pig >> > query: >> > >> > *my_data = LOAD 'hbase://test' using >> > org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:a', '-caching 100') >> as >> > (a:chararray);dump my_data;* >> > >> > i got a error message and my job was failure. If someone could help me to >> > connect hbase with pig that would be great. >> > >> > >> > the problem I am having now is the following exception on the pig log >> file: >> > >> > Backend error message >> > --------------------- >> > org.apache.hadoop.hbase.ZooKeeperConnectionException: >> > org.apache.hadoop.hbase.ZooKeeperConnectionException: >> > org.apache.zookeeper.KeeperException$ConnectionLossException: >> > KeeperErrorCode = ConnectionLoss for /hbase >> > at >> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:991) >> > at >> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:302) >> > at >> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:293) >> > at >> > >> org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:156) >> > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:167) >> > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145) >> > at >> > >> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:302) >> > at >> > >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.mergeSplitSpecificConf(PigInputFormat.java:133) >> > at >> > >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:111) >> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613) >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322) >> > at org.apache.hadoop.mapred.Child$4.run(Child.java:268) >> > at java.security.AccessController.doPrivileged(Native Method) >> > at javax.security.auth.Subject.doAs(Subject.java:396) >> > at >> > >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) >> > at org.apache.hadoop.mapred.Child.main(Child.java:262) >> > Caused by: org.apache.hadoop.hbase.ZooKeeperConnectionException: >> > org.apache.zookeeper.KeeperException$ConnectionLossException: >> > KeeperErrorCode = ConnectionLoss for /hbase >> > at >> > >> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:147) >> > at >> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:989) >> > ... 15 more >> > Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: >> > KeeperErrorCode = ConnectionLoss for /hbase >> > at >> > org.apache.zookeeper.KeeperException.create(KeeperException.java:90) >> > at >> > org.apache.zookeeper.KeeperException.create(KeeperException.java:42) >> > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637) >> > at >> > >> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:902) >> > at >> > >> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:133) >> > ... 16 more >> > >> > Pig Stack Trace >> > --------------- >> > ERROR 2997: Unable to recreate exception from backed error: >> > org.apache.hadoop.hbase.ZooKeeperConnectionException: >> > org.apache.hadoop.hbase.ZooKeeperConnectionException: >> > org.apache.zookeeper.KeeperException$ConnectionLossException: >> > KeeperErrorCode = ConnectionLoss for /hbase >> > >> > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to >> > open iterator for alias my_data. Backend error : Unable to recreate >> > exception from backed error: >> > org.apache.hadoop.hbase.ZooKeeperConnectionException: >> > org.apache.hadoop.hbase.ZooKeeperConnectionException: >> > org.apache.zookeeper.KeeperException$ConnectionLossException: >> > KeeperErrorCode = ConnectionLoss for /hbase >> > at org.apache.pig.PigServer.openIterator(PigServer.java:742) >> > at >> > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612) >> > at >> > >> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303) >> > at >> > >> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165) >> > >> > >> > >> > >> > Thanks for helping >> > >> > >> > Byambajargal >> > >> > >> >> >> -- >> >> -- >> Thanks and Regards, >> Sulabh Choudhury >> >
