1. Can I generate pigperf on pig cdh3 too or will it only work on apache? 2. pigmix2.patch according to Jira only works on pig 0.7 or will it work on pig apache 0.9 too?
Thanks. On Fri, Sep 9, 2011 at 1:09 PM, Alan Gates <[email protected]> wrote: > CDH 3 version of Hadoop uses a different version of HDFS then the client > packaged with Pig 0.8.1. You should use the CDH version of Pig as well to > make them work together. > > Alan. > > On Sep 9, 2011, at 2:44 AM, kiranprasad wrote: > > > Hi > > > > Iam using > > Hadoop version : hadoop-0.20.2-cdh3u0 and PIG : pig-0.8.1 > > > > > > For Cluster I have 3 VMs(10.0.0.61-master, 10.0.0.62,10.0.0.63 - Slaves) > and > > another VM 10.0.0.64 in which I ve installed PIG from which I run the PIG > > commands. > > I ve updated the xmls , please find the below > > mapred site.xml > > -------------- > > <configuration> > > <property> > > <name>mapred.job.tracker</name> > > <value>10.0.0.61:8021</value> > > </property> > > </configuration> > > > > core-site.xml > > ---------- > > <configuration> > > <property> > > <name>fs.default.name</name> > > <value>hdfs://10.0.0.61:8020</value> > > </property> > > </configuration> > > > > Hdfs-site.xml > > ---------------- > > <configuration> > > <property> > > <name>fs.default.name</name> > > <value>hdfs://10.0.0.61:8020</value> > > </property> > > <property> > > <name>mapred.job.tracker</name> > > <value>10.0.0.61:8021</value> > > </property> > > <property> > > <name>dfs.replication</name> > > <value>2</value> > > </property> > > > > masters > > --------- > > 10.0.0.61 > > > > slaves > > -------- > > 10.0.0.62 > > 10.0.0.63 > > > > Have started start-all.sh. and checked with jps command in > master(10.0.0.61) > > I got response as > > > > 4313 Jps > > 3357 NameNode > > 3580 JobTracker > > 3502 SecondaryNameNode > > > > and in slave I got > > > > 2662 DataNode > > 2902 Jps > > 2759 TaskTracker > > > > So the Cluster is working... > > > > But When I tried to connect to PIG i.e. bin/pig (I ve installed PIG in > > 10.0.0.64 this is out of Hadoop Cluster). > > I am getting below exception again. Please help close this. > > > > [kiranprasad.g@pig1 pig-0.8.1]$ bin/pig > > 2011-09-09 17:13:29,507 [main] INFO org.apache.pig.Main - Logging error > > messages to: /home/kiranprasad.g/pig-0.8.1/pig_1315568609504.log > > 2011-09-09 17:13:29,676 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting > > to hadoop file system at: hdfs://10.0.0.61:8020 > > 2011-09-09 17:13:29,804 [main] ERROR org.apache.pig.Main - ERROR 2999: > > Unexpected internal error. Failed to create DataStorage > > > > > > The Error Log shows : > > > > ERROR 2999: Unexpected internal error. Failed to create DataStorage > > > > java.lang.RuntimeException: Failed to create DataStorage > > at > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75) > > at > org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58) > > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214) > > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134) > > at org.apache.pig.impl.PigContext.connect(PigContext.java:183) > > at org.apache.pig.PigServer.<init>(PigServer.java:226) > > at org.apache.pig.PigServer.<init>(PigServer.java:215) > > at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55) > > at org.apache.pig.Main.run(Main.java:452) > > at org.apache.pig.Main.main(Main.java:107) > > Caused by: java.io.IOException: Call to /10.0.0.61:8020 failed on local > exception: java.io.EOFException > > at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) > > at org.apache.hadoop.ipc.Client.call(Client.java:743) > > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) > > at $Proxy0.getProtocolVersion(Unknown Source) > > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) > > at > org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106) > > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207) > > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170) > > at > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82) > > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378) > > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) > > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390) > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196) > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95) > > at > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72) > > ... 9 more > > Caused by: java.io.EOFException > > at java.io.DataInputStream.readInt(DataInputStream.java:375) > > at > org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501) > > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446) > > > ================================================================================ > > > > > > > > Regards > > Kiran.G > > > > IMImobile Plot 770, Rd. 44 Jubilee Hills, Hyderabad - 500033 > > M +91 9000170909 T +91 40 2355 5945 - Ext: 229 www.imimobile.com > > -- Keren Ouaknine Cell: +972 54 2565404 Web: www.kereno.com
