On Fri, Sep 9, 2011 at 3:35 PM, Keren Ouaknine <[email protected]> wrote:
> 1. Can I generate pigperf on pig cdh3 too or will it only work on apache? > Yes > 2. pigmix2.patch according to Jira only works on pig 0.7 or will it work on > pig apache 0.9 too? > Yes > > Thanks. > > On Fri, Sep 9, 2011 at 1:09 PM, Alan Gates <[email protected]> wrote: > > > CDH 3 version of Hadoop uses a different version of HDFS then the client > > packaged with Pig 0.8.1. You should use the CDH version of Pig as well > to > > make them work together. > > > > Alan. > > > > On Sep 9, 2011, at 2:44 AM, kiranprasad wrote: > > > > > Hi > > > > > > Iam using > > > Hadoop version : hadoop-0.20.2-cdh3u0 and PIG : pig-0.8.1 > > > > > > > > > For Cluster I have 3 VMs(10.0.0.61-master, 10.0.0.62,10.0.0.63 - > Slaves) > > and > > > another VM 10.0.0.64 in which I ve installed PIG from which I run the > PIG > > > commands. > > > I ve updated the xmls , please find the below > > > mapred site.xml > > > -------------- > > > <configuration> > > > <property> > > > <name>mapred.job.tracker</name> > > > <value>10.0.0.61:8021</value> > > > </property> > > > </configuration> > > > > > > core-site.xml > > > ---------- > > > <configuration> > > > <property> > > > <name>fs.default.name</name> > > > <value>hdfs://10.0.0.61:8020</value> > > > </property> > > > </configuration> > > > > > > Hdfs-site.xml > > > ---------------- > > > <configuration> > > > <property> > > > <name>fs.default.name</name> > > > <value>hdfs://10.0.0.61:8020</value> > > > </property> > > > <property> > > > <name>mapred.job.tracker</name> > > > <value>10.0.0.61:8021</value> > > > </property> > > > <property> > > > <name>dfs.replication</name> > > > <value>2</value> > > > </property> > > > > > > masters > > > --------- > > > 10.0.0.61 > > > > > > slaves > > > -------- > > > 10.0.0.62 > > > 10.0.0.63 > > > > > > Have started start-all.sh. and checked with jps command in > > master(10.0.0.61) > > > I got response as > > > > > > 4313 Jps > > > 3357 NameNode > > > 3580 JobTracker > > > 3502 SecondaryNameNode > > > > > > and in slave I got > > > > > > 2662 DataNode > > > 2902 Jps > > > 2759 TaskTracker > > > > > > So the Cluster is working... > > > > > > But When I tried to connect to PIG i.e. bin/pig (I ve installed PIG in > > > 10.0.0.64 this is out of Hadoop Cluster). > > > I am getting below exception again. Please help close this. > > > > > > [kiranprasad.g@pig1 pig-0.8.1]$ bin/pig > > > 2011-09-09 17:13:29,507 [main] INFO org.apache.pig.Main - Logging > error > > > messages to: /home/kiranprasad.g/pig-0.8.1/pig_1315568609504.log > > > 2011-09-09 17:13:29,676 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > > Connecting > > > to hadoop file system at: hdfs://10.0.0.61:8020 > > > 2011-09-09 17:13:29,804 [main] ERROR org.apache.pig.Main - ERROR 2999: > > > Unexpected internal error. Failed to create DataStorage > > > > > > > > > The Error Log shows : > > > > > > ERROR 2999: Unexpected internal error. Failed to create DataStorage > > > > > > java.lang.RuntimeException: Failed to create DataStorage > > > at > > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75) > > > at > > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58) > > > at > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214) > > > at > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134) > > > at org.apache.pig.impl.PigContext.connect(PigContext.java:183) > > > at org.apache.pig.PigServer.<init>(PigServer.java:226) > > > at org.apache.pig.PigServer.<init>(PigServer.java:215) > > > at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55) > > > at org.apache.pig.Main.run(Main.java:452) > > > at org.apache.pig.Main.main(Main.java:107) > > > Caused by: java.io.IOException: Call to /10.0.0.61:8020 failed on > local > > exception: java.io.EOFException > > > at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) > > > at org.apache.hadoop.ipc.Client.call(Client.java:743) > > > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) > > > at $Proxy0.getProtocolVersion(Unknown Source) > > > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) > > > at > > org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106) > > > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207) > > > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170) > > > at > > > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82) > > > at > > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378) > > > at > org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) > > > at > org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390) > > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196) > > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95) > > > at > > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72) > > > ... 9 more > > > Caused by: java.io.EOFException > > > at java.io.DataInputStream.readInt(DataInputStream.java:375) > > > at > > org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501) > > > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446) > > > > > > ================================================================================ > > > > > > > > > > > > Regards > > > Kiran.G > > > > > > IMImobile Plot 770, Rd. 44 Jubilee Hills, Hyderabad - 500033 > > > M +91 9000170909 T +91 40 2355 5945 - Ext: 229 www.imimobile.com > > > > > > > -- > Keren Ouaknine > Cell: +972 54 2565404 > Web: www.kereno.com >
