Hey Dan This usually means that you have mismatched Hadoop jar versions somewhere. I encountered a similar problem with Oozie trying to talk to HDFS. Maybe try posting to the Hadoop user list as well. In general, you should just need the same hadoop-core.jar as on your cluster when you run Pig. From Pig all you should need is pig.jar (and piggybank, etc.) and the pre-compiled jar should suffice.
Cheers, Josh On 21 March 2011 22:56, Dan Hendry <[email protected]> wrote: > First off, I am fairly new to both pig and Hadoop. I am having some > problems > connecting pig to a local hadoop cluster. I am getting the following error > in the hadoop namenode logs whenever I try and start up pig: > > > > 2011-03-21 17:48:17,299 WARN org.apache.hadoop.ipc.Server: Incorrect header > or version mismatch from 127.0.0.1:60928 got version 3 expected version 4 > > > > I am using the cloudera deb repository (CDH3b4) installed according to > https://docs.cloudera.com/display/DOC/CDH3+Installation+Guide. The hadoop > version is 20.2 and running in pseudo distributed mode. I am using pig > 0.8.0, both the provided tarball and a clone of the 0.8.0 tag compiled > locally. Any help would be appreciated. I am getting the following error in > the pig logs: > > > > Error before Pig is launched > > ---------------------------- > > ERROR 2999: Unexpected internal error. Failed to create DataStorage > > > > java.lang.RuntimeException: Failed to create DataStorage > > at > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.jav > a:75) > > at > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.j > ava:58) > > at > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecuti > onEngine.java:213) > > at > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecuti > onEngine.java:133) > > at org.apache.pig.impl.PigContext.connect(PigContext.java:183) > > at org.apache.pig.PigServer.<init>(PigServer.java:225) > > at org.apache.pig.PigServer.<init>(PigServer.java:214) > > at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55) > > at org.apache.pig.Main.run(Main.java:462) > > at org.apache.pig.Main.main(Main.java:107) > > Caused by: java.io.IOException: Call to localhost/127.0.0.1:8020 failed on > local exception: java.io.EOFException > > at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) > > at org.apache.hadoop.ipc.Client.call(Client.java:743) > > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) > > at $Proxy0.getProtocolVersion(Unknown Source) > > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) > > at > org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106) > > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207) > > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170) > > at > > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSyste > m.java:82) > > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378) > > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) > > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390) > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196) > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95) > > at > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.jav > a:72) > > ... 9 more > > Caused by: java.io.EOFException > > at java.io.DataInputStream.readInt(DataInputStream.java:375) > > at > org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501) > > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446) > > > ============================================================================ > ==== > >
