Thanks Alan. Yeah, I solved the problem by not including my hadoop jar in the libs folder on S3 and using the one on S3.
--Gayatri. On Fri, Nov 18, 2011 at 10:53 PM, Alan Gates <[email protected]> wrote: > Usually this means your versions of Pig and Hadoop aren't compatible. > What version of each are you running? > > Alan. > > On Nov 14, 2011, at 5:28 PM, Gayatri Rao wrote: > > > Hi, > > > > I am trying to run pig job to read HAR data from S3 and run the job on > ec2 > > cluster and I am getting the following error: > > Any ideas on what could be running > > > > Error before Pig is launched > > ---------------------------- > > ERROR 2999: Unexpected internal error. Failed to create DataStorage > > > > java.lang.RuntimeException: Failed to create DataStorage^M > > at > > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)^M > > at > > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)^M > > at > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214)^M > > at > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134)^M > > at org.apache.pig.impl.PigContext.connect(PigContext.java:183)^M > > at org.apache.pig.PigServer.<init>(PigServer.java:226)^M > > at org.apache.pig.PigServer.<init>(PigServer.java:215)^M > > at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55)^M > > at org.apache.pig.Main.run(Main.java:492)^M > > at org.apache.pig.Main.main(Main.java:107)^M > > Caused by: java.io.IOException: Call to > > ip-10-148-63-198.us-west-1.compute.internal:9000 failed on local > exception: > > java.net.SocketException: Malformed reply from SOCKS > > server^M > > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1142)^M > > at org.apache.hadoop.ipc.Client.call(Client.java:1110)^M > > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)^M > > at $Proxy0.getProtocolVersion(Unknown Source)^M > > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:398)^M > > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:384)^M > > at > > org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:111)^M > > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:213)^M > > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:180)^M > > at > > > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)^M > > at > > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1514)^M > > at > org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:67)^M > > at > > org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1548)^M > > at > org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1530)^M > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:228)^M > > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:111)^M > > at > > > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)^M > > ... 9 more^M > > Caused by: java.net.SocketException: Malformed reply from SOCKS server^M > > at > > java.net.SocksSocketImpl.readSocksReply(SocksSocketImpl.java:90)^M > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:472)^M > > at java.net.Socket.connect(Socket.java:529)^M > > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:406)^M > > at > > > org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:425)^M > > at > > org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:532)^M > > at > > org.apache.hadoop.ipc.Client$Connection.access$2300(Client.java:210)^M > > at org.apache.hadoop.ipc.Client.getConnection(Client.java:1247)^M > > at org.apache.hadoop.ipc.Client.call(Client.java:1078)^M > > > > Thanks > > Gayatri > >
