Hi ,
I am trying to integrate Pig with Hadoop for processing of jobs.
I am able to run Pig in local mode and Hadoop with streaming api perfectly.
But when I try to run Pig with Hadoop I get follwong Error:
Pig Stack Trace
---------------
ERROR 2116: Unexpected error. Could not validate the output specification for:
file:///Users/rahulmalviya/Documents/Pig/dev/main_merged_hdp_out
org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected
exception caused the validation to stop
at
org.apache.pig.impl.plan.PlanValidator.validate(PlanValidator.java:56)
at
org.apache.pig.impl.logicalLayer.validators.InputOutputFileValidator.validate(InputOutputFileValidator.java:49)
at
org.apache.pig.impl.logicalLayer.validators.InputOutputFileValidator.validate(InputOutputFileValidator.java:37)
at
org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:89)
at org.apache.pig.PigServer.validate(PigServer.java:930)
at org.apache.pig.PigServer.compileLp(PigServer.java:910)
at org.apache.pig.PigServer.compileLp(PigServer.java:871)
at org.apache.pig.PigServer.compileLp(PigServer.java:852)
at org.apache.pig.PigServer.execute(PigServer.java:816)
at org.apache.pig.PigServer.access$100(PigServer.java:105)
at org.apache.pig.PigServer$Graph.execute(PigServer.java:1080)
at org.apache.pig.PigServer.executeBatch(PigServer.java:288)
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:109)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:391)
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 2116:
Unexpected error. Could not validate the output specification for:
file:///Users/rahulmalviya/Documents/Pig/dev/main_merged_hdp_out
at
org.apache.pig.impl.logicalLayer.validators.InputOutputFileVisitor.visit(InputOutputFileVisitor.java:93)
at org.apache.pig.impl.logicalLayer.LOStore.visit(LOStore.java:140)
at org.apache.pig.impl.logicalLayer.LOStore.visit(LOStore.java:37)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at
org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:50)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at
org.apache.pig.impl.plan.PlanValidator.validate(PlanValidator.java:50)
... 16 more
Caused by: java.io.IOException: Call to localhost/127.0.0.1:9001 failed on
local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
at org.apache.hadoop.ipc.Client.call(Client.java:743)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at org.apache.hadoop.mapred.$Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:429)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:423)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:410)
at org.apache.hadoop.mapreduce.Job.<init>(Job.java:50)
at
org.apache.pig.impl.logicalLayer.validators.InputOutputFileVisitor.visit(InputOutputFileVisitor.java:89)
... 24 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
================================================================================
Did anyone got the same error. I think it related to connection between pig and
hadoop.
Can someone tell me how to connect Pig and hadoop.
Thanks.
