[
https://issues.apache.org/jira/browse/PIG-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Kamath updated PIG-471:
-------------------------------
Status: Patch Available (was: Open)
Attached patch for the issue. The problem was that when the file is missing,
the job submission fails in getSplit() even BEFORE map or reduce tasks are ever
run. In MapReduceLauncher after we come to know the job has failed, we try to
get mapTaskReport and reduceTaskReport (in Launcher.getStats()) from Hadoop
which results in an IOexception. The fix catches this and gives a warning to
the user IF the job had succeeded
> Verbose (unrelated) exception stack is printed when input file in the Load is
> not present
> -----------------------------------------------------------------------------------------
>
> Key: PIG-471
> URL: https://issues.apache.org/jira/browse/PIG-471
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Pradeep Kamath
> Fix For: types_branch
>
>
> When input file in load is absent the following is printed on the screen:
> {noformat}
> 2008-10-06 10:24:28,048 [main] ERROR
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - java.io.IOException: bla does not exist
> at
> org.apache.pig.backend.executionengine.PigSlicer.validate(PigSlicer.java:105)
> at
> org.apache.pig.impl.io.ValidatingInputFileSpec.validate(ValidatingInputFileSpec.java:59)
> at
> org.apache.pig.impl.io.ValidatingInputFileSpec.<init>(ValidatingInputFileSpec.java:44)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:200)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:742)
> at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:370)
> at
> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
> at
> org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
> at java.lang.Thread.run(Thread.java:595)
> 2008-10-06 10:24:28,061 [main] ERROR org.apache.pig.tools.grunt.GruntParser -
> java.io.IOException: Unable to store for alias: 1 [java.io.IOException:
> java.lang.NullPointerException
> at java.util.TreeMap.getEntry(TreeMap.java:324)
> at java.util.TreeMap.get(TreeMap.java:255)
> at
> org.apache.hadoop.mapred.JobTracker.getMapTaskReports(JobTracker.java:1894)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> ]
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:255)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:647)
> at org.apache.pig.PigServer.execute(PigServer.java:638)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:278)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:439)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:249)
> at
> org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:94)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:282)
> Caused by: org.apache.pig.backend.executionengine.ExecException:
> java.io.IOException: java.lang.NullPointerException
> at java.util.TreeMap.getEntry(TreeMap.java:324)
> at java.util.TreeMap.get(TreeMap.java:255)
> at
> org.apache.hadoop.mapred.JobTracker.getMapTaskReports(JobTracker.java:1894)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> ... 9 more
> Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> java.lang.NullPointerException
> at java.util.TreeMap.getEntry(TreeMap.java:324)
> at java.util.TreeMap.get(TreeMap.java:255)
> at
> org.apache.hadoop.mapred.JobTracker.getMapTaskReports(JobTracker.java:1894)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> at org.apache.hadoop.ipc.Client.call(Client.java:715)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
> at org.apache.hadoop.mapred.$Proxy1.getMapTaskReports(Unknown Source)
> at
> org.apache.hadoop.mapred.JobClient.getMapTaskReports(JobClient.java:935)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:90)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:92)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:245)
> ... 8 more
> 2008-10-06 10:24:28,061 [main] ERROR org.apache.pig.tools.grunt.GruntParser -
> Unable to store for alias: 1 [java.io.IOException:
> java.lang.NullPointerException
> at java.util.TreeMap.getEntry(TreeMap.java:324)
> at java.util.TreeMap.get(TreeMap.java:255)
> {noformat}
> Of this only the first part which actually tells that the input file is
> absent is useful, the rest is misleading and confusing
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
