[
https://issues.apache.org/jira/browse/PIG-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated PIG-3726:
------------------------------------
Attachment: PIG-3726.patch
Sure [~mtelizhyn], here is patch that I was working on just yesterday! As you
can see, it's very similar to yours :-)
The reason why I did not submitted it yet is that I wasn't sure whether my
(our) approach is entirely correct. Method {{saveCounters}} is written in a way
to throw an exception if the RANK counters are not present. I'm assuming that
the counters might be missing from many reasons and the only valid case that we
want to cover is when the input relation is empty. I wanted poke around a bit
to see if that is indeed the case. Will be happy to hear others thoughts.
> Ranking empty records leads to NullPointerException
> ---------------------------------------------------
>
> Key: PIG-3726
> URL: https://issues.apache.org/jira/browse/PIG-3726
> Project: Pig
> Issue Type: Bug
> Reporter: Mykhaylo Telizhyn
> Attachments: PIG-3726.patch, emptyRecordsNPE.patch
>
>
> steps to reproduce:
> 1) create an empty file on hdfs (e.g. /user/root/test/empty.txt)
> 2) execute the following pig script:
> records = LOAD '/user/root/test/empty.txt' AS (Amount: DOUBLE);
> ranked = RANK records BY Amount DESC;
> DUMP ranked;
> 3) on console you will get " ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 2043: Unexpected error during execution." and in pig's logs you will see:
> Pig Stack Trace
> {code}
> ---------------
> ERROR 2043: Unexpected error during execution.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias ranked
> at org.apache.pig.PigServer.openIterator(PigServer.java:880)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:607)
> at org.apache.pig.Main.main(Main.java:156)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias
> ranked
> at org.apache.pig.PigServer.storeEx(PigServer.java:982)
> at org.apache.pig.PigServer.store(PigServer.java:942)
> at org.apache.pig.PigServer.openIterator(PigServer.java:855)
> ... 12 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043:
> Unexpected error during execution.
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1333)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
> at org.apache.pig.PigServer.storeEx(PigServer.java:978)
> ... 14 more
> Caused by: java.lang.RuntimeException: Error to read counters into Rank
> operation counterSize 0
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.saveCounters(JobControlCompiler.java:386)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.updateMROpPlan(JobControlCompiler.java:332)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:387)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
> ... 16 more
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.counters.Limits.filterName(Limits.java:44)
> at org.apache.hadoop.mapreduce.counters.Limits.filterGroupName(Limits.java:52)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.getGroup(AbstractCounters.java:220)
> at org.apache.hadoop.mapred.Counters.getGroup(Counters.java:113)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.saveCounters(JobControlCompiler.java:360)
> ... 19 more
> ================================================================================
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
