[
https://issues.apache.org/jira/browse/PIG-4569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-4569:
------------------------------------
Attachment: PIG-4569-4.patch
[~daijy],
Attached a patch that solves the issue in JobControlCompiler itself by
creating the counter offsets correctly skipping the empty tasks. Instead of
using counterSize index as the taskId in the counterPair keys, it uses the
actual taskIds in ascending order. Makes it easier to understand.
> Fix e2e test Rank_1 failure
> ---------------------------
>
> Key: PIG-4569
> URL: https://issues.apache.org/jira/browse/PIG-4569
> Project: Pig
> Issue Type: Bug
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.15.0
>
> Attachments: PIG-4569-1.patch, PIG-4569-2.patch, PIG-4569-3.patch,
> PIG-4569-4.patch
>
>
> It fails on Hadoop 1, but the issue could exist on Hadoop 2 as well.
> Error message:
> {code}
> org.apache.pig.backend.executionengine.ExecException: ERROR 0: Error while
> executing ForEach at [C[6,4]]
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:325)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:279)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:274)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.RuntimeException: Unable to read counter
> pig.counters.counter_6929257954538808410_2
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PORank.getRankCounterOffset(PORank.java:185)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PORank.addRank(PORank.java:160)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PORank.getNextTuple(PORank.java:141)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:307)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:252)
> ... 11 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
