[
https://issues.apache.org/jira/browse/PIG-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheolsoo Park updated PIG-3719:
-------------------------------
Attachment: PIG-3719-1.patch
Attaching 1st patch that fixes NPE in tez.conf e2e tests.
The root cause of NPE was that POShuffleTezLoad was constructing a new instance
of NullableWritable for minimum key, but PigNullableWritable.newInstance()
didn't set key for NullablePartitionWritable.
Looks like there is no need to construct a new instance in the first place, so
I am reverting the change made by PIG-3626 as follows-
{code}
@@ -108,8 +108,7 @@ public class POShuffleTezLoad extends POPackage implements
TezLoad {
hasData = true;
cur = readers.get(i).getCurrentKey();
if (min == null || comparator.compare(min, cur) > 0) {
- min =
PigNullableWritable.newInstance((PigNullableWritable)cur);
- cur = min;
+ min = (PigNullableWritable) cur;
}
}
}
{code}
I am going to commit this change first so that tez.conf will be fixed.
But nightly.conf SkewedJoin tests still fail due to differences in results. I
will upload another patch for that problem.
> Fix skewed join e2e tests
> -------------------------
>
> Key: PIG-3719
> URL: https://issues.apache.org/jira/browse/PIG-3719
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: tez-branch
>
> Attachments: PIG-3719-1.patch
>
>
> There are two sets of skewed join e2e tests-
> # tez.conf: Join7 and 8
> # nightly.conf: SkewedJoin
> We need to get both passing.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
