[
https://issues.apache.org/jira/browse/PIG-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheolsoo Park updated PIG-3719:
-------------------------------
Attachment: PIG-3719-2.patch
Attaching 2nd patch with the following change-
{code}
@@ -109,7 +110,10 @@ public class POShuffleTezLoad extends POPackage implements
TezLoad {
cur = readers.get(i).getCurrentKey();
if (min == null || comparator.compare(min, cur) > 0) {
min =
PigNullableWritable.newInstance((PigNullableWritable)cur);
- cur = min;
+ if (isSkewedJoin) {
+ ((NullablePartitionWritable)min).setKey(
+
((NullablePartitionWritable)cur).getKey());
+ }
}
}
}
{code}
Basically, I explicitly set key after copying NullableWritable in case of
skewed join. This lets both tez.conf e2e tests and TestAccumulator pass.
> Fix skewed join e2e tests
> -------------------------
>
> Key: PIG-3719
> URL: https://issues.apache.org/jira/browse/PIG-3719
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: tez-branch
>
> Attachments: PIG-3719-1.patch, PIG-3719-2.patch
>
>
> There are two sets of skewed join e2e tests-
> # tez.conf: Join7 and 8
> # nightly.conf: SkewedJoin
> We need to get both passing.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
