[
https://issues.apache.org/jira/browse/PIG-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-2627:
----------------------------
Description:
{code}
A = load 'i3.txt' as (k,v);
L = FILTER A BY k == 'k1';
B = GROUP A BY k PARTITION BY SimpleCustomPartitioner PARALLEL 3;
C = FOREACH B GENERATE FLATTEN(A) PARALLEL 3;
STORE C INTO 'output1';
STORE L INTO 'output2';
{code}
For the above script the custom partitioner mentioned in the GROUP BY statement
is ignored by Pig.
To workaround this issue I had to disable Multiquery or put exec statements in
between.
The configuration is getting missed out while the Physical plan is getting
converted into the MR plan.
(Note the POGlobalRearrange is visited on a different MROper than what the
actual MRPlan returns)
was:
{code}
A = load 'i3.txt' as (k,v);
L = FILTER A BY k == 'k1';
B = GROUP A BY k PARTITION BY SimpleCustomPartitioner PARALLEL 3;
C = FOREACH B GENERATE FLATTEN(A) PARALLEL 3;
STORE C INTO 'output1';
STORE L INTO 'output2';
{code}
For the above script the custom partitioner mentioned in the GROUP BY statement
is ignored by Pig.
To workaround this issue I had to disable Multiquery or put exec statements in
between.
The configuration is getting missed out while the Physical plan is getting
converted into the MR plan.
(Note the POGlobalRearrange is visited on a different MROper than what the
actual MRPlan returns)
Fix Version/s: 0.11
Assignee: Aniket Mokashi
> Custom partitioner not set when POSplit is involved in Plan
> -----------------------------------------------------------
>
> Key: PIG-2627
> URL: https://issues.apache.org/jira/browse/PIG-2627
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.2, 0.10.0
> Reporter: Vivek Padmanabhan
> Assignee: Aniket Mokashi
> Fix For: 0.11
>
> Attachments: PIG-2627.patch
>
>
> {code}
> A = load 'i3.txt' as (k,v);
> L = FILTER A BY k == 'k1';
> B = GROUP A BY k PARTITION BY SimpleCustomPartitioner PARALLEL 3;
> C = FOREACH B GENERATE FLATTEN(A) PARALLEL 3;
> STORE C INTO 'output1';
> STORE L INTO 'output2';
> {code}
> For the above script the custom partitioner mentioned in the GROUP BY
> statement is ignored by Pig.
> To workaround this issue I had to disable Multiquery or put exec statements
> in between.
> The configuration is getting missed out while the Physical plan is getting
> converted into the MR plan.
> (Note the POGlobalRearrange is visited on a different MROper than what the
> actual MRPlan returns)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
