[
https://issues.apache.org/jira/browse/PIG-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13922937#comment-13922937
]
Cheolsoo Park edited comment on PIG-3799 at 3/7/14 1:10 AM:
------------------------------------------------------------
The parallelism is set in POForeach, and it used to overwrite the parallelism
of TezOperator before PIG-3795. But since I got rid of the overwriting logic,
the parallelism of vertex is no longer set.
Attached is a patch that explicitly sets the parallelism of vertex.
was (Author: cheolsoo):
The parallelism used to be set in POForeach, and that value used to overwrite
that of TezOperator before PIG-3795. But since I got rid of the overwriting
logic, the parallelism of vertex is no longer set.
Attached is a patch that explicitly sets the parallelism of vertex.
> TestCustomPartitioner is broken in tez branch
> ---------------------------------------------
>
> Key: PIG-3799
> URL: https://issues.apache.org/jira/browse/PIG-3799
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: tez-branch
>
> Attachments: PIG-3799-1.patch
>
>
> This is a regression of PIG-3795. In TezCompiler, visitDistinct() doesn't set
> the requested parallelism of TezOperator, resulting that only one reducer
> runs for the following query-
> {code}
> A = LOAD 'table_testCustomPartitionerDistinct' as (a0:int, a1:int);
> B = distinct A PARTITION BY
> org.apache.pig.test.utils.SimpleCustomPartitioner3 parallel 2;
> {code}
> The test fails because it sees a single output file while it expects two.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
