[
https://issues.apache.org/jira/browse/PIG-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072753#comment-14072753
]
Rohini Palaniswamy commented on PIG-4064:
-----------------------------------------
Looks good. Can you clarify a doubt though.
TestEvalPipeline2.testLimitAutoReducer - Why for the same reducer bytes, the
parallelism produced by MR and Tez is very different? What is the
differentiating behavior?
Also can you create a separate jira to fix secondary sort optimizer to support
multiple predecessors.
Could you also remove the below comments which are not valid anymore when
checking in.
// MR SecondaryKeyOptimizer currently does not check for this case. (Prashant
fixed MR in PIG-3827)
//Let's add combiner if possible. (Copied over wrongly i believe)
//TODO: Case of from plan leaf being POUnion. (Don't think we have POUnion in
Tez plan at all)
> Fix tez auto parallelism test failures
> --------------------------------------
>
> Key: PIG-4064
> URL: https://issues.apache.org/jira/browse/PIG-4064
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Reporter: Rohini Palaniswamy
> Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4064-1.patch
>
>
> Tests that hang:
> - TestLimitVariable.testLimitVariable2
> - TestEvalPipeline2.testLocalRearrangeInReducer
> - TestEvalPipeline2.testLimitAutoReducer
> - TestEvalPipeline2.testForEachSameOriginColumn1
> Tests that fail:
> - TestEvalPipeline2.testEmptySort (Divide by zero error)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
