[
https://issues.apache.org/jira/browse/PIG-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-3497:
----------------------------
Fix Version/s: 0.12.0
Actually I overlooked the patch. It also moves adjustNumReducers to a much
later stage. And at that time, all POLoad has been removed from plan, so
reduceAdjuster will stop working. Will create another ticket for that.
> JobControlCompiler should only do reducer estimation when the job has a
> reduce phase
> ------------------------------------------------------------------------------------
>
> Key: PIG-3497
> URL: https://issues.apache.org/jira/browse/PIG-3497
> Project: Pig
> Issue Type: Bug
> Reporter: Akihiro Matsukawa
> Assignee: Akihiro Matsukawa
> Priority: Minor
> Fix For: 0.12.0
>
> Attachments: reducer_estimation.patch
>
>
> Currently, JobControlCompiler makes an estimation for the number of reducers
> required (by default based on input size into mappers) regardless of whether
> there is a reduce phase in the job. This is unnecessary, especially when
> running more complex custom reducer estimators.
> Change to only estimate reducers when necessary.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
