Jie Li created PIG-2779:
---------------------------
Summary: Refactoring the code for setting number of reducers
Key: PIG-2779
URL: https://issues.apache.org/jira/browse/PIG-2779
Project: Pig
Issue Type: Bug
Reporter: Jie Li
As PIG-2652 observed, currently the code for setting number of reducers is a
little messy. MapReduceOper.requestedParallelism seems being misused in some
plases, and now we support runtime estimation of #reducer which further
complicates the problem.
For example, if we specify parallel 1 for the order-by, the estimated #reducer
will be used. If we specify parallel 2 while it estimates 4, order-by will fail
due to "Illegal partition for Null". If we specify parallel 4 while it
estimates 2, then some reducers will have nothing to do.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
