Hi, all
Why set mapred.reduce.tasks does not work in hive-0.70-rc1 edition?
The source code follows:
private void setNumberOfReducers() throws IOException {
// this is a temporary hack to fix things that are not fixed in the compiler
Integer numReducersFromWork = work.getNumReduceTasks();
if (work.getReducer() == null) {
console
.printInfo("Number of reduce tasks is set to 0 since there's
no reduce operator");
work.setNumReduceTasks(Integer.valueOf(0));
} else {
if (numReducersFromWork >= 0) {
console.printInfo("Number of reduce tasks determined at compile time: "
+ work.getNumReduceTasks());
} else if (job.getNumReduceTasks() > 0) {
int reducers = job.getNumReduceTasks();
work.setNumReduceTasks(reducers);
console
.printInfo("Number of reduce tasks not specified.
Defaulting to jobconf value of: "
+ reducers);
} else {
int reducers = estimateNumberOfReducers();
work.setNumReduceTasks(reducers);
console
.printInfo("Number of reduce tasks not specified.
Estimated from input data size: "
+ reducers);
}
}
I don't know the reasons. Thanks in advance.
--
Thanks,
Jander
