marton-bod commented on a change in pull request #2163:
URL: https://github.com/apache/iceberg/pull/2163#discussion_r569209357
##########
File path:
mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java
##########
@@ -210,7 +218,7 @@ private void cleanup(JobContext jobContext) throws
IOException {
JobConf conf = jobContext.getJobConf();
// If there are reducers, then every reducer will generate a result file.
// If this is a map only task, then every mapper will generate a result
file.
- int expectedFiles = conf.getNumReduceTasks() != 0 ?
conf.getNumReduceTasks() : conf.getNumMapTasks();
+ int expectedFiles = conf.getNumReduceTasks() > 0 ?
conf.getNumReduceTasks() : conf.getNumMapTasks();
Review comment:
yes, exactly. For map-only tasks, `numReduceTasks` can have -1 value,
which would lead to `expectedFiles` being -1 as well
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
