marton-bod commented on a change in pull request #2161:
URL: https://github.com/apache/hive/pull/2161#discussion_r609854017
##########
File path:
iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java
##########
@@ -224,6 +224,11 @@ private void cleanup(JobContext jobContext) throws
IOException {
// If there are reducers, then every reducer will generate a result file.
// If this is a map only task, then every mapper will generate a result
file.
int expectedFiles = conf.getNumReduceTasks() > 0 ?
conf.getNumReduceTasks() : conf.getNumMapTasks();
+ // For Tez, we can only reliably get the number of tasks from the Tez AM,
not the job conf
Review comment:
I wanted to be considerate of MR (even though it's unsupported in
upstream Hive), but my initial approach was to not touch it here but set the
numReduceTasks to the correct value in the `commitInsertTable`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
