pvary commented on a change in pull request #2161:
URL: https://github.com/apache/hive/pull/2161#discussion_r609860590
##########
File path:
iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java
##########
@@ -224,6 +224,11 @@ private void cleanup(JobContext jobContext) throws
IOException {
// If there are reducers, then every reducer will generate a result file.
// If this is a map only task, then every mapper will generate a result
file.
int expectedFiles = conf.getNumReduceTasks() > 0 ?
conf.getNumReduceTasks() : conf.getNumMapTasks();
+ // For Tez, we can only reliably get the number of tasks from the Tez AM,
not the job conf
Review comment:
If I understand correctly in Hive3 `commitInsertTable` is not called
when the execution engine is MR.
If it would be called anywhere then we are in a big trouble anyway as
`HiveIcebergOutputCommitter.commitJob` would be called twice.
So all-in-all I would go for this change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
