deniskuzZ commented on code in PR #4446:
URL: https://github.com/apache/hive/pull/4446#discussion_r2127071701
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -141,9 +141,10 @@ public void commitTask(TaskAttemptContext originalContext)
throws IOException {
TaskAttemptID attemptID = context.getTaskAttemptID();
JobConf jobConf = context.getJobConf();
Set<Path> mergedPaths = getCombinedLocations(jobConf);
+ String outputId = jobConf.get(HiveIcebergOutputFormat.ICEBERG_OUTPUT_ID);
Review Comment:
`Reason: There can be more than one OutputCommitter per vertex. If a job
writes to 2 Iceberg tables in the same vertex, there can be 2 commit tasks
happening in same vertex.`
can we use separate tasks for multiple outputs? cc @abstractdog
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
