[
https://issues.apache.org/jira/browse/HIVE-26102?focusedWorklogId=755230&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-755230
]
ASF GitHub Bot logged work on HIVE-26102:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Apr/22 13:43
Start Date: 11/Apr/22 13:43
Worklog Time Spent: 10m
Work Description: marton-bod commented on code in PR #3131:
URL: https://github.com/apache/hive/pull/3131#discussion_r847343603
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -325,9 +327,40 @@ private void commitTable(FileIO io, ExecutorService
executor, JobContext jobCont
"numReduceTasks/numMapTasks", jobContext.getJobID(), name);
return conf.getNumReduceTasks() > 0 ? conf.getNumReduceTasks() :
conf.getNumMapTasks();
});
- Collection<DataFile> dataFiles = dataFiles(numTasks, executor, location,
jobContext, io, true);
- boolean isOverwrite = conf.getBoolean(InputFormatConfig.IS_OVERWRITE,
false);
+ if (HiveIcebergStorageHandler.isDelete(conf, name)) {
+ Collection<FilesForCommit> writeResults = collectResults(numTasks,
executor, location, jobContext, io, true);
+ commitDelete(jobContext, table, startTime, writeResults);
+ } else if (HiveIcebergStorageHandler.isWrite(conf, name)) {
+ Collection<FilesForCommit> writeResults = collectResults(numTasks,
executor, location, jobContext, io, true);
+ boolean isOverwrite = conf.getBoolean(InputFormatConfig.IS_OVERWRITE,
false);
+ commitInsert(jobContext, table, startTime, writeResults, isOverwrite);
+ } else {
+ LOG.info("Unable to determine commit operation type for table: {},
jobID: {}. Will not create a commit.",
+ table, jobContext.getJobID());
+ }
+ }
+
+ private void commitDelete(JobContext jobContext, Table table, long
startTime, Collection<FilesForCommit> results) {
Review Comment:
Thanks for checking! I've refactored the `commitDelete` and `commitInsert`
to use an optional Transaction object, which can be passed in case of an update
or merge query.
Issue Time Tracking
-------------------
Worklog Id: (was: 755230)
Time Spent: 16.5h (was: 16h 20m)
> Implement DELETE statements for Iceberg tables
> ----------------------------------------------
>
> Key: HIVE-26102
> URL: https://issues.apache.org/jira/browse/HIVE-26102
> Project: Hive
> Issue Type: New Feature
> Reporter: Marton Bod
> Assignee: Marton Bod
> Priority: Major
> Labels: pull-request-available
> Time Spent: 16.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
