[
https://issues.apache.org/jira/browse/HIVE-26102?focusedWorklogId=755280&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-755280
]
ASF GitHub Bot logged work on HIVE-26102:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Apr/22 15:44
Start Date: 11/Apr/22 15:44
Worklog Time Spent: 10m
Work Description: marton-bod commented on code in PR #3131:
URL: https://github.com/apache/hive/pull/3131#discussion_r847473881
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -325,9 +327,40 @@ private void commitTable(FileIO io, ExecutorService
executor, JobContext jobCont
"numReduceTasks/numMapTasks", jobContext.getJobID(), name);
return conf.getNumReduceTasks() > 0 ? conf.getNumReduceTasks() :
conf.getNumMapTasks();
});
- Collection<DataFile> dataFiles = dataFiles(numTasks, executor, location,
jobContext, io, true);
- boolean isOverwrite = conf.getBoolean(InputFormatConfig.IS_OVERWRITE,
false);
+ if (HiveIcebergStorageHandler.isDelete(conf, name)) {
+ Collection<FilesForCommit> writeResults = collectResults(numTasks,
executor, location, jobContext, io, true);
+ commitDelete(jobContext, table, startTime, writeResults);
+ } else if (HiveIcebergStorageHandler.isWrite(conf, name)) {
+ Collection<FilesForCommit> writeResults = collectResults(numTasks,
executor, location, jobContext, io, true);
+ boolean isOverwrite = conf.getBoolean(InputFormatConfig.IS_OVERWRITE,
false);
+ commitInsert(jobContext, table, startTime, writeResults, isOverwrite);
+ } else {
+ LOG.info("Unable to determine commit operation type for table: {},
jobID: {}. Will not create a commit.",
+ table, jobContext.getJobID());
+ }
+ }
+
+ private void commitDelete(JobContext jobContext, Table table, long
startTime, Collection<FilesForCommit> results) {
Review Comment:
That should allow you to do something like:
```
// update
Transaction transaction = table.newTransaction();
commitDelete(table, Optional.of(transaction), startTime, deleteWriteResults);
commitInsert(table, Optional.of(transaction), startTime, insertWriteResults,
isOverwrite);
transaction.commitTransaction();
```
Issue Time Tracking
-------------------
Worklog Id: (was: 755280)
Time Spent: 17h (was: 16h 50m)
> Implement DELETE statements for Iceberg tables
> ----------------------------------------------
>
> Key: HIVE-26102
> URL: https://issues.apache.org/jira/browse/HIVE-26102
> Project: Hive
> Issue Type: New Feature
> Reporter: Marton Bod
> Assignee: Marton Bod
> Priority: Major
> Labels: pull-request-available
> Time Spent: 17h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
