[
https://issues.apache.org/jira/browse/HIVE-26102?focusedWorklogId=754999&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-754999
]
ASF GitHub Bot logged work on HIVE-26102:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Apr/22 05:33
Start Date: 10/Apr/22 05:33
Worklog Time Spent: 10m
Work Description: pvary commented on code in PR #3131:
URL: https://github.com/apache/hive/pull/3131#discussion_r846716331
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -325,9 +327,40 @@ private void commitTable(FileIO io, ExecutorService
executor, JobContext jobCont
"numReduceTasks/numMapTasks", jobContext.getJobID(), name);
return conf.getNumReduceTasks() > 0 ? conf.getNumReduceTasks() :
conf.getNumMapTasks();
});
- Collection<DataFile> dataFiles = dataFiles(numTasks, executor, location,
jobContext, io, true);
- boolean isOverwrite = conf.getBoolean(InputFormatConfig.IS_OVERWRITE,
false);
+ if (HiveIcebergStorageHandler.isDelete(conf, name)) {
+ Collection<FilesForCommit> writeResults = collectResults(numTasks,
executor, location, jobContext, io, true);
+ commitDelete(jobContext, table, startTime, writeResults);
+ } else if (HiveIcebergStorageHandler.isWrite(conf, name)) {
+ Collection<FilesForCommit> writeResults = collectResults(numTasks,
executor, location, jobContext, io, true);
+ boolean isOverwrite = conf.getBoolean(InputFormatConfig.IS_OVERWRITE,
false);
+ commitInsert(jobContext, table, startTime, writeResults, isOverwrite);
+ } else {
+ LOG.info("Unable to determine commit operation type for table: {},
jobID: {}. Will not create a commit.",
+ table, jobContext.getJobID());
+ }
+ }
+
+ private void commitDelete(JobContext jobContext, Table table, long
startTime, Collection<FilesForCommit> results) {
Review Comment:
After playing around with update, i think we will have 2 methods in the end:
- InsertOverwrite
- RowUpdate (I still have to check what happens, if there is no insert or
delete), but the other part of the code is the same. If rowupdate does not work
we can fall back to Transaction
Issue Time Tracking
-------------------
Worklog Id: (was: 754999)
Time Spent: 15h 40m (was: 15.5h)
> Implement DELETE statements for Iceberg tables
> ----------------------------------------------
>
> Key: HIVE-26102
> URL: https://issues.apache.org/jira/browse/HIVE-26102
> Project: Hive
> Issue Type: New Feature
> Reporter: Marton Bod
> Assignee: Marton Bod
> Priority: Major
> Labels: pull-request-available
> Time Spent: 15h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
