[
https://issues.apache.org/jira/browse/HIVE-25007?focusedWorklogId=582345&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582345
]
ASF GitHub Bot logged work on HIVE-25007:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Apr/21 10:13
Start Date: 14/Apr/21 10:13
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2178:
URL: https://github.com/apache/hive/pull/2178#discussion_r613119272
##########
File path:
iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java
##########
@@ -143,19 +145,30 @@ public void commitJob(JobContext originalContext) throws
IOException {
LOG.info("Committing job has started for table: {}, using location: {}",
table,
generateJobLocation(conf, jobContext.getJobID()));
- FileIO io = HiveIcebergStorageHandler.io(jobContext.getJobConf());
- List<DataFile> dataFiles = dataFiles(jobContext, io, true);
-
- if (dataFiles.size() > 0) {
+ List<DataFile> dataFiles = dataFiles(jobContext, table.io(), true);
+
+ boolean isOverwrite = conf.getBoolean(InputFormatConfig.IS_OVERWRITE,
false);
+ if (isOverwrite) {
+ // Replacing affected partitions (or whole table) since it's an insert
overwrite
+ // We do the replacement even for 0 data files (i.e. empty source)
+ ReplacePartitions overwrite = table.newReplacePartitions();
+ dataFiles.forEach(overwrite::addFile);
+ overwrite.commit();
Review comment:
Is there an easy way to check if we change anything?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 582345)
Time Spent: 20m (was: 10m)
> Implement insert overwrite for Iceberg tables
> ---------------------------------------------
>
> Key: HIVE-25007
> URL: https://issues.apache.org/jira/browse/HIVE-25007
> Project: Hive
> Issue Type: New Feature
> Reporter: Marton Bod
> Assignee: Marton Bod
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
