[
https://issues.apache.org/jira/browse/IMPALA-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noemi Pap-Takacs resolved IMPALA-13501.
---------------------------------------
Fix Version/s: Impala 4.5.0
Resolution: Fixed
> Conflicting commits to Iceberg tables leave uncommitted orphan files
> --------------------------------------------------------------------
>
> Key: IMPALA-13501
> URL: https://issues.apache.org/jira/browse/IMPALA-13501
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Noemi Pap-Takacs
> Assignee: Noemi Pap-Takacs
> Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 4.5.0
>
>
> Iceberg supports multiple writers with optimistic concurrency. Each writer
> can write new files which are then added to the table after a validation
> check to ensure that the commit does not conflict with other modifications
> made during the execution.
> When there was a conflicting change and the newly written files cannot be
> committed, there are 2 ways to proceed: the commit can be retried and rebased
> on top of the latest snapshot. If this cannot resolve the conflict, the
> change cannot be committed and the files become orphan files in the file
> system.
> It would be nice to remove the remaining files from an unsuccessful commit in
> one step. Deleting orphan files later as a table maintenance step is also a
> possible resolution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
