Noemi Pap-Takacs has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22189
Change subject: IMPALA-13501: Clean up uncommitted Iceberg files after validation check failure ...................................................................... IMPALA-13501: Clean up uncommitted Iceberg files after validation check failure Iceberg supports multiple writers with optimistic concurrency. Each writer can write new files which are then added to the table after a validation check to ensure that the commit does not conflict with other modifications made during the execution. When there was a conflicting change which cannot be resolved, it means that the newly written files cannot be committed to the table, so they used to become orphan files in the file system. This change introduces automatic cleanup of uncommitted files after an unsuccessful DML operation. Testing: - E2E test: injected ValidationException with debug option Change-Id: Ibe59546ebf3c639b75b53dfa1daba37cef50eb21 --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/DebugUtils.java M tests/query_test/test_iceberg.py 4 files changed, 82 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/22189/1 -- To view, visit http://gerrit.cloudera.org:8080/22189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ibe59546ebf3c639b75b53dfa1daba37cef50eb21 Gerrit-Change-Number: 22189 Gerrit-PatchSet: 1 Gerrit-Owner: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
