Peter Rozsa has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24394
Change subject: IMPALA-15065: Fix incorrect partition spec identification when removing deletion vectors from Iceberg V3 tables ...................................................................... IMPALA-15065: Fix incorrect partition spec identification when removing deletion vectors from Iceberg V3 tables When Impala removes an old deletion vector via rowDelta.removeDeletes(), it previously passed a DeleteFile reconstructed from the C++ flatbuffer using FileMetadata.Builder. For tables whose partition spec is a void transform (partitioned-to-unpartitioned evolution), FileMetadata.Builder produces partition()=null. The manifest-resident entry for the same DV was written with a non-null (empty) PartitionData. Iceberg's ManifestFilterManager.canContainAny() then fails to match the manifest. This change uses direct manifest entry lookup to locate the old deletion vectors based on the referenced data file path, it guarantees that the DeleteFile is removable in the row delta. Tests: - added regression test to iceberg-v3-merge.test Change-Id: Iba280972e2089e92771c8b3f7355d916d46ad968 --- M fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-merge.test 4 files changed, 201 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/24394/1 -- To view, visit http://gerrit.cloudera.org:8080/24394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iba280972e2089e92771c8b3f7355d916d46ad968 Gerrit-Change-Number: 24394 Gerrit-PatchSet: 1 Gerrit-Owner: Peter Rozsa <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
