Peter Rozsa has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/24394 )
Change subject: IMPALA-15065: Fix incorrect partition spec identification when removing deletion vectors from Iceberg V3 tables ...................................................................... IMPALA-15065: Fix incorrect partition spec identification when removing deletion vectors from Iceberg V3 tables When Impala removes an old deletion vector via rowDelta.removeDeletes(), it previously passed a DeleteFile reconstructed from the C++ flatbuffer using FileMetadata.Builder. For tables whose partition spec is a void transform (partitioned-to-unpartitioned evolution), FileMetadata.Builder produces partition()=null. The manifest-resident entry for the same DV was written with a non-null (empty) PartitionData. Iceberg's ManifestFilterManager.canContainAny() then fails to match the manifest. This change uses direct manifest entry lookup to locate the old deletion vectors based on the referenced data file path, it guarantees that the DeleteFile is removable in the row delta. Tests: - added regression test to iceberg-v3-merge.test Change-Id: Iba280972e2089e92771c8b3f7355d916d46ad968 --- M fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-merge.test 4 files changed, 209 insertions(+), 33 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/24394/4 -- To view, visit http://gerrit.cloudera.org:8080/24394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iba280972e2089e92771c8b3f7355d916d46ad968 Gerrit-Change-Number: 24394 Gerrit-PatchSet: 4 Gerrit-Owner: Peter Rozsa <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
