Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24394 )
Change subject: IMPALA-15065: Fix incorrect partition spec identification when removing deletion vectors from Iceberg V3 tables ...................................................................... IMPALA-15065: Fix incorrect partition spec identification when removing deletion vectors from Iceberg V3 tables When Impala removes an old deletion vector via rowDelta.removeDeletes(), it previously passed a DeleteFile reconstructed from the C++ flatbuffer using FileMetadata.Builder. For tables whose partition spec is a void transform (partitioned-to-unpartitioned evolution), FileMetadata.Builder produces partition()=null. The manifest-resident entry for the same DV was written with a non-null (empty) PartitionData. Iceberg's ManifestFilterManager.canContainAny() then fails to match the manifest. This change uses direct manifest entry lookup to locate the old deletion vectors based on the referenced data file path, it guarantees that the DeleteFile is removable in the row delta. Tests: - added regression test to iceberg-v3-merge.test Change-Id: Iba280972e2089e92771c8b3f7355d916d46ad968 Reviewed-on: http://gerrit.cloudera.org:8080/24394 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-merge.test 4 files changed, 209 insertions(+), 31 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iba280972e2089e92771c8b3f7355d916d46ad968 Gerrit-Change-Number: 24394 Gerrit-PatchSet: 9 Gerrit-Owner: Peter Rozsa <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
