Hello Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24394
to look at the new patch set (#5).
Change subject: IMPALA-15065: Fix incorrect partition spec identification when
removing deletion vectors from Iceberg V3 tables
......................................................................
IMPALA-15065: Fix incorrect partition spec identification when removing
deletion vectors from Iceberg V3 tables
When Impala removes an old deletion vector via rowDelta.removeDeletes(),
it previously passed a DeleteFile reconstructed from the C++ flatbuffer
using FileMetadata.Builder. For tables whose partition spec is a void
transform (partitioned-to-unpartitioned evolution), FileMetadata.Builder
produces partition()=null. The manifest-resident entry for the same DV
was written with a non-null (empty) PartitionData. Iceberg's
ManifestFilterManager.canContainAny() then fails to match the manifest.
This change uses direct manifest entry lookup to locate the old deletion
vectors based on the referenced data file path, it guarantees that the
DeleteFile is removable in the row delta.
Tests:
- added regression test to iceberg-v3-merge.test
Change-Id: Iba280972e2089e92771c8b3f7355d916d46ad968
---
M fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java
M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-merge.test
4 files changed, 209 insertions(+), 29 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/24394/5
--
To view, visit http://gerrit.cloudera.org:8080/24394
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iba280972e2089e92771c8b3f7355d916d46ad968
Gerrit-Change-Number: 24394
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Peter Rozsa <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
