Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/21210 )
Change subject: IMPALA-12600: Schema evolution with equality delete files
......................................................................
IMPALA-12600: Schema evolution with equality delete files
This patch adds test coverage for a table that has equality delete
files and also schema evolution, where the schema changes didn't affect
the primary key columns.
Note, partition evolution on tables with equality deletes is still
not supported.
Testing:
- Added a new test table for this use-case and some E2E tests on that
table.
Change-Id: I125f72bade5b79bad5aaa6b676d6afaf3ca98395
Reviewed-on: http://gerrit.cloudera.org:8080/21210
Reviewed-by: Gabor Kaszab <gaborkas...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
---
M testdata/data/README
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/data/d=2024-03-20/00000-10-e4b47c78-9a7a-4d68-81d9-ab22e44a3630-00002.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/data/d=2024-03-20/3645a3085845c344-9698e59400000000_1309071497_data.0.parq
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/data/d=2024-03-21/00000-10-e4b47c78-9a7a-4d68-81d9-ab22e44a3630-00004.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/data/d=2024-03-21/00000-11-a72caf13-6a91-4fd5-b509-54ec8b16864f-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/data/d=2024-03-21/00000-11-a72caf13-6a91-4fd5-b509-54ec8b16864f-00002.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/data/d=2024-03-21/3645a3085845c344-9698e59400000000_1656341410_data.0.parq
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/data/d=2024-03-22/3645a3085845c344-9698e59400000000_1634747934_data.0.parq
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/94efa501-9664-420d-a524-30535c11d363-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/bf0f2c96-954e-4b3c-a686-8b06e9fd56e8-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/fb7a4022-ee0a-4540-87d8-b8fa8e4c8596-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/fb7a4022-ee0a-4540-87d8-b8fa8e4c8596-m1.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/snap-3986738438831924669-1-bf0f2c96-954e-4b3c-a686-8b06e9fd56e8.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/snap-5816823095034839884-1-fb7a4022-ee0a-4540-87d8-b8fa8e4c8596.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/snap-5816823095034839884-1-fb7a4022-ee0a-4540-87d8-b8fa8e4c8596.avro_tmp
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/snap-7131747670101362192-1-94efa501-9664-420d-a524-30535c11d363.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/v3.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/v4.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/v5.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/v6.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_equality_delete_schema_evolution/metadata/version-hint.text
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M
testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-equality-deletes.test
27 files changed, 1,034 insertions(+), 0 deletions(-)
Approvals:
Gabor Kaszab: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/21210
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I125f72bade5b79bad5aaa6b676d6afaf3ca98395
Gerrit-Change-Number: 21210
Gerrit-PatchSet: 6
Gerrit-Owner: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
