Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18240 )
Change subject: IMPALA-11053: Impala should be able to read migrated partitioned Iceberg tables ...................................................................... Patch Set 4: (4 comments) Looks superb, few comments about tests. http://gerrit.cloudera.org:8080/#/c/18240/4/common/fbs/IcebergObjects.fbs File common/fbs/IcebergObjects.fbs: http://gerrit.cloudera.org:8080/#/c/18240/4/common/fbs/IcebergObjects.fbs@23 PS4, Line 23: AVRO AVRO was added here as a possibility, but there is no test for it yet (if I haven't missed something). It is also not mentioned in the commit mesage. http://gerrit.cloudera.org:8080/#/c/18240/4/testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/metadata/v1.metadata.json File testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/metadata/v1.metadata.json: http://gerrit.cloudera.org:8080/#/c/18240/4/testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/metadata/v1.metadata.json@1 PS4, Line 1: { Can you add some minimal info about these files to https://github.com/apache/impala/blob/master/testdata/data/README ? http://gerrit.cloudera.org:8080/#/c/18240/4/testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/metadata/v1.metadata.json@148 PS4, Line 148: "engine.hive.enabled" : "true", : "MIGRATED_TO_ICEBERG" : "true", Does the Hive we depend on support upgrading to Iceberg? It would be great to add a test for the interop scenario where the Iceberg metadata files are created by Hive during the test or dataload. This would allow us to catch if something changes on the Hive side. http://gerrit.cloudera.org:8080/#/c/18240/4/testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_legacy_partition_schema_evolution_orc/metadata/v1.metadata.json File testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_legacy_partition_schema_evolution_orc/metadata/v1.metadata.json: http://gerrit.cloudera.org:8080/#/c/18240/4/testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_legacy_partition_schema_evolution_orc/metadata/v1.metadata.json@77 PS4, Line 77: "write.format.default" : "parquet", This name of the table contains "orc", but metadata files contain "parquet" -- To view, visit http://gerrit.cloudera.org:8080/18240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac11a02de709d43532056f71359c49d20c1be2b8 Gerrit-Change-Number: 18240 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Gergely Fürnstáhl <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tamas Mate <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Tue, 01 Mar 2022 13:05:25 +0000 Gerrit-HasComments: Yes
