Tamas Mate has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17918
Change subject: WIP IMPALA-10569: Determine Iceberg file format from Icebert metadata ...................................................................... WIP IMPALA-10569: Determine Iceberg file format from Icebert metadata This commit refactors how the Iceberg table's fileformat is obtained. Earlier it was collected from the HMS, however Iceberg stores the file format on DataFile level. Iceberg Metadata is the source of truth in this case and it is beneficial to use it instead of the HMS information. The FlatBuffer FileDescriptor has been extended with the file format, which is initialized with the DataFile's format during FD creation. Currently, only one fileformat per partition is supported by Impala and Iceberg is treated as an un-partitioned table, therefore any FileDescriptor in the FileDescriptor list can be used as file format. Change-Id: I74a583ffe91694a647a37846cd58a99e37d6ce72 --- M common/fbs/CatalogObjects.fbs M common/fbs/IcebergObjects.fbs M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java 11 files changed, 72 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/17918/1 -- To view, visit http://gerrit.cloudera.org:8080/17918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I74a583ffe91694a647a37846cd58a99e37d6ce72 Gerrit-Change-Number: 17918 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate <[email protected]>
