Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18041
Change subject: IMPALA-11022: Impala uses wrong file descriptors for Iceberg tables in local catalog mode ...................................................................... IMPALA-11022: Impala uses wrong file descriptors for Iceberg tables in local catalog mode When local catalog mode is used, Impala retrieves the Iceberg snapshot from CatalogD. The response contains a map of the file descriptors. The file descriptors contain block location information, but the hosts are only referred by indexes. In the Coordinator's local catalog the host indexes might refer to different hosts than in CatalogD. This might lead to unnecessary remote reads as scan ranges are scheduled to random hosts. This patch translates the host index to the coordinators host list, so block locations remain consisten. Testing: * tested manually on a 6-node cluster, and verified that the file locations are consistent with HDFS Change-Id: I253b505846e1cf4d1be445c0d06b2552dc4ba1f8 --- M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/util/ListMap.java 13 files changed, 74 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/18041/1 -- To view, visit http://gerrit.cloudera.org:8080/18041 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I253b505846e1cf4d1be445c0d06b2552dc4ba1f8 Gerrit-Change-Number: 18041 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
