Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24758
Change subject: IMPALA-15291: Fix EC Policy column when block locations are not preloaded ...................................................................... IMPALA-15291: Fix EC Policy column when block locations are not preloaded IMPALA-15229 made the "EC Policy" column (SHOW TABLE STATS, SHOW PARTITIONS, SHOW FILES) derive from FileDescriptor.getIsEc(). That flag was only set on the block-loading path (create()); createWithNoBlocks(), used when block-location preloading is disabled, hardcoded is_ec=false, so erasure-coded files loaded that way reported NONE. createWithNoBlocks now reads the erasure-coding and encryption flags from the FileStatus, like create(). Wherever the listing carries a real (HDFS) FileStatus, getIsEc()/getIsEncrypted() are now correct at no extra cost. Iceberg is the exception: on the no-preload path it builds the FileStatus synthetically (size and path only) to avoid a per-file stat RPC, so it carries no erasure-coding info. Rather than re-add a per-file NameNode RPC that could still be wrong (a directory's policy is not necessarily the file's), SHOW FILES reports "n/a" for such descriptors. Knowing the EC policy in such cases wouldn't have too much added value as we already schedule blockless splits as remote reads, so EC policy doesn't change much for query processing. SHOW TABLE STATS is unaffected: it reads the policy from one RPC on the table location. Testing * FeFsTableTest: descriptors from createWithNoBlocks keep the flags; a partition mixing an EC and a plain file reports MIXED. * iceberg-insert.test SHOW FILES rows assert $EXPECTED_ERASURECODE_POLICY, supplied per driver: the real policy in test_iceberg, "n/a" in TestDisabledBlockLocations. * test_iceberg_smoke/test_iceberg_v3_smoke in TestDisabledBlockLocations on an EC build. Change-Id: Ia7a7c97a20f6ec0efc2150185f4a9e26e3ac43cf Assisted-by: Claude Opus 4.8 <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/FileDescriptor.java M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M fe/src/test/java/org/apache/impala/catalog/FeFsTableTest.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-insert.test M tests/custom_cluster/test_disabled_block_locations.py M tests/query_test/test_iceberg.py 7 files changed, 73 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/24758/1 -- To view, visit http://gerrit.cloudera.org:8080/24758 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia7a7c97a20f6ec0efc2150185f4a9e26e3ac43cf Gerrit-Change-Number: 24758 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
