Fu Lili has uploaded a new patch set (#15). (
http://gerrit.cloudera.org:8080/21608 )
Change subject: IMPALA-13254: Optimize REFRESH for Iceberg tables
......................................................................
IMPALA-13254: Optimize REFRESH for Iceberg tables
Considering that Iceberg's ContentFile is a collection of immutable
files, the current code logic has been simplified. The optimized
process is as follows:
1. For existing ContentFiles, directly reuse the existing file
descriptors.
2. For newly added ContentFiles that do not support block locations,
directly create file descriptors.
3. For newly added ContentFiles that support block locations,
choose between using a listLocatedStatus operation or calling
getFileBlockLocations one by one, based on the number of files.
Change-Id: I8c99a28eb16275efdff52e0ea2711c0c6036719
---
M be/src/common/global-flags.cc
M fe/src/main/java/org/apache/impala/catalog/IcebergFileMetadataLoader.java
M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java
3 files changed, 134 insertions(+), 159 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/21608/15
--
To view, visit http://gerrit.cloudera.org:8080/21608
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8c99a28eb16275efdff52e0ea2711c0c6036719
Gerrit-Change-Number: 21608
Gerrit-PatchSet: 15
Gerrit-Owner: Fu Lili <[email protected]>
Gerrit-Reviewer: Fu Lili <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
