Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24041 )
Change subject: IMPALA-14583: Support partial RPC dispatch for Iceberg tables ...................................................................... Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/24041/3/fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java File fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java: http://gerrit.cloudera.org:8080/#/c/24041/3/fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java@162 PS3, Line 162: if (startOffset == 0 && endOffset == fileDescList_.size()) return toThrift(); : : if (reverseMap_ == null) { : > 'reverseMap' has got some backlash already. What about this: change it to a Adding some more backlash for the reverse map :) A hash map should take around 30-40 bytes per entry, so 30-40MB for a 1 M file table where the pagination would be actually used. This is not ignorable, in the local table I use for benchmark the IcebergContentFileStore takes around 400-450MB, so it's nearly 10% overhead. I would switch back to generating it on each request (my guess is that it is not that slow), or use a soft reference to make it GCable if the catalogd runs out of memory completely. -- To view, visit http://gerrit.cloudera.org:8080/24041 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7f2c058b7cc8efc15bac9fe0e91baadbb7b92cbb Gerrit-Change-Number: 24041 Gerrit-PatchSet: 4 Gerrit-Owner: Mihaly Szjatinya <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Mihaly Szjatinya <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 12 Mar 2026 09:12:21 +0000 Gerrit-HasComments: Yes
