Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21847 )
Change subject: IMPALA-11265: Part1: Clear GroupContentFiles once used ...................................................................... IMPALA-11265: Part1: Clear GroupContentFiles once used GroupContentFiles stores the file descriptors in Iceberg's format and is used for creating file descriptors in Impala's format. Once this creation is done, we no longer have to keep these Iceberg ContentFiles. Dropping these could significantly reduce the memory footprint of an Iceberg table. Measurements: I have a test table that has 110k files. The measurements showed that cleaning the GroupedContentFiles could reduce the memory size of this particular table from 140MB to 80MB. Change-Id: I1efdd2a46c9675f7461535259e5892ed213a6b21 Reviewed-on: http://gerrit.cloudera.org:8080/21847 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java M fe/src/main/java/org/apache/impala/catalog/iceberg/GroupedContentFiles.java 2 files changed, 7 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/21847 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1efdd2a46c9675f7461535259e5892ed213a6b21 Gerrit-Change-Number: 21847 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
