Hello Daniel Becker, Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21847
to look at the new patch set (#2).
Change subject: IMPALA-11265: Part1: Clear GroupContentFiles once used
......................................................................
IMPALA-11265: Part1: Clear GroupContentFiles once used
GroupContentFiles stores the file descriptors in Iceberg's format and is
used for creating file descriptors in Impala's format. Once this
creation is done, we no longer have to keep these Iceberg ContentFiles.
Dropping these could significantly reduce the memory footprint of an
Iceberg table.
Measurements:
I have a test table that has 110k files. The measurements showed that
cleaning the GroupedContentFiles could reduce the memory size of this
particular table from 140MB to 80MB.
Change-Id: I1efdd2a46c9675f7461535259e5892ed213a6b21
---
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/GroupedContentFiles.java
2 files changed, 7 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/21847/2
--
To view, visit http://gerrit.cloudera.org:8080/21847
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1efdd2a46c9675f7461535259e5892ed213a6b21
Gerrit-Change-Number: 21847
Gerrit-PatchSet: 2
Gerrit-Owner: Gabor Kaszab <[email protected]>
Gerrit-Reviewer: Daniel Becker <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
