Misha Dmitriev has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/10982 )
Change subject: IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap. ...................................................................... IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap. This change switches initialization from eager to lazy for the following fields in Table: HashMap colsByName_ and ArrayType type_. In this way, we avoid wasting memory when HashMaps stay empty (an empty HashMap still uses at least 48 bytes in the heap), and when a StructType with its own empty HashMap and ArrayList instances is initialized, but stays unused. I have calculated ArrayType, StructType, two HashMaps and one ArrayList collectively use up 160 bytes in the heap per a Table. Further analysis can be found in the JIRA. This optimization becomes really relevant when a catalog server loads a very large number (millions) of tables, and all/most of them are represented as IncompleteTable instances. Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6 --- M fe/src/main/java/org/apache/impala/catalog/Table.java 1 file changed, 24 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/10982/4 -- To view, visit http://gerrit.cloudera.org:8080/10982 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6 Gerrit-Change-Number: 10982 Gerrit-PatchSet: 4 Gerrit-Owner: Misha Dmitriev <[email protected]> Gerrit-Reviewer: Misha Dmitriev <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]>
