Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11236 )
Change subject: IMPALA-7453. Intern HdfsStorageDescriptors ...................................................................... IMPALA-7453. Intern HdfsStorageDescriptors The number of unique HdfsStorageDescriptors in a warehouse is typically much smaller than the number of partitions. Each object takes 32/40 bytes (with/without compressed OOPs respectively). So, by interning these objects, we can save that amount of memory as well as one object per partition. The overall savings aren't huge (on the order of tens of MBs) but the change is pretty simple so seems worthwhile. This patch also pulls in the errorprone annotations into the pom so that errorprone can ensure that the class can be annotated as Immutable. errorprone checks that classes annotated as Immutable only contain immutable fields. I tested this change by comparing 'jmap -histo:live' on a catalogd before/after. For my local dev environment test warehouse, I had 12055 instances (385kb) before the change and 24 instances (768 bytes) after. Change-Id: I9ef93148d629b060fa9f67c631e9c3d904a0ccf9 Reviewed-on: http://gerrit.cloudera.org:8080/11236 Reviewed-by: Bharath Vissapragada <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/pom.xml M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsStorageDescriptor.java M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalPartitionSpec.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java 6 files changed, 86 insertions(+), 26 deletions(-) Approvals: Bharath Vissapragada: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11236 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9ef93148d629b060fa9f67c631e9c3d904a0ccf9 Gerrit-Change-Number: 11236 Gerrit-PatchSet: 6 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tianyi Wang <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]>
