Todd Lipcon has submitted this change and it was merged. ( )

Change subject: IMPALA-7453. Intern HdfsStorageDescriptors

IMPALA-7453. Intern HdfsStorageDescriptors

The number of unique HdfsStorageDescriptors in a warehouse is typically
much smaller than the number of partitions. Each object takes 32/40 bytes
(with/without compressed OOPs respectively). So, by interning these
objects, we can save that amount of memory as well as one object per

The overall savings aren't huge (on the order of tens of MBs) but the
change is pretty simple so seems worthwhile.

This patch also pulls in the errorprone annotations into the pom so that
errorprone can ensure that the class can be annotated as Immutable.
errorprone checks that classes annotated as Immutable only contain
immutable fields.

I tested this change by comparing 'jmap -histo:live' on a catalogd
before/after. For my local dev environment test warehouse, I had 12055
instances (385kb) before the change and 24 instances (768 bytes) after.

Change-Id: I9ef93148d629b060fa9f67c631e9c3d904a0ccf9
Reviewed-by: Bharath Vissapragada <>
Tested-by: Impala Public Jenkins <>
M fe/pom.xml
M fe/src/main/java/org/apache/impala/catalog/
M fe/src/main/java/org/apache/impala/catalog/
M fe/src/main/java/org/apache/impala/catalog/local/
M fe/src/main/java/org/apache/impala/catalog/local/
M fe/src/test/java/org/apache/impala/catalog/
6 files changed, 86 insertions(+), 26 deletions(-)

  Bharath Vissapragada: Looks good to me, approved
  Impala Public Jenkins: Verified

To view, visit
To unsubscribe, visit

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9ef93148d629b060fa9f67c631e9c3d904a0ccf9
Gerrit-Change-Number: 11236
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <>
Gerrit-Reviewer: Bharath Vissapragada <>
Gerrit-Reviewer: Impala Public Jenkins <>
Gerrit-Reviewer: Tianyi Wang <>
Gerrit-Reviewer: Todd Lipcon <>
Gerrit-Reviewer: Vuk Ercegovac <>

Reply via email to