Todd Lipcon created IMPALA-7453:
-----------------------------------
Summary: Intern HdfsStorageDescriptors
Key: IMPALA-7453
URL: https://issues.apache.org/jira/browse/IMPALA-7453
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Reporter: Todd Lipcon
Every partition currently has an HdfsStorageDescriptor attached. In most cases,
the number of unique storage descriptors in a warehouse is pretty low (most
partitions use the same escaping, file formats, etc). For example, in the
functional test data load, we only have 24 unique SDs across ~10k partitions.
Each object takes 32 bytes (with compressed oops) or 40 (without). So, we can
get some small memory/object-count savings by interning these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
