[
https://issues.apache.org/jira/browse/HIVE-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ádám Szita updated HIVE-22414:
------------------------------
Description:
MultiPartitionCacheTag relies on LinkedLists.
A LinkedList object that holds e.g. 2 nodes consumes 112 bytes roughly in this
composition:
* 16 bytes for LinkedList object header
* 8 bytes for referring head
* 8 bytes for referring tail
* 80 = 2 x (16 bytes for LinkedList$Node header, 3 x 8 bytes for referring
prev, next, item)
This is a lot, so I propose to replace LinkedList in MultiPartitionCacheTag
with a simple String array. (For a similar scenario an array would take 16 + 8
+ 2 x 8 = 40 bytes, as per header, count, and 2 references for our actual
objects).
> Make LLAP CacheTags more memory efficient
> -----------------------------------------
>
> Key: HIVE-22414
> URL: https://issues.apache.org/jira/browse/HIVE-22414
> Project: Hive
> Issue Type: Improvement
> Components: llap
> Reporter: Ádám Szita
> Assignee: Ádám Szita
> Priority: Major
> Attachments: HIVE-22414.0.patch
>
>
> MultiPartitionCacheTag relies on LinkedLists.
> A LinkedList object that holds e.g. 2 nodes consumes 112 bytes roughly in
> this composition:
> * 16 bytes for LinkedList object header
> * 8 bytes for referring head
> * 8 bytes for referring tail
> * 80 = 2 x (16 bytes for LinkedList$Node header, 3 x 8 bytes for referring
> prev, next, item)
> This is a lot, so I propose to replace LinkedList in MultiPartitionCacheTag
> with a simple String array. (For a similar scenario an array would take 16 +
> 8 + 2 x 8 = 40 bytes, as per header, count, and 2 references for our actual
> objects).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
