[
https://issues.apache.org/jira/browse/HIVE-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968441#comment-16968441
]
Ádám Szita commented on HIVE-22414:
-----------------------------------
Also included a feature so that partition keys and values become separable
through encoding - this could be beneficial if e.g. in the future we'd want to
filter buffers based on partition (values).
> Make LLAP CacheTags more memory efficient
> -----------------------------------------
>
> Key: HIVE-22414
> URL: https://issues.apache.org/jira/browse/HIVE-22414
> Project: Hive
> Issue Type: Improvement
> Components: llap
> Reporter: Ádám Szita
> Assignee: Ádám Szita
> Priority: Major
> Attachments: HIVE-22414.0.patch, HIVE-22414.1.patch,
> HIVE-22414.2.patch, HIVE-22414.3.patch, HIVE-22414.4.patch,
> HIVE-22414.5.patch, HIVE-22414.6.patch
>
>
> MultiPartitionCacheTag relies on LinkedLists.
> A LinkedList object that holds e.g. 2 nodes consumes 112 bytes roughly in
> this composition:
> * 16 bytes for LinkedList object header
> * 8 bytes for referring head
> * 8 bytes for referring tail
> * 80 = 2 x (16 bytes for LinkedList$Node header, 3 x 8 bytes for referring
> prev, next, item)
> This is a lot, so I propose to replace LinkedList in MultiPartitionCacheTag
> with a simple String array. (For a similar scenario an array would take 16 +
> 8 + 2 x 8 = 40 bytes, as per header, count, and 2 references for our actual
> objects).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
