[
https://issues.apache.org/jira/browse/HIVE-20321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581881#comment-16581881
]
Matt McCline commented on HIVE-20321:
-------------------------------------
With patch #3, there are 4 different kinds of vector hash key wrapper classes
with these members:
1) VectorHashKeyWrapperEmpty (int hashcode)
2) VectorHashKeyWrapperSingleLong (int hashcode, boolean isNull flag, long
value)
3) VectorHashKeyWrapperTwoLong (int hashcode, 2 X boolean isNull flags 2 X
long values)
4) VectorHashKeyWrapperGeneral (original class with many members including
arrays for all data types, etc)
With the new code "framework", we can add special case classes for other keys
as desired.
> Vectorization: Cut down memory size of 1 col VectorHashKeyWrapper to <1
> CacheLine
> ---------------------------------------------------------------------------------
>
> Key: HIVE-20321
> URL: https://issues.apache.org/jira/browse/HIVE-20321
> Project: Hive
> Issue Type: Bug
> Reporter: Gopal V
> Assignee: Matt McCline
> Priority: Major
> Attachments: HIVE-20321.01.patch, HIVE-20321.02.patch,
> HIVE-20321.03.patch
>
>
> With a full sized LLAP instance, the memory size of the VectorHashKeyWrapper
> is bigger than the low Xmx JVMs.
> {code}
> ***** 64-bit VM: **********************************************************
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper object internals:
> OFFSET SIZE
> TYPE DESCRIPTION VALUE
> 0 16
> (object header) N/A
> 16 4
> int VectorHashKeyWrapper.hashcode N/A
> 20 4
> (alignment/padding gap)
> 24 8
> long[] VectorHashKeyWrapper.longValues N/A
> 32 8
> double[] VectorHashKeyWrapper.doubleValues N/A
> 40 8
> byte[][] VectorHashKeyWrapper.byteValues N/A
> 48 8
> int[] VectorHashKeyWrapper.byteStarts N/A
> 56 8
> int[] VectorHashKeyWrapper.byteLengths N/A
> 64 8
> org.apache.hadoop.hive.serde2.io.HiveDecimalWritable[]
> VectorHashKeyWrapper.decimalValues N/A
> 72 8
> java.sql.Timestamp[] VectorHashKeyWrapper.timestampValues N/A
> 80 8
> org.apache.hadoop.hive.common.type.HiveIntervalDayTime[]
> VectorHashKeyWrapper.intervalDayTimeValues N/A
> 88 8
> boolean[] VectorHashKeyWrapper.isNull N/A
> 96 8
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper.HashContext
> VectorHashKeyWrapper.hashCtx N/A
> Instance size: 104 bytes
> Space losses: 4 bytes internal + 0 bytes external = 4 bytes total
> {code}
> Pulling this up to a parent class allows for this to be cut down to 32 bytes
> for the single column case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
