[
https://issues.apache.org/jira/browse/HIVE-24221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206086#comment-17206086
]
Stamatis Zampetakis commented on HIVE-24221:
--------------------------------------------
There are various ways to create a hash from composite keys/columns. Without
any special effort to derive a perfect hash function we can do the following:
Input columns: A, B, C, D
+Option A:+
{noformat}
hash(hash(hash(A, B), C), D)
{noformat}
+Option B:+
{noformat}
31*(31*(31 * hash(A) + hash(B)) + hash(C)) + hash(D)
{noformat}
The second option is more or less what happens currently when we write hash(A,
B, C, D) in the non-vectorized implementation of GenericUDFMurmurHash.
The first option although it looks simpler is computationally more expensive.
> Use vectorizable expression to combine multiple columns in semijoin bloom
> filters
> ---------------------------------------------------------------------------------
>
> Key: HIVE-24221
> URL: https://issues.apache.org/jira/browse/HIVE-24221
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
> Environment:
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, multi-column semijoin reducers use an n-ary call to
> GenericUDFMurmurHash to combine multiple values into one, which is used as an
> entry to the bloom filter. However, there are no vectorized operators that
> treat n-ary inputs. The same goes for the vectorized implementation of
> GenericUDFMurmurHash introduced in HIVE-23976.
> The goal of this issue is to choose an alternative way to combine multiple
> values into one to pass in the bloom filter comprising only vectorized
> operators.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
