[
https://issues.apache.org/jira/browse/HIVE-20893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16680486#comment-16680486
]
Gopal V commented on HIVE-20893:
--------------------------------
The masks object was rough place-holder for an eventual Long8 type to be
allocated locally (the jdk value types), to stand in for the __m512i variables
Talked to [~bslim] about this and I'm thinking an even simpler solution than
unrolling might work out just fine.
{code}
- masks[wordOffset] |= (1L << bitPos);
+ final long bloomWord = bitSet.data[wordOffset + i];
+ if (0 == (bloomWord & (1L << bitPos))) return false;
+ }
+ return true;
{code}
And re-run your benchmark?
That makes it similar to the original bloom implementation and better than the
ThreadLocal at rejecting rows.
If you do have a linux box with libhsdis, then the output of JMH -prof perfasm
would be super useful to look into.
> BloomK Filter probing method is not thread safe
> -----------------------------------------------
>
> Key: HIVE-20893
> URL: https://issues.apache.org/jira/browse/HIVE-20893
> Project: Hive
> Issue Type: Bug
> Components: storage-api
> Reporter: slim bouguerra
> Priority: Major
>
> As far i can tell this is not an issue for Hive yet (most of the usage of
> probing seems to be done by one thread at a time) but it is an issue of other
> users like Druid as per the following
> issue.[https://github.com/apache/incubator-druid/issues/6546]
> The fix is proposed by the author of
> [https://github.com/apache/incubator-druid/pull/6584] is to make couple of
> local fields as ThreadLocals.
> Idea looks good to me and doesn't have any perf drawbacks.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
