[
https://issues.apache.org/jira/browse/LUCENE-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16207815#comment-16207815
]
Michael McCandless commented on LUCENE-7994:
--------------------------------------------
bq. The current implementation in HPPC uses a different key mixing strategy
\[2\], combined with a unique per-instance seed to minimize the practical
impact of such clashes. The performance cost is there, but it's not huge...
something to consider?
Whoa, this looks very cool [~dweiss]! Do you remember roughly what performance
impact? I think rather than add a full dependency on HPPC, I would copy over
just the specific native maps we are using here; the license is ASL2 (thank
you!!). Does that sound OK?
bq. For the first condition, Isn't taxoReaderSize essentially the cardinality?
Why would we want a sparse hashtable in this low-cardinality case, I would
think the opposite (a simple array should be best, it will be small).
Duh, you're right, the logic is backwards for that first clause -- we should
just use an array if the cardinality is lowish (< 1024). And I agree on the
2nd clause too! I'll fix ...
> Use int/int hash map for int taxonomy facet counts
> --------------------------------------------------
>
> Key: LUCENE-7994
> URL: https://issues.apache.org/jira/browse/LUCENE-7994
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: master (8.0), 7.2
>
> Attachments: LUCENE-7994.patch
>
>
> Int taxonomy facets today always count into a dense {{int[]}}, which is
> wasteful in cases where the number of unique facet labels is high and the
> size of the current result set is small.
> I factored the native hash map from LUCENE-7927 and use a simple heuristic
> (customizable by the user by subclassing) to decide up front whether to count
> sparse or dense. I also made loading of the large children and siblings
> {{int[]}} lazy, so that they are only instantiated if you really need them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
