[
https://issues.apache.org/jira/browse/LUCENE-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16207921#comment-16207921
]
Dawid Weiss edited comment on LUCENE-7994 at 10/17/17 5:07 PM:
---------------------------------------------------------------
bq. Do you remember roughly what performance impact?
Well, in stress tests it was a significant number, but these are fairly
unrealistic in the sense that they don't do anything else *but* map edits... In
reality I don't think it'll play a crucial role. Those hash collisions, on the
other hand, were possible to hit with trivial map-iteration-copying blocks and
they were *nasty* (and people rightfully didn't and couldn't anticipate them to
happen). So I went for "better slower than sorry" direction...
bq. I think rather than add a full dependency on HPPC, I would copy over just
the specific native maps we are using here; the license is ASL2 (thank you!!).
Does that sound OK?
Absolutely, take what you want. {{fastutil}} is also ASL2 and it's very
high-quality too, so feel free to take whatever you wish. {{Koloboke}} has some
nice ideas too (for example storing int key/value as consecutive items in a
single array for better cache locality.
was (Author: dweiss):
bq. I think rather than add a full dependency on HPPC, I would copy over just
the specific native maps we are using here; the license is ASL2 (thank you!!).
Does that sound OK?
Absolutely, take what you want. {{fastutil}} is also ASL2 and it's very
high-quality too, so feel free to take whatever you wish. {{Koloboke}} has some
nice ideas too (for example storing int key/value as consecutive items in a
single array for better cache locality.
> Use int/int hash map for int taxonomy facet counts
> --------------------------------------------------
>
> Key: LUCENE-7994
> URL: https://issues.apache.org/jira/browse/LUCENE-7994
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: master (8.0), 7.2
>
> Attachments: LUCENE-7994.patch
>
>
> Int taxonomy facets today always count into a dense {{int[]}}, which is
> wasteful in cases where the number of unique facet labels is high and the
> size of the current result set is small.
> I factored the native hash map from LUCENE-7927 and use a simple heuristic
> (customizable by the user by subclassing) to decide up front whether to count
> sparse or dense. I also made loading of the large children and siblings
> {{int[]}} lazy, so that they are only instantiated if you really need them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
