[
https://issues.apache.org/jira/browse/ARROW-9873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185601#comment-17185601
]
Yibo Cai commented on ARROW-9873:
---------------------------------
*NOTE: this approach will cause performance regression in certain conditions.*
We need to find min/max values before deciding if we can use this counting
method. If we finally find (max-min) is too large, that step is wasted and
causes performance drop. This drop is trivial normally.
But it can be big in certain conditions: there are few distinct values, but
sparsely distributed(max-min is big). In this condition, hash table will be
very fast, so the penalty of finding min/max is non-trivial. Above 10%
performance drop is observed per my test, see [test
code|https://github.com/cyb70289/mytests/blob/master/cpp/minmax-penalty.cc].
> [C++][Compute] Improve mode kernel for intergers within limited value range
> ---------------------------------------------------------------------------
>
> Key: ARROW-9873
> URL: https://issues.apache.org/jira/browse/ARROW-9873
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Yibo Cai
> Assignee: Yibo Cai
> Priority: Major
> Attachments: mode-range-skylake.png
>
>
> It's possible to improve mode kernel performance for integers within limited
> value range by using a value indexed array instead of general hash table.
> Similar trick is used in sorting kernel ARROW-1571.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
