[
https://issues.apache.org/jira/browse/LUCENE-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Jelinski updated LUCENE-6878:
------------------------------------
Attachment: speedtest.tar.gz
I merged 64 ScoreDoc lists, 100k docs each, and took top 100k results, in 3
different score distributions. For time calculations, each test was repeated 60
times, and I averaged the results of 10 subsequent runs, discarding any
outliers. For the number of lessThan calls in random case, I run the test 3
times and took an average. The number of lessThan calls for case 1 and 2 is
constant.
I tested score lists generated using 3 different methods.
1) All scores equal to 1. This is the case where the patch made a greatest
difference, mostly because of tie breaks in lessThan methods. Results:
Without the patch - 2.66 msec per merge call, 1600057 calls to lessThan
With the patch - 0.32 msec per merge call, 200071 calls to lessThan.
Overall, ~88% savings on time and lessThan calls
2) Each list contains scores 100000,99999,....,1
Without the patch - 3.5 msec per merge call, 1100063 calls to lessThan
With the patch - 2.6 msec per merge call, 1005390 calls to lessThan
Overall, ~25% savings on time, 9% savings on lessThan calls
3) Each list starts with doc with score 100000, score of other docs is
calculated as score of previous doc minus Math.random()
Without the patch - 3.5 msec per merge call, ~1156500 calls to lessThan
With the patch, 2.7 msec per merge call, ~960500 calls to lessThan
Overall, ~23% savings on time, 17% savings on lessThan calls.
In the random case the gain is much less than the advertised double speed, but
it's still a net improvement.
I attached the code I used to measure the speed, in case anyone is interested.
Fair warning, it's not pretty.
> TopDocs.merge should use updateTop instead of pop / add
> -------------------------------------------------------
>
> Key: LUCENE-6878
> URL: https://issues.apache.org/jira/browse/LUCENE-6878
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
> Affects Versions: Trunk
> Reporter: Daniel Jelinski
> Assignee: Adrien Grand
> Priority: Trivial
> Fix For: 6.0, 5.4
>
> Attachments: LUCENE-6878.patch, speedtest.tar.gz
>
>
> The function TopDocs.merge uses PriorityQueue in a pattern: pop, update value
> (ref.hitIndex++), add. JavaDocs for PriorityQueue.updateTop say that using
> this function instead should be at least twice as fast.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
