Hello Rui,
We actually released a change that should make MaxScoreBulkScorer faster on
dense disjunctions in 9.8: https://github.com/apache/lucene/pull/12444.
Your benchmark case is quite specific though as all clauses match all docs
and produce constant scores, so I would expect the scorer to quickly
realize that it can skip all documents once it's scored the first k docs.
This makes me wonder if it bottleneck on skipping blocks of documents
rather than on scoring them. Would you be able to share your whole flame
graph, it looks like it may be truncated a the top?
On Mon, Sep 16, 2024 at 10:01 PM Rui Wu <rui...@mongodb.com> wrote:
> Correction: The index has 3.6 million documents.
>
> On Mon, Sep 16, 2024 at 1:00 PM Rui Wu <rui...@mongodb.com> wrote:
>
>> Dear experts,
>>
>> In our Mongodb Atlas Search performance regression test between Lucene
>> 9.7 and Lucene 9.11, we detect a 43% latency regression in this query shape:
>> 12 SHOULD clause, and each clause matches all of the documents. Each
>> should clause is wrapped in ConstantScoreQuery.
>>
>> The index has 3.6 documents, and every document is identical: Every
>> document is {"path": ["1", "2", "3" ... "12"]}
>> The query shape is a BooleanQuery of SHOULD "1", SHOULD "2", ... SHOULD
>> "12".
>>
>> Our flamegraphs show that most of the time in search() is spent on
>> the MaxScoreBulkScorer class:
>> [image: image.png]
>>
>> We wonder if this extreme test case is expected to be slow on
>> MaxScoreBulkScorer?
>>
>> Thanks a lot!
>>
>> Rui Wu
>> Lead Engineer, MongoDB
>>
>
--
Adrien
