Luca Cavanna created LUCENE-10507:
-------------------------------------
Summary: Should it be more likely to search concurrently in tests?
Key: LUCENE-10507
URL: https://issues.apache.org/jira/browse/LUCENE-10507
Project: Lucene - Core
Issue Type: Improvement
Reporter: Luca Cavanna
As part of LUCENE-10002 we are migrating test usages of
IndexSearcher#search(Query, Collector) to use the corresponding search method
that takes a CollectorManager in place of a Collector. As part of such changes,
I've been paying attention to whether searchers are created through
LuceneTestCase#newSearcher and migrating to it when possible.
This caused some recent test failures following test changes, which were in
most cases test issues, although they were quite rare due to the fact that we
only rarely exercise the concurrent code-path in tests.
One recent failure uncovered LUCENE-10500, which was an actual bug that
affected concurrent searches only, and was uncovered by a test run that indexed
a considerable amount of docs and was lucky enough to get an executor set to
its index searcher as well as get multiple slices.
LuceneTestCase#newIndexSearcher(IndexReader) uses threads only rarely, and even
when useThreads is true, the searcher may not get an executor set. Also, it can
often happen that despite an executor is set, the searcher will hold only one
slice, as not enough documents are indexed. Some nightly tests index enough
documents, and LuceneTestCase also lowers the slice limits but only 50% of the
times and only when wrapWithAssertions is false. Also I wonder if the lower
limits are low enough:
{code:java}
int maxDocPerSlice = 1 + random.nextInt(100000);
int maxSegmentsPerSlice = 1 + random.nextInt(20);
{code}
All in all, I wonder if we should make it more likely for real concurrent
searches to happen while testing across multiple slices. It seems like it could
be useful especially as we'd like users to use collector managers instead of
collectors (although that does not necessarily translate to concurrent search).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
