Hi Mayya, thanks for persisting - I think we need to wrestle this to the ground for sure. In the test I ran, RAM buffer was the default checked in, which is weirdly: 1994MB. I did not specifically set heap size. I used maxConn/M=200. I'll try with larger buffer to see if I can get 9.4 to produce a single segment for the same test settings. I see you used a much smaller M (16), which should have produced quite small graphs, and I agree, should have been a single segment. Were you able to verify the number of segments?
Agree that decrease in recall is not expected when more segments are produced. On Mon, Sep 12, 2022 at 1:51 PM Mayya Sharipova <mayya.sharip...@elastic.co.invalid> wrote: > > Hello Michael, > Thanks for checking. > Sorry for bringing this up again. > First of all, I am ok with proceeding with the Lucene 9.4 release and leaving > the performance investigations for later. > > I am interested in what's the maxConn/M value you used for your tests? What > was the heap memory and the size of the RAM buffer for indexing? > Usually, when we have multiple segments, recall should increase, not > decrease. But I agree that with multiple segments we can see a big drop in > QPS. > > Here is my investigation with detailed output of the performance difference > between 9.3 and 9.4 releases. In my tests I used a large indexing buffer > (2Gb) and large heap (5Gb) to end up with a single segment for both 9.3 and > 9.4 tests, but still see a big drop in QPS in 9.4. > > Thank you. > > > > > > On Fri, Sep 9, 2022 at 12:21 PM Alan Woodward <romseyg...@gmail.com> wrote: >> >> Done. Thanks! >> >> > On 9 Sep 2022, at 16:32, Michael Sokolov <msoko...@gmail.com> wrote: >> > >> > Hi Alan - I checked out the interval queries patch; seems pretty safe, >> > please go ahead and port to 9.4. Thanks! >> > >> > Mike >> > >> > On Fri, Sep 9, 2022 at 10:41 AM Alan Woodward <romseyg...@gmail.com> wrote: >> >> >> >> Hi Mike, >> >> >> >> I’ve opened https://github.com/apache/lucene/pull/11760 as a small bug >> >> fix PR for a problem with interval queries. Am I OK to port this to the >> >> 9.4 branch? >> >> >> >> Thanks, Alan >> >> >> >> On 2 Sep 2022, at 20:42, Michael Sokolov <msoko...@gmail.com> wrote: >> >> >> >> NOTICE: >> >> >> >> Branch branch_9_4 has been cut and versions updated to 9.5 on stable >> >> branch. >> >> >> >> Please observe the normal rules: >> >> >> >> * No new features may be committed to the branch. >> >> * Documentation patches, build patches and serious bug fixes may be >> >> committed to the branch. However, you should submit all patches you >> >> want to commit to Jira first to give others the chance to review >> >> and possibly vote against the patch. Keep in mind that it is our >> >> main intention to keep the branch as stable as possible. >> >> * All patches that are intended for the branch should first be committed >> >> to the unstable branch, merged into the stable branch, and then into >> >> the current release branch. >> >> * Normal unstable and stable branch development may continue as usual. >> >> However, if you plan to commit a big change to the unstable branch >> >> while the branch feature freeze is in effect, think twice: can't the >> >> addition wait a couple more days? Merges of bug fixes into the branch >> >> may become more difficult. >> >> * Only Jira issues with Fix version 9.4 and priority "Blocker" will delay >> >> a release candidate build. >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> >> >> >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> > For additional commands, e-mail: dev-h...@lucene.apache.org >> > >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
