Re: Get knowledge about apache lucene index migrate

2024-09-12 Thread Rui Wu
Maybe a silly question: is it feasible (on your scale) to rebuild your index from your source of truth data? Thanks! On Tue, Aug 6, 2024 at 2:11 PM Michael Sokolov wrote: > Yes, there is no support for upgrading a pre-8.x index to 9 or later. > At some point it was decided that supporting that

KnnQueries and result discrepancy between indexes with the same data

2024-09-12 Thread Marc Davenport
Hello, I've been working on this personalization project using KNN queries and I have a couple questions but one is more pressing for me than the others. 1) Inconsistency between index instances: All of the same documents are loaded into different indexes. They may be loaded in different order, bu

Re: KnnQueries and result discrepancy between indexes with the same data

2024-09-12 Thread Adrien Grand
Indeed, the load order can influence Lucene's approximate nearest neighbor search results. If your two indexes load data sequentially and in the same order, then I believe that you would get the same results. But we consider this an implementation detail rather than a guarantee that Lucene should

Re: KnnQueries and result discrepancy between indexes with the same data

2024-09-12 Thread Michael Sokolov
> If your two indexes load data sequentially and in the same order, then I believe that you would get the same results. But we consider this an implementation detail rather than a guarantee that Lucene should have. You might even still be surprised by nondeterminism arising from concurrency during

Re: KnnQueries and result discrepancy between indexes with the same data

2024-09-12 Thread Marc Davenport
Adrien & Micheal. Thanks for confirming what I suspected. I think in the long run I will be ok as our users have a sticky session to an instance for some other reasons already. Marc On Thu, Sep 12, 2024 at 6:03 PM Michael Sokolov wrote: > > If your two indexes load data sequentially and in the