Re: What exactly returns IndexReader.numDeletedDocs()

2022-12-08 Thread Michael Wechner
So IIUC the information re number of deleted documents is only visible temporarily and only when there are many documents, right? Thanks Michael Am 08.12.22 um 14:21 schrieb Uwe Schindler: If this is a reader with only a few documents the likelyness of all deletes being applied while closing

Re: What exactly returns IndexReader.numDeletedDocs()

2022-12-08 Thread Uwe Schindler
If this is a reader with only a few documents the likelyness of all deletes being applied while closing is high. Uwe Am 08.12.2022 um 11:44 schrieb Michael Wechner: My code at the moment is as follows: Directory dir = FSDirectory.open(Paths.get(vectorIndexPath)); IndexReader reader = Direct

Re: What exactly returns IndexReader.numDeletedDocs()

2022-12-08 Thread Michael Wechner
My code at the moment is as follows: Directory dir = FSDirectory.open(Paths.get(vectorIndexPath)); IndexReader reader = DirectoryReader.open(FSDirectory.open(Paths.get(vectorIndexPath))); int numberOfDocsBeforeDeleting = reader.numDocs(); log.info("Number of documents: " + numberOfDocsBeforeDel

Re: What exactly returns IndexReader.numDeletedDocs()

2022-12-08 Thread Uwe Schindler
You have to reopen the index reader to see deletes from the indexwriter. Am 08.12.2022 um 10:32 schrieb Hrvoje Lončar: Did you call this method before or after commit method? My wild guess would be that you can count deleted documents inside transaction only. On Thu, Dec 8, 2022 at 12:10 AM Mic

Re: What exactly returns IndexReader.numDeletedDocs()

2022-12-08 Thread András Péteri
IIRC, it's the number of documents marked with a "deleted" bit. They are obliterated during merges as segments written during the merge operation no longer include deleted contents. So eg. if you call forceMerge(1), no previous segment is preserved and the deleted count will drop to 0 as a result.

Re: What exactly returns IndexReader.numDeletedDocs()

2022-12-08 Thread Hrvoje Lončar
Did you call this method before or after commit method? My wild guess would be that you can count deleted documents inside transaction only. On Thu, Dec 8, 2022 at 12:10 AM Michael Wechner wrote: > Hi > > I am using Lucen 9.4.2 vector search and everything seems to work fine, > except that when

What exactly returns IndexReader.numDeletedDocs()

2022-12-07 Thread Michael Wechner
Hi I am using Lucen 9.4.2 vector search and everything seems to work fine, except that when I delete some documents from the index, then the method https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/index/IndexReader.html#numDeletedDocs() always returns 0, whereas I would have expect