Re: [I] `FSTCompiler.Builder` should have an option to stream the FST bytes directly to Directory [lucene]

2023-10-05 Thread via GitHub
dungba88 commented on issue #12543: URL: https://github.com/apache/lucene/issues/12543#issuecomment-1749982744 One of the thing I think is missing is that those byte manipulation methods should not be called after calling `#finish()`, but currently there is no such enforcement. -- This

Re: [PR] Allow implementers of AbstractKnnVectorQuery to access final topK results [lucene]

2023-10-05 Thread via GitHub
benwtrent merged PR #12590: URL: https://github.com/apache/lucene/pull/12590 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Allow implementers of AbstractKnnVectorQuery to access final topK results? [lucene]

2023-10-05 Thread via GitHub
benwtrent closed issue #12575: Allow implementers of AbstractKnnVectorQuery to access final topK results? URL: https://github.com/apache/lucene/issues/12575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Make IndexWriter#flushNextBuffer also apply deletes if necessary [lucene]

2023-10-05 Thread via GitHub
s1monw commented on PR #12595: URL: https://github.com/apache/lucene/pull/12595#issuecomment-1749460562 see https://github.com/apache/lucene/issues/12572#issuecomment-1749458532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Make IndexWriter#flushNextBuffer also apply deletes if necessary [lucene]

2023-10-05 Thread via GitHub
s1monw closed pull request #12595: Make IndexWriter#flushNextBuffer also apply deletes if necessary URL: https://github.com/apache/lucene/pull/12595 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Make IndexWriter#flushNextBuffer flush deletes too? [lucene]

2023-10-05 Thread via GitHub
s1monw commented on issue #12572: URL: https://github.com/apache/lucene/issues/12572#issuecomment-1749458532 After digging into this and opening a PR for it I think this is unnecessary. I tried to beef up tests for this and this caused me to refresh my knowledge how stuff works down in the

Re: [I] Make IndexWriter#flushNextBuffer flush deletes too? [lucene]

2023-10-05 Thread via GitHub
s1monw closed issue #12572: Make IndexWriter#flushNextBuffer flush deletes too? URL: https://github.com/apache/lucene/issues/12572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Allow implementers of AbstractKnnVectorQuery to access final topK results [lucene]

2023-10-05 Thread via GitHub
kaivalnp commented on PR #12590: URL: https://github.com/apache/lucene/pull/12590#issuecomment-1749418274 Thanks for all the help! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Allow implementers of AbstractKnnVectorQuery to access final topK results [lucene]

2023-10-05 Thread via GitHub
benwtrent commented on PR #12590: URL: https://github.com/apache/lucene/pull/12590#issuecomment-1749324868 @kaivalnp && @mikemccand I can merge and backport to 9x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Should we explore DiskANN for aKNN vector search? [lucene]

2023-10-05 Thread via GitHub
benwtrent commented on issue #12615: URL: https://github.com/apache/lucene/issues/12615#issuecomment-1749322588 > QDrant has a filter solution however the methodology described in their blog is opaque. QDrant's HNSW filter solution is the exact same as Lucene's. You can look at the

Re: [PR] Allow implementers of AbstractKnnVectorQuery to access final topK results [lucene]

2023-10-05 Thread via GitHub
kaivalnp commented on PR #12590: URL: https://github.com/apache/lucene/pull/12590#issuecomment-1749204748 Hi @benwtrent @mikemccand can someone help merge this in / let me know if there's anything pending? -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Should we explore DiskANN for aKNN vector search? [lucene]

2023-10-05 Thread via GitHub
robertvanwinkle1138 commented on issue #12615: URL: https://github.com/apache/lucene/issues/12615#issuecomment-1749187258 The SPANN paper does not address efficient filtered queries. Lucene's HNSW calculates the similarity score for every record, regardless of the record matching the

Re: [PR] Ability to compute vector similarity scores with DoubleValuesSource [lucene]

2023-10-05 Thread via GitHub
stefanvodita commented on code in PR #12548: URL: https://github.com/apache/lucene/pull/12548#discussion_r1347266594 ## lucene/core/src/java/org/apache/lucene/search/FloatVectorSimilarityValuesSource.java: ## @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Compute better windows in MaxScoreBulkScorer. [lucene]

2023-10-05 Thread via GitHub
jpountz merged PR #12593: URL: https://github.com/apache/lucene/pull/12593 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Use a MergeSorter taking advantage of extra storage for StableMSBRadixSorter [lucene]

2023-10-05 Thread via GitHub
gf2121 merged PR #12623: URL: https://github.com/apache/lucene/pull/12623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Use a MergeSorter taking advantage of extra storage for StableMSBRadixSorter [lucene]

2023-10-05 Thread via GitHub
gf2121 commented on code in PR #12623: URL: https://github.com/apache/lucene/pull/12623#discussion_r1347069694 ## lucene/core/src/java/org/apache/lucene/util/StableMSBRadixSorter.java: ## @@ -78,4 +78,60 @@ protected void reorder(int from, int to, int[] startOffsets, int[]

Re: [PR] TaskExecutor waits for all tasks to complete before returning [lucene]

2023-10-05 Thread via GitHub
javanna commented on PR #12523: URL: https://github.com/apache/lucene/pull/12523#issuecomment-1748362692 Thanks @quux00 ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] TaskExecutor waits for all tasks to complete before returning [lucene]

2023-10-05 Thread via GitHub
javanna merged PR #12523: URL: https://github.com/apache/lucene/pull/12523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] SOLR-16843: Replace timeNs by epochTimeNs in most of autoscaling [lucene-solr]

2023-10-05 Thread via GitHub
psalagnac opened a new pull request, #2679: URL: https://github.com/apache/lucene-solr/pull/2679 [SOLR-16843](https://issues.apache.org/jira/browse/SOLR-16843) # Description Autoscaling framework use timestamps returned by the JVM call System.nanoTime(), but

Re: [PR] Use a MergeSorter taking advantage of extra storage for StableMSBRadixSorter [lucene]

2023-10-05 Thread via GitHub
jpountz commented on code in PR #12623: URL: https://github.com/apache/lucene/pull/12623#discussion_r1346923517 ## lucene/core/src/java/org/apache/lucene/util/StableMSBRadixSorter.java: ## @@ -78,4 +78,60 @@ protected void reorder(int from, int to, int[] startOffsets, int[]