[PR] Add public getter for SynonymQuery#field [lucene]

2024-02-27 Thread via GitHub
mkhludnev opened a new pull request, #13137: URL: https://github.com/apache/lucene/pull/13137 backport (#13077) #13136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[PR] Update CHANGES.txt SynonymQuery.getField() (#13077) [lucene]

2024-02-27 Thread via GitHub
mkhludnev opened a new pull request, #13136: URL: https://github.com/apache/lucene/pull/13136 move #13077 under 9.11.0 ### Description -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Update CHANGES.txt SynonymQuery.getField() (#13077) [lucene]

2024-02-27 Thread via GitHub
mkhludnev merged PR #13136: URL: https://github.com/apache/lucene/pull/13136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Log MockRandomMergePolicy reverse in verbose. (#13117) [lucene]

2024-02-27 Thread via GitHub
mkhludnev opened a new pull request, #13138: URL: https://github.com/apache/lucene/pull/13138 ### Description -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Log MockRandomMergePolicy reverse in verbose. [lucene]

2024-02-27 Thread via GitHub
mkhludnev merged PR #13117: URL: https://github.com/apache/lucene/pull/13117 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Log MockRandomMergePolicy reverse in verbose. [lucene]

2024-02-27 Thread via GitHub
mkhludnev commented on PR #13117: URL: https://github.com/apache/lucene/pull/13117#issuecomment-1966091179 I think it doesn't need to be mentioned in CHANGES -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Add public getter for SynonymQuery#field [lucene]

2024-02-27 Thread via GitHub
mkhludnev merged PR #13137: URL: https://github.com/apache/lucene/pull/13137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add getter for SynonymQuery#field [lucene]

2024-02-27 Thread via GitHub
mkhludnev merged PR #13077: URL: https://github.com/apache/lucene/pull/13077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add getter for SynonymQuery#field [lucene]

2024-02-27 Thread via GitHub
mkhludnev commented on PR #13077: URL: https://github.com/apache/lucene/pull/13077#issuecomment-1966073356 aaahgg... wrong CHANGES.. going to move in unde 9.11 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Add new parallel merge task executor for parallel actions within a single merge action [lucene]

2024-02-27 Thread via GitHub
benwtrent commented on code in PR #13124: URL: https://github.com/apache/lucene/pull/13124#discussion_r1503892704 ## lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java: ## @@ -910,4 +936,58 @@ public void setSuppressExceptions(ConcurrentMergeScheduler

Re: [PR] Allow multiple JDKs in smoke test [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #13108: URL: https://github.com/apache/lucene/pull/13108#issuecomment-1966098725 > I don't think it's going to be so easy - some things just didn't work for me with local artifacts. I'll follow up, perhaps tomorrow. That was always a problem, I know.

Re: [PR] Allow multiple JDKs in smoke test [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #13108: URL: https://github.com/apache/lucene/pull/13108#issuecomment-1966353181 Will you forward port the patch to main branch? I can help with that, have some freetime. This will allow us to make everything Java 21 ready tomorrow, so we have one file

Re: [PR] Log MockRandomMergePolicy reverse in verbose. (#13117) [lucene]

2024-02-27 Thread via GitHub
mkhludnev merged PR #13138: URL: https://github.com/apache/lucene/pull/13138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Allow multiple JDKs in smoke test [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #13108: URL: https://github.com/apache/lucene/pull/13108#issuecomment-1966395258 The other PR looks straight-forward to me. Have you compared the two files directly after merging? They should be mostly identical and only the BASE_VERSION line should be different.

Re: [PR] Allow multiple JDKs in smoke test [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #13108: URL: https://github.com/apache/lucene/pull/13108#issuecomment-1966393719 > Here it is: #13139. I wasn't sure about the difficulties @dweiss was encountering - I expect this to just work. I think Dawid's problems were preexisting, not related to this

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
rmuir commented on code in PR #13139: URL: https://github.com/apache/lucene/pull/13139#discussion_r1504126264 ## dev-tools/scripts/smokeTestRelease.py: ## @@ -39,6 +39,8 @@ from collections import namedtuple import scriptutil +BASE_JAVA_VERSION = "17" Review Comment:

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #13139: URL: https://github.com/apache/lucene/pull/13139#issuecomment-1966420051 Thanks, I compared the 9.x and main branch smoke tester. The files are identical with some minor differences: - The 9.x checks for `CONTRIBUTING.md` to be available in root folder,

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #13139: URL: https://github.com/apache/lucene/pull/13139#issuecomment-1966421869 I will just push the change to main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #13139: URL: https://github.com/apache/lucene/pull/13139#issuecomment-1966427942 Sorry for last 2 commits. In 9.x the CONTRIBUTING.md was missing. All fine. Sorry for this. I compared both files, all is fine and minor changes are expected. -- This is an

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #13139: URL: https://github.com/apache/lucene/pull/13139#issuecomment-1966429392 Thanks for merging this :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Allow multiple JDKs in smoke test [lucene]

2024-02-27 Thread via GitHub
stefanvodita commented on PR #13108: URL: https://github.com/apache/lucene/pull/13108#issuecomment-1966359649 Here it is: #13139. I wasn't sure about the difficulties @dweiss was encountering - I expect this to just work. -- This is an automated message from the Apache Git Service. To

[PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
stefanvodita opened a new pull request, #13139: URL: https://github.com/apache/lucene/pull/13139 Like #13108, but targeting the `main` branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
uschindler commented on code in PR #13139: URL: https://github.com/apache/lucene/pull/13139#discussion_r1504141136 ## dev-tools/scripts/smokeTestRelease.py: ## @@ -39,6 +39,8 @@ from collections import namedtuple import scriptutil +BASE_JAVA_VERSION = "17" Review Comment:

Re: [PR] Allow multiple JDKs in smoke test [lucene]

2024-02-27 Thread via GitHub
stefanvodita merged PR #13108: URL: https://github.com/apache/lucene/pull/13108 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
dweiss commented on PR #13139: URL: https://github.com/apache/lucene/pull/13139#issuecomment-1966405336 Please merge this, I'll just follow-up on top of that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Allow multiple JDKs in smoke test [lucene]

2024-02-27 Thread via GitHub
stefanvodita commented on PR #13108: URL: https://github.com/apache/lucene/pull/13108#issuecomment-1966411059 I have. The diff looks correct. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
stefanvodita merged PR #13139: URL: https://github.com/apache/lucene/pull/13139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Allow multiple JDKs in smoke test on main [lucene]

2024-02-27 Thread via GitHub
stefanvodita commented on PR #13139: URL: https://github.com/apache/lucene/pull/13139#issuecomment-1966411565 Thanks all - merging... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Bump minimum required Java version to 21 [lucene]

2024-02-27 Thread via GitHub
uschindler commented on PR #12753: URL: https://github.com/apache/lucene/pull/12753#issuecomment-1966464023 Hi @rmuir @stefanvodita: I merged main branch into this and resolved the conflict -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Reproduce multi-word synonym regression (SOLR-16652) [lucene]

2024-02-27 Thread via GitHub
mkhludnev closed pull request #12157: Reproduce multi-word synonym regression (SOLR-16652) URL: https://github.com/apache/lucene/pull/12157 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Make Lucene90 postings format to write FST off heap [lucene]

2024-02-27 Thread via GitHub
dungba88 commented on PR #12985: URL: https://github.com/apache/lucene/pull/12985#issuecomment-1966513989 @mikemccand I tried to use `createTempOutput` when the size is beyond some threshold. The logic works, but I got this exception ``` java.lang.IllegalArgumentException: invalid

Re: [PR] Bump minimum required Java version to 21 [lucene]

2024-02-27 Thread via GitHub
rmuir commented on PR #12753: URL: https://github.com/apache/lucene/pull/12753#issuecomment-1966604919 Thank you for merging! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Move synonym map off-heap for SynonymGraphFilter [lucene]

2024-02-27 Thread via GitHub
dungba88 commented on PR #13054: URL: https://github.com/apache/lucene/pull/13054#issuecomment-1966724748 I realized I also need the `saveMetadata` change for https://github.com/apache/lucene/pull/12985. Do you think we should make it a standalone PR and merge first? Otherwise I've

Re: [PR] Make Lucene90 postings format to write FST off heap [lucene]

2024-02-27 Thread via GitHub
mikemccand commented on PR #12985: URL: https://github.com/apache/lucene/pull/12985#issuecomment-1966868122 Hmm, the `.tmp` file should never be part of the file set for the actual segment? It should be transient, and then re-copied into the final result (the actual block tree file that

Re: [PR] Terminate automaton after matched the whole prefix for PrefixQuery. [lucene]

2024-02-27 Thread via GitHub
mikemccand commented on PR #13072: URL: https://github.com/apache/lucene/pull/13072#issuecomment-1966886223 I will have a look -- thanks for the ping @jpountz. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Make Lucene90 postings format to write FST off heap [lucene]

2024-02-27 Thread via GitHub
dungba88 commented on PR #12985: URL: https://github.com/apache/lucene/pull/12985#issuecomment-1966740284 If there is a way to create temp file outside of the search index, then it would work too, but I can't find it as all I/O are accessible from `SegmentWriteState.directory`. -- This

Re: [PR] Make Lucene90 postings format to write FST off heap [lucene]

2024-02-27 Thread via GitHub
mikemccand commented on PR #12985: URL: https://github.com/apache/lucene/pull/12985#issuecomment-1966874257 And you can use the `IndexOutput.getName()` to get the `String` name of the temp file, and delete that file (prolly in a `finally` clause in case disaster strikes) using

Re: [PR] Avoid allocating redundant Strings [lucene]

2024-02-27 Thread via GitHub
sabi0 commented on PR #13085: URL: https://github.com/apache/lucene/pull/13085#issuecomment-1967301383 Thank you, @stefanvodita -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Move synonym map off-heap for SynonymGraphFilter [lucene]

2024-02-27 Thread via GitHub
msfroh commented on PR #13054: URL: https://github.com/apache/lucene/pull/13054#issuecomment-1967141003 > I realized I also need the saveMetadata change for https://github.com/apache/lucene/pull/12985. Do you think we should make it a standalone PR and merge first? Otherwise I've

Re: [PR] Add a nightly workflow to run and verify buildAndPushRelease.py and smokeTestRelease.py [lucene]

2024-02-27 Thread via GitHub
dweiss commented on PR #13141: URL: https://github.com/apache/lucene/pull/13141#issuecomment-1967491480 Ok. I think things can be improved but let's not make it too complicated at the beginning. Everything seems to work:

[PR] Replace Collections.synchronizedSet() with ConcurrentHashMap.newKeySet() [lucene]

2024-02-27 Thread via GitHub
sabi0 opened a new pull request, #13142: URL: https://github.com/apache/lucene/pull/13142 `ConcurrentHashMap.newKeySet()` provides better multi-threaded performance thanks to judicious use of synchronization. Also it does not require "external" synchronization for iteration. --

Re: [PR] Add a nightly workflow to run and verify buildAndPushRelease.py and smokeTestRelease.py [lucene]

2024-02-27 Thread via GitHub
dweiss commented on PR #13141: URL: https://github.com/apache/lucene/pull/13141#issuecomment-1967128468 This is already working (https://github.com/dweiss/lucene/actions/runs/8067320693) but I need to polish a few things before it's worth of your time to review. Hopefully later tonight or

Re: [PR] Make Lucene90 postings format to write FST off heap [lucene]

2024-02-27 Thread via GitHub
dungba88 commented on PR #12985: URL: https://github.com/apache/lucene/pull/12985#issuecomment-1967923651 Oh `Directory.deleteFile` is exactly what I needed! It's silly that I missed that. I'll post another revision soon. -- This is an automated message from the Apache Git Service. To

Re: [PR] Support getMaxScore of DisjunctionSumScorer for non top level scoring clause [lucene]

2024-02-27 Thread via GitHub
mrkm4ntr commented on PR #13066: URL: https://github.com/apache/lucene/pull/13066#issuecomment-1968035921 Thanks. The result is here. ``` TaskQPS baseline StdDevQPS my_modified_version StdDevPct diff p-value

Re: [PR] Make Lucene90 postings format to write FST off heap [lucene]

2024-02-27 Thread via GitHub
dungba88 commented on PR #12985: URL: https://github.com/apache/lucene/pull/12985#issuecomment-1968041186 I've updated to use temp IndexOutput and modify the test. It seems to be working now. I'm open for suggestion of the default block heap threshold and how to configure it. -- This is

Re: [PR] Make FSTPostingFormat to build FST off-heap [lucene]

2024-02-27 Thread via GitHub
dungba88 commented on PR #12980: URL: https://github.com/apache/lucene/pull/12980#issuecomment-1968037819 Thanks @mikemccand for the clarification! Do you think we should still make this change? One benefit is that it can be used for reference. Otherwise I'll close this PR -- This

Re: [PR] Terminate automaton after matched the whole prefix for PrefixQuery. [lucene]

2024-02-27 Thread via GitHub
rmuir commented on PR #13072: URL: https://github.com/apache/lucene/pull/13072#issuecomment-1968087576 I think the optimization may be similar to the one done in AutomatonTermsEnum?

Re: [PR] Fix DV update files referenced by merge will be deleted by concurrent flush [lucene]

2024-02-27 Thread via GitHub
guojialiang92 commented on code in PR #13017: URL: https://github.com/apache/lucene/pull/13017#discussion_r1505452744 ## lucene/core/src/java/org/apache/lucene/index/ReadersAndUpdates.java: ## @@ -782,13 +784,17 @@ synchronized MergePolicy.MergeReader

Re: [PR] Fix DV update files referenced by merge will be deleted by concurrent flush [lucene]

2024-02-27 Thread via GitHub
guojialiang92 commented on code in PR #13017: URL: https://github.com/apache/lucene/pull/13017#discussion_r1505447216 ## lucene/core/src/java/org/apache/lucene/index/IndexWriter.java: ## @@ -3935,7 +3935,11 @@ public void setMergeInfo(SegmentCommitInfo info) {

Re: [PR] Replace Collections.synchronizedSet() with ConcurrentHashMap.newKeySet() [lucene]

2024-02-27 Thread via GitHub
dweiss commented on code in PR #13142: URL: https://github.com/apache/lucene/pull/13142#discussion_r1505502212 ## lucene/replicator/src/java/org/apache/lucene/replicator/nrt/PrimaryNode.java: ## @@ -158,10 +158,7 @@ public long getPrimaryGen() { */ public boolean

Re: [PR] Fix DV update files referenced by merge will be deleted by concurrent flush [lucene]

2024-02-27 Thread via GitHub
guojialiang92 commented on PR #13017: URL: https://github.com/apache/lucene/pull/13017#issuecomment-1968390042 > Thanks! Could you also look at the CHANGES entry I had mentioned? We want to record the fix and credit you for the work. Thanks, I have add entry to CHANGES.txt. --

Re: [PR] Terminate automaton after matched the whole prefix for PrefixQuery. [lucene]

2024-02-27 Thread via GitHub
vsop-479 commented on PR #13072: URL: https://github.com/apache/lucene/pull/13072#issuecomment-1968284534 > I think the optimization may be similar to the one done in AutomatonTermsEnum? Thanks for reminding that, I will dig into `AutomatonTermsEnum`'s optimization. -- This is an

Re: [PR] Fix DV update files referenced by merge will be deleted by concurrent flush [lucene]

2024-02-27 Thread via GitHub
guojialiang92 commented on code in PR #13017: URL: https://github.com/apache/lucene/pull/13017#discussion_r1505450205 ## lucene/core/src/test/org/apache/lucene/index/TestIndexWriterMergePolicy.java: ## @@ -950,4 +957,354 @@ public void setMergeInfo(SegmentCommitInfo info) {

Re: [PR] Replace Collections.synchronizedSet() with ConcurrentHashMap.newKeySet() [lucene]

2024-02-27 Thread via GitHub
dweiss commented on code in PR #13142: URL: https://github.com/apache/lucene/pull/13142#discussion_r1505494792 ## lucene/core/src/java/org/apache/lucene/util/VirtualMethod.java: ## @@ -100,7 +98,7 @@ public VirtualMethod(Class baseClass, String method, Class... parameters)

Re: [PR] Terminate automaton after matched the whole prefix for PrefixQuery. [lucene]

2024-02-27 Thread via GitHub
vsop-479 commented on PR #13072: URL: https://github.com/apache/lucene/pull/13072#issuecomment-1968078407 @mikemccand Thanks for you suggestion, I am trying to implement it. > Have you tried to measure any performance change with this? E.g. you could run a luceneutil benchy with just

Re: [PR] Terminate automaton after matched the whole prefix for PrefixQuery. [lucene]

2024-02-27 Thread via GitHub
mikemccand commented on code in PR #13072: URL: https://github.com/apache/lucene/pull/13072#discussion_r1504523798 ## lucene/core/src/java/org/apache/lucene/util/automaton/Automaton.java: ## @@ -92,6 +93,7 @@ public Automaton() { public Automaton(int numStates, int

Re: [PR] Fix DV update files referenced by merge will be deleted by concurrent flush [lucene]

2024-02-27 Thread via GitHub
stefanvodita commented on PR #13017: URL: https://github.com/apache/lucene/pull/13017#issuecomment-1966914429 Thanks! Could you also look at the CHANGES entry I had mentioned? We want to record the fix and credit you for the work. -- This is an automated message from the Apache Git

Re: [I] `TestIndexWriterOnDiskFull.testAddIndexOnDiskFull` reproducible test failure [lucene]

2024-02-27 Thread via GitHub
mikemccand commented on issue #13116: URL: https://github.com/apache/lucene/issues/13116#issuecomment-1966918485 I like that proposed solution @easyice! Exception handling is hard. Likely many Lucene tests are missing/failing to catch `UncheckedIOException` now... -- This is an

[PR] Fix MultiTermQueryDocValuesWrapper not propagating QueryVisitor [lucene]

2024-02-27 Thread via GitHub
sabi0 opened a new pull request, #13140: URL: https://github.com/apache/lucene/pull/13140 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[PR] Add a nightly workflow to run and verify buildAndPushRelease and smokeTestRelease.py [lucene]

2024-02-27 Thread via GitHub
dweiss opened a new pull request, #13141: URL: https://github.com/apache/lucene/pull/13141 This is an initial attempt to run a gh workflow to at least make sure these two scripts work and things are building. For the time being, I've omitted the tests entirely - nightly tests OOM on gh

Re: [PR] Add a nightly workflow to run and verify buildAndPushRelease and smokeTestRelease.py [lucene]

2024-02-27 Thread via GitHub
dweiss commented on code in PR #13141: URL: https://github.com/apache/lucene/pull/13141#discussion_r1504572039 ## dev-tools/scripts/smokeTestRelease.py: ## @@ -158,9 +158,9 @@ def checkJARMetaData(desc, jarFile, gitRevision, version): break else: if