shbhar opened a new issue, #16950: URL: https://github.com/apache/lucene/issues/16950
### Description All else being equal, a stopword should be invisible and not change results. But with the change in https://github.com/apache/lucene/pull/16446 in the following case, with and without stopword (ex: "the") before AND operator the parsed queries and therefore the returned documents become different which I believe is not expected (or at least is a change in behavior from 10.5.2) ``` public class TestStopwordBeforeAndHits extends LuceneTestCase { public void testStopwordBeforeAndDoesNotChangeHits() throws Exception { MockAnalyzer analyzer = new MockAnalyzer(random(), MockTokenizer.WHITESPACE, true, MockTokenFilter.ENGLISH_STOPSET); try (ByteBuffersDirectory dir = new ByteBuffersDirectory(); IndexWriter w = new IndexWriter(dir, new IndexWriterConfig(analyzer))) { Document doc = new Document(); doc.add(new TextField("body", "apple cherry", Field.Store.NO)); w.addDocument(doc); try (DirectoryReader r = DirectoryReader.open(w)) { IndexSearcher s = new IndexSearcher(r); QueryParser qp = new QueryParser("body", analyzer); // passes as expected (banana and cherry are both required but doc has no banana) assertEquals(0, s.count(qp.parse("apple banana AND cherry"))); // fails in 10.6, passes in 10.5.2 assertEquals(0, s.count(qp.parse("apple banana the AND cherry"))); } } } } ``` This happens because the parsed queries become actually different due to the stopword. Without stopword it is ```body:apple +body:banana +body:cherry``` vs with stopword is ```+(body:apple body:banana) +body:cherry``` A diagram, more verbose AI explanation & suggested fix is here: https://shbhar.github.io/lucene-ai-bug-hunt/bugs/LFV-284.html Note: the bug was found by AI using Z3 based formal verification (https://github.com/apache/lucene/issues/16937) ### Version and environment details branch: branch_10_6 Java: JDK25 OS: AL2 Linux aarch64 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
