Phew, tricky! The patch looks good Steve. Thanks for digging and beasting and finding a separate crab!
Mike McCandless http://blog.mikemccandless.com On Tue, Dec 27, 2016 at 10:00 PM, Steve Rowe <sar...@gmail.com> wrote: > Yes, I’ve posted a couple of them on the issue, but they don’t reproduce. > > My beasting so far (couple thousand iterations) hasn’t yet reproduced. > Beasting did find <https://issues.apache.org/jira/browse/LUCENE-7605> though. > > I have an idea about what might cause the problem: under multi-threaded use, > in lookup(), the suggester’s SearcherManager could be re-assigned in the > middle of an acquire/release cycle as a result of > <https://issues.apache.org/jira/browse/LUCENE-7564>. I’m also beasting a > change to take a local reference and use it for the whole cycle.[1] > > -- > Steve > www.lucidworks.com > > [1] Patch to hold SearcherManager constant in > AnalyzingInfixSuggester.lookup(): > > —— > diff --git > a/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java > > b/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java > index 1e98361..c63726f 100644 > --- > a/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java > +++ > b/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java > @@ -642,7 +642,8 @@ public class AnalyzingInfixSuggester extends Lookup > implements Closeable { > // only retrieve the first num hits now: > Collector c2 = new EarlyTerminatingSortingCollector(c, SORT, num); > List<LookupResult> results = null; > - IndexSearcher searcher = searcherMgr.acquire(); > + SearcherManager mgr = searcherMgr; // use local reference to perform > acquire & release on same SearcherManager > + IndexSearcher searcher = mgr.acquire(); > try { > //System.out.println("got searcher=" + searcher); > searcher.search(finalQuery, c2); > @@ -653,7 +654,7 @@ public class AnalyzingInfixSuggester extends Lookup > implements Closeable { > // hits = searcher.search(query, null, num, SORT); > results = createResults(searcher, hits, num, key, doHighlight, > matchedTokens, prefixToken); > } finally { > - searcherMgr.release(searcher); > + mgr.release(searcher); > } > > //System.out.println((System.currentTimeMillis() - t0) + " msec for > infix suggest"); > —— > > > >> On Dec 26, 2016, at 5:56 PM, Michael McCandless <luc...@mikemccandless.com> >> wrote: >> >> I think (not sure) these failures starting happening after >> https://issues.apache.org/jira/browse/LUCENE-7564 ? >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Mon, Dec 26, 2016 at 4:29 PM, Apache Jenkins Server >> <jenk...@builds.apache.org> wrote: >>> Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/628/ >>> >>> 1 tests failed. >>> FAILED: >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest.testRandomNRT >>> >>> Error Message: >>> Captured an uncaught exception in thread: Thread[id=29, name=Thread-11, >>> state=RUNNABLE, group=TGRP-AnalyzingInfixSuggesterTest] >>> >>> Stack Trace: >>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >>> uncaught exception in thread: Thread[id=29, name=Thread-11, state=RUNNABLE, >>> group=TGRP-AnalyzingInfixSuggesterTest] >>> at >>> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5:C4A3CFCB61E1C079]:0) >>> Caused by: org.apache.lucene.store.AlreadyClosedException: this >>> ReferenceManager is closed >>> at __randomizedtesting.SeedInfo.seed([608DC176393E1CC5]:0) >>> at >>> org.apache.lucene.search.ReferenceManager.acquire(ReferenceManager.java:98) >>> at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:645) >>> at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:457) >>> at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest$LookupThread.run(AnalyzingInfixSuggesterTest.java:533) >>> >>> >>> >>> >>> Build Log: >>> [...truncated 9362 lines...] >>> [junit4] Suite: >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest >>> [junit4] 2> ??? 26, 2016 1:30:30 ?? >>> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler >>> uncaughtException >>> [junit4] 2> WARNING: Uncaught exception in thread: >>> Thread[Thread-11,5,TGRP-AnalyzingInfixSuggesterTest] >>> [junit4] 2> org.apache.lucene.store.AlreadyClosedException: this >>> ReferenceManager is closed >>> [junit4] 2> at >>> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5]:0) >>> [junit4] 2> at >>> org.apache.lucene.search.ReferenceManager.acquire(ReferenceManager.java:98) >>> [junit4] 2> at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:645) >>> [junit4] 2> at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:457) >>> [junit4] 2> at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest$LookupThread.run(AnalyzingInfixSuggesterTest.java:533) >>> [junit4] 2> >>> [junit4] 2> NOTE: reproduce with: ant test >>> -Dtestcase=AnalyzingInfixSuggesterTest -Dtests.method=testRandomNRT >>> -Dtests.seed=608DC176393E1CC5 -Dtests.multiplier=2 -Dtests.slow=true >>> -Dtests.locale=el -Dtests.timezone=SystemV/MST7MDT -Dtests.asserts=true >>> -Dtests.file.encoding=US-ASCII >>> [junit4] ERROR 32.8s J1 | AnalyzingInfixSuggesterTest.testRandomNRT <<< >>> [junit4] > Throwable #1: >>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >>> uncaught exception in thread: Thread[id=29, name=Thread-11, state=RUNNABLE, >>> group=TGRP-AnalyzingInfixSuggesterTest] >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5:C4A3CFCB61E1C079]:0) >>> [junit4] > Caused by: org.apache.lucene.store.AlreadyClosedException: >>> this ReferenceManager is closed >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5]:0) >>> [junit4] > at >>> org.apache.lucene.search.ReferenceManager.acquire(ReferenceManager.java:98) >>> [junit4] > at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:645) >>> [junit4] > at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:457) >>> [junit4] > at >>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest$LookupThread.run(AnalyzingInfixSuggesterTest.java:533) >>> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): >>> {textgrams=PostingsFormat(name=Memory doPackFST= false), >>> text=PostingsFormat(name=Memory doPackFST= false), >>> contexts=PostingsFormat(name=LuceneVarGapFixedInterval), >>> exacttext=PostingsFormat(name=LuceneVarGapFixedInterval)}, >>> docValues:{payloads=DocValuesFormat(name=Memory), >>> textgrams=DocValuesFormat(name=Memory), >>> weight=DocValuesFormat(name=Lucene54), text=DocValuesFormat(name=Memory), >>> contexts=DocValuesFormat(name=Lucene54), >>> exacttext=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=846, >>> maxMBSortInHeap=6.751687589960355, >>> sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=el, >>> timezone=SystemV/MST7MDT >>> [junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation >>> 1.8.0_102 (64-bit)/cpus=4,threads=1,free=158780648,total=343932928 >>> [junit4] 2> NOTE: All tests run in this JVM: [TestContextQuery, >>> TestSuggestStopFilter, TestPrefixCompletionQuery, >>> AnalyzingInfixSuggesterTest] >>> [junit4] Completed [28/32 (1!)] on J1 in 37.17s, 22 tests, 1 error <<< >>> FAILURES! >>> >>> [...truncated 65892 lines...] >>> >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
