The oal.index.Term documentation warns about this issue:
* <p>WARNING: the provided BytesRef is not copied, but used directly.
* Therefore the bytes should not be modified after construction, for
* example, you should clone a copy by {@link BytesRef#deepCopyOf}
* rather than pass reused bytes from a TermsEnum.
But it still worries me a bit: if we fell into this trap in our tests,
then users will probably too?
I opened https://issues.apache.org/jira/browse/LUCENE-6321 for discussion.
On Mon, Mar 2, 2015 at 8:57 AM, Adrien Grand <[email protected]> wrote:
> It does not reproduce but I think I understand what is happening: this
> test iterates over a terms enum and searches over every term. But
> terms from the terms enum are reused and this modifies in-place
> queries which are already in the query cache.
>
> On Mon, Mar 2, 2015 at 6:29 AM, Policeman Jenkins Server
> <[email protected]> wrote:
>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11743/
>> Java: 64bit/jdk1.7.0_76 -XX:-UseCompressedOops -XX:+UseSerialGC
>>
>> 1 tests failed.
>> FAILED: org.apache.lucene.search.TestSearcherManager.testSearcherManager
>>
>> Error Message:
>> Captured an uncaught exception in thread: Thread[id=2055, name=Thread-1806,
>> state=RUNNABLE, group=TGRP-TestSearcherManager]
>>
>> Stack Trace:
>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
>> uncaught exception in thread: Thread[id=2055, name=Thread-1806,
>> state=RUNNABLE, group=TGRP-TestSearcherManager]
>> Caused by: java.lang.RuntimeException:
>> java.lang.ArrayIndexOutOfBoundsException: 24
>> at __randomizedtesting.SeedInfo.seed([C34C31F71B1B64C]:0)
>> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:397)
>> Caused by: java.lang.ArrayIndexOutOfBoundsException: 24
>> at
>> org.apache.lucene.util.StringHelper.murmurhash3_x86_32(StringHelper.java:209)
>> at
>> org.apache.lucene.util.StringHelper.murmurhash3_x86_32(StringHelper.java:230)
>> at org.apache.lucene.util.BytesRef.hashCode(BytesRef.java:131)
>> at org.apache.lucene.index.Term.hashCode(Term.java:126)
>> at org.apache.lucene.search.TermQuery.hashCode(TermQuery.java:220)
>> at java.util.HashMap.hash(HashMap.java:362)
>> at java.util.HashMap.removeEntryForKey(HashMap.java:669)
>> at java.util.HashMap.remove(HashMap.java:656)
>> at
>> java.util.LinkedHashMap$LinkedHashIterator.remove(LinkedHashMap.java:387)
>> at
>> org.apache.lucene.search.LRUQueryCache.evictIfNecessary(LRUQueryCache.java:287)
>> at
>> org.apache.lucene.search.LRUQueryCache.putIfAbsent(LRUQueryCache.java:278)
>> at
>> org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:568)
>> at
>> org.apache.lucene.search.ConstantScoreWeight.scorer(ConstantScoreWeight.java:68)
>> at org.apache.lucene.search.Weight.bulkScorer(Weight.java:127)
>> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:691)
>> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408)
>> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617)
>> at
>> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:602)
>> at
>> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:544)
>> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:450)
>> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:500)
>> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runQuery(ThreadedIndexingAndSearchingTestCase.java:670)
>> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.access$100(ThreadedIndexingAndSearchingTestCase.java:58)
>> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:383)
>>
>>
>>
>>
>> Build Log:
>> [...truncated 1038 lines...]
>> [junit4] Suite: org.apache.lucene.search.TestSearcherManager
>> [junit4] 1> Thread-1806: hit exc
>> [junit4] 1> java.lang.ArrayIndexOutOfBoundsException: 24
>> [junit4] 1> at
>> org.apache.lucene.util.StringHelper.murmurhash3_x86_32(StringHelper.java:209)
>> [junit4] 1> at
>> org.apache.lucene.util.StringHelper.murmurhash3_x86_32(StringHelper.java:230)
>> [junit4] 1> at
>> org.apache.lucene.util.BytesRef.hashCode(BytesRef.java:131)
>> [junit4] 1> at
>> org.apache.lucene.index.Term.hashCode(Term.java:126)
>> [junit4] 1> at
>> org.apache.lucene.search.TermQuery.hashCode(TermQuery.java:220)
>> [junit4] 1> at java.util.HashMap.hash(HashMap.java:362)
>> [junit4] 1> at
>> java.util.HashMap.removeEntryForKey(HashMap.java:669)
>> [junit4] 1> at java.util.HashMap.remove(HashMap.java:656)
>> [junit4] 1> at
>> java.util.LinkedHashMap$LinkedHashIterator.remove(LinkedHashMap.java:387)
>> [junit4] 1> at
>> org.apache.lucene.search.LRUQueryCache.evictIfNecessary(LRUQueryCache.java:287)
>> [junit4] 1> at
>> org.apache.lucene.search.LRUQueryCache.putIfAbsent(LRUQueryCache.java:278)
>> [junit4] 1> at
>> org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:568)
>> [junit4] 1> at
>> org.apache.lucene.search.ConstantScoreWeight.scorer(ConstantScoreWeight.java:68)
>> [junit4] 1> at
>> org.apache.lucene.search.Weight.bulkScorer(Weight.java:127)
>> [junit4] 1> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:691)
>> [junit4] 1> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408)
>> [junit4] 1> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617)
>> [junit4] 1> at
>> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:602)
>> [junit4] 1> at
>> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:544)
>> [junit4] 1> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:450)
>> [junit4] 1> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:500)
>> [junit4] 1> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runQuery(ThreadedIndexingAndSearchingTestCase.java:670)
>> [junit4] 1> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.access$100(ThreadedIndexingAndSearchingTestCase.java:58)
>> [junit4] 1> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:383)
>> [junit4] 2> ??? 02, 2015 4:00:20 ?
>> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
>> uncaughtException
>> [junit4] 2> WARNING: Uncaught exception in thread:
>> Thread[Thread-1806,5,TGRP-TestSearcherManager]
>> [junit4] 2> java.lang.RuntimeException:
>> java.lang.ArrayIndexOutOfBoundsException: 24
>> [junit4] 2> at
>> __randomizedtesting.SeedInfo.seed([C34C31F71B1B64C]:0)
>> [junit4] 2> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:397)
>> [junit4] 2> Caused by: java.lang.ArrayIndexOutOfBoundsException: 24
>> [junit4] 2> at
>> org.apache.lucene.util.StringHelper.murmurhash3_x86_32(StringHelper.java:209)
>> [junit4] 2> at
>> org.apache.lucene.util.StringHelper.murmurhash3_x86_32(StringHelper.java:230)
>> [junit4] 2> at
>> org.apache.lucene.util.BytesRef.hashCode(BytesRef.java:131)
>> [junit4] 2> at
>> org.apache.lucene.index.Term.hashCode(Term.java:126)
>> [junit4] 2> at
>> org.apache.lucene.search.TermQuery.hashCode(TermQuery.java:220)
>> [junit4] 2> at java.util.HashMap.hash(HashMap.java:362)
>> [junit4] 2> at
>> java.util.HashMap.removeEntryForKey(HashMap.java:669)
>> [junit4] 2> at java.util.HashMap.remove(HashMap.java:656)
>> [junit4] 2> at
>> java.util.LinkedHashMap$LinkedHashIterator.remove(LinkedHashMap.java:387)
>> [junit4] 2> at
>> org.apache.lucene.search.LRUQueryCache.evictIfNecessary(LRUQueryCache.java:287)
>> [junit4] 2> at
>> org.apache.lucene.search.LRUQueryCache.putIfAbsent(LRUQueryCache.java:278)
>> [junit4] 2> at
>> org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:568)
>> [junit4] 2> at
>> org.apache.lucene.search.ConstantScoreWeight.scorer(ConstantScoreWeight.java:68)
>> [junit4] 2> at
>> org.apache.lucene.search.Weight.bulkScorer(Weight.java:127)
>> [junit4] 2> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:691)
>> [junit4] 2> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408)
>> [junit4] 2> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617)
>> [junit4] 2> at
>> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:602)
>> [junit4] 2> at
>> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:544)
>> [junit4] 2> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:450)
>> [junit4] 2> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:500)
>> [junit4] 2> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runQuery(ThreadedIndexingAndSearchingTestCase.java:670)
>> [junit4] 2> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.access$100(ThreadedIndexingAndSearchingTestCase.java:58)
>> [junit4] 2> at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:383)
>> [junit4] 2>
>> [junit4] 2> NOTE: reproduce with: ant test
>> -Dtestcase=TestSearcherManager -Dtests.method=testSearcherManager
>> -Dtests.seed=C34C31F71B1B64C -Dtests.multiplier=3 -Dtests.slow=true
>> -Dtests.locale=ar_OM -Dtests.timezone=Pacific/Ponape -Dtests.asserts=true
>> -Dtests.file.encoding=ISO-8859-1
>> [junit4] ERROR 3.04s J1 | TestSearcherManager.testSearcherManager <<<
>> [junit4] > Throwable #1: java.lang.AssertionError
>> [junit4] > at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:551)
>> [junit4] > at
>> org.apache.lucene.search.TestSearcherManager.testSearcherManager(TestSearcherManager.java:60)
>> [junit4] > at java.lang.Thread.run(Thread.java:745)Throwable
>> #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
>> uncaught exception in thread: Thread[id=2055, name=Thread-1806,
>> state=RUNNABLE, group=TGRP-TestSearcherManager]
>> [junit4] > Caused by: java.lang.RuntimeException:
>> java.lang.ArrayIndexOutOfBoundsException: 24
>> [junit4] > at
>> __randomizedtesting.SeedInfo.seed([C34C31F71B1B64C]:0)
>> [junit4] > at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:397)
>> [junit4] > Caused by: java.lang.ArrayIndexOutOfBoundsException: 24
>> [junit4] > at
>> org.apache.lucene.util.StringHelper.murmurhash3_x86_32(StringHelper.java:209)
>> [junit4] > at
>> org.apache.lucene.util.StringHelper.murmurhash3_x86_32(StringHelper.java:230)
>> [junit4] > at
>> org.apache.lucene.util.BytesRef.hashCode(BytesRef.java:131)
>> [junit4] > at
>> org.apache.lucene.index.Term.hashCode(Term.java:126)
>> [junit4] > at
>> org.apache.lucene.search.TermQuery.hashCode(TermQuery.java:220)
>> [junit4] > at java.util.HashMap.hash(HashMap.java:362)
>> [junit4] > at
>> java.util.HashMap.removeEntryForKey(HashMap.java:669)
>> [junit4] > at java.util.HashMap.remove(HashMap.java:656)
>> [junit4] > at
>> java.util.LinkedHashMap$LinkedHashIterator.remove(LinkedHashMap.java:387)
>> [junit4] > at
>> org.apache.lucene.search.LRUQueryCache.evictIfNecessary(LRUQueryCache.java:287)
>> [junit4] > at
>> org.apache.lucene.search.LRUQueryCache.putIfAbsent(LRUQueryCache.java:278)
>> [junit4] > at
>> org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:568)
>> [junit4] > at
>> org.apache.lucene.search.ConstantScoreWeight.scorer(ConstantScoreWeight.java:68)
>> [junit4] > at
>> org.apache.lucene.search.Weight.bulkScorer(Weight.java:127)
>> [junit4] > at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:691)
>> [junit4] > at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:408)
>> [junit4] > at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:617)
>> [junit4] > at
>> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:602)
>> [junit4] > at
>> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:544)
>> [junit4] > at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:450)
>> [junit4] > at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:500)
>> [junit4] > at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runQuery(ThreadedIndexingAndSearchingTestCase.java:670)
>> [junit4] > at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.access$100(ThreadedIndexingAndSearchingTestCase.java:58)
>> [junit4] > at
>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:383)
>> [junit4] 2> NOTE: leaving temporary files on disk at:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.search.TestSearcherManager
>> C34C31F71B1B64C-001
>> [junit4] 2> NOTE: test params are: codec=Lucene50,
>> sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {extra10=DFR
>> I(F)BZ(0.3), extra30=DFR I(ne)L3(800.0), extra22=IB SPL-DZ(0.3), extra20=DFR
>> GB1, extra39=IB LL-L1, extra21=IB SPL-L3(800.0), extra5=DFR I(n)Z(0.3),
>> extra11=DFR I(F)B3(800.0), extra9=DFR I(F)Z(0.3), extra16=DFR I(ne)L2,
>> extra2=DFR I(n)B3(800.0), extra14=DFR GB3(800.0), extra26=DFR I(ne)BZ(0.3),
>> extra34=DFR I(n)L1, extra31=DFR I(ne)1, extra36=DFR I(F)BZ(0.3), extra32=IB
>> LL-LZ(0.3), extra12=DFR I(ne)B1, extra15=IB SPL-D1, extra33=DFR
>> I(F)L3(800.0), extra28=DFR G3(800.0), extra17=DFR I(n)B3(800.0), extra23=DFR
>> I(F)2, extra37=DFR I(F)B3(800.0), extra25=DFR I(n)3(800.0), extra13=IB
>> LL-L1, extra7=LM Jelinek-Mercer(0.100000), extra19=LM
>> Jelinek-Mercer(0.700000), extra18=IB SPL-D3(800.0), body=DFR GB2,
>> extra6=DefaultSimilarity, extra35=DFR I(F)L2, extra8=DFR I(ne)L1, extra27=IB
>> SPL-L2, titleTokenized=IB LL-LZ(0.3), extra24=DFR I(F)3(800.0), extra1=DFR
>> I(ne)L2, extra0=IB SPL-D1}, locale=ar_OM, timezone=Pacific/Ponape
>> [junit4] 2> NOTE: Linux 3.13.0-46-generic amd64/Oracle Corporation
>> 1.7.0_76 (64-bit)/cpus=8,threads=1,free=168409416,total=310468608
>> [junit4] 2> NOTE: All tests run in this JVM: [TestIndexSearcher,
>> TestPrefixInBooleanQuery, TestLucene50FieldInfoFormat, TestTopDocsMerge,
>> TestSpans, TestBagOfPostings, TestConsistentFieldNumbers, Test2BPositions,
>> TestPhraseQuery, TestNRTReaderCleanup, TestRegexpRandom, TestTerm,
>> TestSimpleExplanationsOfNonMatches, TestArrayUtil, TestMultiDocValues,
>> TestBagOfPositions, TestSpanExplanations, TestLevenshteinAutomata,
>> TestFieldReuse, TestGraphTokenizers,
>> TestLucene50StoredFieldsFormatHighCompression, TestFuzzyQuery,
>> TestNumericRangeQuery64, TestSortedNumericSortField, TestStressIndexing2,
>> TestFieldCacheRewriteMethod, TestMockFilesystems,
>> TestSpanMultiTermQueryWrapper, TestLucene50TermVectorsFormat,
>> TestGroupFiltering, TestNewestSegment, TestIndexInput, Nested, Nested,
>> TestSpanSearchEquivalence, FuzzyTermOnShortTermsTest,
>> TestIndexWriterWithThreads, TestPayloadsOnVectors, TestDocIdSet,
>> TestTimSorter, TestSimilarityBase, TestIsCurrent, TestTopFieldCollector,
>> TestConstantScoreQuery, TestEliasFanoDocIdSet, TestRollingBuffer,
>> TestMixedCodecs, TestSpansAdvanced, TestLucene50StoredFieldsFormat,
>> TestMultiTermQueryRewrites, TestIOUtils, TestDirectPacked,
>> TestEarlyTermination, TestPerFieldDocValuesFormat, TestBlockPostingsFormat,
>> TestCrash, TestByteArrayDataInput, TestByteBlockPool, TestMinimize,
>> TestNumericTokenStream, TestDeletionPolicy, TestCodecHoldsOpenFiles,
>> InBeforeClass, InAfterClass, InTestMethod, NonStringProperties,
>> TestAssertingNormsFormat, TestDocValuesRewriteMethod, TestAtomicUpdate,
>> TestNIOFSDirectory, TestNamedSPILoader, TestQueryWrapperFilter,
>> MultiCollectorTest, TestUniqueTermCount, TestSimpleExplanations,
>> TestOrdinalMap, TestTermRangeQuery, TestMultiMMap, TestNoMergePolicy,
>> TestVirtualMethod, TestBlockPostingsFormat3, TestCustomSearcherSort,
>> TestDocValuesTermsQuery, TestRegexpQuery, TestSubScorerFreqs,
>> TestSearchAfter, TestIndexWriterCommit, TestAssertingTermVectorsFormat,
>> TestMockDirectoryWrapper, TestSegmentTermDocs, TestForUtil, TestFlex,
>> TestIntBlockPool, TestDemoParallelLeafReader, TestTermVectors,
>> TestSearchWithThreads, TestIndexWriterThreadsToSegments,
>> TestTermRangeFilter, TestBinaryDocument, TestFieldMaskingSpanQuery,
>> TestDisjunctionMaxQuery, TestNorms, TestTrackingDirectoryWrapper,
>> TestMultiTermConstantScore, TestPerSegmentDeletes, TestMockAnalyzer,
>> Nested1, TestLucene50CompoundFormat, TestIndexWriter, TestIndexWriterReader,
>> TestIndexWriterMerging, TestFSTs, TestBytesStore, TestNumericRangeQuery32,
>> TestLookaheadTokenFilter, TestAddIndexes, TestShardSearching,
>> TestSearcherManager]
>> [junit4] Completed on J1 in 3.10s, 9 tests, 1 error <<< FAILURES!
>>
>> [...truncated 830 lines...]
>> BUILD FAILED
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The
>> following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The
>> following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following
>> error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The
>> following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build.xml:49: The
>> following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1370:
>> The following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:976:
>> There were test failures: 421 suites, 3373 tests, 1 error, 66 ignored (56
>> assumptions)
>>
>> Total time: 35 minutes 56 seconds
>> Build step 'Invoke Ant' marked build as failure
>> [description-setter] Description set: Java: 64bit/jdk1.7.0_76
>> -XX:-UseCompressedOops -XX:+UseSerialGC
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>
>
>
> --
> Adrien
--
Adrien
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
