[
https://issues.apache.org/jira/browse/LUCENE-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe reopened LUCENE-6829:
--------------------------------
My Jenkins found a nightly seed that causes
{{TestStressIndexing..testStressIndexAndSearching()}} to fail nearly 100% of
the time:
{noformat}
[junit4] Suite: org.apache.lucene.index.TestStressIndexing
[junit4] 1> Thread[Thread-66,5,TGRP-TestStressIndexing]: exc
[junit4] 1> java.lang.NullPointerException
[junit4] 1> at
java.util.ComparableTimSort.countRunAndMakeAscending(ComparableTimSort.java:317)
[junit4] 1> at
java.util.ComparableTimSort.sort(ComparableTimSort.java:198)
[junit4] 1> at java.util.Arrays.sort(Arrays.java:1246)
[junit4] 1> at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:653)
[junit4] 1> at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:73)
[junit4] 1> at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
[junit4] 1> at
org.apache.lucene.index.TestStressIndexing$SearcherThread.doWork(TestStressIndexing.java:106)
[junit4] 1> at
org.apache.lucene.index.TestStressIndexing$TimedThread.run(TestStressIndexing.java:48)
[junit4] 1> Thread[Thread-65,5,TGRP-TestStressIndexing]: exc
[junit4] 1> java.lang.NullPointerException
[junit4] 1> at
java.util.ComparableTimSort.binarySort(ComparableTimSort.java:258)
[junit4] 1> at
java.util.ComparableTimSort.sort(ComparableTimSort.java:203)
[junit4] 1> at java.util.Arrays.sort(Arrays.java:1246)
[junit4] 1> at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:653)
[junit4] 1> at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:73)
[junit4] 1> at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
[junit4] 1> at
org.apache.lucene.index.TestStressIndexing$SearcherThread.doWork(TestStressIndexing.java:106)
[junit4] 1> at
org.apache.lucene.index.TestStressIndexing$TimedThread.run(TestStressIndexing.java:48)
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestStressIndexing
-Dtests.method=testStressIndexAndSearching -Dtests.seed=98FCDFF6002C97E1
-Dtests.nightly=true -Dtests.slow=true -Dtests.locale=tr
-Dtests.timezone=Indian/Comoro -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.12s J3 | TestStressIndexing.testStressIndexAndSearching
<<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at
__randomizedtesting.SeedInfo.seed([98FCDFF6002C97E1:7F95878F4DC6F61D]:0)
[junit4] > at
org.apache.lucene.index.TestStressIndexing.runStressTest(TestStressIndexing.java:155)
[junit4] > at
org.apache.lucene.index.TestStressIndexing.testStressIndexAndSearching(TestStressIndexing.java:172)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene53):
{contents=PostingsFormat(name=Memory doPackFST= false),
id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))},
docValues:{}, sim=ClassicSimilarity, locale=tr, timezone=Indian/Comoro
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation
1.8.0_45 (64-bit)/cpus=16,threads=1,free=573503112,total=1469054976
[junit4] 2> NOTE: All tests run in this JVM: [TestNumericRangeQuery32,
TestMmapDirectory, TestSpans, TestIndexWriterDelete, TestIndexReaderClose,
TestSearchForDuplicates, TestStringHelper, TestPackedInts,
TestControlledRealTimeReopenThread, TestDirectory, TestMinimize,
TestNamedSPILoader, TestLucene50SegmentInfoFormat, TestStressIndexing]
{noformat}
I put a debug println in {{SegmentInfos.FindSegmentsFile.run()}} and found that
{{RAMDirectory.listAll()}} can sometimes produce null entries. [~mikemccand],
looks like your commit to {{RAMDirector.listAll()}} on this issue is the
problem:
{code:java}
@@ -111,10 +111,7 @@
// and do not synchronize or anything stronger. it's great for testing!
// NOTE: fileMap.keySet().toArray(new String[0]) is broken in non Sun JDKs,
// and the code below is resilient to map changes during the array
population.
- Set<String> fileNames = fileMap.keySet();
- List<String> names = new ArrayList<>(fileNames.size());
- for (String name : fileNames) names.add(name);
- return names.toArray(new String[names.size()]);
+ return fileMap.keySet().toArray(new String[fileMap.size()]);
}
{code}
I'm guessing that an inconsistency between {{fileMap.size()}} and
{{fileMap.keySet()}} will cause this issue - if the number of entries goes down
inbetween these calls then trailing entries in the resulting array will be null.
> OfflineSorter should use Directory API
> --------------------------------------
>
> Key: LUCENE-6829
> URL: https://issues.apache.org/jira/browse/LUCENE-6829
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: Trunk
>
> Attachments: LUCENE-6829.patch, LUCENE-6829.patch, LUCENE-6829.patch,
> LUCENE-6829.patch
>
>
> I think this is a blocker for LUCENE-6825, because the block KD-tree makes
> heavy use of OfflineSorter and we don't want to fill up tmp space ...
> This should be a straightforward cutover, but there are some challenges, e.g.
> the test was failing because virus checker blocked deleting of files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
