[
https://issues.apache.org/jira/browse/LUCENE-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078627#comment-17078627
]
Simon Willnauer commented on LUCENE-9271:
-----------------------------------------
We ran into a test failure
[here|https://github.com/apache/lucene-solr/pull/1397#issuecomment-610930218]:
{noformat}
org.apache.lucene.index.TestIndexManyDocuments > test suite's output saved to
/home/mike/src/simon/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.index.TestIndexManyDocuments.txt,
copi\
ed below:
1> CheckIndex failed
1> 0.00% total deletions; 10976 documents; 0 deleteions
1> Segments file=segments_1 numSegments=4 version=9.0.0
id=cekay2d5izae12ssuqikoqgoc
1> 1 of 4: name=_d maxDoc=8700
1> version=9.0.0
1> id=cekay2d5izae12ssuqikoqgob
1> codec=Asserting(Lucene84)
1> compound=false
1> numFiles=11
1> size (MB)=0.003
1> diagnostics = {os.version=5.5.6-arch1-1, java.vendor=Oracle
Corporation, source=merge, os.arch=amd64, mergeFactor=10,
java.runtime.version=11.0.6+8-LTS, os=Linux, timestamp=1586347074798, lucene.ve\
rsion=9.0.0, java.vm.version=11.0.6+8-LTS, java.version=11.0.6,
mergeMaxNumSegments=-1}
1> no deletions
1> test: open reader.........OK [took 0.001 sec]
1> test: check integrity.....OK [took 0.000 sec]
1> test: check live docs.....OK [took 0.000 sec]
1> test: field infos.........OK [1 fields] [took 0.000 sec]
1> test: field norms.........OK [1 fields] [took 0.001 sec]
1> test: terms, freq, prox...ERROR: java.lang.AssertionError:
buffer=java.nio.HeapByteBuffer[pos=0 lim=0 cap=0] bufferSize=1024
buffer.length=0
1> java.lang.AssertionError: buffer=java.nio.HeapByteBuffer[pos=0 lim=0
cap=0] bufferSize=1024 buffer.length=0
1> at
org.apache.lucene.store.BufferedIndexInput.setBufferSize(BufferedIndexInput.java:78)
1> at
org.apache.lucene.codecs.MultiLevelSkipListReader.loadSkipLevels(MultiLevelSkipListReader.java:241)
1> at
org.apache.lucene.codecs.MultiLevelSkipListReader.init(MultiLevelSkipListReader.java:208)
1> at
org.apache.lucene.codecs.lucene84.Lucene84SkipReader.init(Lucene84SkipReader.java:103)
1> at
org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.advance(Lucene84PostingsReader.java:837)
1> at
org.apache.lucene.index.FilterLeafReader$FilterPostingsEnum.advance(FilterLeafReader.java:271)
1> at
org.apache.lucene.index.AssertingLeafReader$AssertingPostingsEnum.advance(AssertingLeafReader.java:377)
1> at org.apache.lucene.index.CheckIndex.checkFields(CheckIndex.java:1426)
1> at org.apache.lucene.index.CheckIndex.testPostings(CheckIndex.java:1867)
1> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:720)
1> at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:301)
1> at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:286)
1> at
org.apache.lucene.store.BaseDirectoryWrapper.close(BaseDirectoryWrapper.java:45)
1> at org.apache.lucene.util.IOUtils.close(IOUtils.java:89)
1> at org.apache.lucene.util.IOUtils.close(IOUtils.java:77)
1> at
org.apache.lucene.index.TestIndexManyDocuments.test(TestIndexManyDocuments.java:69)
1> at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
1> at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
1> at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
1> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
1> at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
1> at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
1> at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
1> at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
1> at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
1> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
1> at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
1> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
1> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
1> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
1> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
1> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
1> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
1> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
1> at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
1> at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
1> at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
1> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
1> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
1> at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
1> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
1> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
1> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
1> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
1> at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
1> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
1> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
1> at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
1> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
1> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
1> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
1> at java.base/java.lang.Thread.run(Thread.java:834)
1> test: stored fields.......OK [0 total field count; avg 0.0 fields per
doc] [took 0.001 sec]
1> test: term vectors........OK [0 total term vector count; avg 0.0
term/freq vector fields per doc] [took 0.000 sec]
1> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 NUMERIC;
0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET] [took 0.000 sec]
1> test: points..............OK [0 fields, 0 points] [took 0.000 sec]
1> FAILED
1> WARNING: exorciseIndex() would remove reference to this segment; full
exception:
1> java.lang.RuntimeException: Term Index test failed
1> WARNING: 1 broken segments (containing 8700 documents) detected
1> Took 0.013 sec total.
1>
> java.lang.RuntimeException: CheckIndex failed
> at
__randomizedtesting.SeedInfo.seed([CB92BC7F7BCB5A04:43C683A5D53737FC]:0)
> at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:306)
> at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:286)
> at
org.apache.lucene.store.BaseDirectoryWrapper.close(BaseDirectoryWrapper.java:45)
> at org.apache.lucene.util.IOUtils.close(IOUtils.java:89)
> at org.apache.lucene.util.IOUtils.close(IOUtils.java:77)
> at
org.apache.lucene.index.TestIndexManyDocuments.test(TestIndexManyDocuments.java:69)
> at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
> at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
> at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
> at java.base/java.lang.Thread.run(Thread.java:834)
2> NOTE: reproduce with: ant test -Dtestcase=TestIndexManyDocuments
-Dtests.method=test -Dtests.seed=CB92BC7F7BCB5A04 -Dtests.slow=true
-Dtests.badapples=true -Dtests.locale=sr-Cyrl-ME -Dtests.timezone=A\
merica/Los_Angeles -Dtests.asserts=true -Dtests.file.encoding=UTF-8
2> NOTE: leaving temporary files on disk at:
/home/mike/src/simon/lucene/core/build/tmp/tests-tmp/lucene.index.TestIndexManyDocuments_CB92BC7F7BCB5A04-001
2> NOTE: test params are: codec=Asserting(Lucene84),
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4fe8fb1e),
locale=sr-Cyrl-ME, timezone=America/Los_Angeles
2> NOTE: Linux 5.5.6-arch1-1 amd64/Oracle Corporation 11.0.6
(64-bit)/cpus=128,threads=1,free=249224512,total=268435456
2> NOTE: All tests run in this JVM: [TestSearchForDuplicates, TestCircle2D,
TestLucene80NormsFormatMergeInstance, TestFastDecompressionMode,
TestMultiTermsEnum, TestFieldInvertState, TestFieldReuse, TestC\
oncurrentMergeScheduler, TestTerms, TestFieldUpdatesBuffer,
TestIndexWriterFromReader, TestReadOnlyIndex, TestTragicIndexWriterDeadlock,
TestIndexManyDocuments]
{noformat}
it might be related to this?
> Make BufferedIndexInput work on a ByteBuffer
> --------------------------------------------
>
> Key: LUCENE-9271
> URL: https://issues.apache.org/jira/browse/LUCENE-9271
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Adrien Grand
> Priority: Minor
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Currently {{BufferedIndexInput}} works on a {{byte[]}} but its main
> implementation, in NIOFSDirectory, has to implement a hack to maintain a
> ByteBuffer view of it that it can use in calls to the FileChannel API. Maybe
> we should instead make {{BufferedIndexInput}} work directly on a
> {{ByteBuffer}}? This would also help reuse the existing
> {{ByteBuffer#get(|Short|Int|long)}} methods instead of duplicating them from
> {{DataInput}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
