this seems like a bug but the window is super small. Since you need to be in startCommit just before you add the new "pendingCommit" and be in rollback with another thread already leaving the critical section. In addition you need to have passed the ensureOpen() call with the committing thread before we entered rollbackInternal. I think it would make sense to do a ensureOpen() call before we add the new pending commit in start commit to fix the problem. The unsynced reads robert is mentioning are fine since pendingCommit is volatile though.
simon On Fri, Jun 15, 2012 at 3:37 PM, Robert Muir <[email protected]> wrote: > This seems like a thread hazard in rollBackInternal? it clears > pendingCommit, but the closeInternal is outside of the synchronized > block... so some other thread must be sneaking a pendingCommit in at > that exact moment in this test? > > On Fri, Jun 15, 2012 at 9:24 AM, Robert Muir <[email protected]> wrote: >> I can't reproduce this, on linux or windows: but I think its a bug. >> >> rollback()'s javadocs say "This also clears a previous call to {@link >> #prepareCommit}." >> >> On Fri, Jun 15, 2012 at 9:09 AM, <[email protected]> wrote: >>> Build: >>> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows-Java7-64/70/ >>> >>> 1 tests failed. >>> REGRESSION: >>> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback >>> >>> Error Message: >>> cannot close: prepareCommit was already called with no corresponding call >>> to commit >>> >>> Stack Trace: >>> java.lang.IllegalStateException: cannot close: prepareCommit was already >>> called with no corresponding call to commit >>> at >>> __randomizedtesting.SeedInfo.seed([6DC405A18E10711B:8BE3EA4E2ABF1A65]:0) >>> at >>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:894) >>> at >>> org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:1870) >>> at >>> org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:1792) >>> at >>> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:959) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:601) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) >>> at >>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) >>> at >>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) >>> at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) >>> at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> at >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) >>> at >>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) >>> at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) >>> at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) >>> at >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) >>> at >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> at >>> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) >>> at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> at >>> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) >>> at >>> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) >>> at >>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) >>> at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> at >>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) >>> >>> >>> >>> >>> Build Log: >>> [...truncated 1173 lines...] >>> [junit4] Completed in 0.02s, 2 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.search.TestDocValuesScoring >>> [junit4] Completed in 0.03s, 1 test >>> [junit4] >>> [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler >>> [junit4] Completed in 2.31s, 4 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.search.TestCachingWrapperFilter >>> [junit4] Completed in 0.05s, 5 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.search.TestFieldValueFilter >>> [junit4] Completed in 0.07s, 2 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.util.TestVersion >>> [junit4] Completed in 0.01s, 2 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.index.TestSegmentMerger >>> [junit4] Completed in 0.77s, 3 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.search.TestMultiPhraseQuery >>> [junit4] IGNORED 0.00s | TestMultiPhraseQuery.testMultiSloppyWithRepeats >>> [junit4] > Cause: Annotated @Ignore() >>> [junit4] Completed in 0.34s, 16 tests, 1 skipped >>> [junit4] >>> [junit4] Suite: org.apache.lucene.util.TestRollingBuffer >>> [junit4] Completed in 0.17s, 1 test >>> [junit4] >>> [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy >>> [junit4] Completed in 3.78s, 6 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.index.TestFlex >>> [junit4] Completed in 0.26s, 2 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.TestSearch >>> [junit4] Completed in 0.02s, 1 test >>> [junit4] >>> [junit4] Suite: org.apache.lucene.search.spans.TestPayloadSpans >>> [junit4] Completed in 0.88s, 10 tests >>> [junit4] >>> [junit4] Suite: >>> org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides >>> [junit4] Completed in 0.02s, 2 tests >>> [junit4] >>> [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimatorOnWildAnimals >>> [junit4] Completed in 0.84s, 1 test >>> [junit4] >>> [junit4] Suite: org.apache.lucene.search.TestScorerPerf >>> [junit4] Completed in 0.56s, 1 test >>> [junit4] >>> [junit4] Suite: org.apache.lucene.index.TestAddIndexes >>> [junit4] ERROR 0.50s | TestAddIndexes.testAddIndexesWithRollback >>> [junit4] > Throwable #1: java.lang.IllegalStateException: cannot >>> close: prepareCommit was already called with no corresponding call to commit >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([6DC405A18E10711B:8BE3EA4E2ABF1A65]:0) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:894) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:1870) >>> [junit4] > at >>> org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:1792) >>> [junit4] > at >>> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:959) >>> [junit4] > at >>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> [junit4] > at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> [junit4] > at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> [junit4] > at java.lang.reflect.Method.invoke(Method.java:601) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32) >>> [junit4] > at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) >>> [junit4] > at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) >>> [junit4] > >>> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestAddIndexes >>> -Dtests.method=testAddIndexesWithRollback -Dtests.seed=6DC405A18E10711B >>> -Dtests.locale=nl_BE -Dtests.timezone=America/Kralendijk >>> -Dargs="-Dfile.encoding=Cp1252" >>> [junit4] 2> >>> [junit4] > (@AfterClass output) >>> [junit4] 2> NOTE: test params are: codec=Lucene40: >>> {id=Pulsing40(freqCutoff=19 minBlockSize=13 maxBlockSize=67), >>> content=PostingsFormat(name=Memory doPackFST= true), >>> c=Pulsing40(freqCutoff=19 minBlockSize=13 maxBlockSize=67), >>> f1=MockFixedIntBlock(blockSize=1640), f2=Pulsing40(freqCutoff=3 >>> minBlockSize=13 maxBlockSize=67)}, >>> sim=RandomSimilarityProvider(queryNorm=true,coord=false): {id=DFR >>> I(F)B3(800.0), content=DFR GB2, foo=IB LL-D1}, locale=nl_BE, >>> timezone=America/Kralendijk >>> [junit4] 2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_04 >>> (64-bit)/cpus=2,threads=1,free=203015176,total=286392320 >>> [junit4] 2> NOTE: All tests run in this JVM: [TestDirectoryReader, >>> TestNewestSegment, TestWindowsMMap, TestSameTokenSamePosition, >>> TestSpansAdvanced, TestRAMDirectory, TestSortedVIntList, >>> TestDeterminizeLexicon, TestOmitPositions, TestIsCurrent, TestDocValues, >>> TestPayloadProcessorProvider, TestDocValuesIndexing, TestPriorityQueue, >>> TestSimilarityProvider, TestDoc, TestStressAdvance, >>> TestBooleanMinShouldMatch, TestSort, TestSpansAdvanced2, >>> TestIndexWriterForceMerge, TestCopyBytes, TestIndexInput, >>> TestIndexableField, TestAllFilesHaveCodecHeader, TestNamedSPILoader, >>> TestFixedBitSet, TestByteSlices, TestSloppyPhraseQuery, >>> TestRollingCharBuffer, TestSpanFirstQuery, TestCollectionUtil, >>> TestReuseDocsEnum, TestVirtualMethod, TestPhrasePrefixQuery, Nested, >>> Nested, TestSloppyPhraseQuery2, TestLazyProxSkipping, TestTermsEnum, >>> TestPagedBytes, TestDocIdSet, TestSnapshotDeletionPolicy, TestPrefixQuery, >>> TestImpersonation, TestPrefixInBooleanQuery, TestParallelReaderEmptyIndex, >>> TestTerm, TestByteBlockPool, TestOmitNorms, TestFieldMaskingSpanQuery, >>> TestCheckIndex, TestFieldCacheRangeFilter, TestFileSwitchDirectory, >>> TestDocTermOrds, TestPackedInts, TestNot, TestIndexWriterLockRelease, >>> TestPositiveScoresOnlyCollector, Nested1, TestRollingUpdates, >>> TestSearchWithThreads, TestFieldCache, TestUTF32ToUTF8, TestNumericUtils, >>> TestNRTCachingDirectory, TestOpenBitSet, TestStressIndexing2, >>> TestSearchAfter, TestTopDocsCollector, TestNoMergeScheduler, >>> TestFlushByRamOrCountsPolicy, TestDocumentsWriterStallControl, >>> TestIndexWriterExceptions, TestIdentityHashSet, TestArrayUtil, >>> TestDocValuesTypeCompatibility, TestSpanExplanations, >>> TestCustomSearcherSort, TestUniqueTermCount, TestIndexWriterCommit, >>> TestSegmentTermEnum, TestBasics, TestParallelCompositeReader, >>> TestBackwardsCompatibility, TestBytesRefHash, TestIndexWriterUnicode, >>> TestCompoundFile, TestIndexWriterConfig, TestWildcard, TestRollback, >>> TestMultiValuedNumericRangeQuery, TestWildcardRandom, TestGraphTokenizers, >>> TestSetOnce, TestSimpleExplanationsOfNonMatches, TestReaderClosed, >>> TestRegexpRandom2, TestTermsEnum2, TestRecyclingByteBlockAllocator, >>> TestSimpleAttributeImpl, TestIndexWriterMergePolicy, TestDeterminism, >>> TestMultiTermQueryRewrites, TestBitVector, TestIntsRef, TestDuelingCodecs, >>> ThrowInUncaught, TestStressNRT, TestPrefixCodedTerms, TestSentinelIntSet, >>> TestFieldCacheSanityChecker, TestSearcherManager, InBeforeClass, >>> NonStringProperties, InAfterClass, InTestMethod, TestParallelAtomicReader, >>> TestSpanExplanationsOfNonMatches, TestVersionComparator, TestSumDocFreq, >>> TestTypePromotion, TestByteArrayDataInput, TestFieldCacheTermsFilter, >>> Test2BPostings, TestNoDeletionPolicy, TestMockAnalyzer, >>> TestIndexWriterOnJRECrash, TestExplanations, TestMultiLevelSkipList, >>> TestSimpleSearchEquivalence, TestSimilarityBase, TestLookaheadTokenFilter, >>> TestCachingTokenFilter, TestCompiledAutomaton, TestSurrogates, >>> TestCharTermAttributeImpl, TestDateSort, TestFuzzyQuery, >>> TestNumericTokenStream, TestAttributeSource, TestIndexWriterReader, >>> TestMixedCodecs, TestMergeSchedulerExternal, TestSpans, TestIOUtils, >>> TestDocCount, TestRegexpQuery, TestIndexableBinaryStringTools, >>> TestCustomNorms, TestPerSegmentDeletes, TestDirectoryReaderReopen, >>> TestBasicOperations, TestParallelTermEnum, TestMultiMMap, >>> TestAutomatonQueryUnicode, TestRegexpRandom, TestShardSearching, >>> TestFilteredQuery, TestQueryWrapperFilter, TestFieldInfos, >>> TestSpanSearchEquivalence, TestSimilarity2, TestIndexWriterWithThreads, >>> TestMultiThreadTermVectors, TestComplexExplanationsOfNonMatches, >>> TestPulsingReuse, TestNGramPhraseQuery, TestComplexExplanations, TestToken, >>> TestNorms, TestCodecs, TestTieredMergePolicy, TestPayloads, >>> TestScoreCachingWrappingScorer, TestIndexWriter, TestFilteredSearch, >>> TestLevenshteinAutomata, TestAppendingCodec, TestDoubleBarrelLRUCache, >>> TestMatchAllDocsQuery, TestBufferedIndexInput, TestNeverDelete, >>> TestIndexWriterOnDiskFull, TestBoolean2, TestStressIndexing, >>> TestSegmentTermDocs, MultiCollectorTest, TestFieldsReader, >>> TestNRTReaderWithThreads, Test2BTerms, TestConsistentFieldNumbers, >>> TestNumericRangeQuery32, TestSimpleExplanations, TestForceMergeForever, >>> TestBinaryTerms, TestDisjunctionMaxQuery, TestIndexWriterMerging, >>> TestIndexWriterNRTIsCurrent, TestPerFieldPostingsFormat, TestDateTools, >>> TestTwoPhaseCommitTool, TestBooleanQuery, TestPositionIncrement, >>> TestBooleanOr, TestSpanMultiTermQueryWrapper, TestUnicodeUtil, Nested, >>> Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, >>> Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, >>> Nested, Nested, TestDateFilter, TestDocValuesScoring, >>> TestConcurrentMergeScheduler, TestCachingWrapperFilter, >>> TestFieldValueFilter, TestVersion, TestSegmentMerger, TestMultiPhraseQuery, >>> TestRollingBuffer, TestDeletionPolicy, TestFlex, TestSearch, >>> TestPayloadSpans, Before3, Before3, TestRamUsageEstimatorOnWildAnimals, >>> TestScorerPerf, TestAddIndexes] >>> [junit4] 2> >>> [junit4] Completed in 3.30s, 20 tests, 1 error <<< FAILURES! >>> [...truncated 279 lines...] >>> >>> [...truncated 1553 lines...] >>> >>> [...truncated 1553 lines...] >>> >>> [...truncated 1553 lines...] >>> >>> [...truncated 1553 lines...] >>> >>> [...truncated 1553 lines...] >>> >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >> >> >> >> -- >> lucidimagination.com > > > > -- > lucidimagination.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
