Sounds good. Also note there are more iterator implementations in DocValuesConsumer that still need to be ported / code validated..
-- Itamar Syn-Hershko http://code972.com | @synhershko <https://twitter.com/synhershko> Freelance Developer & Consultant Lucene.NET committer and PMC member On Mon, Jan 26, 2015 at 3:57 AM, Laimonas Simutis <[email protected]> wrote: > The tests have been running a long time ever since LogByteSizeMergePolicy > was fixed. I haven't been able to fully track down what the issue is, but I > suspect fixing the policy is allowing the merge to run fully, which is > exposing another bug in there. What is happening now is that after merge > run, the logic finds another merge that it should do instead of stopping. > And it continues to do so over and over again until some really high limit > of segments is reached. > > Just letting you guys know, in case you are wondering. I will keep an eye > on this as I am fixing the next set of tests, hoping I will figure out > quickly what the issue is. > > > Laimis > > > On Sun Jan 25 2015 at 10:59:37 AM <[email protected]> wrote: > > > Build Lucene.NET :: Lucene.NET Core #105 failed (Execution timeout; tests > > failed: 158 (18 new), passed: 2157, ignored: 96). > > Agent: JetBrains-dotnet-agent01 > > Build results: http://teamcity.codebetter.com/viewLog.html?buildId= > > 179703&buildTypeId=LuceneNet_Core > > > > Failed Tests Summary: > > Newly failed tests (18 tests, alphabetically ordered) > > ================================================================== > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > TestLucene45DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsLastMissingMerge > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Perfield. > > > TestPerFieldDocValuesFormat.BaseIndexFileFormatTestCase.TestMergeStability > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestAddIndexes. > > TestAddIndexesWithRollback > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestAddIndexes. > > TestAddIndexesWithThreads > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestBagOfPositions.Test > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestBagOfPostings.Test > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseDocValuesFormatTestCase.TestBytesTwoDocumentsMerged > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseDocValuesFormatTestCase.TestSortedSetTwoDocumentsFirstMissingMerge > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestForceMergeForever.Test > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriter. > > TestRotatingFieldNames > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterReader. > > TestDuringAddIndexes > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterWithThreads. > > TestIOExceptionDuringAbortWithThreadsOnlyOnce > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterWithThreads. > > TestIOExceptionDuringWriteSegmentWithThreadsOnlyOnce > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestMultiDocValues.TestSorted > > Lucene.Net.Tests.dll: > Lucene.Net.Index.TestStressIndexing2.TestMultiConfig > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestTermsEnum.TestIntersectRandom > > Lucene.Net.Tests.dll: Lucene.Net.Search.TestTermVectors. > > TestFullMergeAddIndexesDir > > Lucene.Net.Tests.dll: Lucene.Net.Store.TestLockFactory. > > TestStressLocksNativeFSLockFactory > > > > Other failed tests (140 tests, alphabetically ordered, only first 100 > > shown) > > ================================================================== > > Lucene.Net.Tests.dll: Lucene.Net.Analysis.TestLookaheadTokenFilter. > > TestNeverCallingPeek > > Lucene.Net.Tests.dll: Lucene.Net.Analysis.TestLookaheadTokenFilter. > > TestRandomStrings > > Lucene.Net.Tests.dll: Lucene.Net.Analysis.TestMockAnalyzer. > > TestRandomRegexps > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene40. > > TestLucene40DocValuesFormat.BaseDocValuesFormatTestCase. > > TestByteNumericsVsStoredFields > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene40. > > TestLucene40DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene40. > > TestLucene40DocValuesFormat.BaseDocValuesFormatTestCase. > > TestTwoDocumentsMerged > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene41.TestBlockPostingsFormat2 > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene41. > > TestBlockPostingsFormat3.Test > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetFixedLengthSingleValuedVsStoredFields > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetFixedLengthVsStoredFields > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetFixedLengthVsUninvertedField > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetOneValue > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTermsEnum > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetThreeValuesTwoDocs > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsFirstMissing > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsFirstMissingMerge > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsLastMissing > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsLastMissingMerge > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsMerged > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoFields > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoValues > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoValuesUnordered > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetVariableLengthSingleValuedVsStoredFields > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetVariableLengthVsStoredFields > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetVariableLengthVsUninvertedField > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene42. > > TestLucene42DocValuesFormat.BaseDocValuesFormatTestCase. > > TestVeryLargeButLegalSortedBytes > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > TestLucene45DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > TestLucene45DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetFixedLengthVsUninvertedField > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > TestLucene45DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > TestLucene45DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetThreeValuesTwoDocs > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > TestLucene45DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsMerged > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > TestLucene45DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetVariableLengthVsUninvertedField > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > > TestLucene45DocValuesFormat.BaseIndexFileFormatTestCase.TestMergeStability > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Perfield. > > TestPerFieldDocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Perfield. > > TestPerFieldDocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetFixedLengthVsUninvertedField > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Perfield. > > TestPerFieldDocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Perfield. > > TestPerFieldDocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetThreeValuesTwoDocs > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Perfield. > > TestPerFieldDocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsMerged > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Perfield. > > TestPerFieldDocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetVariableLengthVsUninvertedField > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestAddIndexes. > > TestAddIndexMissingCodec > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestAtomicUpdate.TestAtomicUpdates > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestBinaryDocValuesUpdates. > > TestTonsOfUpdates > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestBinaryDocValuesUpdates. > > TestUpdateSegmentWithNoDocValues > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestBinaryDocValuesUpdates. > > TestUpdatesAndDeletes > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestCrashCausesCorruptIndex. > > TestCrashCorruptsIndexing > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestCustomNorms.TestFloatNorms > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDirectoryReader.TestNoDir > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseDocValuesFormatTestCase.TestSortedMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseDocValuesFormatTestCase.TestSortedSetFixedLengthVsUninvertedField > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseDocValuesFormatTestCase.TestSortedSetMergeAwayAllValues > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseDocValuesFormatTestCase.TestSortedSetThreeValuesTwoDocs > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseDocValuesFormatTestCase.TestSortedSetTwoDocumentsMerged > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseDocValuesFormatTestCase.TestSortedSetVariableLengthVsUninvertedField > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocValuesFormat. > > BaseIndexFileFormatTestCase.TestMergeStability > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocumentsWriterDeleteQueue > > .TestStressDeleteQueue > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDocumentsWriterStallContro > > l.TestSimpleStall > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestDuelingCodecs.TestEquals > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestFieldsReader.TestExceptions > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestFilterAtomicReader. > > TestOverrideMethods > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriter. > > TestThreadInterruptDeadlock > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriter. > > TestTwoThreadsInterruptDeadlock > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterConfig. > > TestSettersChaining > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterExceptions. > > TestDocumentsWriterAbort > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterExceptions. > > TestExceptionDuringSync > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterExceptions. > > TestExceptionOnMergeInit > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterExceptions. > > TestExceptionsDuringCommit > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterExceptions. > > TestNoLostDeletesOrUpdates > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterExceptions. > > TestSegmentsChecksumError > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterForceMerge. > > TestBackgroundForceMerge > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterReader. > > TestDuringAddDelete > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterReader. > > TestNRTOpenExceptions > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterUnicode. > > TestRandomUnicodeStrings > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestIndexWriterUnicode. > > TestTermUTF16SortOrder > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestLongPostings. > > TestLongPostingsNoPositions > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestMixedDocValuesUpdates. > > TestStressMultiThreading > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestMixedDocValuesUpdates. > > TestTonsOfUpdates > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestMultiDocValues.TestSortedSet > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestMultiDocValues. > > TestSortedSetWithDups > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestMultiLevelSkipList. > > TestSimpleSkip > > Lucene.Net.Tests.dll: Lucene.Net.Index.TestNRTThreads.TestNRTThreads_Mem > > > > > > > > Newly failed tests details (only first 3 failures shown here) > > ================================================================== > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Lucene45. > > TestLucene45DocValuesFormat.BaseDocValuesFormatTestCase. > > TestSortedSetTwoDocumentsLastMissingMerge (new) ================= > > Test(s) failed. Test exceeded Timeout value of 20000ms > > > > Lucene.Net.Tests.dll: Lucene.Net.Codecs.Perfield. > > > TestPerFieldDocValuesFormat.BaseIndexFileFormatTestCase.TestMergeStability > > (new) ================= > > Test(s) failed. Expected: 9782 > > But was: 9622 > > > > at NUnit.Framework.Assert.That(Object actual, IResolveConstraint > > expression, String message, Object[] args) > > at NUnit.Framework.Assert.AreEqual(Object expected, Object actual) > > at Lucene.Net.Util.LuceneTestCase.assertEquals[T,S](IDictionary`2 > > expected, IDictionary`2 actual) in z:\Builds\work\ > > > bcdbe6b8cc677a49\src\Lucene.Net.TestFramework\JavaCompatibility\LuceneTestCase.cs:line > > 64 > > at Lucene.Net.Index.BaseIndexFileFormatTestCase.TestMergeStability() > > in z:\Builds\work\bcdbe6b8cc677a49\src\Lucene.Net.TestFramework\Index\ > > BaseIndexFileFormatTestCase.cs:line 126 > > Lucene.Net.Tests.dll: > Lucene.Net.Index.TestAddIndexes.TestAddIndexesWithRollback > > (new) ================= > > Test(s) failed. Expected: True > > But was: False > > > > at NUnit.Framework.Assert.That(Object actual, IResolveConstraint > > expression, String message, Object[] args) > > at NUnit.Framework.Assert.IsTrue(Boolean condition) > > at Lucene.Net.Index.TestAddIndexes.TestAddIndexesWithRollback() in > > > z:\Builds\work\bcdbe6b8cc677a49\src\Lucene.Net.Tests\core\Index\TestAddIndexes.cs:line > > 1032 > > ------- Stderr: ------- > > at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue > > value, Boolean add) > > at System.Collections.Generic.Dictionary`2.set_Item(TKey key, TValue > > value) > > at Lucene.Net.Index.IndexWriter.ForceMerge(Int32 maxNumSegments, > > Boolean doWait) in z:\Builds\work\bcdbe6b8cc677a49\src\Lucene. > > Net.Core\Index\IndexWriter.cs:line 2153 > > at Lucene.Net.Index.IndexWriter.ForceMerge(Int32 maxNumSegments) in > > > z:\Builds\work\bcdbe6b8cc677a49\src\Lucene.Net.Core\Index\IndexWriter.cs:line > > 2101 > > at Lucene.Net.Index.TestAddIndexes.CommitAndAddIndexes3.DoBody(Int32 > > j, Directory[] dirs) in z:\Builds\work\bcdbe6b8cc677a49\src\Lucene. > > Net.Tests\core\Index\TestAddIndexes.cs:line 914 > > at Lucene.Net.Index.TestAddIndexes.RunAddIndexesThreads. > > ThreadAnonymousInnerClassHelper.Run() in z:\Builds\work\ > > bcdbe6b8cc677a49\src\Lucene.Net.Tests\core\Index\TestAddIndexes.cs:line > > 713 > > see all newly failed tests: http://teamcity.codebetter. > > com/viewLog.html?buildId=179703&buildTypeId=LuceneNet_Core > > Changes included (3 changes) > > ==================================================== > > Change 8520cba87bde2750a9f58943eeedce2d0ef320be by itamar (0 file): > > Merge remote-tracking branch 'laimis/merge_logic_fixes' > > > > > > Change 9776c7060c33b65c35b8b55af1d9b9177432cca4 by laimis (0 file): > > merge with upstream/head, fix conflicts > > > > > > Change 4edb41be48fc2bd73c7ce9260ac7ec692b6861ed by laimis (3 files): > > match Lucene logic for merging values > > > > > > see more information about changed files: http://teamcity.codebetter. > > > com/viewLog.html?tab=buildChangesDiv&buildId=179703&buildTypeId=LuceneNet_ > > Core > > > > ============================================================ > > ================ > > Configure email notifications: http://teamcity.codebetter. > > com/profile.html?init=1&tab=userNotifications > > >
