It's looking much better, there are only a few tests that repeatedly fail, and most of them are failing a small percentage of the time. Here's the short form, full list attached: DO NOT ENABLE LIST: MoveReplicaHDFSTest.testFailedMove MoveReplicaHDFSTest.testNormalFailedMove TestControlledRealTimeReopenThread.testCRTReopen TestICUNormalizer2CharFilter.testRandomStrings TestICUTokenizerCJK TestImpersonationWithHadoopAuth.testForwarding TestLTRReRankingPipeline.testDifferentTopN TestRandomChains
DO NOT ANNOTATE LIST CdcrBidirectionalTest.testBiDir IndexSizeTriggerTest.testMergeIntegration IndexSizeTriggerTest.testMixedBounds IndexSizeTriggerTest.testSplitIntegration IndexSizeTriggerTest.testTrigger InfixSuggestersTest.testShutdownDuringBuild ShardSplitTest.test ShardSplitTest.testSplitMixedReplicaTypes ShardSplitTest.testSplitWithChaosMonkey TestLatLonShapeQueries.testRandomBig TestRandomChains.testRandomChainsWithLargeStrings TestTriggerIntegration.testSearchRate Processing file (History bit 3): HOSS-2019-01-08.csv Processing file (History bit 2): HOSS-2018-12-31.csv Processing file (History bit 1): HOSS-2018-12-24.csv Processing file (History bit 0): HOSS-2018-12-17.csv **Annotated tests that didn't fail in the last 4 weeks. **Tests removed from the next two lists because they were specified in 'doNotEnable' in the properties file MoveReplicaHDFSTest.testNormalFailedMove **Annotations will be removed from the following tests because they haven't failed in the last 4 rollups. **Methods: 11 ComputePlanActionTest.testNodeAdded ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp CustomCollectionTest.testRouteFieldForHashRouter DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplicaLegacy ScheduledTriggerIntegrationTest.testScheduledTrigger SolrRrdBackendFactoryTest.testBasic StreamingTest.testParallelMergeStream StreamingTest.testZeroParallelReducerStream TestMiniSolrCloudClusterSSL.testSslWithCheckPeerName TestStressInPlaceUpdates.stressTest ZkShardTermsTest.testParticipationOfReplicas ********Failures in Hoss' reports for the last 4 rollups. There were 655 unannotated tests that failed in Hoss' rollups. Ordered by the date I downloaded the rollup file, newest->oldest. See above for the dates the files were collected These tests were NOT BadApple'd or AwaitsFix'd All tests that failed 4 weeks running will be BadApple'd unless there are objections Failures in the last 4 reports.. Report Pct runs fails test 0123 0.2 1673 11 MissingSegmentRecoveryTest.testLeaderRecovery 0123 1.6 1484 19 TestSQLHandler.doTest 0123 0.2 1702 18 TestSearchAfter.testQueries 0123 8.1 1656 59 TestSimLargeCluster.testAddNode 0123 0.7 1656 9 TestSimLargeCluster.testSearchRate 0123 2.8 502 12 TestSimTriggerIntegration.testCooldown 0123 1.4 1212 34 TestSimTriggerIntegration.testListeners 0123 37.3 501 160 TestSimTriggerIntegration.testNodeMarkersRegistration As usual, I'll commit on Thursday barring objections. Erick
DO NOT ENABLE LIST: MoveReplicaHDFSTest.testFailedMove MoveReplicaHDFSTest.testNormalFailedMove TestControlledRealTimeReopenThread.testCRTReopen TestICUNormalizer2CharFilter.testRandomStrings TestICUTokenizerCJK TestImpersonationWithHadoopAuth.testForwarding TestLTRReRankingPipeline.testDifferentTopN TestRandomChains DO NOT ANNOTATE LIST CdcrBidirectionalTest.testBiDir IndexSizeTriggerTest.testMergeIntegration IndexSizeTriggerTest.testMixedBounds IndexSizeTriggerTest.testSplitIntegration IndexSizeTriggerTest.testTrigger InfixSuggestersTest.testShutdownDuringBuild ShardSplitTest.test ShardSplitTest.testSplitMixedReplicaTypes ShardSplitTest.testSplitWithChaosMonkey TestLatLonShapeQueries.testRandomBig TestRandomChains.testRandomChainsWithLargeStrings TestTriggerIntegration.testSearchRate Processing file (History bit 3): HOSS-2019-01-08.csv Processing file (History bit 2): HOSS-2018-12-31.csv Processing file (History bit 1): HOSS-2018-12-24.csv Processing file (History bit 0): HOSS-2018-12-17.csv **Annotated tests that didn't fail in the last 4 weeks. **Tests removed from the next two lists because they were specified in 'doNotEnable' in the properties file MoveReplicaHDFSTest.testNormalFailedMove **Annotations will be removed from the following tests because they haven't failed in the last 4 rollups. **Methods: 11 ComputePlanActionTest.testNodeAdded ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp CustomCollectionTest.testRouteFieldForHashRouter DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplicaLegacy ScheduledTriggerIntegrationTest.testScheduledTrigger SolrRrdBackendFactoryTest.testBasic StreamingTest.testParallelMergeStream StreamingTest.testZeroParallelReducerStream TestMiniSolrCloudClusterSSL.testSslWithCheckPeerName TestStressInPlaceUpdates.stressTest ZkShardTermsTest.testParticipationOfReplicas ********Failures in Hoss' reports for the last 4 rollups. There were 655 unannotated tests that failed in Hoss' rollups. Ordered by the date I downloaded the rollup file, newest->oldest. See above for the dates the files were collected These tests were NOT BadApple'd or AwaitsFix'd All tests that failed 4 weeks running will be BadApple'd unless there are objections Failures in the last 4 reports.. Report Pct runs fails test 0123 0.2 1673 11 MissingSegmentRecoveryTest.testLeaderRecovery 0123 1.6 1484 19 TestSQLHandler.doTest 0123 0.2 1702 18 TestSearchAfter.testQueries 0123 8.1 1656 59 TestSimLargeCluster.testAddNode 0123 0.7 1656 9 TestSimLargeCluster.testSearchRate 0123 2.8 502 12 TestSimTriggerIntegration.testCooldown 0123 1.4 1212 34 TestSimTriggerIntegration.testListeners 0123 37.3 501 160 TestSimTriggerIntegration.testNodeMarkersRegistration ************ Will BadApple all tests above this line except ones listed at the top************** ***********************Annotations report Number of AwaitsFix: 49 Number of BadApples: 55 *****************AwaitsFix Annotations: ************Lucene AwaitsFix TestControlledRealTimeReopenThread.testCRTReopen @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5737") TestICUNormalizer2CharFilter.testRandomStrings @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5595") TestICUTokenizerCJK.TestICUTokenizerCJK suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8222") TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-7161") ************Solr AwaitsFix CdcrBidirectionalTest.testBiDir @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12524") CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028") CollectionsAPISolrJTest.testCreateAndDeleteShard @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13021") ComputePlanActionTest.testNodeLost @LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") ConcurrentCreateRoutedAliasTest.ConcurrentCreateRoutedAliasTest suite @LuceneTestCase.AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12386") // "Can't find resource" DocValuesNotIndexedTest.testGroupingSorting @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028") HdfsAutoAddReplicasIntegrationTest.HdfsAutoAddReplicasIntegrationTest suite @LuceneTestCase.AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13060") HdfsCollectionsAPIDistributedZkTest.HdfsCollectionsAPIDistributedZkTest suite @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13060") IndexSizeTriggerTest.testMaxOps @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") IndexSizeTriggerTest.testMergeIntegration @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") IndexSizeTriggerTest.testMixedBounds @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") IndexSizeTriggerTest.testSplitIntegration @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") IndexSizeTriggerTest.testTrigger @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") MoveReplicaHDFSTest.MoveReplicaHDFSTest suite @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13060") MoveReplicaHDFSTest.testFailedMove @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12080") // added 03-Oct-2018 NodeMarkersRegistrationTest.NodeMarkersRegistrationTest suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-13072") ReplaceNodeNoTargetTest.ReplaceNodeNoTargetTest suite @LuceneTestCase.AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11067") ReplaceNodeNoTargetTest.test @LuceneTestCase.AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11067") SSLMigrationTest.SSLMigrationTest suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 17-Mar-2018 SaslZkACLProviderTest.testSaslZkACLProvider @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-13075") ScheduledMaintenanceTriggerTest.testInactiveShardCleanup @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 17-Mar-2018 ScheduledTriggerTest.testTrigger @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 20-Sep-2018 SearchRateTriggerTest.testTrigger @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") Tagger2Test.testStopWords @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-8344") TestCollapseQParserPlugin.testStringCollapse @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-11974") TestHdfsCloudBackupRestore.TestHdfsCloudBackupRestore suite @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12866") TestHierarchicalDocBuilder.testThreeLevelHierarchy @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12801") // this test fails easily under beasting TestImpersonationWithHadoopAuth.testForwarding @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-9893") TestJsonFacetRefinement.testProcessEmptyRefinement @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556") TestJsonFacetRefinement.testSortedSubFacetRefinementWhenParentOnlyReturnedByOneShardProcessEmpty @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12556") TestLTRReRankingPipeline.testDifferentTopN @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11134") TestLocalFSCloudBackupRestore.TestLocalFSCloudBackupRestore suite @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12866") TestMinMaxOnMultiValuedField.testDoubleFieldCache @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709") TestMinMaxOnMultiValuedField.testFloatFieldCache @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709") TestMinMaxOnMultiValuedField.testIntFieldCache @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709") TestMinMaxOnMultiValuedField.testLongFieldCache @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709") TestMiniSolrCloudClusterSSL.after assumeFalse("@AwaitsFix: SOLR-12988 - ssl issues on Java 11/12", Constants.JRE_IS_MINIMUM_JAVA11); TestPullReplica.TestPullReplica suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") TestSSLRandomization.testRandomizedSslAndClientAuth assumeFalse("@AwaitsFix: SOLR-12988 - ssl issues on Java 11/12", Constants.JRE_IS_MINIMUM_JAVA11); TestSimComputePlanAction.testNodeLost @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028") // if you beast this, eventually you will see TestSimLargeCluster.testBasic @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // this test hits a timeout easily TestStressRecovery.TestStressRecovery suite @LuceneTestCase.AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 6-Sep-2018 TestTlogReplica.TestTlogReplica suite @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12313") TimeRoutedAliasUpdateProcessorTest.TimeRoutedAliasUpdateProcessorTest suite @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-12801") *****************BadApple Annotations: ************Lucene BadApple ************Solr BadApple BasicAuthIntegrationTest.testBasicAuth @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 BasicDistributedZkTest.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 BlockCacheTest.BlockCacheTest suite @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 12-Jun-2018 CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 CollectionsAPISolrJTest.testCreateCollWithDefaultClusterPropertiesNewFormat @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 08-Jan-2019 ComputePlanActionTest.testNodeAdded @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 CustomCollectionTest.testRouteFieldForHashRouter @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 09-Aug-2018 DeleteReplicaTest.deleteLiveReplicaTest @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 DeleteReplicaTest.deleteReplicaByCountForAllShards @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplicaLegacy @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 FullSolrCloudDistribCmdsTest.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 HdfsNNFailoverTest.HdfsNNFailoverTest suite @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 12-Jun-2018 HdfsRecoveryZkTest.HdfsRecoveryZkTest suite @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") LBHttpSolrClientTest.testLBHttpSolrClientHttpClientResponseParserStringArray @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 20-Sep-2018 LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 MoveReplicaHDFSTest.testNormalFailedMove @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 MoveReplicaTest.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 MoveReplicaTest.testFailedMove @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 MultiThreadedOCPTest.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 RestartWhileUpdatingTest.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 ScheduledTriggerIntegrationTest.testScheduledTrigger @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 SearchRateTriggerIntegrationTest.testDeleteNode @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 21-May-2018 SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 20-Jul-2018 SolrRrdBackendFactoryTest.testBasic @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 6-Sep-2018 SolrZkClientTest.testSimpleUpdateACLs @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 StreamDecoratorTest.testParallelCommitStream @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 StreamDecoratorTest.testParallelExecutorStream @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 2-Aug-2018 StreamingTest.testParallelMergeStream @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 6-Sep-2018 StreamingTest.testZeroParallelReducerStream @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 6-Sep-2018 TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 6-Sep-2018 TestCloudRecovery.corruptedLogTest @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 TestCollectionStateWatchers.testCanWaitForNonexistantCollection @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 TestDistribIDF.testMultiCollectionQuery @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 TestDistributedStatsComponentCardinality.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 TestInPlaceUpdatesDistrib.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 TestMiniSolrCloudClusterSSL.testSslWithCheckPeerName @LuceneTestCase.BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 2-Aug-2018 TestPullReplica.testCreateDelete @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 21-May-2018 TestSimGenericDistributedQueue.testDistributedQueue @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 TestSimLargeCluster.testNodeLost @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 14-Oct-2018 TestSimPolicyCloud.testCreateCollectionAddReplica @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 TestSimTriggerIntegration.testNodeAddedTriggerRestoreState @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 08-Jan-2019 TestSimTriggerIntegration.testNodeLostTriggerRestoreState @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 08-Jan-2019 TestSkipOverseerOperations.testSkipDownOperations @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 20-Sep-2018 TestStressInPlaceUpdates.stressTest @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // 09-Apr-2018 TestSystemCollAutoCreate.testAutoCreate @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 TestWithCollection.testAddReplicaWithPolicy @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 20-Sep-2018 TestWithCollection.testNodeAdded @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 15-Sep-2018 TriLevelCompositeIdRoutingTest.test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 TriggerIntegrationTest.testTriggerThrottling @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 TriggerSetPropertiesIntegrationTest.testSetProperties @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // annotated on: 24-Dec-2018 ZkShardTermsTest.testParticipationOfReplicas @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028") // added 15-Sep-2018 Process finished with exit code 0
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org