I'm still trying to find the balance on how aggressively to BadApple tests. There's a danger of BadApple-ing tests that _should_ fail or are being worked on. So here's my current proposal:
> I'll collect candidate failed tests through Tuesday of each week, then try to publish the candidate list on Wednesday (Yes, today is Thursday, gimme a break I'm trying to figure it out). > Any tests from that candidate list that are _still_ failing as reported by either Mark's or Hoss' reports on Saturday for the most recent 24 hours are the "true list" and I'll BadApple them. That gives a couple of days for people to look over the candidate list and object. > I'll check the history of the test before BadApple-ing it. If it's been changed recently I'll leave it alone. That should help keep tests that have been corrected, but won't necessarily catch code fixes that the test exposed where the fix is not in the test code. > If you don't think something on the list should be BadApple'd let me know and I won't BadApple it. Feel free to open a JIRA and annotate the test and point to that JIRA if you want. > The failures I collected by day are at the end for reference. > For a week or two the list will be fairly long, but the goal is certainly to have the "BadApple candidate list" be very small to nonexistent. Fails 4, 5, 6 Mar, "BadApple candidate list" org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat.testSortedNumericsSingleValuedMissingVsStoredFields org.apache.lucene.index.TestIndexSorting.testRandom3 org.apache.lucene.index.TestNorms.testMaxByteNorms org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testMultiVariateNormalDistribution org.apache.solr.client.solrj.request.TestV1toV2ApiMapper.testCommands org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState org.apache.solr.cloud.cdcr.CdcrReplicationHandlerTest org.apache.solr.cloud.CollectionsAPISolrJTest.testCreateWithDefaultConfigSet org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test org.apache.solr.cloud.hdfs.StressHdfsTest.test org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderAndMixedReplicas org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderOldReplica org.apache.solr.cloud.LIRRollingUpdatesTest.testNewReplicaOldLeader org.apache.solr.cloud.LIRRollingUpdatesTest.testOldLeaderAndMixedReplicas org.apache.solr.cloud.MoveReplicaHDFSTest org.apache.solr.cloud.MoveReplicaHDFSTest.test org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeaderAfterRestart org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread.testPublishDownState org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_stored_idx org.apache.solr.cloud.TestStressLiveNodes.testStress org.apache.solr.cloud.TestTlogReplica.testOnlyLeaderIndexes org.apache.solr.metrics.reporters.SolrGraphiteReporterTest.testReporter org.apache.solr.update.TestInPlaceUpdatesDistrib.test Fails by day. Fails 4-March org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderAndMixedReplicas org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderOldReplica org.apache.solr.cloud.LIRRollingUpdatesTest.testNewReplicaOldLeader org.apache.solr.cloud.LIRRollingUpdatesTest.testOldLeaderAndMixedReplicas org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeaderAfterRestart org.apache.solr.cloud.TestTlogReplica.testOnlyLeaderIndexes LUCENE test failing: org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat.testSortedNumericsSingleValuedMissingVsStoredFields Fails 5-Mar org.apache.solr.cloud.cdcr.CdcrReplicationHandlerTest org.apache.solr.cloud.CollectionsAPISolrJTest.testCreateWithDefaultConfigSet org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderAndMixedReplicas org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderOldReplica org.apache.solr.cloud.LIRRollingUpdatesTest.testNewReplicaOldLeader org.apache.solr.cloud.MoveReplicaHDFSTest org.apache.solr.cloud.MoveReplicaHDFSTest.test org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader org.apache.solr.cloud.TestStressLiveNodes.testStress org.apache.solr.metrics.reporters.SolrGraphiteReporterTest.testReporter Fails 6-Mar: org.apache.lucene.index.TestIndexSorting.testRandom3 org.apache.lucene.index.TestNorms.testMaxByteNorms org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testMultiVariateNormalDistribution org.apache.solr.client.solrj.request.TestV1toV2ApiMapper.testCommands org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState org.apache.solr.cloud.hdfs.StressHdfsTest.test org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderAndMixedReplicas org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderOldReplica org.apache.solr.cloud.LIRRollingUpdatesTest.testNewReplicaOldLeader org.apache.solr.cloud.LIRRollingUpdatesTest.testOldLeaderAndMixedReplicas org.apache.solr.cloud.MoveReplicaHDFSTest.test org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeaderAfterRestart org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread.testPublishDownState org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_stored_idx org.apache.solr.update.TestInPlaceUpdatesDistrib.test Fails 7-Mar org.apache.lucene.index.TestIndexSorting.testRandom3 org.apache.solr.cloud.BasicDistributedZk2Test.test org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderAndMixedReplicas org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderOldReplica org.apache.solr.cloud.LIRRollingUpdatesTest.testNewReplicaOldLeader org.apache.solr.cloud.LIRRollingUpdatesTest.testOldLeaderAndMixedReplicas org.apache.solr.cloud.MoveReplicaHDFSTest.test org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalMove org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeaderAfterRestart org.apache.solr.cloud.TestPullReplica.testKillLeader org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToPullReplica org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenRenew org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload Fails 8-Mar: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalFailedMove org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader org.apache.solr.cloud.TestPullReplica.testKillPullReplica --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
