Tests take forever to run, and we have totally intermingled unit tests (testing one class or with mocks) with integration tests (spinning up solr clusters, indexing etc), which is not good project hygiene imo. Can we start tagging integration tests in a way so we can choose to leave them out for the quick dev iterations? We already have these properties for test filtering: // filtering [propName: 'tests.filter', value: null, description: "Applies a test filter (see :helpTests)."], [propName: 'tests.slow', value: true, description: "Enables or disables @Slow tests."], [propName: 'tests.nightly', value: false, description: "Enables or disables @Nightly tests."], [propName: 'tests.weekly', value: false, description: "Enables or disables @Weekly tests."], [propName: 'tests.monster', value: false, description: "Enables or disables @Monster tests."], [propName: 'tests.awaitsfix', value: null, description: "Enables or disables @AwaitsFix tests."], [propName: 'tests.badapples', value: null, description: "Enables or disables @BadApple tests."], So perhaps add an @Ingtegrationtest annotation and a -Ptests.integrationtest=true/false flag. Then we don't need to move test files to other folder, enough to annotate them. For a start, all SolrCloudTestCase tests could be annotated (are annotations inherited?) Then I imagine I'd run unit tests frequently and the whole suite occationally right before merging a large feature. It would also be interesting to run only unit tests and then have a look at the JaCoco test coverage stats. I have a suspicion we have not been good enough writing basic unit tests, including for failure and corner cases.
Don't get me wrong. I think it is crucial to test real, live clusters for a project like Solr, and we should keep the tests (and stabilize them). But we cannot focus only on integration. It is developer hostile :) I'd like a "gradlew test" to take 3-5 minutes, at most. Perhaps "gradlew check" could include integration tests while "test" don't? I also know that Mark Miller could write 24 e-mails on this topic and how fast our tests could be. In fact, lightning fast I'm told. But let's not go there in this thread :) Progress over perfection. Jan
