[
https://issues.apache.org/jira/browse/SOLR-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13913112#comment-13913112
]
Robert Muir commented on SOLR-5775:
-----------------------------------
I dont think any test coverage is lost. In an open source project, if nobody
can reliably tell if a test failure is "ok" or not (except maybe a few people),
then the test coverage is 0%. Furthermore, lucene's coverage is suffering
because of these bad apples: the recent javadocs failures are not really recent
at all, its just that jenkins was failing so much in solr tests that it never
got that far.
When a test fails consistently: I think it needs to be disabled. I can't safely
commit anything to the solr codebase because the tests fail every time on my
machine: and I know its not unique to me, jenkins is completely red.
> Disable constantly failing solr tests
> -------------------------------------
>
> Key: SOLR-5775
> URL: https://issues.apache.org/jira/browse/SOLR-5775
> Project: Solr
> Issue Type: Bug
> Components: Build
> Reporter: Robert Muir
>
> Currently, solr tests are failing 90%+ of the time. We've been through this
> before many times, the argument is always that someone is looking at the
> failures and knows which ones are bad.
> This argument is a lie. Nobody is watching these failures, or
> DistributedQueryComponentOptimizationTest would not have failed repeatedly
> for two straight days when the fix was trivial (I fixed this last night:
> http://svn.apache.org/r1571930)
> Its frustrating to me as a committer, solr tests *NEVER* pass on my machine,
> no matter how many times I try. How can i possibly commit something without
> knowing i am making the situation even worse?
> This is all a big problem for developers, release managers, even users of the
> project. The test suite should pass.
> The old argument that "solr tests are allowed to fail" is no longer valid. I
> will disable all constantly failing tests.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
