[
https://issues.apache.org/jira/browse/SOLR-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384036#comment-16384036
]
Hoss Man commented on SOLR-12028:
---------------------------------
I've very confused by some of the changes made under the banner of this issue,
and since SOLR-12016 was evidently deleted, it's hard to find the "long
discussion of this topic" to see if it's explained there (i could have tried to
piece it together from mail archives, but instead i'm just going to take the
summary in *this* jira at face value) ...
I draw attention to this comment...
{quote}This JIRA is a placeholder for BadApple tests to point to between the
times they're identified as BadApple and they're either fixed or changed to
AwaitsFix or assigned their own JIRA.
{quote}
...and yet, when i look at {{ZkControllerTest.testPublishAndWaitForDownStates}}
I can not wrap my head around the changes made to that test as part of this
issue.
Prior to the creation of SOLR-12028...
* {{ZkControllerTest.testPublishAndWaitForDownStates}} was disabled using
{{AwaitsFix}}
* The {{AwaitsFix}} anotation pointed to SOLR-7736 which was an OPEN issue
because the test was known to be broken
** admittedly, the issue didn't have much in the description, but it was
explicitly open knowing that this test was currently broken
* As a result, this completely broken test was never being run and never
failing.
As a result of SOLR-12028...
* SOLR-7736 is now marked "Resolved:Duplicate"
** even though there is no "Duplicates" link to any other jira ??????
* The {{AwaitsFix}} annotation was replaced by a {{BadApple}} annotation which
links here to SOLR-12028 *and* to SOLR-7736...
** even though the guidelines of both annotations (as i've always understood
them) are that they aren't suppose to link to closed issues???
* so now this test is running for all developers, even though it was known in
advance to be broken and "awaiting fix"
In short: this test already met the "end goal" of "either fixed or changed to
AwaitsFix or assigned their own JIRA." and yet now it's in a worse state then
when we started...
*WAT?*
What am i not understanding about this process? what is the point of mucking
with tests like these that were already {{AwaitsFix}} pointing to *OPEN*
issues???
> BadApple and AwaitsFix annotations usage
> ----------------------------------------
>
> Key: SOLR-12028
> URL: https://issues.apache.org/jira/browse/SOLR-12028
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Priority: Major
> Attachments: SOLR-12016-buildsystem.patch,
> SOLR-12028-sysprops-reproduce.patch, SOLR-12028.patch, SOLR-12028.patch
>
>
> There's a long discussion of this topic at SOLR-12016. Here's a summary:
> - BadApple annotations are used for tests that intermittently fail, say < 30%
> of the time. Tests that fail more often shold be moved to AwaitsFix. This is,
> of course, a judgement call
> - AwaitsFix annotations are used for tests that, for some reason, the problem
> can't be fixed immediately. Likely reasons are third-party dependencies,
> extreme difficulty tracking down, dependency on another JIRA etc.
> Jenkins jobs will typically run with BadApple disabled to cut down on noise.
> Periodically Jenkins jobs will be run with BadApples enabled so BadApple
> tests won't be lost and reports can be generated. Tests that run with
> BadApples disabled that fail require _immediate_ attention.
> The default for developers is that BadApple is enabled.
> If you are working on one of these tests and cannot get the test to fail
> locally, it is perfectly acceptable to comment the annotation out. You should
> let the dev list know that this is deliberate.
> This JIRA is a placeholder for BadApple tests to point to between the times
> they're identified as BadApple and they're either fixed or changed to
> AwaitsFix or assigned their own JIRA.
> I've assigned this to myself to track so I don't lose track of it. No one
> person will fix all of these issues, this will be an ongoing technical debt
> cleanup effort.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
