I still think it’s a mistake to try and use all the Jenkins results to
drive ignoring tests. It needs to be an objective measure in a good env.
We also should not be ignoring tests in mass.l without individual
consideration. Critical test coverage should be treated differently than
any random
Alexandre:
Feel free! What I'm struggling with is not that someone checked in
some code that all the sudden started breaking things. Rather that a
test that's been working perfectly will fail once the won't
reproducibly fail again and does _not_ appear to be related to recent
code changes.
In
Just a completely random thought that I do not have deep knowledge for
(still learning my way around Solr tests).
Is this something that Machine Learning could help with? The Github
repo/history is a fantastic source of learning on who worked on which
file, how often, etc. We certainly should be
Shawn:
Trouble is there were 945 tests that failed at least once in the last
4 weeks. And the trend is all over the map on a weekly basis.
e-mail-2018-06-11.txt: There were 989 unannotated tests that failed
e-mail-2018-06-18.txt: There were 689 unannotated tests that failed
On 7/30/2018 11:52 AM, Erick Erickson wrote:
Is anybody paying the least attention to this or should I just stop bothering?
The job you're doing is thankless. That's the nature of the work. I'd
love to have the time to really help you out. If only my employer didn't
expect me to spend so
I was thinking of the challenge with sporadic/random failures the other day
and what would help. I think more and smarter notifications of failures
could help a lot.
(A) Using Git history, a Jenkins plugin could send an email to anyone who
touched the failing test in the last 4 weeks. If that
Hi Erick,
> Is anybody paying the least attention to this or should I just stop bothering?
I think your effort is invaluable, although if not backed by actions
to fix those bugs
it's pointless. I'm paying attention to the Lucene part. As for Solr
tests I admit I gave
up hope a longer while ago.
Steve:
Ok, InfixSuggestersTest.testShutdownDuringBuild is in my "Do not annotate" list.
On Mon, Jul 30, 2018 at 7:33 PM, Steve Rowe wrote:
> Hi Erick,
>
> I think it’s valuable to continue the BadApple process as you’re currently
> running it. I’m guessing most people will not engage, but
Hi Erick,
I think it’s valuable to continue the BadApple process as you’re currently
running it. I’m guessing most people will not engage, but some will, myself
included (though I don’t claim to read the list every week).
I’m working on fixing InfixSuggestersTest.testShutdownDuringBuild
