+1 Dawid

I understand your point Yonik, but the practical consequences are worse
than disabling these tests like Erick pointed out in his initial emails.

If we are concerned about forgetting these disabled tests, which is a
concern I agree, I think Uwe's idea to add a weekly job that runs with
-Dtests.awaitsfix=true
is a good compromise.

Le jeu. 22 févr. 2018 à 07:55, Dawid Weiss <dawid.we...@gmail.com> a écrit :

>
> Don't know, Yonik... If I make a change I am interested in regressions
> from the start state; running flaky tests makes it impossible and
> frustrating (and pointless in my opinion). I don't think my expectations
> are that much off from the average - you may wake up being the only person
> who has the defaults enabled, which seems wrong.
>
> Dawid
>
>
>
> On Feb 22, 2018 01:55, "Chris Hostetter" <hossman_luc...@fucit.org> wrote:
>
>>
>> : * Hoss has worked on aggregating all test failures from the 3 Jenkins
>> : systems (ASF, Policeman, and Steve's), downloading the test results &
>> logs,
>> : and running some reports/stats on failures. He should be ready to share
>> : this more publicly soon.
>>
>> I think Steve's linked to some of this before from jira comments, but it
>> was only recently I realized i've never explicitly said to the list "Hey
>> folks, here's a thing i've been working on" ...
>>
>>   http://fucit.org/solr-jenkins-reports/
>>   https://github.com/hossman/jenkins-reports/
>>
>> The most interesting bit is probably here...
>>
>>   http://fucit.org/solr-jenkins-reports/failure-report.html
>>
>> ...but there are currently a few caveats:
>>
>> 1) there's some noise inthe '7days' data because I wasn't accounting for
>> the way jenkins reports some types of failure -- that will gradually clean
>> itself up
>>
>> 2) I think i've been been blocked by builds.apache.org, so at the moment
>> the data seems to just be from the sarowe & policeman jenkins failures.
>>
>> 3) allthough the system is archiving the past 7 days worth of jenkins logs
>> for any jobs with failures, there is currently no easy way to download
>> the relevant log(s) from that failure report -- you currently have to
>> download a CSV file like this one to corrolate the test failures to the
>> jenkins job, and then go look for that job in the job-data dirs...
>>
>>   http://fucit.org/solr-jenkins-reports/reports/7days-method-failures.csv
>>   http://fucit.org/solr-jenkins-reports/job-data/
>>
>> (My hope is to make #3 trivial from failure-report.html -- so you can say
>> "hey weird, this test has failed X times, let's go download those logs."
>> right from a single screen in your browser)
>>
>>
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Reply via email to