HI ben
you are right. We are all running like mad.
We are focusing on the bootstrap because guille will leave to brussels
end of the month.
Esteban is fighting with FFI to get migrating to Spur.
But yes this issue is a real one.
Stef
Le 23/8/15 04:08, Ben Coman a écrit :
Is there somewhere that statistics of failures from the CI validation
report are gathered?
We've all experienced transient CI-monkey validation failures
unrelated to our submitted fix, where just resubmitting to the monkey
without change works fine. The problem is:
* this may have been introduced several builds ago, but the transient
failure passed the monkey at the time
* it can be hard to identify which build introduced a transient error
* doubly hard if the failure is due to some CI infrastructure change,
or one particular build server
* its hard to get an overview of how many issues get the same failure
- is this a "once in six months" failure or is it happening 50% of the
time in every issue.
* it takes longer for an individual to observes a false-failure is
common enough to worry about, than if reports were merged. Since
different slices should produce different errors, failures common
across all slices should stand out in statistics.
If such statistics are not currently available, in cases where a
simple resubmission passed the monkey, should we:
* have an issue for each such false-failure, to get some early
indication from manually track occurrences
* where such an issue hasn't yet been created, first report comes to the list
------
So, I just got the following false-failures in separate runs for:
https://pharo.fogbugz.com/f/cases/16265
* ProcessTerminateBug>>#testTerminationDuringUnwind
* a CISmallIntRule: Timeout (after 0:00:40:00) occured while loading...
Anyone else?
cheers -ben
P.S. I understand some errors are filtered. Can this list be output as
part of the validation report (or is it already) ?
