Mike Matrigali wrote:
I continue to really appreciate the information in these reports!
I don't know if the following is easy or hard, but thought I would
ask. I just spent some time trying to track down from which change
in the tests did a reoccuring regression come from. This would seem
the last piece of automated information that you could provide in
your current automatic "attempted failure analysis". I just spent
time doing this by hand which involved going to the full history page
and then doing binary search clicking down to into the details a
couple of levels and seeing when the bug started and if it was
consistent on that particular platform.
It would help a lot if the current report said something like:
test failed on xxx platform N out M times, first encountered in
test report XXX with svn changes YYY.
I have had the same wish, and already have the basic mechanisms in
place. I will look into integrating it with the current reporting.
For instance I was trying to do some analysis on the stress memory
heap issue, and found:
test failed on linux platform 100% time (actually I didn't check
every run, just did a few probes), first encountered in
http://www.multinet.no/~solberg/public/Apache/Derby/Limited/testSummary-370210.html
3 changes happened in that test run:
http://www.multinet.no/~solberg/public/Apache/Derby/UpdateInfo/370210.txt
Of which I would look at 370059 as it makes an ij change with respect to
muti-threaded ij.
If the reports automatically recorded this kind of info it probably
would be less likely for regression issues to sit around.
--
Ole Solberg, Database Technology Group,
Sun Microsystems, Trondheim, Norway
