Ole Solberg wrote:
Andrew McIntyre wrote:
On May 4, 2005, at 2:44 PM, Ole Solberg wrote:
We also build and test on a few platforms daily and could provide those results.
My level of ambition would be to just send out the results without any deep analysis. (Just catching and filtering obvious local setup/enviroment blunders etc.)
I think communicating daily regression test results could be a good way to present the state of Derby.
It's great to hear that other derby-dev'rs are building Derby nightly and running the tests! I think it would be a very good thing to be sharing test results, but I'm a bit concerned about sending them to derby-dev itself. I personally feel that nightly automated mail to the list would simply decrease the signal-to-noise ratio on the list and be a bit of a nuisance (and ultimately not likely to be read). But there are alternatives to sending nightly test results to the list, like posting them in a specific location on the Derby website, as I'm currently doing with the doc/javadoc build. Or we could have a page on the Derby website with links to locations where derby-dev'rs are publishing their test results.
I agree that that having a page on the Derby website linking to the actual test reports would be a much better solution than "polluting" derby-dev with lots of e-mails.
Anyone with neccessary rights willing to create such a page?
Again, I'm not sure why there would be "lots of emails" for a nightly build. Also, if we email failures, that would be great, as I know I personally won't be checking the test web site on a regular basis.
In an ideal situation, it would be great to have a tinderbox approach, as David suggested, with constantly active build/test cycles running. But there's always the complicated question of who's going to provide the hardware and put the box out on the net for all to see when going that route.
Which tests should be included in such an approach? What must the maximum turnaround time for this be to be considered useful?
Well, it seems to me you just have a cycle of "pull, build, full regresssions, post results." If we have a powerful enough machine, it shouldn't take that long, and we'd have fresh results every, say, four hours. Not bad!
If we want a faster turnaround, we could, as you suggest, identify a subset of tests.
andrew
-- Ole
