On Wed, Jan 23, 2013 at 4:37 PM, Karl Rupp <rupp at mcs.anl.gov> wrote:
> Dear PETScians, > > I've adopted the HTML summary page script for the style checks a little in > order to have a quick overview of the results of our nightly tests: > > http://krupp.iue.tuwien.ac.at/**petsc-test/<http://krupp.iue.tuwien.ac.at/petsc-test/> > It doesn't do anything fancy, particularly no automatic bisection or the > like, but it gives an overview of the Nightly results in seconds (which are > not overly good at the moment...). > Yep, this is nice. > Assuming that there is no question about the need for improving our > current automatic testing environment, it's merely a question of how far we > want to go. Regarding overall productivity, I made good experiences with > just using CTest and setting the summary page as the start page of my > browser. It's not overly complicated an can be integrated with reasonable > effort into our CMake build system. > > We could also do all the fancy things provided by continuous integration > systems. Sadly, I don't have any experience on that front. Regardless, I > still prefer rather simple systems which are in daily use and fairly robust > rather than a complex, > it-can-do-anything-for-you-if-**you-find-the-right-switch > system that is never set up... > > Any thoughts/experiences? I know Jed has good plans on a testing > environment, so this is not meant to be an assault on him ;-) > I am always skeptical of big programs to overall a large piece of infrastructure that works fairly well. However, there is a really simple thing that we need which would make us much much much better at using our own tests. We need a better way to test numerical output. I am not sure what the right thing to do is, or I would have already done it. The current best solution is to print fewer digits, which judging from the HTML page is not sufficient. I think that current PETSc output is so stylized that parsing output is feasible, and would allow nice diffs with tolerances tailored to the type of output and individual test. Matt > Best regards, > Karli > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130123/87630e25/attachment.html>
