My $0.02 In my variant of helgrind I have a client request saying that we expect a race on a particular address. If the race is detected, the tool is silent. But if the race is *not* detected, the tool complains at the end.
This is not a substitute for the current testing scheme; it is quite orthogonal. The current scheme tests that the output matches the golden file (modulo addresses, tids, etc). My scheme is not applicable to all kinds of errors that valgrind tools may emit and does not check the exact output. But it is immune to changes in the test code and compiler optimizations (inlining). And it does not require any golden file which is useful when the test suite is under development. --kcc On Wed, Feb 27, 2008 at 2:24 PM, Ashley Pittman <[EMAIL PROTECTED]> wrote: > > On Wed, 2008-02-27 at 11:49 +0100, Julian Seward wrote: > > > Maybe I should have made myself more clear. The comment in > > > tests/vg_regtest.pl.in explains that stdout.exp files only can have a > > > numeric suffix ([0-9]*), while stderr.exp files can have any suffix > > > (*). If I understood the tests/vg_regtest.pl.in sourcecode correctly, > > > both stdout.exp and stderr.exp files can now have any suffix. Is this > > > correct ? > > > > I thought I only changed the stderr.exp to allow stderr.exp*, and > > left the others (stdout.exp, post.exp) unchanged. But I can't > > remember now. A bit of poking around with svn ann / svn log might > > make it clearer. > > When I was playing with post-processing of valgrind output a year or so > ago I did get as far as being able to run the regression tests through > my script which has the potential to make the tests simpler and more > robust. > > The basic scheme was to run valgrind in xml mode and have a output > filter to convert from xml back to "standard" output. The primary > purpose of this was for parallel jobs, to have a single output file per > job rather than per process but it also allowed you to pick and choose > which output you wanted to see, it became easy to suppress things like > line number and thread id's if required. Potentially each test could > have a single sample output file and a file listing which elements of > the output should be invariant. > > The code is basically there, the problem I had was a performance one, > for the input size I wanted XML::Simple was taking in excess of a minute > to parse the data, this wouldn't be a problem for the tests however. > > Of course all this might not help drd although it would be nice to see > the xml output extended to allow higher level tools to be built on top > of valgrind. > > Ashley, > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Valgrind-developers mailing list > Valgrind-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/valgrind-developers > ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Valgrind-developers mailing list Valgrind-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-developers
