Martin Sebor wrote: > >> Another option would be to continue testing these sections, but >> separate them out into their own tests. > >Right. That's the solution I keep ending up with as the most robust >one. The main advantage I see is minimizing the scope of the tested >functionality. The biggest disadvantage is the significant overhead >of building and running twice as many (thread safety) tests as we >have now.
If we had one test to verify that method foo() was mt-safe, and another test to verify bar(), we still need a test that foo() and bar() are mt-safe when used together. So that's another test to have. And it brings us back to just testing all of the tests into one again. Duplicate code or file dependencies could be a problem also. >We could eliminate some of the overhead due to building >these tests by building just one binary and rigging our test driver >to invoke it two or more times with different options (I like this >feature in and of itself a lot) but I fear there is little we can >do to deal with the overhead of running the test(s) multiple times. > Are you talking about modifying the nightly build infrastructure so that it will invoke a test several times with different command line arguments to run the test differently, or are you thinking about having the test invoke itself with different arguments or something? I don't think any option other than the first will work. Travis
