I'd suggest looking into KLEE "KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs"
http://llvm.org/pubs/2008-12-OSDI-KLEE.html http://llvm.org/pubs/2008-12-OSDI-KLEE.pdf http://hci.stanford.edu/cstr/reports/2008-03.pdf http://klee.github.io/klee/GetStarted.html Perhaps talk with Daniel Dunbar (who used to work on Blender, and helped write KLEE and works on LLVM). LetterRip On Wed, Mar 26, 2014 at 7:49 AM, Howard Trickey <[email protected]> wrote: > I agree with the sentiments about the value of automated tests. > I see the value every day in my day job. > > Blender has some automated tests, in spite of what that wiki link says. > See source/tests in the source tree, and the test target in the builds. > > Unfortunately, these cover only a small part of functionality (mostly I/O) > and have so lacked attention that more of them are failing than not right > now. > > I have started contributing tests for modeling here, but it would be great > if all of the developers could try to add tests and fix the failing ones. > > > On Wed, Mar 26, 2014 at 11:31 AM, Reed Law <[email protected]> wrote: > >> I was curious about the state of Blender's source code so I checked it out >> and after looking about I was surprised not to find automated tests for >> much of the C/C++ code. Then I came across this FAQ >> item< >> http://wiki.blender.org/index.php/Dev%3aDoc/FAQ#Why_don.27t_you_run_automated_tests.3F >> >that >> spells out the reasons for not running automated tests. None of the >> "reasons" hold water (and they should be called "excuses"). >> >> - Tests that require user interaction are called "integration" or >> "end-to-end" tests. If all of the underlying units are well-tested and >> the >> UI is just a thin layer then this type of testing can be kept to a >> minimum. >> - Tests that take too long to run indicate tightly coupled code. >> - Tests taking too much effort to maintain is an illusion. I would bet >> *not* having tests requires much more effort and leads to the problem >> described in The Mythical Man >> Month<https://en.wikipedia.org/wiki/The_Mythical_Man-Month>: >> >> - The fundamental problem with program maintenance is that fixing a >> defect has a substantial (20-50 percent) chance of introducing >> another. So >> the whole process is two steps forward and one step back. >> - Tests covering areas that hardly ever change is a *good* thing. They >> can help a new contributor understand what the code is doing by showing >> examples of the interface for those classes. >> - Having tests for areas that are immediately noticeable when broken is >> also good because it saves that extra step of noticing the breakage, or >> of >> getting a bug report from someone else. >> - Tests that break but are not bugs should be updated to reflect the new >> features/interface. >> - Tests that break but are not a problem for the end user should also be >> refactored. What is the use case being tested in that case? >> - Although many Blender developers are volunteers, tests can be enforced >> by only merging pull requests that have good test coverage and pass the >> automated test suite run on a continuous integration server for each >> commit. >> >> I realize that the solution to these problems is found in the call for help >> at the bottom of the FAQ item. But there is only so much a single >> contributor can do, especially in light of a long history of untested code. >> Perhaps setting up a continuous integration server with a service such as >> Travis CI would be a first step. >> _______________________________________________ >> Bf-committers mailing list >> [email protected] >> http://lists.blender.org/mailman/listinfo/bf-committers >> > _______________________________________________ > Bf-committers mailing list > [email protected] > http://lists.blender.org/mailman/listinfo/bf-committers _______________________________________________ Bf-committers mailing list [email protected] http://lists.blender.org/mailman/listinfo/bf-committers
