On Mon, Apr 13, 2009 at 7:46 AM, aslak hellesoy <aslak.helle...@gmail.com> wrote: > > So if someone develops a better AutoTest with a plugin architecture, and > that doesn't have to run as a long/lived process, then I'd be very > interested in writing the neural network part - possibly backed by FANN > (http://leenissen.dk/fann/)
In the immortal words of Socrates, "That rocks." The nice thing about separating concerns like this -- the reason why design patterns appeal so much to me -- is that pieces can rather easily be built up incrementally. As Matt said, thinking 'neural networks' with this is way beyond the level of anything I'd had in my head. But it's a damn cool idea. I can think of a few ways to handle this particular chunk, with levels of complexity that would scale up: 1.) DUMB REACTION: Just re-run the tests that failed in this run cycle. Then, periodically, run tests that succeeded in the background as a regression check. (This isn't much beyond what Autotest does now.) 2.) OBSERVERS: Allow handlers to register themselves against certain files, so that when a file is changed, that handlers gets run. Multiple handlers can observe any given file, and a handler can declare multiple rules, including directories or pattern matches. (Again, Autotest has something _sort of_ like this, but not nearly as flexible.) 3.) PERSISTENCE: Track the history of tests and the times they were created, edited, last run, and last failed. Also track file modification times. When a file changes, run the tests first that are either new or have failed since the last time the file was changed. Then run the tests that went from failing to passing in that time. (This could certainly be improved -- I haven't sat down to figure out the actual rule set -- but you get the gist. Know when things changed and set priorities accordingly.) 4.) INTELLIGENCE: Aslak's neural network. Let the system figure out which tests matter to which files, and run what it thinks it ought to run. Maybe use code coverage analysis. It can 'learn' and improve when the full suite is run and it discovers new failures. In all four of these cases, I still think it's imperative to run the full suite. None of these methods are foolproof, and code is tricky and makes weird things happen in weird crevices. That's _why_ testing must be done. But running the suite doesn't have to be a 'blocking' activity, like it is with Autotest now. It can happen in bits and pieces, when nothing else is going on, and it can be configured to only grab your attention when something failed unexpectedly. (That's one of the prime reasons for the 'multiple output views' feature, by the way. When I said output views I wasn't just thinking of the console or a window. I'm also thinking Dashboard widgets, or gauges in the toolbar, or RSS feeds, or dynamic wallpaper, or whatever else anyone can think of. Stuff that stays out of your way until it either needs you or you choose to look at it.) Still making sense? This is starting to sound pretty big and pretty complex -- but I don't think it strictly needs to be. #1 and #2 above are pretty easy. The others don't have to be built before releasing something. And, of course, you wouldn't have to pick just one selection module. You could run or disable all of these depending on your project's needs, or come up with your own system involving Tarot cards and Linux running on a dead badger.(*) I just want to build a core that detects changes in stuff, tells other stuff about it, and passes on what that stuff says about it to a third set of stuff. The rest is implementation-specific details. >8-> (* http://www.strangehorizons.com/2004/20040405/badger.shtml ) -- Have Fun, Steve Eley (sfe...@gmail.com) ESCAPE POD - The Science Fiction Podcast Magazine http://www.escapepod.org _______________________________________________ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
