On Sat Jul 12 09:33:35 2008, coke wrote: > > Another solution here would be to not run them by default. The purpose > of 'make test' > should be to verify that the parrot functionality works on the target > system.
If speed is your concern, you can call 'make coretest'. We've had that functionality available since r23209 (2007-11-28). That lops off not only the pre- and post-configuration tests, but the coding standards tests as well. I've used that many times, and it appears that some of our smoke testers are set up for that as well. particle's original post was one I requested he make as follow-up to a discussion that he, chromatic and I had concerning configuration at YAPC::NA::2008 in Chicago (with additional input from Steve Lembark). That discussion concerned both how Parrot configuration might evolve and how it should be tested. particle's specific comments about processes on Win32 were the first good argument I had heard for fewer (but inherently more sprawling) test files rather than more (but more sharply focused) test files. Testing every aspect of a configuration step class in a single file is made difficult (as I have explained previously on this list) by the fact that the Parrot::Configure object is a singleton. You get one such object per file. Cet. par., if you want to test all the different command-line options and OS variants relevant to a particular configuration step, you need multiple P::C objects and hence multiple files. I have, however, begun to develop a way of running tests for a particular step class, then gutting the P::C object's data structure and restoring it to its state at an earlier point in the test file. This will permit consolidation of multiple test files within a single file. I've begun that work in the 'parallel' branch in our repository. (Anyone care to join me?) There are a number of problems involved in doing this -- the inter::* steps are, predictably, proving troublesome, since you have to simulate command-line interaction -- so it's not ready for trunk quite yet. Far from being adamantly opposed to pulling the pre- and post-configuration tests out of 'make test', I've longed hope that we could do that. After all, I wrote tests for the build tools which are not in 'make test' (and which are not intended to be run there). But I don't think we're there quit yet. The build tools themselves are largely stable code on which few Parrot developers are currently working. The configuration steps, in contrast, are still in a considerable state of flux. When I started writing tests, we had 56 steps; we now have 68 steps. In the short run, that calls for a higher level of visibility. A number of ideas concerning configuration and testing came out of the YAPC discussion mentioned above. I'll eventually open RTs focused on each of them, but particle has indicated that a thorough reimagining of the Parrot configuration process will have to wait until we've achieved several more of our pre-1.0 milestones. And since particle described the particular pain of running tests on Win32 (not just configuration tests, I presume), I've elected to work on that objective first. Thank you very much. kid51
