On Mon, Mar 02, 2009 at 02:38:22PM -0800, Christoph Otto wrote: > Patrick R. Michaud wrote: >> On Mon, Mar 02, 2009 at 12:22:23PM -0800, chromatic wrote: >>> We also need tests for *every* bugfix for HLL breakage. Every time >>> our existing test suite doesn't catch this problem, we know it's >>> incomplete. I didn't see a test added for r37061. >> >> To be honest, I don't think any of us know what caused the r37061 bug >> in the first place to be able to formulate it into a test. > > Correct. The bug was not exposed by Parrot's make test, but only by > Rakudo's build. > > Better HLL testing strikes me as a good idea. However, it will also > significantly increase the time it takes to test a change, especially as > other HLLs are developed. Is there a good rule of thumb for which tests > should be run when testing a change or bugfix, or is the advice "run > rakudo's spectest" every time?
I don't think it's practical (or necessary) to run the HLL tests for every commit to Parrot. But I do think it's important that we test Parrot against a standard set of HLLs prior to a major release (substitute your desired value of "major" here), if only to make sure that the about-to-be-released-major-Parrot isn't the source of any huge blockers for the HLLs. I've gone ahead and created TT #399 for this, as well as placed it as a "critical" item on the ParrotRoadmap page, so that this isn't overlooked or forgotten at the time of the release. I might also suggest we add the step to the release_manager_guide.pod document. Pm _______________________________________________ http://lists.parrot.org/mailman/listinfo/parrot-dev
