-----BEGIN PGP SIGNED MESSAGE----- Moin,
On Monday 20 October 2003 18:35, Tony Bowden wrote: > On Mon, Oct 20, 2003 at 08:34:50AM -0700, Ovid wrote: > > the suite of 15,000 tests takes about an hour to run. > > Devel::Cover issues aside, has anyone else looked at interesting ways of > making test suites run faster? > > We insist on a full regression test on checkin, so having the test suite > take a long time to run puts people off making small changes. :( > > On one project a full regression test takes about 8 minutes, which is at > least an order of magnitude longer than I'd like. I couldn't dream of > having it take an hour! > > Interestingly, a lot of this time is due to the fact that we front-load > a lot of our stuff to compile time (dynamic class generation etc.) as > it's a web-app. This works well under mod_perl, but not so well under > testing as just loading the modules to be tested takes a while - which > is noticable when it has to happen again for each different test file. > > So having some way to have that not need to happen for each test file > would be nice. > > Alternatively we looked at running all the test files in parallel > rather than sequentially. This was looking promising, but we never got > it finished. > > I'm curious as to whether anyone else is doing anything interesting > here... Basically I do: * "optimize" the tests as to not run so many ok() (e.g. run 1000 tests, output on ok(), or the internal test number that failed), although I avoid this "optimization" since it is not so important * optimize the code to be faster (this is a winner) * run the tests in the background, e.g. run them, while working in the foreground and later see if they broke * run the tests only a few times a day. After all, it only matters if the daily build has them running ok or not, not if each change I do breaks the test or not. Some people believe that each individually change must be regression tested, but where do you draw the line? Chaning a file, a line, a word, a comma, a character? I tend to retest after implementing a "feature", e.g. anything that takes minutes to hours to complete, but is still done before the daily build get's pushed out of the door. Oh, I only "update" the dailys build if all tests pass, meanign I can stop working anytime and still have the "latest" daily build which passes all test. Ah, ignore my ramblings... Cheers, Tels - -- Signed on Mon Oct 20 18:51:24 2003 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. "Man, I'm hot." - "Thirsty?" - "No, I mean good looking." -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2-rc1-SuSE (GNU/Linux) Comment: When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. iQEVAwUBP5QSjXcLPEOTuEwVAQFAIAf9FTuz6isjW4n9W+N/5JRif3V9W5LFcKsv pKY++5DiV7PpwyvJwvrqMtk7bMTRpFUAQeX6G6uDM79Kk9gUKJ3CnGR2DTnwIt+R 4eGmyhcr3uZxvURxFFPi0fRYI13If6tqJwbKC0ATHUCrWFnUozJfQlm2HoVD2/Yq wXk2Xw3xHAsrgPLAnJgz3dpoglKLXPdUP1JDLVytRsZIdoTyibWm0b6EtGVlqvE8 gjAvmPkvjVPQV6WQEv5i3mY0ZsHz7cw5E4CdChXnk19yIiQ6/b1w+v8c9SbhFoSj z4LK/DYHksvgfEWgzV2aFcGkEVe4N7x7KvvDe+NmGThma468IveI9Q== =rzUK -----END PGP SIGNATURE-----
