I think we need to have a strategy for managing the amount of time required for running the unittests. Currently, on Windows, the time for all compiler tests + phobos tests is one hour. (The druntime tests are only 30 seconds). By contrast, running all compiler tests + phobos tests on D1, takes about four minutes.
I used to be able to use test-driven development extensively -- running all tests after every change. But this is no longer viable on D2. It's quite terrible to have to wait for one hour before finding that you broke something. A very large fraction of the time is used in testing only a tiny part of the Phobos API. So I have some anxiety over what may happen in the long term -- if current trends continue, we could easily have ten hours of Phobos tests eventually. How about defining a version, eg: version=ExtendedPhobosTests; which contains the more exhaustive, black-box tests, which take almost all of the time. So that the standard tests would consist of (1) regression tests, which should have a corresponding bugzilla entry, unless they were discovered during development of the library (The feature of these tests, is that at some point, they failed); and (2) code coverage tests. Both of these execute quickly, and since they are linear with the number of reported bugs + the number of lines of code, they should always remain manageable. The black-box tests, on the other hand, are potentially unlimited. _______________________________________________ phobos mailing list [email protected] http://lists.puremagic.com/mailman/listinfo/phobos
