The HEAD now has a 'validate' script at the top level, which you can use for
testing that patches in your tree pass minimal testing. It won't work on
Windows yet (no args are passed to configure), and parallelism is fixed at -j2.
'make fast' in the testsuite now runs with zero failures on x86-64/Linux. We
should make this the case on other platforms as a high priority. When we hit
zero failures on most platforms, we can start to announce the new testing regime
for patches to the HEAD.
In the meantime I'd urge people to start using the validate script and switch to
using separate "developing" and "testing" source trees. I've been doing that
for a couple of days now, and it seems to be working quite well.
I did some tests on build times, here's a quick summary. These times are on a
1.2GHz Core2 using 1 CPU only, bootstrapping using GHC 6.6.1, with the "quick"
settings, core packages only:
- 6.6.1: 17.7 minutes
- HEAD: 19.0 minutes
Looking in more detail it seems that the difference is mostly in the libraries,
although I'd like to do some more analysis. It's not clear cut because we used
to do a single configure step at the top, and now we configure the libraries
later in the build. Also HEAD has fewer libraries in core-packages than 6.6.1.
A full validate on a fast 2-core machine should take about 20 minutes. We can
get that down to 15 minutes or so using THREADS=2 for the testsuite, but
currently a few tests fail when I do that (due to bogus ordering requirements,
presumably).
Cheers,
Simon
Simon Marlow wrote:
I've just been chatting with Simon about this, and basically we're
willing to try a more rigorous testing regime for changes to the HEAD.
Yesterday Ian and I uncovered a problem with the "known buildable"
repositories idea: the build slave has a partial repository, so it can't
necessarily push to the buildable repository because it doesn't
necessarily have the patches it needs to push. We could solve this by
giving the builders full repositories, but it's yet more fiddling around
with the buildbot setup.
So we're thinking of putting this idea to one side for now, and trying
the suggestions from this thread instead. The choices aren't mutually
exclusive, so we can still do the "known buildable" repositories later
(when darcs' lazy partial repositories are available it'll be much easier).
Here's our draft game plan. There would be a minimal testing
requirement for patches to HEAD, consisting of
- a 2-stage bootstrap, probably with the "quick" settings, core libs only
- a 'make fast' in testsuite, with zero failures
in addition, if you make changes that might break something that isn't
tested by the above (multiple platforms, older versions of GHC,
profiling, unregisterised, GHCi, binary distributions, HPC etc.) then it
is your responsibility to make a reasonable attempt to test it. We'll
only hand out the pointy hats to people who fail to do minimal testing,
though.
To make testing easier, we'll add an automated 'validate' script which
does the complete build:
- make distclean
- sh boot
- configure
- put an appropriate build.mk in place
- make
- make -C testsuite fast
and if it gets all the way through and reports zero failures, you're ok
to push.
The standard workflow would be to keep two trees: one for developing in,
and one for testing. You record patches in developing, and occasionally
pull into testing and run 'validate'. When it succeeds, you push from
testing. Of course you can have multiple developing trees and a single
testing tree. Your developing trees can lag behind HEAD, but the
testing tree should be up to date with HEAD before running validate.
Right now the fast testsuite doesn't have zero failures (it has about
20), we'll need to get that down to zero first. The fast testsuite
currently runs in about 8 minutes on a fast Linux machine; it was about
5 minutes when I first did it, so we probably need some tuning there. I
have no idea why the build is currently slower than it was (indeed I
wasn't aware that it was), so we should look into that too.
Comments?
Cheers,
Simon x 2
_______________________________________________
Cvs-ghc mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/cvs-ghc
