Re: legitimate parallel make check?
IainS develo...@sandoe-acoustics.co.uk writes: that's really neat indeed - I should have spotted the potential looking at the code in contrib ... although the site.exp is hardwired in btest.sh at present ; I guess one might be able to use .dejagnurc - just need to check on the order that the files are processed. Far easier: just set the DEJAGNU environment variable to the absolute path to the size.exp file. runtest knows about that. hmm. I see that the make check would work using that neat method above... ... however, 1/ I want a different language list than the default... 2/ also some other different configure options... I've got some local patches for btest-gcc.sh which allow just that (e.g. support for Ada testing), which I'll submit once I'm reasonably confident they are general enough. but mainly 3/ ... As I read it - btest.sh strips everything but the testname from the PASSES and FAILS files. (the awk lines that print $2 for a match of /FAIL: / etc.) This means (e.g.) that if a test passes at m32 and fails at m64 I think it will appear in both PASSES and FAILS.. ... this seems destined to result in confusion... You can still use make mail-report.log in the build subdir to get the complete summary; besides, there's still contrib/dg-cmp-results.sh (which I haven't checked out yet). Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University
Re: legitimate parallel make check?
On 10 Mar 2010, at 09:12, Rainer Orth wrote: IainS develo...@sandoe-acoustics.co.uk writes: that's really neat indeed - I should have spotted the potential looking at the code in contrib ... although the site.exp is hardwired in btest.sh at present ; I guess one might be able to use .dejagnurc - just need to check on the order that the files are processed. Far easier: just set the DEJAGNU environment variable to the absolute path to the size.exp file. runtest knows about that. the DEJAGNU env. var is set in btest.sh hmm. I see that the make check would work using that neat method above... ... however, 1/ I want a different language list than the default... 2/ also some other different configure options... I've got some local patches for btest-gcc.sh which allow just that (e.g. support for Ada testing), which I'll submit once I'm reasonably confident they are general enough. OK - (but I'm happy that my local mods for this are also working). but mainly 3/ ... As I read it - btest.sh strips everything but the testname from the PASSES and FAILS files. (the awk lines that print $2 for a match of /FAIL: / etc.) This means (e.g.) that if a test passes at m32 and fails at m64 I think it will appear in both PASSES and FAILS.. ... this seems destined to result in confusion... You can still use make mail-report.log in the build subdir to get the complete summary; besides, there's still contrib/dg-cmp-results.sh (which I haven't checked out yet). contrib/compare_tests is good (I was thinking to merge the capability of that with the capability of btest.sh) meetings today... so until later I can't find a hardware fault -- but I'm exploring a hunch about a FBKAC Iain
Re: legitimate parallel make check?
On 10 Mar 2010, at 09:22, IainS wrote: Far easier: just set the DEJAGNU environment variable to the absolute path to the size.exp file. runtest knows about that. the DEJAGNU env. var is set in btest.sh my mistake. duh - that's what comes of working with code that was forked a long time ago .. .. it isn't set in the current version.. Iain
Re: legitimate parallel make check?
IainS develo...@sandoe-acoustics.co.uk writes: Far easier: just set the DEJAGNU environment variable to the absolute path to the size.exp file. runtest knows about that. the DEJAGNU env. var is set in btest.sh It's not: $ grep DEJAGNU contrib/regression/btest-gcc.sh # DEJAGNU: should point to a site.exp suitable for testing I've got some local patches for btest-gcc.sh which allow just that (e.g. support for Ada testing), which I'll submit once I'm reasonably confident they are general enough. OK - (but I'm happy that my local mods for this are also working). Without doubt, but why should everyone trying to use this have to reinvent the wheel? Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University
Re: legitimate parallel make check?
FWIW; the bus errors were consistently coming from expect in a strcpy [about 100 tcl levels down] the actual fault was a corrupted dejagnu installation - it's not clear how that happened - nothing present in syslogs to indicate disk problems. a clean install of dejagnu appears to have cleared the problem. On 10 Mar 2010, at 09:47, Rainer Orth wrote: IainS develo...@sandoe-acoustics.co.uk writes: I've got some local patches for btest-gcc.sh which allow just that (e.g. support for Ada testing), which I'll submit once I'm reasonably confident they are general enough. OK - (but I'm happy that my local mods for this are also working). Without doubt, but why should everyone trying to use this have to reinvent the wheel? indeed, I have every intention of sharing my changes too (at least those that are of any merit). thanks for the help, and apologies that it was noise in the end... Iain
Re: legitimate parallel make check?
IainS develo...@sandoe-acoustics.co.uk writes:
on my mere 8 cores :-) I'm still getting sporadic Bus errors on:
make -k -j8 check RUNTESTFLAGS ...
[from the command line on a bootstrapped clean trunk @157307]
so there's something else wrong somewhere...
Probably make gets SIGBUS, which obviously won't be in the runtest output.
... I guess a target-related issue ({i686,powerpc}-apple-darwin9)
Or simply a hardware issue?
Rainer
--
-
Rainer Orth, Center for Biotechnology, Bielefeld University
Re: legitimate parallel make check?
On 9 Mar 2010, at 12:10, Rainer Orth wrote: IainS develo...@sandoe-acoustics.co.uk writes: Am I trying something that is unsupported - or is this is a bug? === make -k -j8 check is not particularly helpful (a) because it tests gmp/mpfr/mpc every time (b) any redirected output is hard to scan for problems. What you're trying is far too complicated: just use -k -jN check from the toplevel and forget about the make check output, which, as you say, is interleaved and useless, just like the make -jN bootstrap output. OK. Instead, run make mail-report.log afterwards and check that. Although, I note that contrib/test_summary only reports what gets into the *.sum files -- i.e. tests that complete or timeout. It doesn't log problems in the actual make process itself - to find those one still has to trawl through the interleaved stuff. It would be nice to allow the apparently independent targets [e.g. gcc- c,fortran,c++ etc.] to be (explicitly) make-checked in parallel. thanks, Iain
Re: legitimate parallel make check?
IainS develo...@sandoe-acoustics.co.uk writes: Instead, run make mail-report.log afterwards and check that. Although, I note that contrib/test_summary only reports what gets into the *.sum files -- i.e. tests that complete or timeout. It doesn't log problems in the actual make process itself - to find those one still has to trawl through the interleaved stuff. No, you haven't. You need to look at the corresponding *.log files instead; most of this isn't in the regular make check output anyway. It would be nice to allow the apparently independent targets [e.g. gcc- c,fortran,c++ etc.] to be (explicitly) make-checked in parallel. This is exactly what happens when you invoke make -jN -k check, provided that the parallelism specified is big enough (and the machine can handle the load). Otherwise, you may end up running (say) the c testsuite in parallel, which isn't bad either. As I wrote before, I'm going to use this on an (effectively) 64-core machine and hope to achieve to use all cores in parallel :-) Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University
Re: legitimate parallel make check?
On 3/9/2010 4:28 AM, IainS wrote: It would be nice to allow the apparently independent targets [e.g. gcc-c,fortran,c++ etc.] to be (explicitly) make-checked in parallel. On certain targets, it has been necessary to do this explicitly for a long time, submitting make check-gcc, make check-fortran, make check-g++ separately. Perhaps a script could be made which would detect when the build is complete, then submit the separate make check serial jobs together. -- Tim Prince
Re: legitimate parallel make check?
On 9 Mar 2010, at 12:46, Rainer Orth wrote:
IainS develo...@sandoe-acoustics.co.uk writes:
Instead, run make mail-report.log afterwards and check that.
... snip
suite in parallel, which isn't bad either.
As I wrote before, I'm going to use this on an (effectively) 64-core
machine and hope to achieve to use all cores in parallel :-)
on my mere 8 cores :-) I'm still getting sporadic Bus errors on:
make -k -j8 check RUNTESTFLAGS ...
[from the command line on a bootstrapped clean trunk @157307]
so there's something else wrong somewhere...
... I guess a target-related issue ({i686,powerpc}-apple-darwin9)
(note that this error does *not* get into the .log files, AFAICS)
Iain
Re: legitimate parallel make check?
IainS develo...@sandoe-acoustics.co.uk writes: Am I trying something that is unsupported - or is this is a bug? === make -k -j8 check is not particularly helpful (a) because it tests gmp/mpfr/mpc every time (b) any redirected output is hard to scan for problems. What you're trying is far too complicated: just use -k -jN check from the toplevel and forget about the make check output, which, as you say, is interleaved and useless, just like the make -jN bootstrap output. Instead, run make mail-report.log afterwards and check that. Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University
Re: legitimate parallel make check?
On 9 Mar 2010, at 19:13, Janis Johnson wrote: To run all of the compiler tests in parallel you can do make -jn -k check-gcc from the top level and let the existing build machinery take care of running chunks of tests in parallel and putting the results back together. that's fine (I couldn't see why not from looking at Makefile.in) (maybe Rainer is right and I have a memory fault on my 8-core box .. :-( ) -- just backing up prior to a single user check of that ... .. I don't seem to get the bus errors on a 4CPU g5 or a Core 2 duo .. but .. the 8-core machine is faster .. so ... race conditions are more likely to manifest there. You'd still have to handle the library tests you want to run. I assume that make -jn -k check-target RUNTESTFLAGS is equally reasonable? Perhaps what you want is a way to skip the host library testing, although if you're building them with your new compiler then testing them is a good part of testing that compiler. I do build gmp/mpfr/mpc in-tree... given that I almost always run the whole testsuite - I'm torn between thinking it's a good idea.. ... and that I'm essentially proving little when the versions of gmp/ mpfr/and mpc are static. -- FWIW: I'm trying to update contrib/btest.sh script to handle m32 m64 plus a few other things ... Iain
Re: legitimate parallel make check?
On Tue, Mar 9, 2010 at 2:27 PM, IainS develo...@sandoe-acoustics.co.uk wrote: I do build gmp/mpfr/mpc in-tree... How? Last I tried, it didn't work, as mpc used the system gmp/mpfr, not the just-built in-tree versions. Therefore, it's not really an in-tree build, and you can't build on a system that doesn't already have gmp/mpfr installed. At least, that's what it was... does that work now?
Re: legitimate parallel make check?
IainS develo...@sandoe-acoustics.co.uk writes:
.. I don't seem to get the bus errors on a 4CPU g5 or a Core 2 duo .. but
.. the 8-core machine is faster .. so ... race conditions are more likely
to manifest there.
But race conditions don't manifest themselves in make SEGVs ;-( I'm
regularly running make -k -j128 on a T5220, or -j32 on a Sun Fire X4450
with 4 4-core CPUs, without any problems.
You'd still have to handle the library tests you want to run.
I assume that make -jn -k check-target RUNTESTFLAGS is equally
reasonable?
Indeed, as well as using a site.exp and pointing the DEJAGNU environment
variable at it, as I've learned from looking at the regression tester
sources in contrib/regression. Here's what I use for that:
global target_list
case $target_triplet in {
{ i?86-*-solaris2.1[0-9] } {
# FIXME: Disable multilib testing if the host cannot execute AMD64
# binaries. Should be exceedingly rare now (cf. erebus).
set target_list { unix{,-m64} }
}
{ mips-sgi-irix6* } {
# FIXME: Disable multilib testing if the host cannot execute N64
# binaries. We cannot selectively disable only one multilib during
# the build.
set target_list { unix{,-mabi=32,-mabi=64} }
}
{ sparc*-*-solaris2* } {
set target_list { unix{,-m64} }
}
default {
# Works for alpha*-*-osf*, i?86-*-solaris2.[89] and mips-sgi-irix5*
# testing.
set target_list { unix }
}
}
I do build gmp/mpfr/mpc in-tree...
given that I almost always run the whole testsuite - I'm torn between
thinking it's a good idea..
... and that I'm essentially proving little when the versions of gmp/
mpfr/and mpc are static.
I'd keep gmp/mpfr/mpc at a fixed version, test them once, and be done
with it. Testing the compiler and runtime libraries on its own takes
enough time already.
FWIW: I'm trying to update contrib/btest.sh script to handle m32 m64 plus
a few other things ...
No need: works already via DEJAGNU described above.
Rainer
--
-
Rainer Orth, Center for Biotechnology, Bielefeld University
Re: legitimate parallel make check?
On 9 Mar 2010, at 19:31, NightStrike wrote: On Tue, Mar 9, 2010 at 2:27 PM, IainS develo...@sandoe-acoustics.co.uk wrote: I do build gmp/mpfr/mpc in-tree... How? Last I tried, it didn't work, as mpc used the system gmp/mpfr, not the just-built in-tree versions. Therefore, it's not really an in-tree build, and you can't build on a system that doesn't already have gmp/mpfr installed. At least, that's what it was... does that work now? it's been fixed for a while (resolution of PR42424) I keep one copy of gmp mpfr and mpc in a directory at the root of my source disk and ln -s to that from within the different gcc source trees. Iain
Re: legitimate parallel make check?
On 9 Mar 2010, at 19:36, Rainer Orth wrote:
IainS develo...@sandoe-acoustics.co.uk writes:
.. I don't seem to get the bus errors on a 4CPU g5 or a Core 2
duo .. but
.. the 8-core machine is faster .. so ... race conditions are more
likely
to manifest there.
But race conditions don't manifest themselves in make SEGVs ;-( I'm
regularly running make -k -j128 on a T5220, or -j32 on a Sun Fire
X4450
I'm running memchecks on my 8-core machine ..
Indeed, as well as using a site.exp and pointing the DEJAGNU
environment
variable at it, as I've learned from looking at the regression tester
sources in contrib/regression. Here's what I use for that:
global target_list
case $target_triplet in {
{ i?86-*-solaris2.1[0-9] } {
# FIXME: Disable multilib testing if the host cannot execute AMD64
# binaries. Should be exceedingly rare now (cf. erebus).
set target_list { unix{,-m64} }
}
{ mips-sgi-irix6* } {
# FIXME: Disable multilib testing if the host cannot execute N64
# binaries. We cannot selectively disable only one multilib during
# the build.
set target_list { unix{,-mabi=32,-mabi=64} }
}
{ sparc*-*-solaris2* } {
set target_list { unix{,-m64} }
}
default {
# Works for alpha*-*-osf*, i?86-*-solaris2.[89] and mips-sgi-irix5*
# testing.
set target_list { unix }
}
}
that's really neat indeed - I should have spotted the potential
looking at the code in contrib
... although the site.exp is hardwired in btest.sh at present ;
I guess one might be able to use .dejagnurc - just need to check on
the order that the files are processed.
FWIW: I'm trying to update contrib/btest.sh script to handle m32
m64 plus
a few other things ...
No need: works already via DEJAGNU described above.
hmm. I see that the make check would work using that neat method
above...
... however,
1/ I want a different language list than the default...
2/ also some other different configure options...
but mainly
3/
... As I read it - btest.sh strips everything but the testname from
the PASSES and FAILS files. (the awk lines that print $2 for a match
of /FAIL: / etc.)
This means (e.g.) that if a test passes at m32 and fails at m64 I
think it will appear in both PASSES and FAILS..
... this seems destined to result in confusion...
I would like to identify regressions separately for m32 and m64 -
esp. in obj-c/c++ where there are huge differences on NeXT runtime
systems.
if I've missed the blindingly obvious (the day job is taking up most
of my mental resources ;-)) please point it out :-)
Iain
