Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de> writes: > > | > > | - (a) and (b) need GNU make for parallel execution, > > I should clarify: (a) is likely to cope with parallel non-GNU make, > while (b) will ignore non-GNU make parallelism. For example, BSD makes > typically allow parallel execution as well, but do not implement a job > server.
Yes, that helps me understand your intent - exposing the parallel tasks to any make's dependency engine, vs. hooking directly into the guts of GNU make's job server. > > > | - AFAICS (b) currently needs a shell that understands 'read -n1', > > | - (d) differs from (c) in that one subprocess executes more than one > > | test group, thus is potentially faster because it forks less, > > | - (b) has the nice feature that it allows to parallelize across multiple > > | test suites, and across testing and other, independent build activity. > > | That means, while (c) and (d) allow > > | make check TESTSUITEFLAGS='-j3' > > | to speed up things, (b) allows > > | gmake -j3 check > > | to profit. > > > > Interesting trades. I like that (c) and (d) can do parallel execution > > when the testsuite is run manually (without make); > > Good point. I would not want to have (b) without any of (c), (d); > rather, I thought of adding (c) and maybe also (b). I was hoping that you could combine the good parts of (b) and (c). > > The goal of still shipping a single 'testsuite' file that contains > > everything needed to create the multiple tests is nice. I agree that > > blindly calling 'testsuite n' for n parallel tests is too much overhead. > > On the other hand, it would sure be nice to call testsuite once with the > > user's TESTSUITEFLAGS to generate the subset of individual files to run, > > then turn around and run those individual files in parallel. > > I don't quite understand what the last sentence in this paragraph is > supposed to mean. Our current recommended formula for including an autotest suite in Makefile is to call it with TESTSUITEFLAGS as the user's hook to select a subset of tests. So I was trying to convey my thoughts on a flow that could be something like this: At configure time, config.status generates testsuite and also a makefile fragment that contains dependencies for each test's n.log, which is then included in the user's Makefile (automake supports makefile fragment inclusion even for non-GNU makes, or the fragment could be a multi-line AC_SUBST rather than an external file). At make time, each n.log depends on n.test and a first stamp file. Running 'testsuite --parallel-start TESTSUITEFLAGS=-10' creates the stamp file, creates an independent shell script n.test for tests 1-10 (each with much less startup overhead than testsuite; for example, it can be hardcoded to run with the better shell detected by testsuite, rather than having to repeat the gyrations of detecting a better shell itself), and a no-op n.test for the remaining test groups (merely touching n.log). After the stamp file exists, make can then run the various n.test files in parallel to create n.log files. Finally, a second stamp file depends on all of the n.log files; running 'testsuite --parallel-collect' gathers the results and creates the second stamp. That way, all you have to list for the dependency of check-local is the second stamp file. But that still doesn't answer the question of whether the startup costs for n shell scripts is reasonable, in relation to forking off the main driver shell script that has already collected a bunch of startup state. > > In each of (b), (c), you can take any TESTSUITEFLAGS you would currently > use; with (c), add '-j3' to TESTSUITEFLAGS, with (b), add '-j3' to the > gmake command line directly. And I suppose with (b) and (c) together, you could use: gmake -j3 check TESTSUITEFLAGS=-j3 but then you have two job servers both competing for processor load without communicating with each other on how to balance the load. > > > Is there a way to write a Makefile include fragment which gets included if > > we detect GNU make, but is portably ignored for other makes, where we can > > then exploit gmake features to make parallel execution easier? > > As I understand this question, the at_parse_makeflags snippet shown in > patch 3/3 does exactly that: it should be a no-op for non-GNU parallel > make, as none of them use '--jobserver-fds='. > > If your question is about a general way to include something for GNU > make only, one possibility is to have a GNUmakefile which includes > Makefile plus extra, GNU make-specific code. Whatever we decide, we should probably make it easy for package writers to call a single macro in configure.ac to generate the AC_SUBST that then spits out all the Makefile magic, rather than the current approach with several lines of documentation on how to edit Makefile to add a testsuite. For that matter, it might be worth installing /bin/autotest as a thin shell wrapper around /bin/autom4te --language=autotest. > Fully agreed; I've seen such fishy behavior, too, but mostly ignored it > up to now. I fear that the behavior depends quite a bit on the sh and > make implementations involved. And I think some of it stems from the fact that we base success on the contents of $at_status_file, which can easily be out-of-date if a test group is killed and no trap exists to update the file accordingly. > I think 1.5.25. My usual procedure is to boot w32, fire up cygwin > setup.exe, run its update process without changing any settings > manually, then do testing. I assume that, for experimental 1.7.0 > I'd have to change some setting, no? Your guess is right - since you haven't done anything special, you're running cygwin 1.5.25. -- Eric Blake
