Re: combined report for make check?
On 01/22/2013 12:09 AM, Karl Berry wrote: This would require to change the 'check-recursive' targets not to share the same code with the other '*-recursive' targets. I really don't want to go there. Totally reasonable :). The best solution is on the user-side IMHO: fix the build system to use less (ideally none) make recursion. I don't know what you mean. Texinfo has several separate programs, each with their own tests. Isn't subdirectory recursion the natural way to do it? How else? With a non-recursive, top level Makefile. Quick example: bin_PROGRAMS = old/info old/c-makeinfo old_info_SOURCES = old/info-reader.c old/term.c old_c_makeinfo_SOURCES = old/makeinfo.c bin_SCRIPTS = new/texi2any new/texi2any: new/texi2any.pl rm -f $@ $@-t sed -e 's|@PERL@|$(PERL)|' new/texi2any.pl $@-t chmod a+x,a-w $@-t mv -f $@-t $@ TESTS = \ old/tests/info.test \ old/tests/c-makeinfo.test \ old/tests/known-regression.test \ new/tests/texi2dvi.test \ new/tests/texi2info.test \ new/tests/texi2txt.test The GNU coreutils build system, now (almost) totally non recursive, can be of help there. See in particular the last lines of: http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=Makefile.am the testsuite Makefile: http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=tests/local.mk and the helper makefile for peoples who want to run make check from the 'tests/' subdirectory: http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=tests/GNUmakefile See also: http://miller.emu.id.au/pmiller/books/rmch/ http://www.gnu.org/software/automake/manual/automake.html#Alternative (yes, I'm noticing the latter is quite inadequate to show examples of a real non-recursive build systems; patches would be welcome). If, after taking a look at those resources, you have more precise questions, I'll be happy to try to answer them. For that, a screen scraping script wrapping the 'make check' invocation Yeah. My related idea was that his build script could run something like make check || echo *** tests failed or whatever ... Thanks, Karl Regards, Stefano
Re: combined report for make check?
With a non-recursive, top level Makefile. Quick example: So, replace a nicely separated source setup with munging together the information about everything in one place? Thanks for the suggestion, but that seems like a major step backwards in maintainability to me, so I'll decline. Life goes on. k
Re: combined report for make check?
Hi Karl. On 01/20/2013 11:20 PM, Karl Berry wrote: Especially for packages which run make check in several subdirectories (e.g., texinfo), Nelson Beebe made the suggestion that it would be helpful if there was an overall textual report of success or failure, and not just the exit status. Something like ALL TESTS PASSED vs. FAILED TESTS. Leaving it to the user to go back and examine the full log to see which ones in which directories. This would require to change the 'check-recursive' targets not to share the same code with the other '*-recursive' targets. I really don't want to go there. The best solution is on the user-side IMHO: fix the build system to use less (ideally none) make recursion. Both the parallel and serial testsuite harness should support that setup OOTB. The idea being that people (like Nelson) building on many systems could just grep for that string in the build logs and determine which ones need further inspection. For that, a screen scraping script wrapping the 'make check' invocation might the simplest and most backward-compatible solution (that is, it will work also with all packages bootstrapped using older Automake releases). If anyone wants to take a shot at such a script, and optimize it for use with Automake-generated testsuite harnesses, I'd be very happy to carry it in the 'contrib/' subdirectory of the Automake git repository. I'm not sure how feasible this is, but I thought I'd at least bring it up. Well done; doing so is always a good idea, whatever the final outcome is. Thanks, Stefano
Re: combined report for make check?
This would require to change the 'check-recursive' targets not to share the same code with the other '*-recursive' targets. I really don't want to go there. Totally reasonable :). The best solution is on the user-side IMHO: fix the build system to use less (ideally none) make recursion. I don't know what you mean. Texinfo has several separate programs, each with their own tests. Isn't subdirectory recursion the natural way to do it? How else? For that, a screen scraping script wrapping the 'make check' invocation Yeah. My related idea was that his build script could run something like make check || echo *** tests failed or whatever ... Thanks, Karl
Re: combined report for make check?
Stefano Lattarini stefano.lattar...@gmail.com writes: The best solution is on the user-side IMHO: fix the build system to use less (ideally none) make recursion. Both the parallel and serial testsuite harness should support that setup OOTB. It would be nice if automake had some more features for that... E.g., if I have a directory foo that has sources etc, and builds some specific targets, then I can isolate the automake stuff for foo by using an include file foo/Makefile.am.inc or something, and then putting an appropriate include in the top-level Makefile.am. But it's a bit annoying, in that AFAICT, all filenames, etc, in foo's Makefile fragment must explicitly include the directory name. E.g., if it builds a library, foo/Makefile.am.inc might look like: libfoo_a_SOURCES = foo/oink.c foo/barf.c foo/barf.h ... For longish directory names, this can really bloat things up... It would be really cool if there was some way of telling automake hey, for every filename mentioned in this file, try to use a prefix of ... I dunno whether that would be associated with the include directive, with the makefile fragment, or what, but... anyway. Does automake have some feature like this that I've missed? Or has anybody thought about it? Thanks, -miles -- I'd rather be consing.
