Reference: <http://lists.gnu.org/archive/html/automake-patches/2011-04/msg00141.html>
On Tuesday 19 April 2011, Stefano Lattarini wrote: > Wow, this is an oooold one. References: > <http://lists.gnu.org/archive/html/automake-patches/2009-12/msg00026.html> > <http://lists.gnu.org/archive/html/automake-patches/2009-12/msg00045.html> > > -*-*- > > maintcheck: consistency of list of test scripts > > * CheckListOfTests.am: New file. > (maintainer-check-list-of-tests): New target, > check for consistency between list of tests defined in the > including Makefile and list of tests on the filesystem. > (clean-maintcheck-testslist-tmp): New rule, to clean up the > temporary files that might be left around by the rules associated > with the previous target. > (clean-local): Depend on it. > * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'. > (maintainer-check): Added dependency from the new target > `maintainer-check-list-of-tests'. > * tests/Makefile.am: Likewise. Also ... > (TEST_EXTENSIONS): New, define to `.test'. > --- > ChangeLog | 21 ++++++++++++ > CheckListOfTests.am | 70 > ++++++++++++++++++++++++++++++++++++++++ > Makefile.am | 21 +++++++++++- > Makefile.in | 18 ++++++++++- > lib/Automake/tests/Makefile.am | 5 ++- > lib/Automake/tests/Makefile.in | 68 ++++++++++++++++++++++++++++++++------ > tests/Makefile.am | 5 +++ > tests/Makefile.in | 52 ++++++++++++++++++++++++++++-- > 8 files changed, 243 insertions(+), 17 deletions(-) > create mode 100644 CheckListOfTests.am > > -*-*- > > OK for maint? I'll push in 72 hours if there is no objection. > > Regards, > Stefano > OK, I had to revise this patch a bit now that in the 'testsuite-work' branch we have stuff in $(TESTS) that doesn't correspond to existing files (by a conscious choice). The updated patch, still based of off maint, is attached. I've merged in into 'testsuite-work' and pushed. I hope it will be accepted also in maint some not-too-distant day. Regards, Stefano
From 7b6ab07442b28a62389fc4dc8f35276f31348c13 Mon Sep 17 00:00:00 2001 Message-Id: <7b6ab07442b28a62389fc4dc8f35276f31348c13.1304442594.git.stefano.lattar...@gmail.com> From: Stefano Lattarini <stefano.lattar...@gmail.com> Date: Sat, 30 Jan 2010 01:23:44 +0100 Subject: [PATCH] maintcheck: consistency of list of test scripts * CheckListOfTests.am: New file. (maintainer-check-list-of-tests): New target, check for consistency between list of tests defined in the including Makefile and list of tests on the filesystem. (clean-maintcheck-testslist-tmp): New rule, to clean up the temporary files that might be left around by the rules associated with the previous target. (clean-local): Depend on it. * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'. (maintainer-check): Added dependency from the new target `maintainer-check-list-of-tests'. * tests/Makefile.am: Likewise. * Makefile.am (maintainer-check-list-of-test): New target, calling recursively into `tests/' and `lib/Automake/tests/', using ... (TEST_SUBDIRS): ... this new variable. --- ChangeLog | 20 ++++++++++ CheckListOfTests.am | 76 ++++++++++++++++++++++++++++++++++++++++ Makefile.am | 21 ++++++++++- Makefile.in | 18 +++++++++- lib/Automake/tests/Makefile.am | 5 ++- lib/Automake/tests/Makefile.in | 70 +++++++++++++++++++++++++++++++------ tests/Makefile.am | 3 ++ tests/Makefile.in | 52 ++++++++++++++++++++++++++- 8 files changed, 249 insertions(+), 16 deletions(-) create mode 100644 CheckListOfTests.am diff --git a/ChangeLog b/ChangeLog index 99e4f11..846b569 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,23 @@ +2011-05-03 Stefano Lattarini <stefano.lattar...@gmail.com> + + maintcheck: consistency of list of test scripts + * CheckListOfTests.am: New file. + (maintainer-check-list-of-tests): New target, + check for consistency between list of tests defined in the + including Makefile and list of tests on the filesystem. + (clean-maintcheck-testslist-tmp): New rule, to clean up the + temporary files that might be left around by the rules associated + with the previous target. + (clean-local): Depend on it. + * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'. + (maintainer-check): Added dependency from the new target + `maintainer-check-list-of-tests'. + * tests/Makefile.am: Likewise. + * Makefile.am (maintainer-check-list-of-test): New target, + calling recursively into `tests/' and `lib/Automake/tests/', + using ... + (TEST_SUBDIRS): ... this new variable. + 2011-04-24 Stefano Lattarini <stefano.lattar...@gmail.com> tests: fix spurious failure in a test on TESTS (VPATH-related) diff --git a/CheckListOfTests.am b/CheckListOfTests.am new file mode 100644 index 0000000..28cb795 --- /dev/null +++ b/CheckListOfTests.am @@ -0,0 +1,76 @@ +## -*- Automake -*- +## +## Copyright (C) 2011 Free Software Foundation, Inc. +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2, or (at your option) +## any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program. If not, see <http://www.gnu.org/licenses/>. +## + +## Temporary files used in the `check-list-of-tests' target. +am__tmk = tests-in-makefile-list.tmp +am__tfs = tests-on-filesystem-list.tmp +am__tdf = diff-in-tests-lists.tmp + +## Check that the list of tests given in the Makefile is equal to the +## list of all test scripts in the Automake testsuite. +.PHONY: maintainer-check-list-of-tests +maintainer-check-list-of-tests: +## Prefer unified diffs over plain diffs, for readability. + @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \ + diff='diff -u'; \ + else \ + diff='diff'; \ + fi; \ +## Determine which files are to be considered test scripts. +## Do not blindly look for `$(TEST_EXTENSIONS)', because the parallel +## testsuite driver provided by Automake allow entries in `$(TESTS)' +## not to be real files, but rather paramaters for the corresponding +## LOG_COMPILER (well, sort of). + set X $(checked_test_extensions); shift; \ + if test $$# -eq 0; then \ + set X $(TEST_EXTENSIONS); shift; \ + fi; \ + if test $$# -eq 0; then \ + echo '$@: internal error: $$(checked_test_extensions) and' \ + '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \ + exit 255; \ + fi; \ +## List of tests in Makefile. + lst='$(expected_list_of_tests)'; \ + test -n "$$lst" || lst='$(TESTS)'; \ + for t in $$lst; do \ + echo "$$t"; \ + done | sort >$(am__tmk); \ +## List of tests on filesystem. Be careful to cater for VPATH builds too. + for ext in $$*; do \ + ls *$$ext 2>/dev/null; \ + if test $(srcdir) != $(builddir); then \ + (cd $(srcdir) && ls *$$ext 2>/dev/null); \ + fi; \ + done | sort | uniq >$(am__tfs); \ +## Compare the two lists, complain if they differ. + if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \ + result=0; \ + else \ + echo '$@: list of tests in Makefile an on filesystem differ' >&2; \ + echo "+ $$diff in-makefile on-filesystem" >&2; \ + cat $(am__tdf) >&2; \ + result=1; \ + fi; \ + rm -f $(am__tmk) $(am__tfs) $(am__tdf); \ + exit $$result; + +.PHONY: clean-maintcheck-testslist-tmp +clean-local: clean-maintcheck-testslist-tmp +clean-maintcheck-testslist-tmp: + rm -f $(am__tmk) $(am__tfs) $(am__tdf) diff --git a/Makefile.am b/Makefile.am index 6e8b7df..aeff613 100644 --- a/Makefile.am +++ b/Makefile.am @@ -24,6 +24,8 @@ ## run aclocal and automake. SUBDIRS = lib . doc m4 tests +TEST_SUBDIRS = tests lib/Automake/tests + bin_SCRIPTS = automake aclocal CLEANFILES = $(bin_SCRIPTS) @@ -150,7 +152,24 @@ sc_at_in_texi .PHONY: $(syntax_check_rules) $(syntax_check_rules): automake aclocal -maintainer-check: $(syntax_check_rules) + +## Check that the list of tests given in the Makefile is equal to the +## list of all test scripts in the Automake testsuite. +.PHONY: maintainer-check-list-of-tests +maintainer-check-list-of-tests: + @failcom='exit 1'; \ + for f in x $$MAKEFLAGS; do \ + case $$f in \ + *=* | --[!k]*);; \ + *k*) failcom='fail=yes';; \ + esac; \ + done; \ + for subdir in $(TEST_SUBDIRS); do \ + (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $@) || eval $$failcom; \ + done; \ + test -z "$$fail" + +maintainer-check: $(syntax_check_rules) maintainer-check-list-of-tests ## Look for test whose names can cause spurious failures when used as ## first argument to AC_INIT (chiefly because they might contain an diff --git a/Makefile.in b/Makefile.in index 49c35f5..ea63d6b 100644 --- a/Makefile.in +++ b/Makefile.in @@ -236,6 +236,7 @@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ SUBDIRS = lib . doc m4 tests +TEST_SUBDIRS = tests lib/Automake/tests bin_SCRIPTS = automake aclocal CLEANFILES = $(bin_SCRIPTS) AUTOMAKESOURCES = automake.in aclocal.in @@ -928,7 +929,22 @@ dist-hook: .PHONY: $(syntax_check_rules) $(syntax_check_rules): automake aclocal -maintainer-check: $(syntax_check_rules) + +.PHONY: maintainer-check-list-of-tests +maintainer-check-list-of-tests: + @failcom='exit 1'; \ + for f in x $$MAKEFLAGS; do \ + case $$f in \ + *=* | --[!k]*);; \ + *k*) failcom='fail=yes';; \ + esac; \ + done; \ + for subdir in $(TEST_SUBDIRS); do \ + (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $@) || eval $$failcom; \ + done; \ + test -z "$$fail" + +maintainer-check: $(syntax_check_rules) maintainer-check-list-of-tests sc_test_names: @m4_builtin_rx=`echo $(m4_builtins) | sed 's/ /|/g'`; \ m4_macro_rx="\\<($$m4_builtin_rx)\\>|\\<_?(A[CUMHS]|m4)_"; \ diff --git a/lib/Automake/tests/Makefile.am b/lib/Automake/tests/Makefile.am index c5e53d2..fad3d7e 100644 --- a/lib/Automake/tests/Makefile.am +++ b/lib/Automake/tests/Makefile.am @@ -1,6 +1,7 @@ ## Process this file with automake to create Makefile.in -# Copyright (C) 2002, 2003, 2008, 2009 Free Software Foundation, Inc. +# Copyright (C) 2002, 2003, 2008, 2009, 2010 Free Software Foundation, +# Inc. # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -28,3 +29,5 @@ Version.pl \ Wrap.pl EXTRA_DIST = $(TESTS) + +include $(top_srcdir)/CheckListOfTests.am diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in index b4940db..bdf7219 100644 --- a/lib/Automake/tests/Makefile.in +++ b/lib/Automake/tests/Makefile.in @@ -15,7 +15,8 @@ @SET_MAKE@ -# Copyright (C) 2002, 2003, 2008, 2009 Free Software Foundation, Inc. +# Copyright (C) 2002, 2003, 2008, 2009, 2010 Free Software Foundation, +# Inc. # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -47,8 +48,9 @@ NORMAL_UNINSTALL = : PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ +DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in \ + $(top_srcdir)/CheckListOfTests.am subdir = lib/Automake/tests -DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \ $(top_srcdir)/m4/auxdir.m4 $(top_srcdir)/m4/init.m4 \ @@ -279,11 +281,14 @@ Version.pl \ Wrap.pl EXTRA_DIST = $(TESTS) +am__tmk = tests-in-makefile-list.tmp +am__tfs = tests-on-filesystem-list.tmp +am__tdf = diff-in-tests-lists.tmp all: all-am .SUFFIXES: .SUFFIXES: .html .log .pl -$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(top_srcdir)/CheckListOfTests.am $(am__configure_deps) @for dep in $?; do \ case '$(am__configure_deps)' in \ *$$dep*) \ @@ -518,7 +523,7 @@ maintainer-clean-generic: @echo "it deletes files that may require special tools to rebuild." clean: clean-am -clean-am: clean-generic mostlyclean-am +clean-am: clean-generic clean-local mostlyclean-am distclean: distclean-am -rm -f Makefile @@ -586,17 +591,60 @@ uninstall-am: recheck-html .PHONY: all all-am check check-TESTS check-am check-html clean \ - clean-generic distclean distclean-generic distdir dvi dvi-am \ - html html-am info info-am install install-am install-data \ - install-data-am install-dvi install-dvi-am install-exec \ - install-exec-am install-html install-html-am install-info \ - install-info-am install-man install-pdf install-pdf-am \ - install-ps install-ps-am install-strip installcheck \ - installcheck-am installdirs maintainer-clean \ + clean-generic clean-local distclean distclean-generic distdir \ + dvi dvi-am html html-am info info-am install install-am \ + install-data install-data-am install-dvi install-dvi-am \ + install-exec install-exec-am install-html install-html-am \ + install-info install-info-am install-man install-pdf \ + install-pdf-am install-ps install-ps-am install-strip \ + installcheck installcheck-am installdirs maintainer-clean \ maintainer-clean-generic mostlyclean mostlyclean-generic pdf \ pdf-am ps ps-am recheck recheck-html uninstall uninstall-am +.PHONY: maintainer-check-list-of-tests +maintainer-check-list-of-tests: + @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \ + diff='diff -u'; \ + else \ + diff='diff'; \ + fi; \ + set X $(checked_test_extensions); shift; \ + if test $$# -eq 0; then \ + set X $(TEST_EXTENSIONS); shift; \ + fi; \ + if test $$# -eq 0; then \ + echo '$@: internal error: $$(checked_test_extensions) and' \ + '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \ + exit 255; \ + fi; \ + lst='$(expected_list_of_tests)'; \ + test -n "$$lst" || lst='$(TESTS)'; \ + for t in $$lst; do \ + echo "$$t"; \ + done | sort >$(am__tmk); \ + for ext in $$*; do \ + ls *$$ext 2>/dev/null; \ + if test $(srcdir) != $(builddir); then \ + (cd $(srcdir) && ls *$$ext 2>/dev/null); \ + fi; \ + done | sort | uniq >$(am__tfs); \ + if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \ + result=0; \ + else \ + echo '$@: list of tests in Makefile an on filesystem differ' >&2; \ + echo "+ $$diff in-makefile on-filesystem" >&2; \ + cat $(am__tdf) >&2; \ + result=1; \ + fi; \ + rm -f $(am__tmk) $(am__tfs) $(am__tdf); \ + exit $$result; + +.PHONY: clean-maintcheck-testslist-tmp +clean-local: clean-maintcheck-testslist-tmp +clean-maintcheck-testslist-tmp: + rm -f $(am__tmk) $(am__tfs) $(am__tdf) + # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. .NOEXPORT: diff --git a/tests/Makefile.am b/tests/Makefile.am index 49d8942..e6d3098 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -847,6 +847,9 @@ $(parallel_tests) EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS) +include $(top_srcdir)/CheckListOfTests.am +maintainer-check-list-of-tests: $(parallel_tests) + # Each test case depends on defs, aclocal, and automake. check_SCRIPTS = defs aclocal-$(APIVERSION) automake-$(APIVERSION) diff --git a/tests/Makefile.in b/tests/Makefile.in index 78d723c..76800aa 100644 --- a/tests/Makefile.in +++ b/tests/Makefile.in @@ -50,7 +50,7 @@ POST_UNINSTALL = : build_triplet = @build@ DIST_COMMON = README $(srcdir)/Makefile.am $(srcdir)/Makefile.in \ $(srcdir)/aclocal.in $(srcdir)/automake.in $(srcdir)/defs.in \ - $(srcdir)/parallel-tests.am + $(srcdir)/parallel-tests.am $(top_srcdir)/CheckListOfTests.am subdir = tests ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \ @@ -1117,6 +1117,9 @@ yflags2.test \ $(parallel_tests) EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS) +am__tmk = tests-in-makefile-list.tmp +am__tfs = tests-on-filesystem-list.tmp +am__tdf = diff-in-tests-lists.tmp # Each test case depends on defs, aclocal, and automake. check_SCRIPTS = defs aclocal-$(APIVERSION) automake-$(APIVERSION) @@ -1124,7 +1127,7 @@ all: all-am .SUFFIXES: .SUFFIXES: .html .log .test -$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am $(am__configure_deps) +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am $(top_srcdir)/CheckListOfTests.am $(am__configure_deps) @for dep in $?; do \ case '$(am__configure_deps)' in \ *$$dep*) \ @@ -1455,6 +1458,51 @@ $(parallel_tests): $(parallel_tests:-p.test=.test) Makefile.am < $(srcdir)/$$input >$@ chmod a+rx $@ +.PHONY: maintainer-check-list-of-tests +maintainer-check-list-of-tests: + @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \ + diff='diff -u'; \ + else \ + diff='diff'; \ + fi; \ + set X $(checked_test_extensions); shift; \ + if test $$# -eq 0; then \ + set X $(TEST_EXTENSIONS); shift; \ + fi; \ + if test $$# -eq 0; then \ + echo '$@: internal error: $$(checked_test_extensions) and' \ + '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \ + exit 255; \ + fi; \ + lst='$(expected_list_of_tests)'; \ + test -n "$$lst" || lst='$(TESTS)'; \ + for t in $$lst; do \ + echo "$$t"; \ + done | sort >$(am__tmk); \ + for ext in $$*; do \ + ls *$$ext 2>/dev/null; \ + if test $(srcdir) != $(builddir); then \ + (cd $(srcdir) && ls *$$ext 2>/dev/null); \ + fi; \ + done | sort | uniq >$(am__tfs); \ + if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \ + result=0; \ + else \ + echo '$@: list of tests in Makefile an on filesystem differ' >&2; \ + echo "+ $$diff in-makefile on-filesystem" >&2; \ + cat $(am__tdf) >&2; \ + result=1; \ + fi; \ + rm -f $(am__tmk) $(am__tfs) $(am__tdf); \ + exit $$result; + +.PHONY: clean-maintcheck-testslist-tmp +clean-local: clean-maintcheck-testslist-tmp +clean-maintcheck-testslist-tmp: + rm -f $(am__tmk) $(am__tfs) $(am__tdf) + +maintainer-check-list-of-tests: $(parallel_tests) + clean-local: clean-local-check .PHONY: clean-local-check clean-local-check: -- 1.7.2.3
