On 12 March 2014 04:25, Assaf Gordon <[email protected]> wrote: > I'm trying to convert my project non-recursive makefile, and encountered a > strange problem with the tests. > > I managed to reproduce and isolate it using GNU hello, so I figured perhaps > you can help me. > > The problem is this: > 1. GNU Hello, cloned from latest repository. > 2. locally (on old ubuntu), "make check" works just fine. > 3. Using "Travis-CI" (the website for continuous integration), the tests > fail. > > I've managed to pinpoint the issue to the following: > Inside Makefile.am, the test environment is set using AM_TESTS_ENVIRONMENT. > For some strange reason, this is ignored/bypassed on Travis-CI's servers > (They are using Ubuntu 12.04 which should be standard). > When changed to TESTS_ENVIRONMENT, the tests pass. > > ========= > The steps I've used: > 1. Cloned GNU Hello, put in GitHub, setup Travis-CI. > "make check" fails, all tests can't find the "hello" executable. > > This is the log, scroll to line 2139 to see the tests results: > https://travis-ci.org/agordon/gnuhello/jobs/20581026#L2139 > > 2. Modified "tests/greeting-2", to print the environment and always fail: > https://github.com/agordon/gnuhello/commit/6a5af85e99afe69cdec6ccb015ccd9b95502bff4 > > This is the log, scroll to line 2168 to see the printed environment: > https://travis-ci.org/agordon/gnuhello/jobs/20581280#L2168 > Note especially, the PATH does not contain "." and there is no "HELLO" > variable (both are set in AM_TESTS_ENVIRONMENT). > > 3. Modified "Makefile.am", changed "AM_TESTS_ENVIRONMENT" to > "TESTS_ENVIRONMENT" > https://github.com/agordon/gnuhello/commit/0b54a2fee303f42f85f99284e00b8aad3a667232 > > This is the log after the change, scroll to line 2161 to see the updated > printed environment: > https://travis-ci.org/agordon/gnuhello/jobs/20581441#L2161 > > After the change, PATH is valid and HELLO is set, and the tests pass (except > 'greeting-2' which fails on purpose). > ========= > > So my question - > What's going on, and is changing "AM_TESTS_ENVIRONMENT" the correct > solution?
Hi Gordon, Thank you for good bug report. Short answer: changing AM_TESTS_ENVIRONMENT seems to be the correct solution. Long answer: unless someone objects I'll push the following to upstream. --->8---- From: Sami Kerola <[email protected]> Date: Thu, 13 Mar 2014 17:18:36 +0000 Subject: [PATCH] build: parallel-tests require of TESTS_ENVIRONMENT * Makefile.am: Tests failed in Travis-CI's servers because missing environment variables. Looking the automake documentation the AM_TESTS_ENVIRONMENT cannot be used in combination with parallel-tests, and such cases TESTS_ENVIRONMENT should be used instead. Reported by Assaf Gordon. https://www.gnu.org/software/automake/manual/html_node/Serial-Test-Harness.html --- Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.am b/Makefile.am index eb02a4c..6285887 100644 --- a/Makefile.am +++ b/Makefile.am @@ -68,7 +68,7 @@ BUILT_SOURCES = LOG_COMPILER = $(SHELL) -AM_TESTS_ENVIRONMENT = \ +TESTS_ENVIRONMENT = \ top_srcdir=$(top_srcdir) \ PATH=.$(PATH_SEPARATOR)$(PATH_SEPARATOR)$$PATH \ HELLO=`echo hello | sed '$(transform)'`; \ -- 1.9.0
