Hallo all:

While trying to automate the building and testing of several OpenRISC 
components, I have noticed that the rtl-tests target in the ORPSoC project 
always yields a zero exit code (an indication of success), even when all tests 
fail.

I tracked this issue down to the following code in sim/bin/Makefile:

  # Main RTL test loop
  rtl-tests:
    $(Q)for test in $(TESTS); do \
        export TEST=$$test; \
        $(MAKE) rtl-test-with-check-no-print; \
        if [ $$? -ne 0 ]; then break; fi; \
        echo; echo "\t### $$test test OK ###"; echo; \
    done

We cannot force maintainers to visually look for regression test errors every 
time, especially in a daily build scenario.

I also noticed that, although the top-level test driver is a makefile, it 
breaks when running in parallel mode (when invoking GNU Make with option -j N). 
The tests must be run sequentially, so that only one CPU ever gets busy.

Regards,
  R. Diez
_______________________________________________
OpenRISC mailing list
[email protected]
http://lists.openrisc.net/listinfo/openrisc

Reply via email to