I have been tweaking scripts/comprehensive_test.sh to produce more informative and well-documented results, and I am now (revision 11233) fairly satisfied with it. Thus, I encourage all those here who would like to help out with testing before the release this weekend (and who have ~20GB of free disk space!) to try out this script.
To find out more about what is possible with this script, run scripts/comprehensive_test.sh --help There are many options there to cut down the scope of what is tested (say if you are short of disk space for the non-interactive tests or in case you don't want to do the interactive tests), but I would urge testers not to use those options in order to create as comprehensive a test as possible. Here is how I typically set up before running this script export CFLAGS='-O3 -fvisibility=hidden' export CXXFLAGS='-O3 -fvisibility=hidden' export FFLAGS='-O3 -fvisibility=hidden' The -fvisibility=hidden option minimizes the number of symbols in our libraries and reduces startup latency for every one of our PLplot examples so it should be an efficiency win for these comprehensive tests to some degree although I haven't checked how much. The -O3 optimization makes the builds longer but the tests shorter. My feeling is it will be a net efficiency win, but I haven't checked that. I also then put the cmake version I want to test on my PATH then run scripts/comprehensive_test.sh \ --cmake_added_options '-DPLD_pdf=ON -G "Unix Makefiles"' \ --build_command 'make -j4' The --build_command option is mandatory and allows users to specify any make parallel build options that are optimum for their system. (-j4 for GNU make seems to be pretty efficient for my own two-cpu Intel core duo system.) The --build_command option theoretically (not tested yet) also allows some other build command such as nmake to be used on Windows systems with bash (either winbash or MSYS bash) installed. The -G "Unix Makefiles" part of the above illustrates how to specify a cmake generator, but it is, of course, redundant on Linux since that is the default cmake generator. I specified the -DPLD_pdf=ON option above because I wanted to test that non-default driver. (It is still non-default because of problems with the vanilla libharu library upon which it is based, but I have patched the libharu build following instructions in cmake/external/libharu/README.libharu so I can specify -DPLD_pdf=ON in this case without introducing test errors.) WARNING, by default to above test is comprehensive (as its name implies). For each of the three major build configurations (shared library/dynamic devices, shared library/nondynamic devices, and static library/nondynamic devices) it does 7 major tests (ctest, test_noninterative, and test_interactive in the build tree; test_noninterative, and test_interactive in the installed examples tree with the cmake-based build system there; and test_noninterative, and test_interactive in the installed examples tree using the traditional (Makefile + pkg_config) build system there. The 4 non-interactive tests for the 3 major build configurations consumes something like 25GB of disk space for all the plotted file results so only those with sufficient spare disk space should run this script. Also, the point of the interactive part of these tests (run after the noninteractive part is completed for each of the three major configurations) is simply to check for any major errors (segfaults and the like) with all the configured interactive device drivers and interactive environments). So to avoid driving you mad by doing lots of clicking to get through these tests, I have used the -np (no pause) option wherever I could. So an awful lot of different interactive plots scream by when running these tests so don't expect to use them to check for any subtle rendering issues. For both the non-interactive and interactive part of this testing any major errors (e.g., segfaults) are stored in the *.out files the above script collects. So after the script completes you can test for such major errors in both the noninteractive and interactive testing as follows: softw...@raven> grep -i error \ comprehensive_test_disposeable/*/output_tree/*.out comprehensive_test_disposeable/nondynamic/output_tree/traditional_make_interactive.out:All interactive tests completed without major errors comprehensive_test_disposeable/shared/output_tree/traditional_make_interactive.out:All interactive tests completed without major errors comprehensive_test_disposeable/static/output_tree/traditional_make_interactive.out:All interactive tests completed without major errors Last night I completed the above test for cmake-2.8.2 with no major errors revealed by the above grep statement. I am currently going through the whole process again with cmake-2.6.4. (Testing for two different versions of cmake is overkill for most testers, but I felt it should be done by at least one tester like me for due diligence until we bump our minimum CMake version to 2.8.x.) I fully expect those cmake-2.6.4 results to also be good, and in any case I intend to report all 42 major tests using the simple table at http://www.miscdebris.net/plplot_wiki/index.php?title=Testing_PLplot. If I have time before the release I may also try running the comprehensive test script with -G "MSYS Makefiles" for MinGW/MSYS under Wine and, of course, reporting the results on our Wiki as well. No one tester can test all combinations of cmake options and build platforms so I invite anyone else wanting to help out with testing this release to run the script (which by default does 21 major tests) on our svn trunk version (what is about to become 5.9.7) and report the results at the above page on our wiki. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
