On 2008-08-07 16:20+0100 Andrew Ross wrote: > > I have added a new script to the ctest suite. This will run after all > the existing tests and will compare the postscript output from the > different language bindings. > > For each binding present (other than C) it will list > 1) Examples that are missing compared to C > 2) Examples that produce different results compared to C > > To run the script just needs sh / bash, diff and tail. All 3 will be > checked for by cmake and as long as they are found the test will be run > along with the other tests. > > Results can be seem either by looking in Testing/Temporary/LastTest.log > in the build tree, or by running ctest with the -V option. > > Hopefully this will help ensure consistency between the examples and > identify any problems.
Thanks for that extremely useful additional test for ctest. That new test result inspired me to work most of the day on PLplot as a result. :-) Here are the results on my Debian testing platform. [EMAIL PROTECTED]> ctest --verbose -R compare .... c++ Missing examples : 27 Differing examples : f77 Missing examples : 24 25 26 27 Differing examples : 20 23 f95 Missing examples : 24 25 26 27 Differing examples : 20 java Missing examples : 19 27 Differing examples : octave Missing examples : 19 27 Differing examples : python Missing examples : 19 27 Differing examples : 20 21 tcl Missing examples : 19 20 21 23 24 25 26 27 28 29 30 Differing examples : 01 02 03 04 05 06 07 08 09 10 11 12 13 15 16 18 ada Missing examples : 20 Differing examples : 19 21 ocaml Missing examples : 09 10 12 13 15 16 18 20 21 22 23 24 25 26 27 28 29 30 Differing examples : 11 Ada example 19 is different for me (but not Jerry) because it stops prematurely due to a run-time error. Because of that run-time error I don't think we can label Ada as stable, yet. Otherwise, Ada is in pretty good shape compared to the others and run-time errors don't block anything else. Therefore, as agreed with Jerry off list, I have enabled Ada by default to encourage much wider testing. I understand Hez is actively working on the missing ocaml examples and also bringing example 11 into conformity with the C version. Of the "mature" languages, it appears Tcl is in the worst shape: tcl Missing examples : 19 20 21 23 24 25 26 27 28 29 30 Differing examples : 01 02 03 04 05 06 07 08 09 10 11 12 13 15 16 18 I hope somebody with an interest in Tcl will step forward to add the additional standard examples (and also add the necessary API those standard examples exercise). I looked in detail at the tcl diffs using (for LIST in `ls x??t.psc |sed 's?t.psc??'`; do echo $LIST; diff ${LIST}[tc].psc; done) |less Most of the differing examples reported by the new test are false positives because the tcl examples do some sort of extra reset (of colour?). But there are real issues I found for examples 2, 11, 15, and 16 with the above command. * Example 2 has incorrect colours for tcl. A typical diff stanza is < 0.9961 0.6275 0.0745 C Z --- > 1.0000 0.6275 0.0745 C Z I looked deeper into this by printing out intermediate results. What is going on is the C example is calculating a maximum colour of 254 while the Tcl example is calculating it as 255 for the case when plhlsrgb is returning a floating point number that my printout to 9 figures indicates is 1.0000... I assume the difference is caused by the respective codes truncating (1 + epsilon) * 255 to the nearest integer where epsilon is a very small negative number for C and 0 or positive for Tcl. (It is well known that Tcl floating-point calculations are done quite differently than for compiled languages.) I don't think we will ever get rid of this general Tcl floating-point difference with C, but one can avoid such close truncation decisions in the example by using the factor 255.001 rather than 255. I made that change to all the language implementations of example 2, and they now (including Tcl except for the extra reset) all give identical results. (I have not tested perl, but I did make that simple change there, and I am sure when somebody tests it against the C result it will be fine.) * Example 11 appears to have simple rounding differences (PostScript errors in the last digit). If we used ndiff (fuzzy diff taking into account small floating point differences, see http://www.math.utah.edu/~beebe/software/ndiff/) instead of diff we could presumably get this Tcl example to pass the test (aside from the reset issue). N.B. Andrew, I don't think this one example justifies the complications (cmake find of ndiff, and fall back to diff otherwise with different options in the two cases) of the ndiff possibility. However, I brought it up because ndiff is something to keep in mind if people find rounding issues on other platforms that are more severe than they seem to be for Linux. * Examples 15 and 16 have quite complex diff patterns. I expect there is some real implementation difference for both these examples that a Tcl expert should look at. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel