On 2008-08-07 16:20+0100 Andrew Ross wrote:
>
> I have added a new script to the ctest suite. This will run after all
> the existing tests and will compare the postscript output from the
> different language bindings.
>
> For each binding present (other than C) it will list
> 1) Examples that are missing compared to C
> 2) Examples that produce different results compared to C
>
> To run the script just needs sh / bash, diff and tail. All 3 will be
> checked for by cmake and as long as they are found the test will be run
> along with the other tests.
>
> Results can be seem either by looking in Testing/Temporary/LastTest.log
> in the build tree, or by running ctest with the -V option.
>
> Hopefully this will help ensure consistency between the examples and
> identify any problems.
Thanks for that extremely useful additional test for ctest. That new test
result inspired me to work most of the day on PLplot as a result. :-)
Here are the results on my Debian testing platform.
[EMAIL PROTECTED]> ctest --verbose -R compare
....
c++
Missing examples : 27
Differing examples :
f77
Missing examples : 24 25 26 27
Differing examples : 20 23
f95
Missing examples : 24 25 26 27
Differing examples : 20
java
Missing examples : 19 27
Differing examples :
octave
Missing examples : 19 27
Differing examples :
python
Missing examples : 19 27
Differing examples : 20 21
tcl
Missing examples : 19 20 21 23 24 25 26 27 28 29 30
Differing examples : 01 02 03 04 05 06 07 08 09 10 11 12 13 15 16 18
ada
Missing examples : 20
Differing examples : 19 21
ocaml
Missing examples : 09 10 12 13 15 16 18 20 21 22 23 24 25 26 27 28 29 30
Differing examples : 11
Ada example 19 is different for me (but not Jerry) because it stops
prematurely due to a run-time error. Because of that run-time error I don't
think we can label Ada as stable, yet. Otherwise, Ada is in pretty good
shape compared to the others and run-time errors don't block anything else.
Therefore, as agreed with Jerry off list, I have enabled Ada by default to
encourage much wider testing.
I understand Hez is actively working on the missing ocaml examples and also
bringing example 11 into conformity with the C version.
Of the "mature" languages, it appears Tcl is in the worst shape:
tcl
Missing examples : 19 20 21 23 24 25 26 27 28 29 30
Differing examples : 01 02 03 04 05 06 07 08 09 10 11 12 13 15 16 18
I hope somebody with an interest in Tcl will step forward to add the
additional standard examples (and also add the necessary API those standard
examples exercise).
I looked in detail at the tcl diffs using
(for LIST in `ls x??t.psc |sed 's?t.psc??'`; do echo $LIST; diff
${LIST}[tc].psc; done) |less
Most of the differing examples reported by the new test are false positives
because the tcl examples do some sort of extra reset (of colour?).
But there are real issues I found for examples 2, 11, 15, and 16
with the above command.
* Example 2 has incorrect colours for tcl. A typical diff stanza is
< 0.9961 0.6275 0.0745 C Z
---
> 1.0000 0.6275 0.0745 C Z
I looked deeper into this by printing out intermediate results. What is
going on is the C example is calculating a maximum colour of 254 while the
Tcl example is calculating it as 255 for the case when plhlsrgb is returning
a floating point number that my printout to 9 figures indicates is 1.0000...
I assume the difference is caused by the respective codes truncating (1 +
epsilon) * 255 to the nearest integer where epsilon is a very small negative
number for C and 0 or positive for Tcl. (It is well known that Tcl
floating-point calculations are done quite differently than for compiled
languages.) I don't think we will ever get rid of this general Tcl
floating-point difference with C, but one can avoid such close truncation
decisions in the example by using the factor 255.001 rather than 255. I
made that change to all the language implementations of example 2, and they
now (including Tcl except for the extra reset) all give identical results.
(I have not tested perl, but I did make that simple change there, and I am
sure when somebody tests it against the C result it will be fine.)
* Example 11 appears to have simple rounding differences (PostScript errors
in the last digit). If we used ndiff (fuzzy diff taking into account small
floating point differences, see
http://www.math.utah.edu/~beebe/software/ndiff/) instead of diff we could
presumably get this Tcl example to pass the test (aside from the reset
issue).
N.B. Andrew, I don't think this one example justifies the complications
(cmake find of ndiff, and fall back to diff otherwise with different options
in the two cases) of the ndiff possibility. However, I brought it up because
ndiff is something to keep in mind if people find rounding issues on other
platforms that are more severe than they seem to be for Linux.
* Examples 15 and 16 have quite complex diff patterns. I expect there is
some real implementation difference for both these examples that a Tcl
expert should look at.
Alan
__________________________
Alan W. Irwin
Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).
Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________
Linux-powered Science
__________________________
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Plplot-devel mailing list
Plplot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plplot-devel
