Re: [Plplot-devel] Wishlist: a verbose testing option
On Tue, Aug 12, 2008 at 03:45:39PM -0700, Alan Irwin wrote: > On 2008-08-11 19:59-0700 Alan W. Irwin wrote: > > > On 2008-08-11 14:43-0600 Orion Poplawski wrote: > > > >> Wishlist: a verbose testing option that > >> would output the actual command executed for each test. > > I have implemented a proof of concept of this idea (revision 8636) for our C > examples. Please try it out and let me know what you think. > > Andrew, as a major user of the ctest command, I especially ask you to have a > look at how I did this (since I am no scripting expert) and the results for > ctest --verbose. Please feel free to change any part of my implementation. > If you like what you now see for the C examples (and nobody else objects to > how much output there is now with the ctest --verbose command), I will > propagate the idea to the rest of the test_*.sh.in scripts except for the > octave one. Alan, In principal I like the idea of being able to see the examples as they are exectuted. I wonder whether the whole command line is necessary though. Perhaps just echoing the example name would suffice and would prevent too much unnecessary clutter in the output? For testing purposes this is what is required. For a given test you know what the driver is and what the output files will be. I find looking through Testing/Temporary/LastTest.log useful and all this extra information could make it more difficult to find useful error messages. > For test_octave.sh.in I will rely on you for that needed propagation since > octave expertise is required to read in the verbose_test environment > variable into octave and echo the commands to generate the various "p" and > "x" octave examples. > > Alternatively, it might be a good idea to reduce test_octave.sh.in as much > as possible to the form of the other test scripts so, e.g., the looping > through the various examples is done in the shell rather than in octave. > Obviously, there will be a bit more latency with that approach, but > hopefully it would not be noticeable, and I think it would be valuable to > remove some of the mystery from test_octave.sh.in by doing as much as > possible in the shell. Note, I committed the original version of > test_octave.sh, but that was under heavy instruction from our octave expert > at that time, Jaoa Cardoso, and I will be the first to admit I had no clue > what I was doing. :-) So I think a complete rewrite of test_octave.sh.in by > our current octave expert (you) would be a good idea. I am inclined to stay with the current single instance of octave. The overhead of starting octave ~40 times is not insignificant. Once we agree on a format I can add the necessary octave code. Andrew - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] ndiff trial
On Tue, Aug 12, 2008 at 07:48:54PM -0700, Alan Irwin wrote: > On 2008-08-11 21:39-0700 Alan W. Irwin wrote: > > > On 2008-08-11 22:09+0100 Andrew Ross wrote: > >> For example 21, some of the differences are just rounding errors. I do > >> however see some additional line at the end like. This is from plot3d. > >> I also see similar extra lines from the java and octave versions on a > >> different machine, so again I suspect a rounding error. > > > > I am going to have a look at ndiff > > (http://www.math.utah.edu/~beebe/software/ndiff/) which is advertised to > > ignore rounding errors. I will report back later on its suitability for > > our ctest needs. > > This project seems to be built with autoconf only so it has a nostalgic feel > (the latest release appears to be from 2000). The build, check, and install > were pretty straightforward, but I had to create $prefix/bin and > $prefix/man/man1 directories before the install would work properly for the > special install prefix I specified to the configure script. > > One of the ndiff command line options is to match lines if the relative > differences are less than some limit you specify. > > [snip] > > So from this you know there are some real structural differences between the > two files, and you get a nice summary of the maximum relative difference > that was less than 1.d-03 at the end which shows in this case that most > of the differences between the Tcl and C results are due to rounding. > > So ndiff may be an optional alternative to diff that we want to try for our > last ctest, but before we make that decision, I encourage the developers > here to try it since it is at least useful for quickly spotting real > differences between files which are ordinarily obfuscated by rounding > differences. > > BTW, ndiff is GPLed software. Andrew, there apparently is a name clash with > another Debian package called ndiff, but that sort of issue should be > solvable (say by adopting a variant of the name). Thus, I don't really > understand why this useful variant of diff has not been packaged for Debian. Alan, Following your recommendation I also took a look at the sight. It certainly sounds useful for our purposes. I am tempted to add support for ndiff where available. A third test for examples with only small differences would be useful to sort out the real programming differences from the rounding errors. Andrew - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Wishlist: a verbose testing option
Andrew Ross wrote: > In principal I like the idea of being able to see the examples as they > are exectuted. I wonder whether the whole command line is necessary > though. Perhaps just echoing the example name would suffice and would > prevent too much unnecessary clutter in the output? For testing purposes > this is what is required. For a given test you know what the driver is > and what the output files will be. I find looking through > Testing/Temporary/LastTest.log useful and all this extra information > could make it more difficult to find useful error messages. Perhaps another --debug option? What I find hard is when a test is segfaulting finding the command I need to execute to reproduce it directly so that I can run it under gdb. I agree for "normal" operation the command is not necessary and may be harmful. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA DivisionFAX: 303-415-9702 3380 Mitchell Lane [EMAIL PROTECTED] Boulder, CO 80301 http://www.cora.nwra.com - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Plplot examples and ifort
On Mon, Aug 11, 2008 at 09:39:41PM -0700, Alan Irwin wrote: > On 2008-08-11 22:09+0100 Andrew Ross wrote: > > > > > I've been testing the fortran examples using a rather old version of > > ifort (8.1) on 64-bit linux. In order to get the examples to compile and > > run I encountered two issues > > > > 1) Example 20 requires the -assume byterecl flag to ifort in order to > > get byte length records. Otherwise reading lena.pgm fails with an error. > > This is in accord with the comment in the example. > > > > 2) Example 21 requires the -O0 flag, otherwise the compiler optimises > > away the test (x.ne.x) in the function myisnan. > > > > For 1) it would be nice to automatically add this option if ifort is > > detected. For 2) the flag to turn off optimisation is a bit of a fix. > > ifort does have an isnan intrinsic (along with many other compilers) > > which we could use instead. This requires some extra cmake logic though. > > > > Any suggestions? > > (1) Normally, I believe it should be the user's responsibility to set the > flags appropriate for their compiler. > > So what happens if you simply use > > FC='ifort -assume byterecl' > > ? > > If that works for all examples without messing them up, then perhaps we > should recommend that ifort option in the wiki along with an explanation > that example 20 requires byte-length records. That seems simpler to me than > detecting ifort and fooling around with a special compile option for example > 20 for that compiler. However, I don't feel strongly about it so do what > you think is best. I think you are probably right. The other option is to try and come up with a way of reading in lena.pgm which is not dependent on the record length. > > (2) I think your idea of detecting whether the isnan intrinsic is available > is a good one since -O0 does seem like a workaround. Most of the work is > already done using the CheckFortranFunctionExists.cmake file for CMake-2.6.0 > (which I know about since I wrote the first version of it for the FreeEOS > project). For now, we can simply copy that file from CMake-2.6.0 modules to > our cmake/modules directory so that those with 2.4.8 can still use PLplot. I agree this is a much better solution. On the TODO list, unless anyone else gets there first. Andrew - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Wishlist: a verbose testing option
On 2008-08-13 22:13+0100 Andrew Ross wrote: > In principal I like the idea of being able to see the examples as they > are exectuted. I wonder whether the whole command line is necessary > though. Perhaps just echoing the example name would suffice and would > prevent too much unnecessary clutter in the output? For testing purposes > this is what is required. For a given test you know what the driver is > and what the output files will be. Thanks for that feedback which has convinced me terse is better. I have therefore changed test_c.sh.in accordingly. I have also made a change there so the extra output you get in verbose mode occurs in the natural position before the execution of the command rather than afterward. As of revision 8644 I have used a similar style for all the other test*.in files except test_octave.sh.in which I made no changes to. > [...]Once we agree on a format I can add the necessary octave code. Thanks in advance. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Wishlist: a verbose testing option
On 2008-08-13 15:19-0600 Orion Poplawski wrote: > Andrew Ross wrote: >> In principal I like the idea of being able to see the examples as they >> are exectuted. I wonder whether the whole command line is necessary >> though. Perhaps just echoing the example name would suffice and would >> prevent too much unnecessary clutter in the output? For testing purposes >> this is what is required. For a given test you know what the driver is >> and what the output files will be. I find looking through >> Testing/Temporary/LastTest.log useful and all this extra information >> could make it more difficult to find useful error messages. > > Perhaps another --debug option? What I find hard is when a test is > segfaulting finding the command I need to execute to reproduce it directly so > that I can run it under gdb. I agree for "normal" operation the command is > not necessary and may be harmful. Sorry, Orion, I missed your comment before I made the changes. There are two possible use cases here. Just as a general principle, I think debugging in the install tree is just a whole lot cleaner than in the build tree so I highly recommend it. Doing gdb sessions in the install tree is quite simple because no environment variables have to be set, and you know where everything is. For that case, your suggested --debug option for plplot-test.sh only has limited usefulness for those who are already familiar with running examples in the install tree. However, I suspect you were considering just the build-tree case when you made your proposal, and see below for the limitations in that case. Doing gdb sessions in the build tree is pretty tricky because of all the setup (e.g., setting environment variables) you have to do first. So I don't recommend it except for rare emergencies where you get errors in the build tree but not the install tree. Furthermore, ctest doesn't have a facility to pass options on to plplot-test.sh so we have to choose the plplot-test.sh options for the build tree at cmake time which would not be very convenient for you. So for the above reasons I am not completely keen on your idea. However, if you still are keen after considering what I have said above, then please send me a patch with the implementation (not only the changes to plplot-test.sh.in, but the extra logic required in the various test_*.sh.in files). Assuming the patch looks okay, I can deal with the cmake details so that there is a cmake option such that ctest runs plplot-test.sh with both the --verbose and --debug options. Alan __ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
Re: [Plplot-devel] Plplot examples and ifort
Andrew Ross wrote: (1) Normally, I believe it should be the user's responsibility to set the flags appropriate for their compiler. So what happens if you simply use FC='ifort -assume byterecl' ? If that works for all examples without messing them up, then perhaps we should recommend that ifort option in the wiki along with an explanation that example 20 requires byte-length records. That seems simpler to me than detecting ifort and fooling around with a special compile option for example 20 for that compiler. However, I don't feel strongly about it so do what you think is best. I think you are probably right. The other option is to try and come up with a way of reading in lena.pgm which is not dependent on the record length. The problem with the PGM P5 format from the point of view of Fortran is, that it is a strange mixture of a formatted file (all the header lines) and a binary file (the actual image data). Also, the file is not closed with an end-of-line character sequence. All these details make it very awkward to read it properly - from a Fortran program. Having said that, I think it _is_ possible to adapt the current program to read it even if the record length for direct access files is measured in words rather than bytes. The problem is merely a matter of getting the logic right (detect the record length in terms of bytes, read the records and extract the individual bytes). Regards, Arjen - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/___ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
