On 13 May 2014 10:48, <[email protected]> wrote: > Hello, > > are here developers in the group who actively test (the routines they use) > e.g. by throwing random input at them? > (like in ticket http://trac.sagemath.org/ticket/1136 ) > > If yes, which other techniques do you use to detect bugs? > > Thanks,
I have done a couple of related things, as I have sometimes had concerns about the Sage testing - I think more so in the past than now. Some of the code was of very dubious quality in my opinion. I recall an email from one developer telling me he did not see much point in testing much, as he would just fix bugs when reported !!! I have several times advocated the use of Sage releases where no new functionality is added - only bugs were fixed. There has been little appetite among developers for this. Anyway, a couple of things I have done. #1 I developed some code to send random data to commands and uncovered some which don't behave as expected. cremona_letter_code for example should only work for non-negative integers, but I found that sage: cremona_letter_code(23.23) # Invalid input, gives output. 'x' sage: cremona_letter_code(-3) # Invalid input, hangs using lots of CPU time. That was http://trac.sagemath.org/ticket/10105 I found a number of examples of functions which could mis-behave if fed invalid input. #2 Some time ago I suggested that we could compare Sage output with output from WolframAlpha to use as a note in doc tests, giving the WolframAlpha URL that gives the same result. I felt WolframAlpha was preferable to Mathematica, since everyone had access to it. Alex Ghitza in particular thought we might be breaking the terms of the license, and on reading it, he could well have had a valid point. So I asked Wolfram Reseach to clarify the license, and if we were permitted to use WolrramAlpha to make comparisons with Sage. They basically said yes, although we can't make automated querying of their servers. That is hardly surprising, as calling their servers thousands or millions of times would load their servers. You can read their response here. https://groups.google.com/forum/#!topic/sage-devel/tijz3svNNiE I don't know if anyone has actually done this, making notes in doc tests of the WolframAlpha / Mathematica output. But it would give some confidence that results are the same. I do recall one function, related to prime testing, where Mathematica was a lot faster than Sage. On closer inspection, it was revealed that the Mathematica algorithm had been proved write for N < $some_value, but there was no proof it worked for all values, whereas the Sage test was more thorough. I reported this to Wolfram Research, who said they would amend the documentation to reflect this. I can't recall whether they did. I think there is some value to comparing Sage and Mathematica output for tests, and perhaps also noting the CPU time used. If Sage is significantly slower, it might indicate there are better algorithms in existence, and those could be investigated. Dr. David Kirkby G8WRB http://www.vnacalibration.co.uk/ Economical & accurate VNA calibration kits. Coefficients available for HP, Agilent, Anritsu, Rohde & Schwarz and VNWA network analyzers. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
