Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
On Jun 11, 2010, at 2:42 AM, Minh Nguyen wrote: Hi Florent, On Fri, Jun 11, 2010 at 7:33 PM, Florent Hivert florent.hiv...@univ-rouen.fr wrote: SNIP I like this way of seeing. However, I was speaking about module or functions which are not tested nor deprecated and nowhere used into sage (easy to check using grep). Does it make sens to remove them without a deprecation warning ? Many code seems to had been put here, just in case it is useful, and was never used by the sage lib itself, but maybe by some users... Do we agree on the policy: - If a user need a code, he should take care to document and test it. - Corollary: any code which is not tested, nor used can be safely removed without a deprecation warning. There are functions, classes, methods that were introduced into the Sage library well before the policy of 100% doctest coverage was implemented and os completely lack testing. It can be difficult to know if a piece of orphaned code should be removed. I think we need to consider your proposed policy on a module by module basis. +1. Much of that code has been around for years, and so is the least safe to deprecate without warning. Of course, there's a lot of dead code that could be pruned/cleaned, but lets put at least some deprecation warnings in sooner rather than later. Perhaps, if your thinking about coverage, it would be fair to not count deprecated code in that number. - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Is there still a wiki page for people to sign up to deal with one or more of these? Or a standard for trac ticket titles to ensure that effort is not duplicated? I intend to deal with interfaces/mwrank.py (2/10) and databases/cremona.py (17/40) (at least to start with!). John On 12 June 2010 05:26, Minh Nguyen nguyenmi...@gmail.com wrote: Hi Robert, On Fri, Jun 11, 2010 at 9:47 PM, Robert Miller r...@rlmiller.org wrote: SNIP Yes, exactly. Or 5 modules, or 100. I want to go down the list and start writing doctests for the first module I see there which I feel relatively comfortable working on. See the updated coverage report at http://sage.math.washington.edu/home/mvngu/doctest-coverage/ It now has a section called Strategic reports that has various lists of modules. For example, the first list is a strategic list of 180 modules, if all of which were to have full coverage, then the 90% coverage goal would be met. At the moment, 180 modules is the lowest I could get with a quick-and-dirty approach via itertools.combinations() and then get the very first list that satisfies the 90% goal. The problem essentially boils down to the subset sum problem. But my approach so far has been quick-and-dirty. I wanted to present an overview of where in the Sage library one could devote attention to in working towards the 90% goal of Sage 5.0. There are 3 processes running on mod.math at the moment, trying to decrease the list down to say 170, 160, and 150 strategic modules. From the look of it, it could take hours or days before any of these three processes return a lower strategic list than 180 modules. -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi John, I don't have an answer to your questions, but... On Sat, 12 Jun 2010 14:38:52 +0100, John Cremona john.crem...@gmail.com wrote: I intend to deal with interfaces/mwrank.py (2/10) and databases/cremona.py (17/40) (at least to start with!). ... have a look at #9223, I have just posted a patch that brings the coverage of databases/cremona.py to 34/40. Best, Alex -- Alex Ghitza -- http://aghitza.org/ Lecturer in Mathematics -- The University of Melbourne -- Australia -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi David, On Fri, Jun 11, 2010 at 7:21 AM, Dr. David Kirkby david.kir...@onetel.net wrote: SNIP I think their test procedures are a bit over the top, but it certainly brings in to perspective how some developers feel about testing. More testing is good. The SQLite team certainly has a good variety of tests. It's something we need to learn from. I must admit, reading that Wolfram Research page, the statement that The standards of correctness for Mathematica are certainly much higher than for typical mathematical proofs is extremely stupid, when they don't define typical and they provide no evidence of it. (It was not me he spotted that, but it is extremely dumb thing to write) Perhaps this [1] could explain that dumb statement. But really the specification, implementation and testing should be done by different people. In practice, that is not going to happen in Sage, though I would not be surprised if that happens with Mathematica, since it is pretty standard technique in software engineering. That is what I have been trying to do with Nathann Cohen's graph theory patches. He implements features, I write tests and documentation and try to break his code. That way, at least two people know about the new feature that's going into Sage. I recently wrote a reviewer patch [2] that is more than twice the size of his feature patch. So far, I have yet to hear Nathann complain about the number and size of the tests I wrote corresponding to his new features. Of course, I also tried to do code coverage testing: each execution branch should be tested. [1] http://www1.qainsight.net:8080/2006/08/31/SoftwareDevelopmentComicWithQAAddition.aspx [2] http://trac.sagemath.org/sage_trac/ticket/8922 -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi David, On Fri, Jun 11, 2010 at 8:32 AM, Dr. David Kirkby david.kir...@onetel.net wrote: SNIP Consider two areas # interfaces/tachyon.py: 0% (0 of 4) # graphs/generic_graph.py: 99% (200 of 201) Where would it be most useful to add one doc test? At least from my very little understanding of this, Having 89% coverage would be better than 90% coverage, if those 89% were well targeted. From test-oriented software development: Any function that is not tested is considered broken. That includes functions, classes, methods defined using cdef and cpdef. -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi Florent, On Fri, Jun 11, 2010 at 10:07 AM, Florent Hivert florent.hiv...@univ-rouen.fr wrote: SNIP They all looks like they should be deprecated and removed... If it's true I rather improving the doctest coverage by removing them than adding doctests... However I'd like to have the confirmation that they are indeed obsolete... We are aiming for a Sage 5.0 release. The major version number says it: don't expect backwards compatibility. So whatever functions, methods, classes, modules that are deprecated should be removed in Sage 5.0. Now it's time to take stock of deprecated stuff that are over 60 months old and plan to remove them. Let's be brutal wherever we can :-) -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi Minh, They all looks like they should be deprecated and removed... If it's true I rather improving the doctest coverage by removing them than adding doctests... However I'd like to have the confirmation that they are indeed obsolete... We are aiming for a Sage 5.0 release. The major version number says it: don't expect backwards compatibility. So whatever functions, methods, classes, modules that are deprecated should be removed in Sage 5.0. Now it's time to take stock of deprecated stuff that are over 60 months old and plan to remove them. Let's be brutal wherever we can :-) I like this way of seeing. However, I was speaking about module or functions which are not tested nor deprecated and nowhere used into sage (easy to check using grep). Does it make sens to remove them without a deprecation warning ? Many code seems to had been put here, just in case it is useful, and was never used by the sage lib itself, but maybe by some users... Do we agree on the policy: - If a user need a code, he should take care to document and test it. - Corollary: any code which is not tested, nor used can be safely removed without a deprecation warning. Cheers, Florent -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi Florent, On Fri, Jun 11, 2010 at 7:33 PM, Florent Hivert florent.hiv...@univ-rouen.fr wrote: SNIP I like this way of seeing. However, I was speaking about module or functions which are not tested nor deprecated and nowhere used into sage (easy to check using grep). Does it make sens to remove them without a deprecation warning ? Many code seems to had been put here, just in case it is useful, and was never used by the sage lib itself, but maybe by some users... Do we agree on the policy: - If a user need a code, he should take care to document and test it. - Corollary: any code which is not tested, nor used can be safely removed without a deprecation warning. There are functions, classes, methods that were introduced into the Sage library well before the policy of 100% doctest coverage was implemented and os completely lack testing. It can be difficult to know if a piece of orphaned code should be removed. I think we need to consider your proposed policy on a module by module basis. -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi Robert, On Fri, Jun 11, 2010 at 12:34 PM, Robert Miller r...@rlmiller.org wrote: Minh, Can you make a report which lists the files which, if brought up to 100% coverage, would benefit overall coverage the most? Here is my understanding of what you want. Let's say the Sage community has enough time to write tests for 20 modules. Which 20 modules could we choose to write tests for such that it results in the greatest overall weighted coverage for the Sage library. In that case, I think such a report can be implemented. Please tell me if I have misunderstood your feature request. -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi, I like this way of seeing. However, I was speaking about module or functions which are not tested nor deprecated and nowhere used into sage (easy to check using grep). Does it make sens to remove them without a deprecation warning ? Many code seems to had been put here, just in case it is useful, and was never used by the sage lib itself, but maybe by some users... Do we agree on the policy: - If a user need a code, he should take care to document and test it. - Corollary: any code which is not tested, nor used can be safely removed without a deprecation warning. There are functions, classes, methods that were introduced into the Sage library well before the policy of 100% doctest coverage was implemented and os completely lack testing. It can be difficult to know if a piece of orphaned code should be removed. I think we need to consider your proposed policy on a module by module basis. Ok ! That was the purpose of my first e-mail. Let's do it: sage/monoids/monoid.py sage/structure/element_py.py sage/structure/element_verify.py sage/misc/typecheck.py They all looks like they should be deprecated and removed... If it's true I rather improving the doctest coverage by removing them than adding doctests... However I'd like to have the confirmation that they are indeed obsolete... This remark also hold for the following typecheck function in sage/misc/misc.py # # Type checking # def typecheck(x, C, var=x): Check that x is of instance C. If not raise a TypeError with an error message. if not isinstance(x, C): raise TypeError, %s (=%s) must be of type %s.%(var,x,C) Florent -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Minh, On Fri, Jun 11, 2010 at 2:49 AM, Minh Nguyen nguyenmi...@gmail.com wrote: Here is my understanding of what you want. Let's say the Sage community has enough time to write tests for 20 modules. Which 20 modules could we choose to write tests for such that it results in the greatest overall weighted coverage for the Sage library. In that case, I think such a report can be implemented. Please tell me if I have misunderstood your feature request. Yes, exactly. Or 5 modules, or 100. I want to go down the list and start writing doctests for the first module I see there which I feel relatively comfortable working on. -- Robert L. Miller http://www.rlmiller.org/ -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi Florent, On Fri, Jun 11, 2010 at 9:32 PM, Florent Hivert florent.hiv...@univ-rouen.fr wrote: SNIP sage/monoids/monoid.py I think this module should stay put. Here is a dependency chart based on that module: monoids.monoid.Monoid_class -- monoids.free_monoid.FreeMonoid_class -- monoids.string_monoid.StringMonoid_class where the relation a -- b means that a is used by b. From monoids.string_monoid.StringMonoid_class, various alphabets are defined. These alphabets are extensively used in various cryptography modules, e.g. * crypto/classical_cipher.py * crypto/classical.py * crypto/cryptosystem.py * crypto/stream_cipher.py * crypto/util.py * crypto/public_key/blum_goldwasser.py * crypto/block_cipher/miniaes.py * crypto/block_cipher/sdes.py sage/structure/element_py.py I have looked through that file and have doctested the Sage 4.4.4.alpha0 library with that file removed. So far no failures, which doesn't mean that structure/element_py.py isn't used anywhere else. But then again, I don't understand the logic of this module. Some people maintain that the documentation is in the code and terseness is desirable. In the case of structure/element_py.py, I just feel stupid and ignorant about what it's supposed to do. Remember, a reason why you meticulously document your code is that months later, when you're no longer interested in maintaining that piece of code, you don't want to hear complaints about how to use your code and/or the purpose it serves. Here's a worst case scenario taken from “Lecture 12: FOSS Culture” of the ANU course [1]: quote From: x Date: Fri, 9 Feb 2001 21:07:09 +0900 To: ru...@rustcorp.com.au Subject: Dear Sir, I have always wanted to flame you, however I never really had any good reason to do so. While your software creations were always egoistical and underdocumented, nothing really pushed the mark, yet. Until today, when I downloaded your “Linux 2.4″ call graph utility. So, when is your next absolute-next-to-worthless but oh-so-cool-because-it’s-from-rusty-russell piece of software coming out? Why the f*** don’t you focus on documenting things instead of writing useless shit that “takes about 8 hours to run on my mobile pII laptop” or “generates about 180mb of vector postscript”. How about another example. I heard you were involved with that atrocity called “netfilter”. Sure, it might have nice features and I am still considering using it, but WRITE SOME F***ING DOCS before you release complex shit like this for people to use! When I goto that netfilter site, I don’t give a flying raging f*** who submitted 31337 lines of code to whatever f***ing netfilter module. That’s your egoistical shit, and I could care less about it. …… (more rants delted) … Uhm, that’s all I can think of right now. xxx (name deleted) /quote Indeed, a complaint with strong language. But you get the general idea about having good documentation. For reference, I have collected some resources [2] on how to write good documentation. sage/structure/element_verify.py I get the feeling that this module is about code analysis for Cython modules. I think it might come in handy later on when someone needs to write a code analysis framework. They might not use all of structure/element_verify.py, but I suspect that that module could provide a starting point on how to proceed. But then again, there are various open source code analysis tools around for Python. I'm not sure about Cython. For Python, I often use these tools when I review patches or improve the Sage library: * pep8 * pychecker * pyflakes * pylint * flymake They're good at catching common mistakes, errors, omissions, etc. I would like to know of any similar tools for Cython. In the absence of such tools for Cython, I guess I would continue on spending lots of time meticulously study Cython code, essentially manually trying to perform tasks that could otherwise be automated via a lint-like tool. sage/misc/typecheck.py As with structure/element_py.py, the module misc/typecheck.py just makes me feel stupid when it comes time for me to try to use it. A tool that makes one feel stupid isn't making a good impression on that person, and would drive the person away from using it. To me, the module is very under-documented, doesn't follow PEP 008 coding conventions, and very badly formatted. I let others in the Sage community form their own judgements about the removal of this module. This remark also hold for the following typecheck function in sage/misc/misc.py # # Type checking # def typecheck(x, C, var=x): Check that x is of instance C. If not raise a TypeError with an error message. if not isinstance(x, C): raise TypeError, %s (=%s) must be of type %s.%(var,x,C) This looks to me like, how can I put it, redundant code. I'm basing my judgement on the idea that I would
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi Minh, Thanks for carefully investigating those: sage/monoids/monoid.py I think this module should stay put. Here is a dependency chart based on that module: monoids.monoid.Monoid_class -- monoids.free_monoid.FreeMonoid_class -- monoids.string_monoid.StringMonoid_class where the relation a -- b means that a is used by b. From monoids.string_monoid.StringMonoid_class, various alphabets are defined. These alphabets are extensively used in various cryptography modules, e.g. * crypto/classical_cipher.py * crypto/classical.py * crypto/cryptosystem.py * crypto/stream_cipher.py * crypto/util.py * crypto/public_key/blum_goldwasser.py * crypto/block_cipher/miniaes.py * crypto/block_cipher/sdes.py Right now your are right ! However if you read the doc, it says that it should be removed after making a proper use of categories... So it should be deprecated in the long run... sage/structure/element_py.py I have looked through that file and have doctested the Sage 4.4.4.alpha0 library with that file removed. So far no failures, which doesn't mean that structure/element_py.py isn't used anywhere else. One more indication: there isn't any reference to element_py anywhere in sage: tomahawk-*el/sage-combinat $ grep element_py **/*.py* Fichier binaire build/sage/structure/element_py.pyc concorde Indeed, a complaint with strong language. But you get the general idea about having good documentation. For reference, I have collected some resources [2] on how to write good documentation. Sure !!! You don't need to convince me. sage/structure/element_verify.py I get the feeling that this module is about code analysis for Cython modules. I think it might come in handy later on when someone needs to write a code analysis framework. They might not use all of structure/element_verify.py, but I suspect that that module could provide a starting point on how to proceed. It seems to me completely redudant with the TestSuite/category framework. Once again let's keep it until things got adapted to those framework. sage/misc/typecheck.py As with structure/element_py.py, the module misc/typecheck.py just makes me feel stupid when it comes time for me to try to use it. A tool that makes one feel stupid isn't making a good impression on that person, and would drive the person away from using it. To me, the module is very under-documented, doesn't follow PEP 008 coding conventions, and very badly formatted. I let others in the Sage community form their own judgements about the removal of this module. This remark also hold for the following typecheck function in sage/misc/misc.py # # Type checking # def typecheck(x, C, var=x): Check that x is of instance C. If not raise a TypeError with an error message. if not isinstance(x, C): raise TypeError, %s (=%s) must be of type %s.%(var,x,C) This looks to me like, how can I put it, redundant code. I'm basing my judgement on the idea that I would write such code at the point where I want to do sanity checking, instead of calling a function that does the same thing, but doesn't allow me to customize my exception message. I would personally vote for removal of this function. Let's wait for more vote... Cheers, Florent -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
On Jun 11, 2010, at 2:33 AM, Florent Hivert wrote: Hi Minh, They all looks like they should be deprecated and removed... If it's true I rather improving the doctest coverage by removing them than adding doctests... However I'd like to have the confirmation that they are indeed obsolete... We are aiming for a Sage 5.0 release. The major version number says it: don't expect backwards compatibility. So whatever functions, methods, classes, modules that are deprecated should be removed in Sage 5.0. Now it's time to take stock of deprecated stuff that are over 60 months old and plan to remove them. Let's be brutal wherever we can :-) This brings up an orthogonal issue--if we're going to try to get lots of publicity for 5.0, wouldn't it be better to focus on stability than use the chance to break backwards compatibility? I like this way of seeing. However, I was speaking about module or functions which are not tested nor deprecated and nowhere used into sage (easy to check using grep). Does it make sens to remove them without a deprecation warning ? Many code seems to had been put here, just in case it is useful, and was never used by the sage lib itself, but maybe by some users... Do we agree on the policy: - If a user need a code, he should take care to document and test it. - Corollary: any code which is not tested, nor used can be safely removed without a deprecation warning. The only code without doctests is (should be) old code that went in before the 100% doctest policy, so I don't think we can just go and delete things like this (especially really old code, which often but not always has lots of stuff sitting on top of it.) sage/monoids/monoid.py I'm pretty sure this is actually used. sage/structure/element_py.py Deprecate for sure. sage/structure/element_verify.py Probably deprecate, putting any relevant functionality into the coverage script. sage/misc/typecheck.py This was added just last version? Perhaps ask the author Dmitry Dvoinikov. Note that just deleting a file and running tests isn't sufficient, as one would have to purge the .py and .pyc (or .so) files from all the build directories as well. - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi Robert, On Fri, Jun 11, 2010 at 9:47 PM, Robert Miller r...@rlmiller.org wrote: SNIP Yes, exactly. Or 5 modules, or 100. I want to go down the list and start writing doctests for the first module I see there which I feel relatively comfortable working on. See the updated coverage report at http://sage.math.washington.edu/home/mvngu/doctest-coverage/ It now has a section called Strategic reports that has various lists of modules. For example, the first list is a strategic list of 180 modules, if all of which were to have full coverage, then the 90% coverage goal would be met. At the moment, 180 modules is the lowest I could get with a quick-and-dirty approach via itertools.combinations() and then get the very first list that satisfies the 90% goal. The problem essentially boils down to the subset sum problem. But my approach so far has been quick-and-dirty. I wanted to present an overview of where in the Sage library one could devote attention to in working towards the 90% goal of Sage 5.0. There are 3 processes running on mod.math at the moment, trying to decrease the list down to say 170, 160, and 150 strategic modules. From the look of it, it could take hours or days before any of these three processes return a lower strategic list than 180 modules. -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
On Thu, Jun 10, 2010 at 2:21 PM, Dr. David Kirkby david.kir...@onetel.net wrote: On 06/10/10 09:25 PM, Minh Nguyen wrote: Hi folks, One of the main goals of the upcoming Sage 5.0 release is to get doctest coverage of the Sage library up to at least 90%. As of Sage 4.4.4.alpha0, the overall weighted coverage is 82.7%. Seems we are a long way off. It seems to me, rather than pick a number like 90%, the areas should be targeted carefully. The goal for Sage-5.0 is 90% coverage. And we should aim for 100% by the end of the year. -- William -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
On 06/10/10 09:25 PM, Minh Nguyen wrote: Hi folks, One of the main goals of the upcoming Sage 5.0 release is to get doctest coverage of the Sage library up to at least 90%. As of Sage 4.4.4.alpha0, the overall weighted coverage is 82.7%. Seems we are a long way off. It seems to me, rather than pick a number like 90%, the areas should be targeted carefully. To get a sense of which modules in the Sage library need work on their coverage scores, you could use the coverage script as follows: $ ./sage -coverage /path/to/module.py[x] Or you could do the following to get the coverage scores of all modules, including a coverage summary: $ ./sage -coverageall You might be interested in knowing which modules have a certain coverage percentage, in which case you could save the output of -coverageall to a text file and then grep that file for certain coverage scores. At this repository [1] is a script to generate various types of coverage analysis reports. You can also find the script at [3]. The script currently supports the following reports * The coverage summary of all modules. * Modules with 100% coverage. * Modules with zero coverage. I don't really understand these docs tests well, but to me at least, concentrating on modules which have zero coverage would seem best, even if only one doctest/module is added. My logic being something could be totally broken, and we would never know about it. At least if there is even one doctest, it shows it is not totally broken. (Though one could argue being totally broken is better than being partially broken. At least one finds out in use.) It was recently discovered that certain modules (matrix, class, mgcv, nnet, rpart, spatial, and survival) in R are not building on Solaris. http://trac.sagemath.org/sage_trac/ticket/9201 Had there been even one doctest for each module, this would have been obvious. There is an interesting information about the SQlite database (incluced in Sage) is tested. The number of lines of test code is 679 times that of the actual source code of the project. The number of lines of test code is 45.7 million, the number of lines of source code in the database is 67000! (Both exclude comments and blank lines) http://www.sqlite.org/testing.html I think their test procedures are a bit over the top, but it certainly brings in to perspective how some developers feel about testing. If what is written at http://reference.wolfram.com/mathematica/tutorial/TestingAndVerification.html about testing Mathematica is true, then the following statement is intesting There is also a special instrumented version of Mathematica which is set up to perform internal consistency tests. This version of Mathematica runs at a small fraction of the speed of the real Mathematica, but at every step it checks internal memory consistency, interruptibility, and so on I must admit, reading that Wolfram Research page, the statement that The standards of correctness for Mathematica are certainly much higher than for typical mathematical proofs is extremely stupid, when they don't define typical and they provide no evidence of it. (It was not me he spotted that, but it is extremely dumb thing to write) Of course we can't verify the claims made by Wolfram Reserach, but we can verify what the SQlite developers say, that the number of lines of test code is 679 x the amount of actual code in the database. Even I would refuse to write 679 lines of test code for every line of code I wrote! But really the specification, implementation and testing should be done by different people. In practice, that is not going to happen in Sage, though I would not be surprised if that happens with Mathematica, since it is pretty standard technique in software engineering. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
On 06/10/10 10:27 PM, William Stein wrote: On Thu, Jun 10, 2010 at 2:21 PM, Dr. David Kirkby david.kir...@onetel.net wrote: On 06/10/10 09:25 PM, Minh Nguyen wrote: Hi folks, One of the main goals of the upcoming Sage 5.0 release is to get doctest coverage of the Sage library up to at least 90%. As of Sage 4.4.4.alpha0, the overall weighted coverage is 82.7%. Seems we are a long way off. It seems to me, rather than pick a number like 90%, the areas should be targeted carefully. The goal for Sage-5.0 is 90% coverage. And we should aim for 100% by the end of the year. -- William Consider two areas # interfaces/tachyon.py: 0% (0 of 4) # graphs/generic_graph.py: 99% (200 of 201) Where would it be most useful to add one doc test? At least from my very little understanding of this, Having 89% coverage would be better than 90% coverage, if those 89% were well targeted. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Hi Minh, And you're done. Here [2] is a report generated by the script. The idea is to provide an overview of which modules need work. I'd be interested to know what other types of doctest coverage reports people would like to see. Comments, suggestions, critiques, etc. are welcome. This reports definitely looks like a good idea ! However, I tried to pick-up some random files in the lists: sage/monoids/monoid.py sage/structure/element_py.py sage/structure/element_verify.py sage/misc/typecheck.py They all looks like they should be deprecated and removed... If it's true I rather improving the doctest coverage by removing them than adding doctests... However I'd like to have the confirmation that they are indeed obsolete... This remark also hold for the following typecheck function in sage/misc/misc.py # # Type checking # def typecheck(x, C, var=x): Check that x is of instance C. If not raise a TypeError with an error message. if not isinstance(x, C): raise TypeError, %s (=%s) must be of type %s.%(var,x,C) Cheers, Florent -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
Minh, Can you make a report which lists the files which, if brought up to 100% coverage, would benefit overall coverage the most? On Thu, Jun 10, 2010 at 1:25 PM, Minh Nguyen nguyenmi...@gmail.com wrote: Hi folks, One of the main goals of the upcoming Sage 5.0 release is to get doctest coverage of the Sage library up to at least 90%. As of Sage 4.4.4.alpha0, the overall weighted coverage is 82.7%. To get a sense of which modules in the Sage library need work on their coverage scores, you could use the coverage script as follows: $ ./sage -coverage /path/to/module.py[x] Or you could do the following to get the coverage scores of all modules, including a coverage summary: $ ./sage -coverageall You might be interested in knowing which modules have a certain coverage percentage, in which case you could save the output of -coverageall to a text file and then grep that file for certain coverage scores. At this repository [1] is a script to generate various types of coverage analysis reports. You can also find the script at [3]. The script currently supports the following reports * The coverage summary of all modules. * Modules with 100% coverage. * Modules with zero coverage. * Modules with between 1% and 9% coverage. * Modules with between 10% and 19% coverage. * Modules with between 20% and 29% coverage. * Modules with between 30% and 39% coverage. * Modules with between 40% and 49% coverage. * Modules with between 50% and 59% coverage. * Modules with between 60% and 69% coverage. * Modules with between 70% and 79% coverage. * Modules with between 80% and 89% coverage. * Modules with between 90% and 99% coverage. Each report has links to detailed reports for individual modules. To run the script, copy it to the SAGE_ROOT of a Sage source or binary installation and do [mv...@sage sage-4.4.4.alpha0]$ ./coverage-status.py Coverage report of all modules... Summary of doctest coverage... Modules with 0% coverage... Modules with 100% coverage... Coverage reports within certain ranges... Detailed coverage report for all modules... Format the detailed coverage reports... Format the summary reports... Generate index.html... And you're done. Here [2] is a report generated by the script. The idea is to provide an overview of which modules need work. I'd be interested to know what other types of doctest coverage reports people would like to see. Comments, suggestions, critiques, etc. are welcome. [1] http://bitbucket.org/mvngu/coverage [2] http://sage.math.washington.edu/home/mvngu/doctest-coverage/ [3] http://sage.math.washington.edu/home/mvngu/apps/coverage/ -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org -- Robert L. Miller http://www.rlmiller.org/ -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] pushing towards 90% doctest coverage for Sage 5.0
On Jun 10, 2010, at 2:21 PM, Dr. David Kirkby wrote: On 06/10/10 09:25 PM, Minh Nguyen wrote: Hi folks, One of the main goals of the upcoming Sage 5.0 release is to get doctest coverage of the Sage library up to at least 90%. As of Sage 4.4.4.alpha0, the overall weighted coverage is 82.7%. Seems we are a long way off. It seems to me, rather than pick a number like 90%, the areas should be targeted carefully. 90% is a very nice global goal to have--it's something concrete to shoot for while still letting everyone work on what they like/think is most important. That being said, I agree with you that I'd rather people write tests for more valuable areas rather than easy ones is certainly worthwhile. To get a sense of which modules in the Sage library need work on their coverage scores, you could use the coverage script as follows: $ ./sage -coverage /path/to/module.py[x] Or you could do the following to get the coverage scores of all modules, including a coverage summary: $ ./sage -coverageall You might be interested in knowing which modules have a certain coverage percentage, in which case you could save the output of -coverageall to a text file and then grep that file for certain coverage scores. At this repository [1] is a script to generate various types of coverage analysis reports. You can also find the script at [3]. The script currently supports the following reports * The coverage summary of all modules. * Modules with 100% coverage. * Modules with zero coverage. I don't really understand these docs tests well, but to me at least, concentrating on modules which have zero coverage would seem best, even if only one doctest/module is added. My logic being something could be totally broken, and we would never know about it. At least if there is even one doctest, it shows it is not totally broken. (Though one could argue being totally broken is better than being partially broken. At least one finds out in use.) +1, though of course some files (e.g. tachyon) are indirectly tested, so it's hard to tell just looking at the numbers alone. - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
