On 11/10/10 04:28 PM, Roger Binns wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 11/10/2010 05:53 AM, Dr. David Kirkby wrote: >> Someone recently said he felt that a test:code ratio of 1:1 is about optimal, > > That of course is bunk.
That was my feeling too. > The optimal amount depends on what the project > does, the resources available, the consequences of bugs, and the values of > the developers. For example for your project I'd guess that accuracy is > very important while running out of memory is not. Yes, both are true, though memory leaks are a sign of bad coding, and should be stopped. > If the project is run on > a buggy FDIV pentium would it even notice? I doubt it. If I believed it was a serious issue, I could write a test for it, but there are FAR more serious issues to worry about in Sage than the possibility of someone having a really old Pentium and stumbling across the bug. > SQLite is a library and hence has no control over the application it runs > in. There may be a lot of memory or it may be a small device with tens of > kilobytes. It may be an application that doesn't really matter if it > crashes, or in one that should never crash. The data may be randomly > generated or very important. Yes, I can see that. > The SQLite developers decided their library will always be reliable and > greatly care about data integrity hence the amount of testing. I wish the Sage developers would take as much care. One recently said something to the effect of "I'd rather not spend hours worrying about how code might fail, when it is so easy to create patches when someone reports a bug" I really think that's a stupid approach, but it's an uphill struggle to get that message across. Part of the problem is most developers are mathmaticians and have very little knowledge of computer science. I try to encourage them to read about software engineering, but that too is difficult. But as I noted before, I believe I've made some inroads, and the lead developer is taking this more seriously now. >> Of course there are practical issues with having a lot of test code - if you >> spend a lot of time writing that, you have less time to add functionality. > > If you have code that is unexercised by your test suite then does it make a > sound when falling in the woods? Sorry, I don't understand the question. > Sorry I mean can you make any assumptions > about it at all? Not 100% reliable ones. But any non-trivial program will have bugs. I don't believe it is totally possible to eliminate all bugs in any non-trivial program. > You are of course using your users as the inefficient test > and hoping they would notice any problems. Depending on the application > this may be ok. Well, its unfortunate that in some cases users wont notice the bug. If I use a word processor and it outputs incorrect text, assuming I proof read it, I will know about the bug. In contrast, if I perform a calculation, such as an integral and the answer is wrong, I may well not know about it. I do personally see it as a serious problem, but I'm also aware that there needs to be an acceptance by users that you can't trust the software 100%. > Note that the standard test suite is written in TCL. Languages like TCL, > Python, Lua etc are significantly more productive especially for being test > harnesses. There is generally no boilerplate since you can automate that > away. > However some environments can not adequately run TCL to test SQLite which is > why there is TH3 that does the tests in C. That C code is generated with > control over how it is done as for example having it all done at once may > produce something to big for the platform to run at once. I see. > At the end of the day the best thing to do is be honest with your users. On > the web site say how much test code there is. Say what percentage of the > codebase it covers. Say if that is just lines or decision points (MCDC). > Say what kinds of testing there are (eg correctness, memory allocation, data > integrity etc). Thank you. That is a good suggestion. We have well over 1000 open bugs. Someone who is an expert in testing computer algebra systems has agreed to use his closed-source code to find us more. Our bug list is public of course http://trac.sagemath.org/ > Roger Thank you Roger. -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? Dave _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
