Niranjan Kulkarni wrote: > > -------Original Message------- > > From: David Hamill > Date: 01/07/09 10:25:44 > To: [email protected] > Subject: Re: [c-prog] Re: Good books for software testing > > John wrote: >> I quite like the idea of writing the tests first. > > The problem with this is that you don't know what to test > For until you've written the code. > > Maybe some much more organised people than me can specify > Exactly what they want upfront, then write the tests to > Reflect the specification, then write the code to pass the ... > > Hi David, > > I liked the response and I think the similar. > It is really difficult to anticipate how software will turn out when it is > completed. > But according to ideal process, we should write test cases depending on use > cases before coding right? > Of course we have to assume that the use cases and specification document is > perfect and will tno change later. > > Thanks, > Niranjan.
While I generally agree with the above, the one area that is a HUGE pet peeve for me are the Standards. I'm not talking about just C/C++, I'm talking about ALL Standards. And the IETF RFCs are the worst of the lot. You would think that, after the 5 zillionth incorrect implementation of the HTTP protocol Standard, someone at the IETF would have the "brilliant" idea of writing tests and creating a test suite for each and every Standard. The most tests I've seen in an IETF RFC is three tests - but tests don't belong within a Standard itself. This is, among a multitude of other reasons, why I'm wholly convinced that idiots run the IETF. To date I still cannot search Google for "[Standard name] test suite" and actually find a complete test suite. Or "test suite database" and find a database of test suites, which tells me that it doesn't exist. I'll find bits and pieces of things people have put together...if I'm even that lucky. Which is pretty much never. The most complete, intact, easy to use test suite I have ever downloaded was for the BMP file format (there are about 500 different ways to screw up a BMP implementation and every wrong way has been implemented). But I digress. We would have far fewer implementation issues - and users would have fewer headaches - if each and every last Standard pointed at its test suite and said, "Go forth programmers and behold! The test suite of awesomeness to validate your code against! Make the world a better place!" Stuff like that. Public, free, complete, self-contained (e.g. ZIP file), cross-platform, easy-to-use (e.g. ZIP includes a script to run the tests) test suites are near impossible to find for Standards. Those that do exist basically tell you, "Yes, you CAN use these tests BUT you have to write the program to actually do the testing. And, as a bonus, we even put the tests in this custom file format we made up that we figured would be sooo infinitely helpful." I'm ranting. Sorry. Been working with some Standards lately. Unless, of course, you ARE a member of a Standards body and DON'T have a readily accessible, easy-to-use test suite for each of your Standards and "Annexes". Then I'm simply mocking you. :) This would make a GREAT project for someone here on the list - create a centralized test suite database capable of housing tests for every Standard. Allow everyone out there to collaborate to _easily_ and constantly improve each test suite. AND include the necessary software to run tests against implementations. Everything also needs to be free (as in Creative Commons or Public Domain). And it really needs to reside on neutral territory. Educational institutions tend to be fairly neutral - nothing is truly neutral, but that is as close as it gets. Ask your teachers if you want to implement this project idea. This won't be an easy project and really should be a team effort. Despite sounding simple, there are LOTS of hurdles to overcome. A first pass of such a project could be something like a mashup of a Wiki and a bug tracker that somehow (i.e. gets hacked up) outputs ZIP files of test suites and tools. I realize this isn't exactly a C/C++ project, but most Standards typically get implemented in C/C++. My recommendation for the tools for a first pass attempt is to use a scripting language (e.g. PHP) and just assume a command-line interface for validation of an implementation against a test suite. The user specifies the executable to run the test suite against and the script runs the tests. I'm definitely open to using such a system as are probably most of the rest of those on this list. Being able to claim compliance to a Standard and simply point at a central, community-driven test suite as proof would be a huge plus to businesses and users around the globe by creating more consistent, stable, verifiable software products. Just a few thoughts. -- Thomas Hruska CubicleSoft President Ph: 517-803-4197 *NEW* MyTaskFocus 1.1 Get on task. Stay on task. http://www.CubicleSoft.com/MyTaskFocus/
