On 2/21/06, Stephan Richter <[EMAIL PROTECTED]> wrote: > > Most testing I do is not about handling cases at all, it's about > > testing specific functionality, making sure that old bugs doesn't pop > > up again and stuff like that. Most of these tests would in doctest > > format provide no documentation at all. > > Ah, that is in the eye of the beholder. I would argue that bug fixes, for > example, should be either tested by integrating text in the documentation or > tested using a docstring test, which SchoolTool commonly does. The idea is > not to say: "This test has been written to test the fix for issue 12234", but > rather write something like: "A special case is XYZ, which works as follows:"
Well, possibly, although I'm not convinced. > Any test should provide documentation. That depends on what you mean with "provide". Of course you need in a comment explain why you do the test, and what it tests, but that does not mean that the test makes sense anywhere in what would normally be called "the documentation". > > For my calendar, many of the > > tests are test to do things as migrate data from the old calendar > > product, making sure that the attendee source for CPS does what it > > should, but it has no educational use in itself, > > Well, a doctest could explain the migration test and what has changed. Nothing "changed". It's two different calendar products. It's basically an import/export from the old calendar to the new. There is nothing to explain. > But for translation consistency you have a different test setup anyways. Eh, no... I don't think I follow you here. It's a test, it's run by the test runner. If you are referring to the setUp code, then I actually don't understand what that has to do with it. > Doctests are for testing code, not translations or similar things. Well, then we at least agree on one exception to the rule. :-) I don't agree doctest is for testing code. I think it's for testing documentation. > > Many tests require setup/teardown > > functionality, which gets hard to do in the inherently linear format > > of doctests. > > I used to think this way too until Jim convinced me that showing all the setup > steps and teardown is actually useful to the reader. It shows how much is > involved in getting a particular piece of code running. Alternatively, you > can of course define setup and teardown functions when instantiating the > suite. That's not the problem, the problem is that I need to run the setup/teardown many times. I could split the tests I have now so that each test is a separate doctest file, and use the setup/teardown during instantiation, but is it really reasonable to have say, ten doctest files that do very similar things, all saying things like "The calendar can migrate from the old calendar ever if it has french accents", and then having the code for this (which is rather long). That's not much of a doctest. It is definitely not any sort of documentation. And, it also means that the setup and teardown steps are NOT useful to the reader. In fact, it is my now repeatedly stated opinion that several of CPSSharedCalendars tests are not useful to any reader. They make sure it actually works, they don't provide any insight into how to use CPSSharedCalendar. You use it by clicking around on a website, not by writing python code. I think you are stuck in library-development mode. ;-) But this proposal isn't only made for libraries, right? It seems to me that several of the requirements have to do with site integration and such, things you get for finished products that you can install and use. They need to be documented of course, and part of that documentation can very well be in doctest format, but most of the documentation that is needed (and currently sadly missing) is for end-users. A doctest requirement wouldn't help the documentation in any reasonable way, and it would slow down the test-writing. And why should I pretend that the migration tests are documentation of how to write a migration, when it is in fact a test that migration works even with non-ascii characters? There is today no documentation of how to write migration code from the old calendar to the new, because that code is written. I don't need to document how to do it again. :-) > SchoolTool solves this by simply storing an old ZODB database and then run the > generation scripts, because this is what happens in the real world anyways. Well, I solve it by not using doctests, which I like better, because it enables me to easily add and change the tests according to me finding something new that needs testing, withouthaving to change a binary file which can't be diffed. And how do you fit the ZODB into the documentation? :-) Tests are not always documentation. It's nice if they can be both, but I don't think it's a good idea to require it to be both, because quite often, it isn't. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/ _______________________________________________ Zope3-dev mailing list Zope3email@example.com Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com