Re: [Zope3-dev] Re: The bug fixing problem
Jim Fulton wrote: [snip] I would say that there are two bugs in the case you are describing: the one you meant to fix and the one which is the lack of any tests for the module / class / whatever. I would bet that spending your thirty minutes adding minimal tests to such a module is a *higher* value activity than fixing most bugs, because it makes it easier for you (or someone else) to fix that bug and others in that module. Good point. The PyPy project actually works with many tests that are not working. They have an infrastructure where such tests can be in the code and explicitly disabled. In some cases, the bug-reporter may be able to write a test and not fix it. Or, alternatively, the person who goes and tries to fix a bug can write tests but doesn't have time to fix them. In such case it would be nice to be able to add tests that are explicitly disabled and thus does not show up in the normal test run. Only when turning a knob these buggy tests show up, and a bug fixer can then easily go and try to fix them. One danger is that this can be used to temporarily disable tests that *used to work*. Then again, that's not hard to do now either. Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: The bug fixing problem
On Fri, Jul 07, 2006 at 08:48:39PM +0200, [EMAIL PROTECTED] wrote: In many cases, I can convince myself that a fix does actually work without performing a test -- at least in the sense that it removes one bug. In many cases I have made completely trivial bug fixes that I was absolutely confident about. I wrote unit tests for those bug fixes only out of habit. Quite often I was surprised when my superfluous tests caught bugs in my trivial bugfixes. Marius Gedminas -- Look! Before our very eyes, the future is becoming the past. signature.asc Description: Digital signature ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: The bug fixing problem
On Jul 6, 2006, at 12:29 PM, Philipp von Weitershausen wrote: [EMAIL PROTECTED] wrote: Christian Theune wrote at 2006-7-5 11:46 +0200: ... Another thing are the rules about unit tests. Some bugs touch areas that are poorly tested. When I fix a bug over there, do I have to work harder to introduce the fix because I have to start introducing tests? We should find and announce a reasonable answer for the procedure in this case. Although I have (so far) never fixed a bug in Zope 3 (but posted several patches for Zope 2), I can confirm this: There are bugs that do not need a test once they are fixed. All kinds of NameError and AttributeError fall into this category. Requiring to write a unit test for these or similarly trivial bugs is silly -- especially if there is not yet a testing file for the module (such that a trivial test would suffice). I disagree. How would you make sure that your fix for even a trivial NameError actually works? Perhaps you introduced another typo in the bugfix? Or perhaps another problem pops up in the same codepath. Clearly, since the NameError didn't occur in any other tests, the codepath hasn't been tested yet, so it should be no matter what. There's another aspect to tests for bugs: reproduceability. Especially when fixing bugs I tend to write tests first in order to be absolutely sure that I can reproduce the problem in an automated manner. Then fixing the bug is easy: Just make the test pass... Well said. I agree 100% Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: The bug fixing problem
On Jul 6, 2006, at 2:03 PM, Lennart Regebro wrote: On 7/6/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: When I have introduced secondary bugs in my fixes (which occasionally happened), then a unit test would not have helped. The reason was then that the affected code was used in unanticipated ways -- and because it was unanticipated, I would not have written a test for it. Sure. The point of the tests is not to prevent this. The point of them is mainly to make sure that the thing you fixed says fixed. Or that you actually fixed the thing you set out to fix, because, as part of the test writing process you has to actually reproduce the problem. I often learn quite a bit about bugs when trying to reproduce them. Bugs that appear shallow often turn out to be much deeper during this process. The security problem we dealt with the other day turned out this way. It was only through testing that Tres found out that the problem was much different than it appeared on the surface. Also, I think the habit of testing is as important as anything else. We should always feel a bit queasy about making a change without tests. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: The bug fixing problem
On Jul 6, 2006, at 10:11 PM, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Christian Theune wrote: Hi, Marius Gedminas wrote: I do not think that the requirements to 4. Write unit tests 5. Merge bugfixes from trunk to the release branch 6. Wait for the incredibly slow updates on the collector discourage me all that much. Right. They don't discourage me either, but there is a special case in 4) which I hit several times lately, where the unit tests need very special effort. I think that if more bug reports had a solution outlined in English, I'd be more likely to go fix them every now and them. Indeed! I would say that there are two bugs in the case you are describing: the one you meant to fix and the one which is the lack of any tests for the module / class / whatever. I would bet that spending your thirty minutes adding minimal tests to such a module is a *higher* value activity than fixing most bugs, because it makes it easier for you (or someone else) to fix that bug and others in that module. Good point. OTOH, well I'll mention the other hand in a separate post. :) Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: The bug fixing problem
Hi, Jim Fulton wrote: On Jul 6, 2006, at 12:29 PM, Philipp von Weitershausen wrote: [EMAIL PROTECTED] wrote: Christian Theune wrote at 2006-7-5 11:46 +0200: ... Another thing are the rules about unit tests. Some bugs touch areas that are poorly tested. When I fix a bug over there, do I have to work harder to introduce the fix because I have to start introducing tests? We should find and announce a reasonable answer for the procedure in this case. Although I have (so far) never fixed a bug in Zope 3 (but posted several patches for Zope 2), I can confirm this: There are bugs that do not need a test once they are fixed. All kinds of NameError and AttributeError fall into this category. Requiring to write a unit test for these or similarly trivial bugs is silly -- especially if there is not yet a testing file for the module (such that a trivial test would suffice). I disagree. How would you make sure that your fix for even a trivial NameError actually works? Perhaps you introduced another typo in the bugfix? Or perhaps another problem pops up in the same codepath. Clearly, since the NameError didn't occur in any other tests, the codepath hasn't been tested yet, so it should be no matter what. There's another aspect to tests for bugs: reproduceability. Especially when fixing bugs I tend to write tests first in order to be absolutely sure that I can reproduce the problem in an automated manner. Then fixing the bug is easy: Just make the test pass... Well said. I agree 100% Just to make sure nobody got me wrong: I'm not arguing that tests are a bad thing. They belong to the best thing that came up in the last years. :) I was merely pointing to a small edge-case in bugfixing, that might be even smaller then anticipated, where I got frustrated in the last time. +1 for the tests-are-good-recap :) Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: The bug fixing problem
On Jul 7, 2006, at 7:52 AM, Christian Theune wrote: Hi, Jim Fulton wrote: On Jul 6, 2006, at 12:29 PM, Philipp von Weitershausen wrote: [EMAIL PROTECTED] wrote: Christian Theune wrote at 2006-7-5 11:46 +0200: ... Another thing are the rules about unit tests. Some bugs touch areas that are poorly tested. When I fix a bug over there, do I have to work harder to introduce the fix because I have to start introducing tests? We should find and announce a reasonable answer for the procedure in this case. Although I have (so far) never fixed a bug in Zope 3 (but posted several patches for Zope 2), I can confirm this: There are bugs that do not need a test once they are fixed. All kinds of NameError and AttributeError fall into this category. Requiring to write a unit test for these or similarly trivial bugs is silly -- especially if there is not yet a testing file for the module (such that a trivial test would suffice). I disagree. How would you make sure that your fix for even a trivial NameError actually works? Perhaps you introduced another typo in the bugfix? Or perhaps another problem pops up in the same codepath. Clearly, since the NameError didn't occur in any other tests, the codepath hasn't been tested yet, so it should be no matter what. There's another aspect to tests for bugs: reproduceability. Especially when fixing bugs I tend to write tests first in order to be absolutely sure that I can reproduce the problem in an automated manner. Then fixing the bug is easy: Just make the test pass... Well said. I agree 100% Just to make sure nobody got me wrong: I'm not arguing that tests are a bad thing. They belong to the best thing that came up in the last years. :) I was merely pointing to a small edge-case in bugfixing, that might be even smaller then anticipated, where I got frustrated in the last time. +1 for the tests-are-good-recap :) And just to make sure that I was clear, when trying to urgently go through bugs for a release, I think it's OK to provide a test for just the fix. If, in the course of a fix, you notice that a package is short of tests, it would be great to add more then, but it would be OK to just turn that lack of tests into a separate non-urgent bug entry. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: The bug fixing problem
On 7/6/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: When I have introduced secondary bugs in my fixes (which occasionally happened), then a unit test would not have helped. The reason was then that the affected code was used in unanticipated ways -- and because it was unanticipated, I would not have written a test for it. Sure. The point of the tests is not to prevent this. The point of them is mainly to make sure that the thing you fixed says fixed. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/ ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: The bug fixing problem
Lennart Regebro wrote at 2006-7-6 20:03 +0200: On 7/6/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: When I have introduced secondary bugs in my fixes (which occasionally happened), then a unit test would not have helped. The reason was then that the affected code was used in unanticipated ways -- and because it was unanticipated, I would not have written a test for it. Sure. The point of the tests is not to prevent this. The point of them is mainly to make sure that the thing you fixed says fixed. I hear (also) other sounds than unit tests ;-) Thus, my fixes can say fixed differently (without a passing unit test)... -- Dieter ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com