Andreas, I wasn't trying to pick on you or anyone for that matter. It is something I've been meaning to bring up for some months actually.
What I would hope to see is follows: (1) if there is a bug where a given input FO file causes an unhandled exception, and the bug fix is to fix the underlying problem causing the exception, then that input FO file, or a subset thereof, should be added as a test case, verified to fail prior to the fix, and verified not to fail after the fix; a regression after the fix would cause the exception to occur once again; (2) similarly, if the bug is reporting a failure to produce expected results for an existing feature; in this case, one or more test cases should be added to verify the expected results; (3) similarly, if a bug is requesting a new feature, and the bug fix is to add the feature, then there should be some reasonable attempt to add tests for that new feature; i'm not asking for 100% coverage of the feature, but something more than nothing is a good thing; (4) in the case of a performance fix, i would not say a test case (demonstrating the improvement) is required; of course, if someone was energetic enough to include a test case showing the performance delta, then that would be welcome; Of the above, I'm mostly concerned with the first and second cases. Regards, Glenn On Mon, Jan 24, 2011 at 3:42 PM, Andreas Delmelle < andreas.delme...@telenet.be> wrote: > > On 24 Jan 2011, at 23:17, Glenn Adams wrote: > > Hi Glenn > > > Is there a reason why *all* bug fixes are not accompanied by one or more > new test cases that demonstrate the presence and absence of the bug before > and after the fix? Adding such test cases should be mandatory for all bug > fix commits. I wouldn't quite go that far for performance and clean up > fixes, however, but for legitimate bugs, there should be a reliable way to > ensure that (1) the bug and its fix are testable and (2) that future > regressions do not occur. I realize it takes more effort, but it is worth it > in the long term, both in actual improvements in quality and the ability to > demonstrate consistently good practice to maintain that quality. > > FWIW, in case you are referring to some of my recent commits... > Any specific ones I should be taking a look at? Just following old habits, > I'm afraid, and try to add tests wherever appropriate, even if the fix > affects only a single line of code. > > Performance fixes may be rather difficult to test for, but even then one > could conjure up a way to test whether adding nodes remains a linear > operation, for example. We have no real framework set up for that type of > thing, but I guess I could invest some time in that. > > > > > Many Apache projects require this process be followed; I would urge the > FOP project to adopt a similar criterion for bug fix commits. > > > > > Basically agreed with your viewpoint, so point out where exactly you feel > something was missing and I'll see if I can accommodate you. > > > Regards, > > Andreas > --- > >
