Re: Should patches also update functional tests?

[Jack Klebanoff]

A good test suite is invaluable. It is very important that it be kept 
working.  Then when one makes a code change that causes a test to fail 
you can be fairly sure that your change broke something. You can fix it 
before the change is submitted, keeping Derby's quality up. If the test 
suite is weak or if it shows failures then Derby's quality will 
deteriorate. It will be very hard to get it back up.

I think that we should never accept a patch that breaks a test. If the 
test is in error then the patch should include a fix for the test.

We should strongly encourage all patches to include tests. A committer 
should think twice, or three times, before committing a patch without tests.

Jack
Satheesh Bandaram wrote:
I would like to seek suggestions or comments regarding submitting 
patches with appropriate test changes. Since Derby now has functional 
tests, should we suggest or even require patches to be submitted along 
with a test case added? This will only apply if a test case is 
available and doesn't require undue amount of time and resources to 
test the patch.

Obvious advantages are:
   1. We can be sure we are indeed addressing the problem. Unless 
committers write a test case based on the code changed, we may not 
know for sure we did fix the problem correctly.
   2. Can prevent future regressions in the code, where the same 
defect is reintroduced because of another patch or enhancement.
   3. Strengthens Derby funtional test suites and test coverage.

This may introduce additional burden on patch submissions, hence, we 
could make this voluntary to start with. How do other Apache projects 
handle this?

Satheesh