Hm, it's tricky to just count on the developer submitting the patch to
run derbyall. A common mistake, for example, is that the developer
hasn't done 'svn add' for a new file, so their tests work, but the patch
submitted doesn't have all the necessary files. Also, the developer may
have old classes sitting around or who knows what funny stuff in their
environment that makes it work for them. Also also, the developer
usually tests on a specific VM and OS, and it would be great if at least
one other VM or OS were tried before committing.
What we *really* need is a good full nightly regression across all
platforms and VMS, but until we have that, I think we need to have at
least *one* person run derbyall besides the submitter of the patch.
This does not necessarily have to be a committer.
I'm not sure how this could be managed; perhaps each developer could
"buddy up" with someone, and each of them agree to test out each other's
patches...
The other thing I have noticed is that sometimes a developer forgets to
run 'svn update' and 'ant clobber' and 'ant all' before running tests.
We should probably write up a "checkin checklist", vote on it, and add
it to the web site...
David
P.S. I did run derbyall on the two recent patches I submitted. I'll add
that to my comments when I submit patches from now on.
Mike Matrigali wrote:
me too. I run derbyALL for any thing I submit myself, but as
satheesh has said lately I have not been able to run derbyAll for
all the patches I have been committing for others. I do a full
rebuild, review the change and
run tests that look specific to the patch, and count on the submitter
to have run all tests. Also please check the results posted to
the list the day after your patch goes in, to see if you may have
introduced a problem in a different environment.
Remember that there are now a lot of developers out there depending
on clean test runs, so they can figure out what affect their changes
are having.
Satheesh Bandaram wrote:
+10, if there is anything like that. :-)
I used to run 'derbyAll' for each patch I tried to commit, but not able
to do that anymore with so many patches being submitted these days. It
should be everyone's responsibility to run all tests before submitting a
patch. I will start asking for list of tests run before committing
patches, if not provided already.
Also, new people tend to miss updating multiple versions of master
files, if the test has different masters. (say for DerbyNet or
DerbyNetClient frameworks) It is also possible to have different master
files based on JVM being used or version.
Satheesh
Bernt M. Johnsen wrote:
Hi all,
The last week or two, derbyall has failed a lot the time. This makes
it hard to verify that I haven't goofed up things when doing changes
in the code (I run derbyall daily as a quality measure). To be able to
work efficiently, many of us are dependent on a reliable test suite.
So, I urge everyone who submits a patch to the Derby to run derbyall
on at least one platform/vm and not only the "relevant" tests for the
patch, even if you "know" that it should not be necessary. I think
this would save us all a lot of work (even more time saved for the
committers, I guess :-).
I know this won't guarantee that derbyall runs correctly all the time,
but the frequency of problems should at least go down.
Thanks
