On 11/12/2010 7:53 AM, Rick Hillegas wrote:
Thank you Rick for your work on the release and Derby in general and
sharing your thoughts on testing and releases. I've been thinking about
what you said and thought I would share a few of my own thoughts below.
Buddy-testing = Integration test
I disagree here. I think buddy testing is just that, another
developer, looking at a feature and the documentation with a fresh
perspective in an ad hoc way. It used to be that a community member
would incorporate new features and data types into nstest, mailjdbc,
sttest, and other tests to provide integration testing. Sadly, nobody
seems to have bandwidth for that anymore, so we don't have the benefit
of good integration testing, but I don't think that would claim buddy
testing is a replacement. We should acknowledge the hole.
Release candidate vetting = Alpha test
First release from a new branch = Beta test
I think any release we make needs to be production quality, especially
with regard to data integrity and upgrade. If a beta is required, we
should vote on a beta release, with all the normal disclaimers that go
with it.
Second release from a branch = Production-quality release
Everyone has their own comfort level in terms of how long to wait before
upgrade to a new branch, but I think if we make a release it should be
production quality or if we think it is not, we should label it
appropriately as beta. Pushing quality standards down the line like
this can have a cascade effect, where of course whoever used to wait for
the second release, will now wait for the third,etc.
It was sad I thought when we could no longer release snapshots and
release candidates to the user community as we lost lots of alpha and
beta testers and it seems contrary to the whole open source model to me,
but given that restriction, perhaps we just need to adapt with a
lightweight process on alpha and beta releases and label them as such.
I think the main thing is that we have to have a vote. Never do I think
we should call something a release that we think is not worthy of the title.
Kathey
