On Wed, May 02, 2001 at 06:58:57PM +0100, James Moore wrote:
> > >I would be very against this.. to me it seems silly, the current QA Team
> > >will have to spend 90% of their time running through the (maybe
> > hundreds) of
> > >reports rather than testing. It makes more sense to me to try and attract
> > >more people who know what they are doing to the QA Team rather
> > than having a
> > >fairly (maybe more :)) disorderly group of people testing from
> > people who
> > >do not really know what they are doing..
> >
> > You have tens of thousands of people testing releases today. What's the
> > difference?
>
> The big difference is during a release process is the time scale. There are
> likley to be more bugs in an RC as well as people reporting bugs more
> rigerously (As well as probably reporting lots of bogus/dup bugs, which are
> very tedious to trawl through).
Exactly. While it would be nice to think that broadening the audience
of the testing cycle would yield more bug finding, I fear it would
inundate the bug database with only marginally useful bug reports.
The number of bug reports that boil down to "My script doesn't work!"
would certainly increase. Reports that extension X doesn't work, but
provide no substantial information would increase, and so on.
I don't see inviting this wider audience as providing enough beneficial
information to justify the work of clearing away the less useful
reports.
> Normally I test RC1 massively then if there are problems I check for them in
> later RC's where people have said they have been fixed (or its decided that
> the bug should be fixed before the release).
>
> This time this didnt work for the single reason Phanto was unresposible and
> commited a huge (700 line commit) to RC7 and DIDNT test it. I asked him (as
> I asked sascha too) to when we decided to have RC8 (I think I cc'd the list)
> to test his changes throughly as I would not have time due to "real" work.
> Now Phanto obviously didnt do this, maybe someone should have caught it but
> I feel that by not testing Phanto invalidated a lot of hard work by the rest
> of the team to make 4.0.6 stable.
This sort of thing simply shouldn't happen, imho. If the QA team's word
is to mean anything, they need to have the authority to say "No, that
needs to be removed." If they don't have that authority, then how can
they be expected to say "We have tested RC8, and it is ready to be
released."?
> I am certainly pissed off that this has happened as a lot of people put a
> lot of work into making sure 4.0.5 was stable and the problem here is not
> the testing but the developers commiting unneeded stuff to the RC branch.
>
> I feel we should only have x people commiting to the branch and if somthing
> is commited as late as the COM stuff was its up to the developer to test
> throughly otherwise its their head on the block.
I can understand why it would be frustrating to see what could amount
to an undermining of the testing cycle, but I don't think shifting
blame to the developers is the best solution. This will only lead to
animosity over the quality of releases.
What I'd like to see instead is for the QA team to own the release
cycle. That is, as soon as RC's start being rolled, QA has the right
to determine if a commit shouldn't go in. Without that control, it
seems to me that their hands are being tied.
Matt
--
PHP Development Mailing List <http://www.php.net/>
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]
