Hi Joost, *,
On Tue, Jun 30, 2009 at 11:42 AM, Joost Andrae<[email protected]> wrote:
> [...]
> To improve this we need people who write Testtool testscripts.
I disagree. The problem is not finding the bugs. The question is on
how you decide what to fix and what to ignore.
Testtool scripts might prevent regressions or other stuff, but they
won't solve the issue with the big pile of unresolved issues.
And honestly, I'm /very/ disappointed of the automatic testing,
admittedly not the testtool, but smoketestoo_native.
Smoketestoo_native failed to
* Detect a regression where the installation would not run /at all/
because of wrong permissions
* Detect the PDF-export breaker
Those two simple cases alone describe that testscripts by itself won't
really help in all cases.
> Confirming issues is important.
All of the issues we're talking about in this szenario (rotting pile)
are of course already confirmed. There are enough of those.
> If a developer get's some sort of cook book
> to reproduce an issue the better he/she will be able to find a solution to
> fix this issue.
This is true, and without that step you cannot complain about an issue
not being fixed. I completely agree.
> But it is not the job of the QA team to define the issues to
> be fixed.
And that is one of the major problems. QA (at least is true for
QA-volunteers) are very close to the userbase, have an idea of what
the problems of OOo in daily use is, have specialized in a
category/module or two.
They /know/ the product. They /know/ how to use it. They /know/ what
bothers them in their daily use.
I just thow "mail merge" into the crowd...
> This must be a common understanding between QA, development and
> other stakeholders and more important it's influenced by the sponsor of the
> developer.
This is way to easy ("excuse" was the term used in other postings). Of
course ther must be an agreement. But that's just common sense.
The problem is that in the eyes of the community that common
understanding of those who /can/ fix bugs differs from those of the
community itself.
> [...]
>> Yes and on a side note, I've already heard from some developers that QA is
>> not within their responsibility, so yes, it links to what you said above, a
>> better work together.
I second that statement. Developers should not be bothered with QA.
But of course in my view of what "QA" is in the OpenOffice.org
project.
Developers should never have to deal with bugs that doesn't have
enough info to be processed. (But Andre already mentioned bugs where
you need developer's knowledge to process it, so there is no clear
border)
The developers should of course be interested in QA of the work they
have done. That doesn't mean they have to conduct it themselves, but
it should be clear what QA-members can or should do to verify the
changes. In the cws and in future to prevent regressions.
(This is also very important when talking about new features so that
documentation can know what to write in the first place).
Other people have different views about QA, so it is always hard when
generalizing.
But as this is not the point of my post at all, don't get too much
into it, the point above is far more important to me.
> QA and development must work together to get a common understanding about
> what needs to be fixed within the next version. On the other hand we have
> restricted man power on both sides (Development and QA) so we need to talk
> about what can be fixed and what needs too much time and what's more
> important. It's always a negotiation to define what get's in and what get's
> out of the upcoming release.
QA and development is a little too narrow. The supporters on the
mailing-lists/IRC should have a voice as well. (of course not
individually, that would be chaotic).
> - automation (testtool, api)
> - manual tests
As mentioned above, I don't see the bottleneck there. Sure OOo might
have a few user-visible bugs that are not catched during Release-QA,
but still there are enough already that cannot be dealt with.
> - localization tests
Oh, a story with lots and lots of disappointment. Improved in the last
years, but still not very satisfying.
> - verifying issues of just integrated child workspaces within the current
> master build (to verify there are no integration related issues regarding
> this CWS)
Jup, nice janitorial task that takes off some workload of the other
QA/Devs that can focus on other tasks
+1
> - confirming unconfirmed issues (and to check if the description is easy to
> reproduce)
confirming is not enough. The other part is way more important:
Provide easy, step-by-step instructions, attach sample documents even
when they are easy to create in less than 3 Minutes.
And often enough forgotten (I blame myself here too): Check for
duplicates. Often enough an unconfirmed issue is a duplicate of
another issue (be it unconfirmed or not). Also check for related
issues that directly touch the same are (this is quite hard, not
always applicable)
ciao
Christian
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
