Hi,
I wrote some comments in this thread already. But I was working on
a longer mail with my collected thoughts about this topic.
What are my thoughts to this topic. My first thought was, there is
nothing (really) different in this release as in the past releases. But
this doesn't mean, that this is a good news.
My second thought was, to write down all the point which I identified
which was (perhaps) different as in the past releases. Some of the
points I will list I can support by data, others are rumor in the teams,
which I heard over the past months or years.
1. The number of regression issues for OOo 3.1
For this release the stopper issue was opened long before code freeze.
It was in November 08 and 10-15 issues were added before code freeze
begin of March. But was isn't typically for this release is, that in
the past days a mass of new incoming regressions were added and the
quality of the regressions shows a bad view on the product.
So the whole number isn't that bad for the long time, but I expect a
quality issues with the product, when so many stoppers reports (~20)
were reported in the releases list about in the past 4 days!
2. What is special in the release plan for OOo 3.1
- the first integration of CWSs was on 30. July 2008 (DEV300m29)
- Feature Freeze for OOo 3.1 was on 18th of December 08
=> started on 11th of December in build DEV300m38 and m39 56 CWS
with 403 issues were integrated for Feature Freeze
(the cloned CWS from OOO300 code line are taken out)
- Code Freeze for OOo 3.0.1 was on 11th of December 08
=> started on 11th of December in build OOO300m14 19 CWS with 88
issues were integrated for Code Freeze
This means a very high number of CWSs were handled/finalized by DEV and
QA in a very short time frame - especially before Christmas => most of
the full time engineer at Sun wanted to go on vacation for 2 weeks).
For me it's the first time, that such dates were so near together.
3. What's new in the Build Environment
Started with build DEV300m33 the Source Control Management (SCM) was
switched to SubVersion. SubVersion wasn't as good as estimated. And
it has some bugs and challenges. I read a lot of internal and external
mails, that processes were broken, features wasn't supported and some
need only information how to make this or that. This was another reason
for additional regressions in code on the master code line.
4. External CWS (not handled by Sun developers)
a) The Sun QA team gets more and more external CWSes or only work in Sun
internal CWSs in the past months. The numbers aren't so high, but these
CWSs bind resources in Sun QA team, but often not so much in the
Development team. This could lead to an unequal balance between the
teams.
b) I heard the rumor in the corridors here at Sun, that some external
CWS leads to broken functionality. If this is correct, why the QA
representative couldn't identify these regressions? Who are the QA
representatives etc. Or do we have to change the CWS policies, where
code review is one possible solution for approving a CWS?
5. General quality of the code
a) I also heard the rumor in the corridors here at Sun, that some
feature aren't completely ready until feature freeze date. But the L10N
teams need the UI for translation. Then strings will be integrated first
and functionality will be checked in with another CWS later.
If this is really done, this leads to the problem, that the iTeam do
not have enough time for regression testing. Because the functionality
testing can start too short before CodeFreeze or the first Release
Candidate. Also the time for bug fixing is too short.
b) The number of issues which marked as regressions in IssueTracker
doesn't go down in the past years. We still have a rate of 7-8% of
all reported issues which are marked as regression. For me this mean,
that we aren't going to be better with the developed code, but we
aren't going be worser. But I think, when I delete ~25% duplicate
issues, 10-15% features and enhancements for all reported issues,
the regressions are still more relevant. What does it mean 7-8%.
This means that for 50 developers 2 are working on the regressions
of the other developers only.
c) As I talked in another thread. The rate how often a CWS returns
back to development is ~25-30%. It is still that high over the past
years. And remember we do not work much with an iteration process.
Often the CWS returns because of process violation, bugs aren't fixed
or new bugs raised.
6. What features are important for a release
Do you know the features for the next release. I don't! I am surprised
every time, when I create the feature lists for general and L10N
testing. For me it looks like, that everybody can work on a feature,
which he like most. And then this feature has the highest priority
for him and is a must for the next release. On the other side the
QA, the Release Engineering and the Program Manager doesn't know,
what features they have to worked first. Because all feature have
the same priority or they have all no priority.
My resume is, that we are as good/worst as in the past. Perhaps
this time the release of OOo 3.0.1 and OOo 3.1 feature freeze
in parallel and the change to SubVersion (SCM) make it a little bit
worser.
We have to increase many things to get a better. But it isn't one
single task we can address, I think.
Thorsten
-------- Original-Nachricht --------
Betreff: [dev] amount of stopper / regressions for 3.1 release
Datum: 11 Mar 2009 15:56:11 +0100
Von: Martin Hollmichel <martin.hollmic...@sun.com>
Antwort an: dev@openoffice.org
Organisation: StarOffice / Sun Microsystems, Inc.
An: Mathias Bauer <mathias.ba...@sun.com>, dev@openoffice.org
Newsgruppen: openoffice.dev
*Hi,
so far we have got reported almost 40 regression as stopper for 3.1
release, see query
http://tinyurl.com slash cgsm3y .
for 3.0 ( **http://tinyurl.com slash ahkosf ) we had 27 of these issues,
for 2.4 (**http://tinyurl.com slash c86n3u** ) we had 23.
we are obviously getting worse and I would like to know about the
reasons for this. They are too much issues for me to evaluate the root
cause for every single issue so I would like ask the project and qa
leads to do an analysis for the root causes and to come with suggestions
for avoiding them in the future.
additionally there might be other ideas or suggestions on how to detect
and fix those issues earlier in our release process.
From my perspective one reason for the high amount of regression is the
high amount of integrated child workspaces short before a feature
freeze. In the moment the ITeam (the QA representative) does the
nomination before feature freeze. As an immediate action (for the
upcoming 3.2 release) from my side I will limit this freedom until 4
weeks before feature freeze, in the last 4 weeks before feature freeze,
I or other members from the release status meeting will do the
nomination of the cws for the upcoming release or decide to postpone it
to the then coming release.
**
Martin
*
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@openoffice.org
For additional commands, e-mail: dev-h...@openoffice.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@openoffice.org
For additional commands, e-mail: dev-h...@openoffice.org
