Andreas,
You have some good ideas here and I'd
just like to add some ideas about how we
can recognize and escape the circular dependencies
that get created if we hold to extremes of
zero tolerance policies.
We currently have
topcrash : 39 bugs found.
crash : 413 bugs found.
I betcha the 413 have many dups, unreproducable crashes,
and crashes in obscure features or strange scenarios that
most users are unlikely to encounter.
If we wait to fix every last "known" crasher in the 413 we may
actually retard the progress to improving MTBF.
If the goal is to "improve stability" then the way to
hit this goal for the widest number of users is really
to focus on top crash bugs.
We have whittled away and now most of the bugs on
the topcrash list are now of the "unable to reproduce" variety;
meaning we are actually "pretty close" to your criteria.
The topcrash bugs are the most visible and most frequent
problems that folks are actually encountering given statistical
analysis of talkback data. If we understood them
and fixed these bugs we would definitely make biggest impact on
improved stability. The way we most often fix
these kinds complex, frequent, but unreproducable bugs is to
get a high volume of people using the release. From this high
volume stack traces, configuration, user comments and test cases
information in talkback data we eventually hit upon the
right set of circumstances to recreate or understand
the problem and develop a fix. So the question is what is the easiest
and fastest way to get the highest volume of people using a release?
Its usually to put a line in the sand and declare something like a regular
0.x or 1.0 milestone.
In this case some flexibility in the standard and applying good
judgment really helps us to make the fastest possible progress
toward the main goal which in this case is improving stability
to the highest possible levels for the widest number of users.
We have generally held to the "no known topcrash bugs"
for all the mozilla milestones and pay special attention to it
at key milestones or branch points that we really want
a lot of users to get their hands on. We have made
pretty good progress toward improving MTBF using
this strategy and expect this trend to continue as
we keep knocking off top crashers.
chris h.
Andreas Franke wrote:
> Asa Dotzler wrote:
>
> > I doubt that Netscape6 release had anything to do with the delay of a
> > mozilla 1.0 doc. If you think a particular feature or level of
> > support is important then you can help by finding the bugs in that
> > feature or standard and nominating them for a fix in mozilla0.9 or
> > mozilla1.0 Don't sit around waiting for someone to hand you a list of
> > what Mozilla 1.0 should be, [...]
>
> Brendan Eich wrote on Wed, 27 Sep 2000 in message
> news://news.mozilla.org:119/39D262E5.12C167AE%40meer.net
> in the thread "Another roadmap draft is out" on n.p.m.general:
> > Look for a document defining Mozilla 1.0 soon
> > from [EMAIL PROTECTED], typed up by me.
>
> I've been looking for such a document for more than 2 and a half months now, but
> I haven't seen one. I still think the need for such a document is obvious, but
> I'll be happy to be proved wrong. If you can point me to a newsgroup message
> where people agreed that such a plan is not needed, please do.
>
> I agree that everyone should nominate bugs for moz0.8/0.9/1.0 even in absence of
> such a document, but I don't think this is enough. Here's one example: I feel
> that it is a fundamental part of open source software that the resulting software
> is stable. It simply doesn't crash. Therefore I think it's necessary to follow a
> "zero known crashers" strategy. This doesn't mean there will ever be a release
> with zero crasher bugs. It only means that there will be a release (hopefully
> mozilla 1.0) without any _known_ crashers. The question is how you define what a
> known crasher is. I would define it as a crasher where the steps to reproduce are
> known and reliable.
> Currently, there are more than 400 open crasher bugs ("crash" keyword or "crash"
> in summary) in the "Browser" product, and 70 MailNews crashers. Of course, not
> all of them are "known crashers" according to the definition above. But before
> these bugs can be triaged efficiently, the classification criteria have to be
> decided upon, and there has to be a way to track progress. To me, it seem a bad
> strategy to just add mozilla0.x keywords to all of these crasher bugs, and
> randomly adding keywords to some of them doesn't seem right to me (it would be
> some kind of "equal protection violation" if you understand what I mean.. ;)
>
> I believe that the area of crashers is only one of many where well-defined
> standards are needed.
>
> Andreas
