2.16 release criteria

2012-03-20 Thread David Kastrup

Well, the good news is that we have not had anything holding up the
release without a fix in the pipeline for at least a month (or so it
feels).  The bad news is that the pipelines still see quite a bit of
action.  It will be luck more than anything else if we have a pause in
the stream of newly discovered regressions that is long enough for
making a long-term release.

I have the fear that the desire to get to this state might prompt some
regression fixes that have not necessarily gotten all the diligence that
would have been desirable.  So I am not sure that a timed release
scheme would not possibly lead to a higher quality/time ratio, even
though our current scheme attempts to have monotonically increasing
quality over the subset of known bugs.

On the plus side, regressions are being addressed vigorously right now.
Other bugs, however, get to see this vigor as well, leading to more
regressions in their wake.

-- 
David Kastrup


___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel


Re: 2.16 release criteria

2012-03-20 Thread Graham Percival
On Wed, Mar 21, 2012 at 01:06:42AM +0100, David Kastrup wrote:
 
 I have the fear that the desire to get to this state might prompt some
 regression fixes that have not necessarily gotten all the diligence that
 would have been desirable.

This is a valid fear in general, but I haven't seen it happen yet.
Granted, I don't usually review scheme or C++ patches, so perhaps
people have been sneaking bad bugfixes in that way?  But I
somewhat doubt that.

 So I am not sure that a timed release

No.  Absolutely not.

Yes, it might be good to change the release policy.  But I will
not accept any discussion along those lines.  We discussed matters
to death in GOP.  It hasn't even been 12 months!  What's the point
of having a serious policy discussion if it's going to change in a
few months?

In the summer, I will begin GOP2, and we will begin by reviewing
every single policy decision made in GOP.  It will be understood
that whatever policies we agree upon in GOP 2 will hold for at
least the next year.  We may end up having a yearly review of such
policies.

 On the plus side, regressions are being addressed vigorously right now.
 Other bugs, however, get to see this vigor as well, leading to more
 regressions in their wake.

I think we're looking at about 30% Critical regressions due to
code in the past year.  Solution?  More eyes on reviews and/or
more regtests.

The bulk of Critical regressions happened during the long 2.13
process.  Those block a stable release on the basis of last year's
policy discussions.  To make matters worse, we've begun a big
review of the regression tests.  I guessimate that we currently
have between 5 and 20 broken regtests; the regtest review will
probably find those.

In the long term, I think we're doing fine.  For the first time
ever, we're not regularly breaking regtests.  I cannot emphasize
how important this is -- back when I was handling bugs, I would
see 1-2 broken regtests every devel release.  Unforuntately we're
in for some more pain in the next few months as we discover
previously-broken tests, but once that's shaken down, we'll have a
trustworthy set of regression tests.

- Graham

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel