On Fri, Aug 16, 2013 at 9:47 AM, Rob Weir <robw...@apache.org> wrote:
> On Fri, Aug 16, 2013 at 12:30 PM, Kay Schenk <kay.sch...@gmail.com> wrote:
> > On Wed, Aug 14, 2013 at 3:35 PM, sebb <seb...@gmail.com> wrote:
> >
> >> On 14 August 2013 23:10, Rob Weir <robw...@apache.org> wrote:
> >> > On Wed, Aug 14, 2013 at 4:24 PM, Marcus (OOo) <marcus.m...@wtnet.de>
> >> wrote:
> >> >> Am 08/14/2013 10:05 PM, schrieb Hagar Delest:
> >> >>
> >> >>> Le 14/08/2013 21:39, Rob Weir a écrit :
> >> >>>>
> >> >>>> Maybe we need to call an earlier build the "RC" so it will get more
> >> >>>> attention? We had a complete test build that we were testing for
> >> >>>> over a month. But maybe it is ignored unless we call it an "RC"? In
> >> >>>> other words, there were many opportunities for users to help try
> 4.0
> >> >>>> before it was released, but maybe there opportunities were not well
> >> >>>> known.
> >> >>>
> >> >>>
> >> >>> I think that it was too difficult to get to the dev versions and it
> was
> >> >>> too much complicated. There was no clear link to download a dev
> version
> >> >>> (I had to rely on the url in the messages from the dev list).
> >> >>
> >> >>
> >> >> This was intended.
> >> >>
> >> >> We hadn't published the dev builds via Apache or SF mirrors but only
> on
> >> 2
> >> >> people accounts. Apache policy says it's not allowed to publish them
> >> for a
> >> >> wider audience to save the servers from a high traffic load. It's the
> >> job of
> >> >> the mirrors to handle this.
> >> >>
> >> >>
> >> >>> What I see (from a standard user point of view) for a RC:
> >> >>> - When a dev version is almost done, rename it RC and make it known
> >> >>> (blog and we would relay the announcement in the forums of course)
> >> >>> - Have a link visible under the main download button of the download
> >> >>
> >> >>
> >> >> Both can be done, depending where the install files are located.
> >> >>
> >> >>
> >> >>> page (perhaps a similar button as a dedicated entry)
> >> >>> - Make that RC installable in parallel with a stable version
> >> >>> - No file association allowed for that RC by design
> >> >>
> >> >>
> >> >> IMHO the last both points doesn't apply to a RC [1] as it wouldn't
> be a
> >> RC
> >> >> anymore. One of the RC attributes is to change it into the final
> release
> >> >> with, e.g., just a file name change. But this has to be done without
> any
> >> >> code changes. Otherwise you have to change code parts, build again,
> test
> >> >> again, ... ;-)
> >> >>
> >> >> But a Beta release could go this separated way.
> >> >>
> >> >
> >> > Right. A release is a release is a release. The basic requirements
> >> > for every release still apply:
> >> >
> >> > 1) 3 PMC +1 votes
> >> > 2) Must include source files
> >> > 3) Digital signatures, hash files, etc.
> >> >
> >> > But we can have a "beta release", that follows these rules, and it
> >> > would be acceptable. We can then host on the mirrors, publicize, etc.
> >>
> >> There would likely be some restrictions on how many extra downloads
> >> are permitted.
> >> For example, the ASF mirrors probably could not cope with a set of
> >> betas of all the languages for all the OSes in addition to the current
> >> GA release.
> >>
> >
> > Looking again at this thread, and knowing the amount of time and
> resources
> > it takes to actually mount an approved release, I wonder if some of the
> > issues we had with the 4.0 release could have been addressed by a longer
> RC
> > vote timeframe -- like 2 weeks or so. The vote this time was relatively
> > short as I recall on both the original RC and RC2. We have over 400
> people
> > on this list. I would hope given a longer test period, we might receive
> > better testing and feedback if we took a bit longer for approval.
> >
>
> You can answer that question yourself. Look at the bugs that we're
> fixing in 4.0.1. Which of those would have been found if the same
> project members spent two more weeks reviewing the same code?
>
Well this was my point. Not the same members, but different members on this
list who, due to the timeframe we had for voting on the RC, were not
available for more extensive testing. It is the psychological difference
between a "developer snapshot" and an RC candidate.
> I doubt it would have made much of a difference. The slow Excel
> saving bug, for example, was in the code since November 2012. So this
> is not question of time. I'm hoping we can all look closely at the
> facts and come to a similar conclusion.
>
> The basic concept here is "defect yield", which is a measure of what %
> of the defects in the code are found by a given testing technique.
> We can consider the formal QA test cases as one technique. We can
> consider the PMC voting period as another "testing technique",
> although an informal one. We could consider beta testing, automated
> testing, etc., all as different techniques.
>
> Hopefully it is clear that the defect yield of a technique does not
> increase greatly by spending more time doing it. For example, suppose
> our test cases were capable of finding 25% of the bugs in the code.
> So we run the test cases once and find 23% of the bugs. We don't get
> 25% because of human error in testing. We could double the amount of
> time spent testing and run all the tests again. That might get us to
> 24%. Not a very efficient use of incremental time.
>
> Ditto for PMC voting. The informal testing is going to just poke at
> the surface. It will have some bug yield, less than the formal
> testing did. But we all have our usage patterns, the subset of
> features that we use. It is very likely that in two weeks we
> exercise only slightly more of the features than we did in a one week
> voting period. Diminishing returns. In any case, we need to escape
> the incorrect notion that a RC vote is an invitation to start testing.
> It is not. It is a sign-off period to confirm that testing has
> already been satisfactorily completed and that there are no known
> remaining serious issues. If anyone wants to help testing this should
> be done far before the RC vote.
>
> In any case, I hope you see the pattern here. Repeating a testing
> technique or doesn't really increase defect yields. It is the curse
> of diminishing returns. If you want to find more bugs then you need
> to do something like:
>
> 1) Write more test cases
>
> 2) Introduce a new testing technique that is orthogonal to the ones
> already in use. A performance test would certainly be orthogonal to
> the functional tests.
>
> 3) An early beta test cycle could help here, since that introduces a
> new class of users.
>
>
> Regards,
>
> -Rob
>
>
> > I realize this is not the same as a beta -- we would still be using
> > "development snapshots", but the RC could be provided in all languages we
> > intend to use, etc.
> >
> >
> >
> >
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
> >> For additional commands, e-mail: dev-h...@openoffice.apache.org
> >>
> >>
> >
> >
> > --
> >
> -------------------------------------------------------------------------------------------------
> > MzK
> >
> > "When in doubt, cop an attitude."
> > -- Cat laws
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
> For additional commands, e-mail: dev-h...@openoffice.apache.org
>
>
--
-------------------------------------------------------------------------------------------------
MzK
"When in doubt, cop an attitude."
-- Cat laws
