On 12/17/07, Tim Ellison <[EMAIL PROTECTED]> wrote: > Stepan Mishura wrote: > > What criteria do you use to define if the build is stable enough? > > The opinion of people who work in each area saying it is stable. >
Sure, the final decision is made by people. > The tests and scenarios are there to give us good information to work > with, but in the end it is 'people over process' that decide. > But if regressions have tendency to get accumulated then the next questions start to disturb me: what is wrong with the process? when we should try to make everything 'green' on the snapshot status page? Thanks, Stepan. > > It is not clear from me how we define 'stable build' - the published > > test results showed several regressions (there is not 1 only or 2 > > issues for evaluation). > > For example, I spend a whole day evaluating/reproducing results for > > Eclipse and Geromino unit tests and reliability suite. Running the > > Eclipse tests are bit tricky (I can not get stable results yet). The > > Geromino unit tests IMO regressed since M2. (we have HARMONY-4895 and > > HARMONY-5302. Both claim that there are issues with Geromino but the > > all tests pass with Sun's JDK 1.5.0. So I think more investigation is > > required). And I'm able to reproduce tests crashes in the reliability > > suite. But currently I can not say how critical they are but it looks > > like we have problems in ThreadManager. > > Ok. Weldon or somebody who has been working in that area should give us > a view of whether the code in that area is stable enough for us to > promote to a milestone. I haven't been working in that code to know. > > The Harmony code base is so large that we have to rely on people > agreeing on a milestone date and converging on it for a stable > publication. Continuous integration and testing helps ensure each area > doesn't break everyone, though of course there will be new issues found > during this stability period. But while the code is frozen we can't > make progress. > > > Also is it possible to define a formal criteria for publishing a stable > > build? > > (Say as we did for publishing user's snapshots: the snapshots are > > published if classlib,drlvm.jdktools and EWHA pass on Linux/Windows > > ia32 platform.) > > We could try and do that in future but, as I wrote above, I put more > faith in people. > > Regards, > Tim
