Stepan, I've noticed that EHWA scenario started to fail this week. First, this helped me to discover a usability issue that the log [1] is not sufficient to understand why this scenario fails.
The thing which can be understood from this log is that instead of using perfectly portable java-based HelloAuto plug-in [2] developed by Anton Luht we are using GUI automation tools with all their glitches and support issues. Why? [1] http://people.apache.org/~smishura/r589837/Linux_x86/ehwa/ [2] http://issues.apache.org/jira/browse/HARMONY-752 On 10/22/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > Stepan, > Thanks for explanation! I believe continuous EHWA runs are quite assuring. > Thanks. > > On 10/22/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > On 10/19/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > > > Stepan, > > > > > > I wonder why EHWA (and EGA) are omitted here for Windows platform: > > > http://people.apache.org/~mloenko/snapshot_testing/script/r585787/index.html > > > > > > Are there any issues with these test suites? > > > > > > > There was an issue with host that runs EGA and the host was rebooted. > > As for EHWA - I'm going to add it soon. > > > > Thanks, > > Stepan. > > > > > Thanks. > > > > > > On 10/19/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > On 10/18/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > > Hi Stepan, > > > > > > > > > > Thanks for your reply. Yes - I was meaning successful as not only > > > > > compiled > > > > > succesfully but passed all the required tests. Required tests would > > > > > be a > > > > > set of tests that we aimed to be GREEN all the time. Then if > > > > > something is > > > > > checked in that breaks one of these tests it would either be fixed > > > > > fairly > > > > > quickly (say within a day or two?) or rolled back. We could publish > > > > > any (or > > > > > every) build that passed this subset, which should mean that there's > > > > > never a > > > > > long period of time without a build being published. > > > > > > > > > > I think this would also help with reducing the freeze period for a > > > > > milestone > > > > > because it would mean we would never get too far away from something > > > > > that's > > > > > working, so stabilization would be easier. It's also easier to fix > > > > > something that's just been committed than something that you did 2 or > > > > > 3 > > > > > months ago. > > > > > > > > > > I like the suggestion of using the integrity suite as a start - it > > > > > seems > > > > > like a good basic set of tests to aim for. But if you don't think it's > > > > > realistic to keep these green most the time during a development > > > > > cycle, then > > > > > maybe just start with the classlib, drlvm and drlvm-regression tests? > > > > > These > > > > > are all green at the moment on 32-bit Linux and Windows and I think > > > > > the > > > > > Harmony community is already quite good at trying to keep those tests > > > > > passing. > > > > > > > > > > > > > I agree with 32-bit Linux and Windows platforms. But I think that a > > > > set of suites should include Eclipse Hello World Application (EHWA) > > > > suite and may be jdktools too. Say if EHWA is broken than most > > > > probably that there is serious regression. And IMHO such build > > > > shouldn't be published as users build - it can not run simple Eclipse > > > > scenario. > > > > > > > > Thanks, > > > > Stepan. > > > > > > > > > Thanks, > > > > > > > > > > Sian > > > > > > > > > > > > > > > > > > > > On 18/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > > On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > > > > That's good - it sounds like we are aiming for the same thing > > > > > > > after > > > > > > all! It > > > > > > > would be nice if this could be automated at some point, so > > > > > > > successful > > > > > > builds > > > > > > > are published automatically. Also should the build be considered > > > > > > > broken > > > > > > if > > > > > > > we have an unsuccessful build? It just seems to me like > > > > > > > something is > > > > > > wrong > > > > > > > if there hasn't been a 'successful' build for more than 2 weeks. > > > > > > > What > > > > > > do > > > > > > > you think? > > > > > > > > > > > > > > > > > > > Hi Sian, > > > > > > > > > > > > Do you mean 'successful'=='tested' build? > > > > > > > > > > > > I agree that we need to work out a criteria for publishing tested > > > > > > builds for users. Of cause if everything became GREEN on the testing > > > > > > page then the build is published as users build. But it is not so > > > > > > probable to see during active code development. Also I agree that we > > > > > > should define time frame - if we can not publish users build for a > > > > > > long time that we need to stop and fix all failures. I'd suggest to > > > > > > use a set of suites run by integrity testing [1]. So if classlib > > > > > > tests, drlvm and drlvm-regression tests, jdktools and Eclipse hello > > > > > > world application pass on snapshot testing them we publish the > > > > > > build. > > > > > > > > > > > > [1] http://people.apache.org/~varlax/harmony-integrity/ > > > > > > > > > > > > Thanks, > > > > > > Stepan. > > > > > > > > > > > > <SNIP> > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Unless stated otherwise above: > > > > > IBM United Kingdom Limited - Registered in England and Wales with > > > > > number > > > > > 741598. > > > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire > > > > > PO6 3AU > > > > > > > > > > > > > -- > > > With best regards, > > > Alexei, > > > ESSD, Intel > > > > > -- > With best regards, > Alexei, > ESSD, Intel > -- With best regards, Alexei, ESSD, Intel
