[testing] EHWA GUI scenario fails on the snapshot (Was: Re: [build?] Problem with snapshot build or download page?)
On 10/30/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > Stepan, > > I've noticed that EHWA scenario started to fail this week. First, this > helped me to discover a usability issue that the log [1] is not > sufficient to understand why this scenario fails. > You are welcome to improve the scenario's log. > The thing which can be understood from this log is that instead of > using perfectly portable java-based HelloAuto plug-in [2] developed by > Anton Luht we are using GUI automation tools with all their glitches > and support issues. Why? > Yes, it may be possible that a scenario fails because of GUI automation tools issues. But I have never seen such issues for this scenario - before M3 it also failed under CC and the failures were always reproducible by hands. Thanks, Stepan. > [1] http://people.apache.org/~smishura/r589837/Linux_x86/ehwa/ > [2] http://issues.apache.org/jira/browse/HARMONY-752 > >
Re: [build?] Problem with snapshot build or download page?
Stepan, I've noticed that EHWA scenario started to fail this week. First, this helped me to discover a usability issue that the log [1] is not sufficient to understand why this scenario fails. The thing which can be understood from this log is that instead of using perfectly portable java-based HelloAuto plug-in [2] developed by Anton Luht we are using GUI automation tools with all their glitches and support issues. Why? [1] http://people.apache.org/~smishura/r589837/Linux_x86/ehwa/ [2] http://issues.apache.org/jira/browse/HARMONY-752 On 10/22/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > Stepan, > Thanks for explanation! I believe continuous EHWA runs are quite assuring. > Thanks. > > On 10/22/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > On 10/19/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > > > Stepan, > > > > > > I wonder why EHWA (and EGA) are omitted here for Windows platform: > > > http://people.apache.org/~mloenko/snapshot_testing/script/r585787/index.html > > > > > > Are there any issues with these test suites? > > > > > > > There was an issue with host that runs EGA and the host was rebooted. > > As for EHWA - I'm going to add it soon. > > > > Thanks, > > Stepan. > > > > > Thanks. > > > > > > On 10/19/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > On 10/18/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > > Hi Stepan, > > > > > > > > > > Thanks for your reply. Yes - I was meaning successful as not only > > > > > compiled > > > > > succesfully but passed all the required tests. Required tests would > > > > > be a > > > > > set of tests that we aimed to be GREEN all the time. Then if > > > > > something is > > > > > checked in that breaks one of these tests it would either be fixed > > > > > fairly > > > > > quickly (say within a day or two?) or rolled back. We could publish > > > > > any (or > > > > > every) build that passed this subset, which should mean that there's > > > > > never a > > > > > long period of time without a build being published. > > > > > > > > > > I think this would also help with reducing the freeze period for a > > > > > milestone > > > > > because it would mean we would never get too far away from something > > > > > that's > > > > > working, so stabilization would be easier. It's also easier to fix > > > > > something that's just been committed than something that you did 2 or > > > > > 3 > > > > > months ago. > > > > > > > > > > I like the suggestion of using the integrity suite as a start - it > > > > > seems > > > > > like a good basic set of tests to aim for. But if you don't think it's > > > > > realistic to keep these green most the time during a development > > > > > cycle, then > > > > > maybe just start with the classlib, drlvm and drlvm-regression tests? > > > > > These > > > > > are all green at the moment on 32-bit Linux and Windows and I think > > > > > the > > > > > Harmony community is already quite good at trying to keep those tests > > > > > passing. > > > > > > > > > > > > > I agree with 32-bit Linux and Windows platforms. But I think that a > > > > set of suites should include Eclipse Hello World Application (EHWA) > > > > suite and may be jdktools too. Say if EHWA is broken than most > > > > probably that there is serious regression. And IMHO such build > > > > shouldn't be published as users build - it can not run simple Eclipse > > > > scenario. > > > > > > > > Thanks, > > > > Stepan. > > > > > > > > > Thanks, > > > > > > > > > > Sian > > > > > > > > > > > > > > > > > > > > On 18/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > > On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > > > > That's good - it sounds like we are aiming for the same thing > > > > > > > after > > > > > > all! It > > > > > > > would be nice if this could be automated at some point, so > > > > > > > successful > > > > > > builds > > > > > > > are published automatically. Also should the build be considered > > > > > > > broken > > > > > > if > > > > > > > we have an unsuccessful build? It just seems to me like > > > > > > > something is > > > > > > wrong > > > > > > > if there hasn't been a 'successful' build for more than 2 weeks. > > > > > > > What > > > > > > do > > > > > > > you think? > > > > > > > > > > > > > > > > > > > Hi Sian, > > > > > > > > > > > > Do you mean 'successful'=='tested' build? > > > > > > > > > > > > I agree that we need to work out a criteria for publishing tested > > > > > > builds for users. Of cause if everything became GREEN on the testing > > > > > > page then the build is published as users build. But it is not so > > > > > > probable to see during active code development. Also I agree that we > > > > > > should define time frame - if we can not publish users build for a > > > > > > long time that we need to stop and fix all failures. I'd suggest to > > > > > > use a set of suites run by integrity testing [1]. So if classlib > > > > > > tests, drlvm and drlvm-regress
Re: [build?] Problem with snapshot build or download page?
Stepan, Thanks for explanation! I believe continuous EHWA runs are quite assuring. Thanks. On 10/22/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > On 10/19/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > > Stepan, > > > > I wonder why EHWA (and EGA) are omitted here for Windows platform: > > http://people.apache.org/~mloenko/snapshot_testing/script/r585787/index.html > > > > Are there any issues with these test suites? > > > > There was an issue with host that runs EGA and the host was rebooted. > As for EHWA - I'm going to add it soon. > > Thanks, > Stepan. > > > Thanks. > > > > On 10/19/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > On 10/18/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > Hi Stepan, > > > > > > > > Thanks for your reply. Yes - I was meaning successful as not only > > > > compiled > > > > succesfully but passed all the required tests. Required tests would be > > > > a > > > > set of tests that we aimed to be GREEN all the time. Then if something > > > > is > > > > checked in that breaks one of these tests it would either be fixed > > > > fairly > > > > quickly (say within a day or two?) or rolled back. We could publish > > > > any (or > > > > every) build that passed this subset, which should mean that there's > > > > never a > > > > long period of time without a build being published. > > > > > > > > I think this would also help with reducing the freeze period for a > > > > milestone > > > > because it would mean we would never get too far away from something > > > > that's > > > > working, so stabilization would be easier. It's also easier to fix > > > > something that's just been committed than something that you did 2 or 3 > > > > months ago. > > > > > > > > I like the suggestion of using the integrity suite as a start - it seems > > > > like a good basic set of tests to aim for. But if you don't think it's > > > > realistic to keep these green most the time during a development cycle, > > > > then > > > > maybe just start with the classlib, drlvm and drlvm-regression tests? > > > > These > > > > are all green at the moment on 32-bit Linux and Windows and I think the > > > > Harmony community is already quite good at trying to keep those tests > > > > passing. > > > > > > > > > > I agree with 32-bit Linux and Windows platforms. But I think that a > > > set of suites should include Eclipse Hello World Application (EHWA) > > > suite and may be jdktools too. Say if EHWA is broken than most > > > probably that there is serious regression. And IMHO such build > > > shouldn't be published as users build - it can not run simple Eclipse > > > scenario. > > > > > > Thanks, > > > Stepan. > > > > > > > Thanks, > > > > > > > > Sian > > > > > > > > > > > > > > > > On 18/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > > > > > > > On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > > > That's good - it sounds like we are aiming for the same thing after > > > > > all! It > > > > > > would be nice if this could be automated at some point, so > > > > > > successful > > > > > builds > > > > > > are published automatically. Also should the build be considered > > > > > > broken > > > > > if > > > > > > we have an unsuccessful build? It just seems to me like something > > > > > > is > > > > > wrong > > > > > > if there hasn't been a 'successful' build for more than 2 weeks. > > > > > > What > > > > > do > > > > > > you think? > > > > > > > > > > > > > > > > Hi Sian, > > > > > > > > > > Do you mean 'successful'=='tested' build? > > > > > > > > > > I agree that we need to work out a criteria for publishing tested > > > > > builds for users. Of cause if everything became GREEN on the testing > > > > > page then the build is published as users build. But it is not so > > > > > probable to see during active code development. Also I agree that we > > > > > should define time frame - if we can not publish users build for a > > > > > long time that we need to stop and fix all failures. I'd suggest to > > > > > use a set of suites run by integrity testing [1]. So if classlib > > > > > tests, drlvm and drlvm-regression tests, jdktools and Eclipse hello > > > > > world application pass on snapshot testing them we publish the build. > > > > > > > > > > [1] http://people.apache.org/~varlax/harmony-integrity/ > > > > > > > > > > Thanks, > > > > > Stepan. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Unless stated otherwise above: > > > > IBM United Kingdom Limited - Registered in England and Wales with number > > > > 741598. > > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > > > > 3AU > > > > > > > > > -- > > With best regards, > > Alexei, > > ESSD, Intel > -- With best regards, Alexei, ESSD, Intel
Re: [build?] Problem with snapshot build or download page?
On 10/19/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > Stepan, > > I wonder why EHWA (and EGA) are omitted here for Windows platform: > http://people.apache.org/~mloenko/snapshot_testing/script/r585787/index.html > > Are there any issues with these test suites? > There was an issue with host that runs EGA and the host was rebooted. As for EHWA - I'm going to add it soon. Thanks, Stepan. > Thanks. > > On 10/19/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > On 10/18/07, Sian January <[EMAIL PROTECTED]> wrote: > > > Hi Stepan, > > > > > > Thanks for your reply. Yes - I was meaning successful as not only > > > compiled > > > succesfully but passed all the required tests. Required tests would be a > > > set of tests that we aimed to be GREEN all the time. Then if something is > > > checked in that breaks one of these tests it would either be fixed fairly > > > quickly (say within a day or two?) or rolled back. We could publish any > > > (or > > > every) build that passed this subset, which should mean that there's > > > never a > > > long period of time without a build being published. > > > > > > I think this would also help with reducing the freeze period for a > > > milestone > > > because it would mean we would never get too far away from something > > > that's > > > working, so stabilization would be easier. It's also easier to fix > > > something that's just been committed than something that you did 2 or 3 > > > months ago. > > > > > > I like the suggestion of using the integrity suite as a start - it seems > > > like a good basic set of tests to aim for. But if you don't think it's > > > realistic to keep these green most the time during a development cycle, > > > then > > > maybe just start with the classlib, drlvm and drlvm-regression tests? > > > These > > > are all green at the moment on 32-bit Linux and Windows and I think the > > > Harmony community is already quite good at trying to keep those tests > > > passing. > > > > > > > I agree with 32-bit Linux and Windows platforms. But I think that a > > set of suites should include Eclipse Hello World Application (EHWA) > > suite and may be jdktools too. Say if EHWA is broken than most > > probably that there is serious regression. And IMHO such build > > shouldn't be published as users build - it can not run simple Eclipse > > scenario. > > > > Thanks, > > Stepan. > > > > > Thanks, > > > > > > Sian > > > > > > > > > > > > On 18/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > > > > > On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > > That's good - it sounds like we are aiming for the same thing after > > > > all! It > > > > > would be nice if this could be automated at some point, so successful > > > > builds > > > > > are published automatically. Also should the build be considered > > > > > broken > > > > if > > > > > we have an unsuccessful build? It just seems to me like something is > > > > wrong > > > > > if there hasn't been a 'successful' build for more than 2 weeks. What > > > > do > > > > > you think? > > > > > > > > > > > > > Hi Sian, > > > > > > > > Do you mean 'successful'=='tested' build? > > > > > > > > I agree that we need to work out a criteria for publishing tested > > > > builds for users. Of cause if everything became GREEN on the testing > > > > page then the build is published as users build. But it is not so > > > > probable to see during active code development. Also I agree that we > > > > should define time frame - if we can not publish users build for a > > > > long time that we need to stop and fix all failures. I'd suggest to > > > > use a set of suites run by integrity testing [1]. So if classlib > > > > tests, drlvm and drlvm-regression tests, jdktools and Eclipse hello > > > > world application pass on snapshot testing them we publish the build. > > > > > > > > [1] http://people.apache.org/~varlax/harmony-integrity/ > > > > > > > > Thanks, > > > > Stepan. > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Unless stated otherwise above: > > > IBM United Kingdom Limited - Registered in England and Wales with number > > > 741598. > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > > > -- > With best regards, > Alexei, > ESSD, Intel
Re: [build?] Problem with snapshot build or download page?
Stepan, I wonder why EHWA (and EGA) are omitted here for Windows platform: http://people.apache.org/~mloenko/snapshot_testing/script/r585787/index.html Are there any issues with these test suites? Thanks. On 10/19/07, Stepan Mishura <[EMAIL PROTECTED]> wrote: > On 10/18/07, Sian January <[EMAIL PROTECTED]> wrote: > > Hi Stepan, > > > > Thanks for your reply. Yes - I was meaning successful as not only compiled > > succesfully but passed all the required tests. Required tests would be a > > set of tests that we aimed to be GREEN all the time. Then if something is > > checked in that breaks one of these tests it would either be fixed fairly > > quickly (say within a day or two?) or rolled back. We could publish any (or > > every) build that passed this subset, which should mean that there's never a > > long period of time without a build being published. > > > > I think this would also help with reducing the freeze period for a milestone > > because it would mean we would never get too far away from something that's > > working, so stabilization would be easier. It's also easier to fix > > something that's just been committed than something that you did 2 or 3 > > months ago. > > > > I like the suggestion of using the integrity suite as a start - it seems > > like a good basic set of tests to aim for. But if you don't think it's > > realistic to keep these green most the time during a development cycle, then > > maybe just start with the classlib, drlvm and drlvm-regression tests? These > > are all green at the moment on 32-bit Linux and Windows and I think the > > Harmony community is already quite good at trying to keep those tests > > passing. > > > > I agree with 32-bit Linux and Windows platforms. But I think that a > set of suites should include Eclipse Hello World Application (EHWA) > suite and may be jdktools too. Say if EHWA is broken than most > probably that there is serious regression. And IMHO such build > shouldn't be published as users build - it can not run simple Eclipse > scenario. > > Thanks, > Stepan. > > > Thanks, > > > > Sian > > > > > > > > On 18/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > > > On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > That's good - it sounds like we are aiming for the same thing after > > > all! It > > > > would be nice if this could be automated at some point, so successful > > > builds > > > > are published automatically. Also should the build be considered broken > > > if > > > > we have an unsuccessful build? It just seems to me like something is > > > wrong > > > > if there hasn't been a 'successful' build for more than 2 weeks. What > > > do > > > > you think? > > > > > > > > > > Hi Sian, > > > > > > Do you mean 'successful'=='tested' build? > > > > > > I agree that we need to work out a criteria for publishing tested > > > builds for users. Of cause if everything became GREEN on the testing > > > page then the build is published as users build. But it is not so > > > probable to see during active code development. Also I agree that we > > > should define time frame - if we can not publish users build for a > > > long time that we need to stop and fix all failures. I'd suggest to > > > use a set of suites run by integrity testing [1]. So if classlib > > > tests, drlvm and drlvm-regression tests, jdktools and Eclipse hello > > > world application pass on snapshot testing them we publish the build. > > > > > > [1] http://people.apache.org/~varlax/harmony-integrity/ > > > > > > Thanks, > > > Stepan. > > > > > > > > > > > > > > > > > -- > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -- With best regards, Alexei, ESSD, Intel
Re: [build?] Problem with snapshot build or download page?
I agree with EHWA, and I don't feel that strongly either way about jdktools if you think it's achievable to get it back to green on those two platforms fairly soon. I was going to offer to investigate the jdktools failures today, but looking at the report it looks like it's in the VM area (a debugging problem?) and I'm afraid I would be totally out of my depth there. Is anyone else able to volunteer? Thanks, Sian On 19/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > On 10/18/07, Sian January <[EMAIL PROTECTED]> wrote: > > Hi Stepan, > > > > Thanks for your reply. Yes - I was meaning successful as not only > compiled > > succesfully but passed all the required tests. Required tests would be > a > > set of tests that we aimed to be GREEN all the time. Then if something > is > > checked in that breaks one of these tests it would either be fixed > fairly > > quickly (say within a day or two?) or rolled back. We could publish any > (or > > every) build that passed this subset, which should mean that there's > never a > > long period of time without a build being published. > > > > I think this would also help with reducing the freeze period for a > milestone > > because it would mean we would never get too far away from something > that's > > working, so stabilization would be easier. It's also easier to fix > > something that's just been committed than something that you did 2 or 3 > > months ago. > > > > I like the suggestion of using the integrity suite as a start - it seems > > like a good basic set of tests to aim for. But if you don't think it's > > realistic to keep these green most the time during a development cycle, > then > > maybe just start with the classlib, drlvm and drlvm-regression > tests? These > > are all green at the moment on 32-bit Linux and Windows and I think the > > Harmony community is already quite good at trying to keep those tests > > passing. > > > > I agree with 32-bit Linux and Windows platforms. But I think that a > set of suites should include Eclipse Hello World Application (EHWA) > suite and may be jdktools too. Say if EHWA is broken than most > probably that there is serious regression. And IMHO such build > shouldn't be published as users build - it can not run simple Eclipse > scenario. > > Thanks, > Stepan. > > > Thanks, > > > > Sian > > > > > > > > On 18/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > > > On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > > > > That's good - it sounds like we are aiming for the same thing after > > > all! It > > > > would be nice if this could be automated at some point, so > successful > > > builds > > > > are published automatically. Also should the build be considered > broken > > > if > > > > we have an unsuccessful build? It just seems to me like something > is > > > wrong > > > > if there hasn't been a 'successful' build for more than 2 > weeks. What > > > do > > > > you think? > > > > > > > > > > Hi Sian, > > > > > > Do you mean 'successful'=='tested' build? > > > > > > I agree that we need to work out a criteria for publishing tested > > > builds for users. Of cause if everything became GREEN on the testing > > > page then the build is published as users build. But it is not so > > > probable to see during active code development. Also I agree that we > > > should define time frame - if we can not publish users build for a > > > long time that we need to stop and fix all failures. I'd suggest to > > > use a set of suites run by integrity testing [1]. So if classlib > > > tests, drlvm and drlvm-regression tests, jdktools and Eclipse hello > > > world application pass on snapshot testing them we publish the build. > > > > > > [1] http://people.apache.org/~varlax/harmony-integrity/ > > > > > > Thanks, > > > Stepan. > > > > > > > > > > > > > > > > > -- > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU > -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [build?] Problem with snapshot build or download page?
Stepan Mishura wrote: > I agree with 32-bit Linux and Windows platforms. But I think that a > set of suites should include Eclipse Hello World Application (EHWA) > suite and may be jdktools too. Say if EHWA is broken than most > probably that there is serious regression. And IMHO such build > shouldn't be published as users build - it can not run simple Eclipse > scenario. Just adding my +1 to all that has been said so far :-) Regards, Tim
Re: [build?] Problem with snapshot build or download page?
On 10/18/07, Sian January <[EMAIL PROTECTED]> wrote: > Hi Stepan, > > Thanks for your reply. Yes - I was meaning successful as not only compiled > succesfully but passed all the required tests. Required tests would be a > set of tests that we aimed to be GREEN all the time. Then if something is > checked in that breaks one of these tests it would either be fixed fairly > quickly (say within a day or two?) or rolled back. We could publish any (or > every) build that passed this subset, which should mean that there's never a > long period of time without a build being published. > > I think this would also help with reducing the freeze period for a milestone > because it would mean we would never get too far away from something that's > working, so stabilization would be easier. It's also easier to fix > something that's just been committed than something that you did 2 or 3 > months ago. > > I like the suggestion of using the integrity suite as a start - it seems > like a good basic set of tests to aim for. But if you don't think it's > realistic to keep these green most the time during a development cycle, then > maybe just start with the classlib, drlvm and drlvm-regression tests? These > are all green at the moment on 32-bit Linux and Windows and I think the > Harmony community is already quite good at trying to keep those tests > passing. > I agree with 32-bit Linux and Windows platforms. But I think that a set of suites should include Eclipse Hello World Application (EHWA) suite and may be jdktools too. Say if EHWA is broken than most probably that there is serious regression. And IMHO such build shouldn't be published as users build - it can not run simple Eclipse scenario. Thanks, Stepan. > Thanks, > > Sian > > > > On 18/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > > > On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > > > That's good - it sounds like we are aiming for the same thing after > > all! It > > > would be nice if this could be automated at some point, so successful > > builds > > > are published automatically. Also should the build be considered broken > > if > > > we have an unsuccessful build? It just seems to me like something is > > wrong > > > if there hasn't been a 'successful' build for more than 2 weeks. What > > do > > > you think? > > > > > > > Hi Sian, > > > > Do you mean 'successful'=='tested' build? > > > > I agree that we need to work out a criteria for publishing tested > > builds for users. Of cause if everything became GREEN on the testing > > page then the build is published as users build. But it is not so > > probable to see during active code development. Also I agree that we > > should define time frame - if we can not publish users build for a > > long time that we need to stop and fix all failures. I'd suggest to > > use a set of suites run by integrity testing [1]. So if classlib > > tests, drlvm and drlvm-regression tests, jdktools and Eclipse hello > > world application pass on snapshot testing them we publish the build. > > > > [1] http://people.apache.org/~varlax/harmony-integrity/ > > > > Thanks, > > Stepan. > > > > > > > > > > -- > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [build?] Problem with snapshot build or download page?
Hi Stepan, Thanks for your reply. Yes - I was meaning successful as not only compiled succesfully but passed all the required tests. Required tests would be a set of tests that we aimed to be GREEN all the time. Then if something is checked in that breaks one of these tests it would either be fixed fairly quickly (say within a day or two?) or rolled back. We could publish any (or every) build that passed this subset, which should mean that there's never a long period of time without a build being published. I think this would also help with reducing the freeze period for a milestone because it would mean we would never get too far away from something that's working, so stabilization would be easier. It's also easier to fix something that's just been committed than something that you did 2 or 3 months ago. I like the suggestion of using the integrity suite as a start - it seems like a good basic set of tests to aim for. But if you don't think it's realistic to keep these green most the time during a development cycle, then maybe just start with the classlib, drlvm and drlvm-regression tests? These are all green at the moment on 32-bit Linux and Windows and I think the Harmony community is already quite good at trying to keep those tests passing. Thanks, Sian On 18/10/2007, Stepan Mishura <[EMAIL PROTECTED]> wrote: > > On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > > That's good - it sounds like we are aiming for the same thing after > all! It > > would be nice if this could be automated at some point, so successful > builds > > are published automatically. Also should the build be considered broken > if > > we have an unsuccessful build? It just seems to me like something is > wrong > > if there hasn't been a 'successful' build for more than 2 weeks. What > do > > you think? > > > > Hi Sian, > > Do you mean 'successful'=='tested' build? > > I agree that we need to work out a criteria for publishing tested > builds for users. Of cause if everything became GREEN on the testing > page then the build is published as users build. But it is not so > probable to see during active code development. Also I agree that we > should define time frame - if we can not publish users build for a > long time that we need to stop and fix all failures. I'd suggest to > use a set of suites run by integrity testing [1]. So if classlib > tests, drlvm and drlvm-regression tests, jdktools and Eclipse hello > world application pass on snapshot testing them we publish the build. > > [1] http://people.apache.org/~varlax/harmony-integrity/ > > Thanks, > Stepan. > > > -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [build?] Problem with snapshot build or download page?
On 10/17/07, Sian January <[EMAIL PROTECTED]> wrote: > That's good - it sounds like we are aiming for the same thing after all! It > would be nice if this could be automated at some point, so successful builds > are published automatically. Also should the build be considered broken if > we have an unsuccessful build? It just seems to me like something is wrong > if there hasn't been a 'successful' build for more than 2 weeks. What do > you think? > Hi Sian, Do you mean 'successful'=='tested' build? I agree that we need to work out a criteria for publishing tested builds for users. Of cause if everything became GREEN on the testing page then the build is published as users build. But it is not so probable to see during active code development. Also I agree that we should define time frame - if we can not publish users build for a long time that we need to stop and fix all failures. I'd suggest to use a set of suites run by integrity testing [1]. So if classlib tests, drlvm and drlvm-regression tests, jdktools and Eclipse hello world application pass on snapshot testing them we publish the build. [1] http://people.apache.org/~varlax/harmony-integrity/ Thanks, Stepan.
Re: [build?] Problem with snapshot build or download page?
That's good - it sounds like we are aiming for the same thing after all! It would be nice if this could be automated at some point, so successful builds are published automatically. Also should the build be considered broken if we have an unsuccessful build? It just seems to me like something is wrong if there hasn't been a 'successful' build for more than 2 weeks. What do you think? Thanks, Sian On 17/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > 2007/10/17, Sian January <[EMAIL PROTECTED]>: > > On 17/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > > > > > 2007/10/16, Sian January <[EMAIL PROTECTED]>: > > > > Hi Mikhail, > > > > > > > > Thanks for clarifying that - I had thought that the development page > had > > > > milestone builds for users and snapshots for developers (or users > who > > > want > > > > the absolute latest build). Just out of interest, how do you decide > > > whether > > > > to promote a snapshot to the download page? > > > > > > If the results are not worse than the previous ones > > > > > > I'm just a little confused > > > > about why we have them as well as milestone builds, since we have > quite > > > > frequent milestone releases. I guess I would probably prefer to see > > > > downloads available for the latest successful build. E.g. if a user > > > raises > > > > a bug you would be able to get them to try the latest build if it > was > > > > available to download, and then get them to try the latest again > when a > > > fix > > > > goes in etc. > > > > > > That's right. The question is how we define "successful" build. If it > > > does not run > > > "Hello world", is it successful? > > > > > > > > I think a "successful" build would at least pass all the Harmony tests > > (maybe on a subset of platforms). I guess other test suites could be > added > > in if they're passing reliably and they don't take too long, but > publishing > > every build that passes our own test suite would make sense to me, at > least > > to start with. > > I beleive Stepan uses similar criteria: successful is the build which > passes > some reasonable set of tests. > > Thanks, > Mikhail > > > > > > > > > > Thanks, > > > Mikhail > > > > > > > > > > > Thanks, > > > > > > > > Sian > > > > > > > > > > > > On 16/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > > > > > > > > > The idea is we have snapshots for developers that are provided > with > > > > > the test results > > > > > and snapshots for users who does not care which to download and > need > > > > > "a snapshot" > > > > > > > > > > Download page points to some recent snapshot that is a bit more > > > stable, > > > > > while testing page points to snapshots built according to the > fixed > > > > > schedule > > > > > > > > > > Thanks, > > > > > Mikhail > > > > > > > > > > 2007/10/16, Sian January <[EMAIL PROTECTED]>: > > > > > > Hello, > > > > > > > > > > > > I have just noticed that the latest snapshot available from the > > > download > > > > > > page is still r580985 (i.e. M3), but it looks like two snapshots > > > have > > > > > been > > > > > > created since then (see > > > > > http://people.apache.org/builds/harmony/snapshots/). > > > > > > Is this a bug in the download page or has something gone wrong > when > > > the > > > > > new > > > > > > snapshots were uploaded or built? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Sian > > > > > > > > > > > > -- > > > > > > Unless stated otherwise above: > > > > > > IBM United Kingdom Limited - Registered in England and Wales > with > > > number > > > > > > 741598. > > > > > > Registered office: PO Box 41, North Harbour, Portsmouth, > Hampshire > > > PO6 > > > > > 3AU > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Unless stated otherwise above: > > > > IBM United Kingdom Limited - Registered in England and Wales with > number > > > > 741598. > > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire > PO6 > > > 3AU > > > > > > > > > > > Thanks, > > > > Sian > > > > -- > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU > > > -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [build?] Problem with snapshot build or download page?
2007/10/17, Sian January <[EMAIL PROTECTED]>: > On 17/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > > > 2007/10/16, Sian January <[EMAIL PROTECTED]>: > > > Hi Mikhail, > > > > > > Thanks for clarifying that - I had thought that the development page had > > > milestone builds for users and snapshots for developers (or users who > > want > > > the absolute latest build). Just out of interest, how do you decide > > whether > > > to promote a snapshot to the download page? > > > > If the results are not worse than the previous ones > > > > I'm just a little confused > > > about why we have them as well as milestone builds, since we have quite > > > frequent milestone releases. I guess I would probably prefer to see > > > downloads available for the latest successful build. E.g. if a user > > raises > > > a bug you would be able to get them to try the latest build if it was > > > available to download, and then get them to try the latest again when a > > fix > > > goes in etc. > > > > That's right. The question is how we define "successful" build. If it > > does not run > > "Hello world", is it successful? > > > > I think a "successful" build would at least pass all the Harmony tests > (maybe on a subset of platforms). I guess other test suites could be added > in if they're passing reliably and they don't take too long, but publishing > every build that passes our own test suite would make sense to me, at least > to start with. I beleive Stepan uses similar criteria: successful is the build which passes some reasonable set of tests. Thanks, Mikhail > > > > > Thanks, > > Mikhail > > > > > > > > Thanks, > > > > > > Sian > > > > > > > > > On 16/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > > > > > > > The idea is we have snapshots for developers that are provided with > > > > the test results > > > > and snapshots for users who does not care which to download and need > > > > "a snapshot" > > > > > > > > Download page points to some recent snapshot that is a bit more > > stable, > > > > while testing page points to snapshots built according to the fixed > > > > schedule > > > > > > > > Thanks, > > > > Mikhail > > > > > > > > 2007/10/16, Sian January <[EMAIL PROTECTED]>: > > > > > Hello, > > > > > > > > > > I have just noticed that the latest snapshot available from the > > download > > > > > page is still r580985 (i.e. M3), but it looks like two snapshots > > have > > > > been > > > > > created since then (see > > > > http://people.apache.org/builds/harmony/snapshots/). > > > > > Is this a bug in the download page or has something gone wrong when > > the > > > > new > > > > > snapshots were uploaded or built? > > > > > > > > > > Thanks, > > > > > > > > > > Sian > > > > > > > > > > -- > > > > > Unless stated otherwise above: > > > > > IBM United Kingdom Limited - Registered in England and Wales with > > number > > > > > 741598. > > > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire > > PO6 > > > > 3AU > > > > > > > > > > > > > > > > > > > > > -- > > > Unless stated otherwise above: > > > IBM United Kingdom Limited - Registered in England and Wales with number > > > 741598. > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > > 3AU > > > > > > > Thanks, > > Sian > > -- > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >
Re: [build?] Problem with snapshot build or download page?
On 17/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > 2007/10/16, Sian January <[EMAIL PROTECTED]>: > > Hi Mikhail, > > > > Thanks for clarifying that - I had thought that the development page had > > milestone builds for users and snapshots for developers (or users who > want > > the absolute latest build). Just out of interest, how do you decide > whether > > to promote a snapshot to the download page? > > If the results are not worse than the previous ones > > I'm just a little confused > > about why we have them as well as milestone builds, since we have quite > > frequent milestone releases. I guess I would probably prefer to see > > downloads available for the latest successful build. E.g. if a user > raises > > a bug you would be able to get them to try the latest build if it was > > available to download, and then get them to try the latest again when a > fix > > goes in etc. > > That's right. The question is how we define "successful" build. If it > does not run > "Hello world", is it successful? I think a "successful" build would at least pass all the Harmony tests (maybe on a subset of platforms). I guess other test suites could be added in if they're passing reliably and they don't take too long, but publishing every build that passes our own test suite would make sense to me, at least to start with. > Thanks, > Mikhail > > > > > Thanks, > > > > Sian > > > > > > On 16/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > > > > > The idea is we have snapshots for developers that are provided with > > > the test results > > > and snapshots for users who does not care which to download and need > > > "a snapshot" > > > > > > Download page points to some recent snapshot that is a bit more > stable, > > > while testing page points to snapshots built according to the fixed > > > schedule > > > > > > Thanks, > > > Mikhail > > > > > > 2007/10/16, Sian January <[EMAIL PROTECTED]>: > > > > Hello, > > > > > > > > I have just noticed that the latest snapshot available from the > download > > > > page is still r580985 (i.e. M3), but it looks like two snapshots > have > > > been > > > > created since then (see > > > http://people.apache.org/builds/harmony/snapshots/). > > > > Is this a bug in the download page or has something gone wrong when > the > > > new > > > > snapshots were uploaded or built? > > > > > > > > Thanks, > > > > > > > > Sian > > > > > > > > -- > > > > Unless stated otherwise above: > > > > IBM United Kingdom Limited - Registered in England and Wales with > number > > > > 741598. > > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire > PO6 > > > 3AU > > > > > > > > > > > > > > > -- > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU > > > Thanks, Sian -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [build?] Problem with snapshot build or download page?
2007/10/16, Sian January <[EMAIL PROTECTED]>: > Hi Mikhail, > > Thanks for clarifying that - I had thought that the development page had > milestone builds for users and snapshots for developers (or users who want > the absolute latest build). Just out of interest, how do you decide whether > to promote a snapshot to the download page? If the results are not worse than the previous ones I'm just a little confused > about why we have them as well as milestone builds, since we have quite > frequent milestone releases. I guess I would probably prefer to see > downloads available for the latest successful build. E.g. if a user raises > a bug you would be able to get them to try the latest build if it was > available to download, and then get them to try the latest again when a fix > goes in etc. That's right. The question is how we define "successful" build. If it does not run "Hello world", is it successful? Thanks, Mikhail > > Thanks, > > Sian > > > On 16/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > > > The idea is we have snapshots for developers that are provided with > > the test results > > and snapshots for users who does not care which to download and need > > "a snapshot" > > > > Download page points to some recent snapshot that is a bit more stable, > > while testing page points to snapshots built according to the fixed > > schedule > > > > Thanks, > > Mikhail > > > > 2007/10/16, Sian January <[EMAIL PROTECTED]>: > > > Hello, > > > > > > I have just noticed that the latest snapshot available from the download > > > page is still r580985 (i.e. M3), but it looks like two snapshots have > > been > > > created since then (see > > http://people.apache.org/builds/harmony/snapshots/). > > > Is this a bug in the download page or has something gone wrong when the > > new > > > snapshots were uploaded or built? > > > > > > Thanks, > > > > > > Sian > > > > > > -- > > > Unless stated otherwise above: > > > IBM United Kingdom Limited - Registered in England and Wales with number > > > 741598. > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > > 3AU > > > > > > > > > -- > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >
Re: [build?] Problem with snapshot build or download page?
Hi Mikhail, Thanks for clarifying that - I had thought that the development page had milestone builds for users and snapshots for developers (or users who want the absolute latest build). Just out of interest, how do you decide whether to promote a snapshot to the download page? I'm just a little confused about why we have them as well as milestone builds, since we have quite frequent milestone releases. I guess I would probably prefer to see downloads available for the latest successful build. E.g. if a user raises a bug you would be able to get them to try the latest build if it was available to download, and then get them to try the latest again when a fix goes in etc. Thanks, Sian On 16/10/2007, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > The idea is we have snapshots for developers that are provided with > the test results > and snapshots for users who does not care which to download and need > "a snapshot" > > Download page points to some recent snapshot that is a bit more stable, > while testing page points to snapshots built according to the fixed > schedule > > Thanks, > Mikhail > > 2007/10/16, Sian January <[EMAIL PROTECTED]>: > > Hello, > > > > I have just noticed that the latest snapshot available from the download > > page is still r580985 (i.e. M3), but it looks like two snapshots have > been > > created since then (see > http://people.apache.org/builds/harmony/snapshots/). > > Is this a bug in the download page or has something gone wrong when the > new > > snapshots were uploaded or built? > > > > Thanks, > > > > Sian > > > > -- > > Unless stated otherwise above: > > IBM United Kingdom Limited - Registered in England and Wales with number > > 741598. > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU > > > -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [build?] Problem with snapshot build or download page?
The idea is we have snapshots for developers that are provided with the test results and snapshots for users who does not care which to download and need "a snapshot" Download page points to some recent snapshot that is a bit more stable, while testing page points to snapshots built according to the fixed schedule Thanks, Mikhail 2007/10/16, Sian January <[EMAIL PROTECTED]>: > Hello, > > I have just noticed that the latest snapshot available from the download > page is still r580985 (i.e. M3), but it looks like two snapshots have been > created since then (see http://people.apache.org/builds/harmony/snapshots/). > Is this a bug in the download page or has something gone wrong when the new > snapshots were uploaded or built? > > Thanks, > > Sian > > -- > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >
