2007/4/5, Alexey Varlamov <[EMAIL PROTECTED]>:
2007/4/5, Mikhail Loenko <[EMAIL PROTECTED]>:
> 2007/4/4, Vladimir Ivanov <[EMAIL PROTECTED]>:
> > On 4/3/07, Stepan Mishura <[EMAIL PROTECTED]> wrote:
> > > On 4/3/07, Vladimir Ivanov wrote:
> > > <SNIP>
> > > >
> > > > > I'd like to propose the next approach that may help us to know about
> > > > > instabilities: develop (or take existing one, for example, Eclipse
> > > > > hello world) a scenario for testing stability and configure CC to run
> > > > > it at all times. The stability scenario must be the only one scenario
> > > > > for CC; it must be short (no longer then an hour), test JRE in stress
> > > > > conditions and cover most of functionality. If the scenario fails then
> > > > > all newly committed updates are subject for investigation and fix (or
> > > > > rollback).
> > > >
> > > > Actually, I prefer something without GUI or at least without using
> > > > special 'GUI testing" tools. It should improve quality of this testing
> > > > (than less tools than more predictable results :)) Current "Eclipse
> > > > hello world" scenario based on the AutoIT for Win and X11GuiTest for
> > > > Linux platform. Also we have this scenario based on API calls which
> > > > should emulate GUI scenario. From these 2 approaches I prefer second
> > > > to minimize 'false alarms'. Or may be some other scenarios (non-GUI)?
> > > >
> > >
> >
> > > Did I understand you correctly that there may be 'false alarms' caused
> > > by using external 'GUI testing' tools? If yes which kind of 'false
> > > alarms' are there?
> >
> > Seems, in some cases the input from the X11GuiTest can be lost in the
system :(
> > Also timeouts between different symbols may depends on the system load etc
>
> How often does EHWA fail? Given that this scenario exercises VM very well
> and the scenario itself is pretty fast we may run it twice and report
> a problem if
> it fails both attempts.
As Vladimir already mentioned, drlvm build provides automated EHWA
test driven via Eclipse APIs, it is reliable and very fast - why not
use it instead of GUI robots, indeed?
But it would be nice to have some non-Eclipse GUI testing since
Eclipse does not use awt and swing.
SY, Alexey
