Yes, exactly. To (hopefully) clarify, running Gremlin 0..1 for 1,000,000 events
with a switching depth of 10,000 should give the same results as Gremlin 0..1
for 1,000,000 events with a switching depth of 100,000 or even running Gremlin 0
for 1,000,000 events followed by running Gremlin 1 for 1,000,000 events.
-- Keith
Richard Hartman <[EMAIL PROTECTED]> on 12/10/99 10:15:00 AM
Please respond to [EMAIL PROTECTED]
Sent by: Richard Hartman <[EMAIL PROTECTED]>
To: "'palm-dev-forum @3com.com'" <[EMAIL PROTECTED]>
cc: (Keith Rollin/HQ/3Com)
Subject: RE: POSE 3.0a3 - Unreliable ???
Ok ... let me see if I got this straight -- you aren't
merely swapping between pseudo-random sequences generating
taps on the same instance of the app running on the emulator;
you are actually doing complete context swapping, effectively
running multiple, separate, instances of the app on the
emulator, each with it's own gremlin and time-slicing between
them using the "switching depth" as the trigger?
--
-Richard M. Hartman
[EMAIL PROTECTED]
186,000 mi./sec ... not just a good idea, it's the LAW!
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, December 09, 1999 7:26 PM
> To: [EMAIL PROTECTED]
> Subject: RE: POSE 3.0a3 - Unreliable ???
>
>
>
>
> The point isn't to interleave two (or more) different
> sequences for the benefit
> of randomness. The point is to try find as many bugs as
> possible. With Gremlins
> Classic, there were two problems with trying to do that:
>
> Problem #1: Running a Gremlin and then leaving for the night
> would often result
> in the Gremlin finding a bug and then sitting there for the
> rest of the night,
> idling when it could be off doing other things like running
> other Gremlins. Even
> if it didn't find a bug and ran for 1,000,000 events, it
> could still be sitting
> there for hours until you came back.
>
> Solution #1: Gremlin Hordes, where Poser switches to a new
> Gremlin after a
> previous Gremlin finds a bug or runs to completion.
>
> Problem #2: We've noticed that a Gremlin would often find a
> bug immediately,
> within the first 30,000 events, or not at all. Thus, the
> Horde mechanism would
> benefit (that is, find the most number of bugs) by doing some
> sort of "breadth
> first" search instead of running a single Gremlin to
> completion first. After
> all, if a Gremlin didn't turn up a bug in the first 30,000
> events, why spend
> time running it for the next 970,000 events if you could just
> switch to another
> Gremlin and increase the chance that you'll find a bug?
>
> Solution #2: The "Switching Depth", where you can specify how
> deep the "breadth
> first" search is.
>
> -- Keith Rollin
> -- Palm OS Emulator engineer
>
>
>
>
>
>
> Richard Hartman <[EMAIL PROTECTED]> on 12/09/99 07:04:58 PM
>
> Please respond to [EMAIL PROTECTED]
>
> Sent by: Richard Hartman <[EMAIL PROTECTED]>
>
>
> To: "'palm-dev-forum @3com.com'" <[EMAIL PROTECTED]>
> cc: (Keith Rollin/HQ/3Com)
> Subject: RE: POSE 3.0a3 - Unreliable ???
>
>
>
>
> I'll agree that you may need to test with more
> than just gremlin #0 ... but interleaving two
> different sequences every 1000 values? I'm still
> not sure I see that that is anything more than
> just a different pseudorandom sequence. It seems
> as if it is just an extra level of complexity that
> does not add value. You can get all sorts of
> pseudorandom sequences depending upon the seed
> you choose ... is interleaving two different
> pseudorandom sequences any "more" random than
> one sequence?
>
> --
> -Richard M. Hartman
> [EMAIL PROTECTED]
>
> 186,000 mi./sec ... not just a good idea, it's the LAW!
>
>
> > -----Original Message-----
> > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, December 09, 1999 10:34 AM
> > To: [EMAIL PROTECTED]
> > Subject: RE: POSE 3.0a3 - Unreliable ???
> >
> >
> >
> >
> > Is any pseudo-random number generator random enough? There
> > are plenty of
> > case-examples where an application will survive 1 million
> > events at Gremlin #0,
> > but fail on other Gremlins. Here at Palm, we like to do as
> > much testing as
> > possible.
> >
> > There's even the (very valid) argument that Gremlins testing
> > it not enough when
> > it comes to automated testing. That's why the scripting
> > facility was added -- so
> > that you can ensure that all parts of your application are
> > testing in every
> > conceivable way for which you feel like writing a test case.
> >
> > -- Keith Rollin
> > -- Palm OS Emulator engineer
> >
> >
> >
> >
> >
> >
> > Richard Hartman <[EMAIL PROTECTED]> on 12/09/99 09:25:22 AM
> >
> > Please respond to [EMAIL PROTECTED]
> >
> > Sent by: Richard Hartman <[EMAIL PROTECTED]>
> >
> >
> > To: "'palm-dev-forum @3com.com'" <[EMAIL PROTECTED]>
> > cc: (Keith Rollin/HQ/3Com)
> > Subject: RE: POSE 3.0a3 - Unreliable ???
> >
> >
> >
> >
> > And what is the point of this? Is the random tapping
> > of a single gremlin sequence not random enough?
> >
> > --
> > -Richard M. Hartman
> > [EMAIL PROTECTED]
> >
> > 186,000 mi./sec ... not just a good idea, it's the LAW!
> >
> >
> > > -----Original Message-----
> > > From: Frykholm, Niklas [mailto:[EMAIL PROTECTED]]
> > > Sent: Wednesday, December 08, 1999 11:43 PM
> > > To: '[EMAIL PROTECTED]'
> > > Subject: RE: POSE 3.0a3 - Unreliable ???
> > >
> > >
> > > >5) I looked at the count and it was 395,000. Some time
> > later it was
> > > >400,000. Then a few minutes later it was back to 390,000. Very
> > > >irratic.
> > >
> > > It's not a bug, it's a feature! The latest version of POSE
> > > has something
> > > called "gremlin hordes", which means it switches between
> > > several different
> > > gremlins. If the switching depth is set to 10000 (which I
> > > believe is the
> > > default) POSE runs each Gremlin for 10000 events and then
> > > switches to the
> > > next.
> > >
> > > This is probably what happened in your case. POSE ran events
> > > 390000-400000
> > > of gremlin #0 and then events 390000-400000 of gremlin #1.
> > > That is why the
> > > counter was seemingly set back. (If you studied the dialog
> > > box carefully you
> > > would notice that the gremlin number in the upper right
> > > corner changed.)
> > >
> > > // Niklas
> > >
> >
> >
> >
> >
> >
> >
>
>
>
>
>
>
