How important is it to run gremlins on different ROMs on POSE? And, since PalmSim runs so much faster, is it important to run gremlins on both POSE and PalmSim? If so, what additional information is gathered?
It seems to me that all of these questions could be combined into "How important is it to test on as many different platforms as possible?" I'd think that the answer would be "pretty important".
Obviously, there are some diminishing returns. Testing on one platform provides, say 90% effectiveness, testing a different platform takes up some amount of the remaining 10% (with that "some amount" increasing the more different that second platform is from the first, such as testing on Palm Simulator or an ARM device after first testing on a 68K device or Poser). At some point, you have to decide if you have sufficiently covered your bases enough. Most people will test against a representative sample of devices and OSes; I don't think you'll find anyone doing thorough testing against *all* devices and OSes that have come out.
Does gremlins give different results on debug ROMs vs standard ROMs?
I'm not sure, but I don't think the answer to the question is particularly useful. I mean, what if I said "yes"? Then what would you do? Include that in your testing mix, with the intent of throwing as many different variables at your application as possible? If that's the intent, I'd suggest focusing on different OS versions (e.g. 3.x, 4.x, and 5.x) instead of Debug vs. Release.
But keep in mind that using Debug ROMs should flush out more problems than when using Release ROMs, so if you need to decide between one or the other, use the Debug ROMs.
-- Keith Rollin -- Palm OS Emulator engineer
-- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
