On Tue, 22 Mar 2005 19:02:51 +0000, David Cantrell <[EMAIL PROTECTED]> wrote: > On Mon, Mar 21, 2005 at 06:21:38PM -0800, Ben Tilly wrote: [...] > > A fun issue is popups. Everything works fine and then someday > > there is an unexpected error and a popup stops everything in its > > tracks. Sure, you can probably put in some kind of search for > > popups that makes them go away, but do you dare? Until you see > > them do you know whether it is safe to ignore this popup? > > > > The issue here isn't even gui vs command line, the same problem > > was the bane of expect scripts. It is fairly simple to teach the > > computer what happens if everything goes right. But in > > manipulating someone else's user interface you discover > > boundary cases the hard way - one by one. The result is very > > fragile. > > Whenever your test script meets something it has not been taught to > expect that is either a bug in what you are testing, or a bug in the > test script. In either case, the only correct thing to do is stop.
I almost commented on this in my previous email. What you say is absolutely correct for test scripts. It most emphatically is NOT correct for production jobs. When you have a chain of dependencies, halting the whole process dead in its tracks at the first sign of trouble leads to a process that does not complete. Instead you want to log anything that seems minor, pause and alert someone on anything that seems major (or is unknown), and be able to be told to continue, told to stop, and be able to be restarted from a known clean state without having lost all of your work so far. If you have frequent intermediate states that you can restart from, then it may be OK to always stop and just get restarted from there. Adding that extra logic adds a lot of complexity. Adding it while working remotely through your (possibly incompletely understood) interface adds a lot more complexity. And once written, even minor UI tweaks are likely to cause havoc, so you cannot easily upgrade the program being driven. If you're calling a library (by contrast), that extra logic is usually easier to add. (For one thing you don't have to categorize errors - they already come as warnings or fatal.) The API tends to be a lot simpler than the UI was, and API changes from one version to the next are far less likely to cause problems, so you are much more able to upgrade. If your job is testing software, then driving a UI is necessary, there is no other way to test the UI. If your job is maintaining a production system, you really want a programmatic API. Cheers, Ben PS Disclaimer: I've had to run nightly production jobs. I have never been responsible for automated testing. Assume appropriate biases. _______________________________________________ Boston-pm mailing list [email protected] http://mail.pm.org/mailman/listinfo/boston-pm
