Hello Mark,
> The true intent here is to find* real bugs *in the interpreter.
I totally agree with you 100%!
It is obvious Rick and you have been doing this long enough to (A) simply
know or (B) follow your gut on what are "real bugs" in the interpreter. I
would like to thank you for being so patient with me and my reports,
especially those the "acceptable" error / failure reports.
At this point, I should ask: How do I learn to tell the difference between
"real bugs" and those errors / failures we can expect and thus do not need
to report? (I guess: How do new testers gain you knowledge and experience
quickly?)
> In most cases, in my opinion, it is better to keep the test then to drop
it.
I completely agree! You can never have enough regression tests and as long
as you know the expected results and where to ignore errors /
failures..... AFTER ALL, so tests should return error conditions as the
suit should insure the application catches expected generated failures /
errors.
The thing to remember, though, is: As you expand the pool of people who
run the test suite and have to evaluate the results at the "local level,"
the more documentation one needs OR the smarter the code needs to be in
reporting failure.
> It is better to keep the tests, because they some of the only tests that
> test the OLEObject class, than to worry about them. And, the tests work
> fine for me.
I agree 100%. Never get rid of a regression test! :-)
> Don't write such long e-mails. ;-) Don't use the -s flag. The only use
> for that flag is if things are not running to completion. If the entire
> test suite finishes, there is no need for it. Just report the results.
Ah! Thank you! As I say, I need to learn more about the test suit. Right
now, I'm just running it without understanding it. (Thus my above
requests.)
Take care,
Bert.
P.S. This PS is on the topic of getting more people to run the test suit
and may be way off base. Ignore it, as you desire. I'm not here to push
an agenda other than trying to suggest ways to help get more people running
the test suite.
I'm thinking there are at least two reasons people aren't running the test
suite:
1) They're not sure what commitment they'll have to make to help. They may
well be willing to help out, but don't want to buy a "pig in the poke" or
as some might say, "I want to count the cost before agreeing to do it." (I
came up with the last one after listening to a church sermon.)
2) They would participate in the test suite, if it didn't take up so much
time and/or was easier to figure out what to report, when to update, etc.
A while back I suggested a distributed testing method that would test on
people's machines. In that case, the results wouldn't go to the local
user, but back through the network to the server and whomever reviews it
there.
I know this isn't something you're going to offer anytime soon and until
there are lots of people willing to run the test case it doesn't make
sense. The ONLY reason I'm even bringing this up is to suggest ways to
increase the number of people who run the test suite and use the
candidates. (Sort of like "build it and they will come" (preventing a
catch-22 situation)).
Here is a thought to consider for the future on how one could increase the
number of people who use the test suit.
After creating a distributed testing system (lots of work, I know), each
release candidate would come with the framework. During the installation
users would have several choices:
1) Not install the test suite at all.
2) Install only the manual test suite (with or without automatic updating
enabled).
3) Install both the manual test suite and distributed framework.
In the case of #2 and #3 the test suite would NOT come the candidate
release, but only the ability to download the current test suit.
If the user accepts #2 (s)he could decline automatic updating. Even with
declining the system would still let the user know when a new test suit
exists, but not download / install it.
Choosing #3 would mean accepting automatic updating, as well as #2 could
accept automatic downloading / installing. It would also mean the user
agreed to allow a remote server to run the test cases and report back.
As part of insuring security (#3), a run would never occur without specific
permission at the time the run occurs. This way the user would be
expecting to see stuff flash on his desktop / screen. Yet, wouldn't be
needing to involve him or herself beyond this point.
Anyway, as I say this would be the future and is just a thought /
suggestion. The ONLY reason I'm even bringing this up is to suggest ways
to increase the number of people who run the test suite and use the
candidates.
Finally, an enhancement thought for ooRexx: Include a way for the product
to tell the end-user a new release or candidate release is available would
be great! Even include an automatic download and optional installation
start would be wonderful. It is just a thought.
------------------------------------------------------------------------------
Android apps run on BlackBerry 10
Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
Now with support for Jelly Bean, Bluetooth, Mapview and more.
Get your Android app in front of a whole new audience. Start now.
http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
_______________________________________________
Oorexx-users mailing list
Oorexx-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-users
