Rick pointed out that I forgot to include how to do negative tests.
This is really an important part of testing, and since I spent a lot
of time in test, I'm kicking myself for overlooking it here. <grin>
We'll continue with the STREAM.testGroup and look at what the doc says
about the second arg. Which is exactly: "The second argument can be
one of the following strings ..." and it the lists exactly 3 strings:
State, Description, and Command.
So a negative test is what happens when you don't follow a
requirement. (Well there are other types of negative tests, but let's
not get too complicated here.)
The requirement is that the second arg be one of only 3 different
things. What we want to test, is what happens if we don't use of the
3 specified args. Say if we use FILESTATE ?
Staying with the same STREAM.testGroup. How do we test this?
::method test_wrong_2nd_arg
parse source junk notUsed streamName
retDiscrpt = stream(streamName, "FILESTATE")
Maybe something along those lines? Output:
[error] [20080701 12:06:53.206000]
Test: TEST_WRONG_2ND_ARG
Class: STREAM.testGroup
File: C:\work.ooRexx\ooRexxUnit\3.2.0\ooRexx\STREAM.testGroup
Event: [SYNTAX 40.904] raised unexpectedly.
STREAM argument 2 must be one of SDC; found "FILESTATE"
Line: 83
83 *-* retDiscrpt = stream(streamName, "FILESTATE")
Now if you look closely at that, it was not a failure, it was an error.
If you remember back to what I said about terminology, I said with a
test case you expect one of 2 outcomes. You either expect the test to
pass, or you expect the test to fail.
Errors are something totally unexpected. We can see from the output
that the interpreter is handling this the way it should. But, what we
want is a test case for this that passes.
In this case we *expect* the error condition. So our test case will
pass when we get an error. The framework provides for these types of
test with the expectSyntax() method.
We code our test like this:
::method test_wrong_2nd_arg
parse source junk notUsed streamName
self~expectSyntax(40.904)
retDiscrpt = stream(streamName, "FILESTATE")
And the output we get is:
C:\ooRexxUnit.3.2.0>rexx testOORexx.rex -R ooRexx -f stream
..
Interpreter: REXX-ooRexx_3.2.0(MT) 6.02 30 Oct 2007
ooRexxUnit: 2.0.0_3.2.0 ooTest: 1.0.0_3.2.0
Tests ran: 5
Assertions: 3
Failures: 0
Errors: 0
Skipped files: 0
File search: 00:00:00.160000
Suite construction: 00:00:00.000000
Test execution: 00:00:00.000000
Total time: 00:00:00.160000
Great, we now have 3 tests coded and they all pass.
What else can we test in this way? We see from the doc that the first
arg to stream is required. What happens if we leave it out? We
should get some type of error. And the interpreter should not blow
up.
Now how to code it? Similar to what we just did:
::method test_no_args
--self~expectSyntax(40.904)
retDiscrpt = stream()
But, what syntax error do we expect? The best thing to do, is to look
up the error numbers and descriptions and decide for yourself what you
think it *should* be.
The error message list is in the back of the doc, in an appendix. I
don't want to drag this out, so I'm going to say, since the first
error was in the 40. range, that is a good place to start. Looking
there we see:
001
External routine "routine" failed
Okay, that seems reasonable. Though, there is also:
005
Missing argument in invocation of routine; argument argument_number is required
Aha, that is it. Argument 1 is required, says the doc. So the
message: "Missing argument in invocation of STREAM; argument 1 is
required" seems perfect.
Out test looks like:
::method test_no_args
self~expectSyntax(40.005)
retDiscrpt = stream()
and our output looks like this:
[error] [20080701 12:27:15.323000]
Test: TEST_NO_ARGS
Class: STREAM.testGroup
File: C:\work.ooRexx\ooRexxUnit\3.2.0\ooRexx\STREAM.testGroup
Event: [SYNTAX 40.3] raised unexpectedly.
Not enough arguments in invocation of STREAM; minimum expected is 1
Line: 90
90 *-* retDiscrpt = stream()
Eureka! We found a bug! This is what it is all about, finding bugs. <grin>
But, let's investigate a little. What is 40.3?
003
Not enough arguments in invocation of routine; minimum expected is number
Making the message: Not enough arguments in STREAM; minimum expected is 1
Well, okay, that message also seems to fit. What to do? If you find
something like this, that you think is a bug, then the thing to do is
bring it up on the developer's list and lay out your case for why you
think this is a bug.
In this instance, you would not prevail with your case. The reason
being that 40.3 is applicable and it is what has been used.
Therefore, changing it because you think 40.5 is better will most
likely break existing code that is checking for this specific error
code.
So, the test looks like this:
::method test_no_args
self~expectSyntax(40.003)
retDiscrpt = stream()
output:
[error] [20080701 12:39:59.063000]
Test: TEST_NO_ARGS
Class: STREAM.testGroup
File: C:\work.ooRexx\ooRexxUnit\3.2.0\ooRexx\STREAM.testGroup
Event: [SYNTAX 40.3] raised unexpectedly.
Not enough arguments in invocation of STREAM; minimum expected is 1
Line: 90
90 *-* retDiscrpt = stream()
Great!! Now we REALLY found a bug! It is supposed to be error 40.003.
Well, again, do a little research. What the doc says is:
Some errors have associated subcodes. A subcode is a one- to
three-digit decimal extension to the error number, for example, 115 in
40.115
A subcode is a *one* to three digit.
Okay, no bug. Our test looks like this:
::method test_no_args
self~expectSyntax(40.3)
retDiscrpt = stream()
output:
Interpreter: REXX-ooRexx_3.2.0(MT) 6.02 30 Oct 2007
ooRexxUnit: 2.0.0_3.2.0 ooTest: 1.0.0_3.2.0
Tests ran: 6
Assertions: 4
Failures: 0
Errors: 0
Skipped files: 0
File search: 00:00:00.160000
Suite construction: 00:00:00.000000
Test execution: 00:00:00.000000
Total time: 00:00:00.160000
Now we have 4 tests of an area that hadn't been touched. The 4
additional tests add 0 seconds to the overall test execution time. 2
of the tests are positive tests, 2 are negative.
--
Mark Miesfeld
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
