Does it include output? Yes
Felix> Your original version of "B" included the response string to make it
Felix> more verbose and more tester friendly, so they are -- in my eyes --
Felix> rather equivalent in output.
A will print:
AssertionError: Response message should not contain FileNotFoundException
at ...
at ...
B will print
AssertionError: Response must not contain FNFE
at ...
at ...
C will print something like
AssertionError: the string shot not contain "FileNotFoundException",
but the actual value was "..>><<<!!!!.. "
In this regard, C is way better since it enables to cross-check the
output even without looking into the sources.
For instance, the actual FileNotFoundException might include some
weird file name or path or whatever.
In that regard options A&B just print "there was some error"
Note, one can use A* which would be "as good as C":
A*) Assert.fail("Response must not contain FNFE, actual response was:
" + sample.getResponseDataAsString());
Felix>which I expect a tester to be familiar with. I find myself often
Felix> struggling to find the right words to state the expectation of the
Felix> assertion. Should I print out the thing I want, the thing that failed or
Felix> both?
Typically, there are 3 items:
1) input data
2) action
3) output
4) expected output
Good failure should include all of the above in one way or another.
Felix>Than we could write "Assert.assertThat(adjustedDelay,
Felix> ToleranceMatcher.isAbout(1000L, 150L))" which could result in an error
Felix> message like
>
Felix> Expected: is about <1000L> with tolerance of <150L>
Felix> but was: <800L>
Would you be glad to receive bug report Bug 29562: Expected: is about
<1000L> with tolerance of <150L> but was: <800L> ?
Does the message include input data? No
Does it include action? No
Does it include output? Yes
Does it include expected output? Yes
All the items might be described in a human-understandable way
(especially in case of boolean-like results).
For instance: "When test duration is set to Long.MAX_VALUE, we don't
want the engine to cap or modify timer pauses, so the sleep delay of
1sec should be executed as ~1s"
Note: I have no idea the intention behind that test. It did took me a
while to browse the code back and forth to produce ANY meaningful
text, and I have absolutely no idea if my text has anything with
actual intention behind the test.
There might be similar test side by side with message like "When sleep
delay falls outside of the scheduled interval, we want to limit the
sleep to prevent threads waiting more than test duration"
Those kind of messages do include input data and action however those
are included in a free-form fashion.
Felix>Should I print out the thing I want, the thing that failed or
Felix> both?
For cases like "csv split test", or "regex test" or similar I like to
have function+its arguments as the message, so the failure looks like
AssertionError: split("1,2", delim=,) expected: {1, 2}, got: {1}
Just note: the message includes both input values, the action
("split"), and the output values.
One might think it is "duplication of the code" or that "it takes
enormous time to write the test", however:
1) The code is much more often read than written
2) For a given function (e.g. stringSplit) you create a single
function with assert call, then you use that checker to write trivial
looking tests:
checkSplit("1,2", 1, 2);
checkSplit("1", 1);
checkSplit("1,1,2", 1, 1, 2);
etc.
Of course there are Spock tests, there are @Parameterized, etc,
however my idea is even plain old JUnit 4 tests are pretty simple to
write and read.
By the way, as sebb said, test class and method name helps as well.
However Java method name with 200 chars and spaces does not sound
quite right to me :)
Felix> refactored and as stated above, the programmer has do good texting.
Undestandable code commends simplify refactoring, don't they?
Treat assert messages like code comments, except it is way better to
have the text in assert message rather than in a comment above.
The comment won't be visible in CI log
Felix> As always thoughtful and well put.
Thanks,
Vladimir
