Anton Avtamonov wrote:
On 3/27/06, Geir Magnusson Jr <[EMAIL PROTECTED]> wrote:
[SNIP]
I don't think it's easier to write test cases for internals though a
public API, because you can call the internal methods directly to see if
they do as they promise, where it's not so easy to always have the same
thing happen through an indirect public API.
Also, wrt things breaking..... yes, tests that ensure that something
works as expected will need to change if the expectation changes.
That also is exactly what you want, right? To know if you broke a
'internal contract' in a class?
Hmmm... Lemme support Richard's point.
At the first, internals not just specify 'internal functional
contracts', but design decisions. Making redesign you will probably
have some (design-dependent) unit tests failing, despite functionality
is not changed. If so we will have to change the tests, right? If we
changing the tests we cannot be sure they still tests the original
behavior. IMHO, tests are the formal and executable way to represent
requirements (XP approach) so they should reflect the 'requirements
level'. In our case that is public API.
1) We're not doing XP. We actually have predefined requirements.
2) For the sake of argument, I'll grant you that tests are the formal
and executable way to represent the requirements, but that would be the
spec/API tests (which we need as well).
2) The test failing is the correct thing, because the functionality did
change.
Suppose you made a change that you *thought* didn't change the
functionality, but actually did. Wouldn't you want a fine-grained unit
test to find that early?
At the second, what we really need to satisfy is API behavior only.
In the end, yes.
Internal implementation is not significantly important if public part
works fine.
Mmmmm. I'm not so sure.
I fond of XP and test-driven development and don't feel
that we need to test something which is not part of requirements :-).
If something is not accessible from the usage scenarios of public API
level, it is not important how it works (I believe this idea appeared
several times already in this thread).
And I don't agree, since I don't believe that you can make a test suite
w/ sufficient coverage. The only real opportunity that we have is
something like gump, which punts on the formal test approach, and does
it a different way - by running anything and everything in a
"monte-carlo"-like technique which hopes to cover the parameter space.
IMO, Gump is a fantastic tool for testing cross-project API contracts,
for example, and will be very helpful for assisting us in finding
problems, but I still believe that a deliberate testing approach both at
the implementation level as well as the spec level would serve us well.
I completely understand the issue of complexity if we test big amout
of code referenced by public API, however I believe that in the
majority of cases the behavior of each API method is not very
complicated and can be tested by few scenarios. At least I haven't
seen spec for some method of several pages long :-) Usually the
behavior is comparative simple.
At the end (but not least important) you have to 'verify' your tests
somehow. You prepare them basing on specification, however you should
check them on RI. Just to make sure test is correct, spec corresponds
to RI, etc. I cannot understand how it could be done for
'internals'...
It's irrelevant for internals, because internals our our implementation.
No need to verify that our internals tests work on anything else.
geir
