Stepan Mishura wrote:
On 3/30/06, Richard Liang wrote:
Dears,
I notice that we put all the test code into one big test method (for
example,
org.apache.harmony.tests.java.util.jar.test_putLjava_lang_ObjectLjava_lang_Object
).
This way we will lose some benefits of junit and even unit test:
1. Test code cannot share configuration code through setUp and tearDown
2. We have to add redundant code, such as, "Assert 1:", "Assert 2: ...."
to make the test results more comprehensive
3. It makes the test code more complex
Shall we just use small test cases?
You may want to read the description at:
http://www.javaworld.com/javaworld/jw-12-2000/jw-1221-junit_p.html
*Keep tests small and fast*
Executing every test for the entire system shouldn't take hours. Indeed,
developers will more consistently run tests that execute quickly.
Without regularly running the full set of tests, it will be difficult to
validate the entire system when changes are made. Errors will start to
creep back in, and the benefits of unit testing will be lost. This means
stress tests and load tests for single classes or small frameworks of
classes shouldn't be run as part of the unit test suite; they should be
executed separately.
Hi Richard,
IMHO, this relates to "stress tests and load tests" only. This means that we
shouldn't put such kind of tests in a 'regular test suite'. The 'regular
test suite' is used to verify regressions only. Returning back to a test's
size, I think it is up to developer - we can only recommend not to test all
functionality in one test case and split independent parts into a number of
test case. But IMHO we can not fully avoid creating 'redundant code', such
as, "Assert 1:", "Assert 2: ....". For example, if there is a constructor
with several parameters and get-methods to return provided parameters then I
wouldn't create 3 tests instead of the next one:
public void test_Ctor() {
Ctor c = new Ctor(param1, param2, param3);
assertEquals("Assert 1", param1, c.getParam1());
assertEquals("Assert 2", param2, c.getParam3());
assertEquals("Assert 3", param3, c.getParam2());
}
Hello Stepan,
Sometimes we do have to use several assert to check the expected
situation. It's true. However, what I mean is, let's take your
"test_Ctor" as an example, if we want to design many test cases to
verify the behavior of the constructor, the number of assert may
increase dramatically.
Do you need just one test method or 4 methods for the following test?
public void test_Ctor() {
Ctor c = new Ctor(param1, param2, param3);
assertEquals("Assert 1", param1, c.getParam1());
assertEquals("Assert 2", param2, c.getParam3());
assertEquals("Assert 3", param3, c.getParam2());
Ctor c2 = new Ctor(null, param2, param3);
assertNull("Assert 4", c2.getParam1());
assertEquals("Assert 5", param2, c2.getParam3());
assertEquals("Assert 6", param3, c3.getParam2());
Ctor c3 = new Ctor(param1, null, param3);
assertEquals("Assert 7", param1, c3.getParam1());
assertNull("Assert 8", c3.getParam3());
assertEquals("Assert 9", param3, c3.getParam2());
Ctor c4 = new Ctor(param1, param2, null);
assertEquals("Assert 10", param1, c4.getParam1());
assertEquals("Assert 11", param2, c4.getParam3());
assertNull("Assert 12", c4.getParam2());
}
Thanks,
Stepan.
Thanks a lot.
Richard Liang wrote:
Dears,
As I cannot find similar pages about testing convention, I just create
one with my rough ideas
http://wiki.apache.org/harmony/Testing_Convention, so that we can
document our decision timely & clearly.
Geir Magnusson Jr wrote:
Leo Simons wrote:
Gentlemen!
On Mon, Mar 27, 2006 at 11:07:51AM +0200, mr A wrote:
On Monday 27 March 2006 10:14, mr B wrote:
On 3/27/06, mr C wrote:
[SNIP]
[SNIP]
[SNIP]
On 1/1/2006, mr D wrote:
[SNIP]
Hmmm... Lemme support [SNIP]
Now let me support [SNIP].
The ASF front page says
(...) "The Apache projects are characterized by a collaborative,
consensus
based development process, " (...)
That's not just some boilerplate. Consensus is a useful thing.
"How should we organize our tests?" has now been the subject of
debate for
*months* around here, and every now and then much of the same
discussion is
rehashed.
And we're making progress. IMO, it really helped my thinking to
distinguish formally between the implementation tests and the spec
tests, because that *completely* helped me come to terms with the
whole o.a.h.test.* issue.
I now clearly see where o.a.h.test.*.HashMapTest fits, and where
java.util.HashMapTest fits.
I don't think our issues were that obvious before, at least to me.
Now, I see clearly.
I think it would be more productive to look for things to agree on
(such as,
"we don't know, but we can find out", or "we have different ideas on
that,
but there's room for both", or "this way of doing things is not the
best one
but the stuff is still useful so let's thank the guy for his work
anyway")
than to keep delving deeper and deeper into these kinds of
disagreements.
Of course, the ASF front page doesn't say that "apache projects are
characterized by a *productive* development process". Its just my
feeling that
for a system as big as harmony we need to be *very* productive.
You don't think we're making progress through these discussions?
Think about it. Is your time better spent convincing lots of other
people to do
their testing differently, or is it better spent writing better tests?
The issue isn't about convincing someone to do it differently, but
understanding the full scope of problems, that we need to embrace
both approaches, because they are apples and oranges, and we need
both apples and oranges. They aren't exclusionary.
geir
--
Richard Liang
China Software Development Lab, IBM
--
Thanks,
Stepan Mishura
Intel Middleware Products Division
--
Richard Liang
China Software Development Lab, IBM
