Emmanuel Lecharny wrote:
Yeah, running integration tests is time consumming. For 1.0.1, it's
acceptable, as it takes only 3min (depending on your computer), but in
1.5, it's around 5 times slower.
We have many more tests about 40% more.
We may consider finding out why it takes so long in 1.5 (server
initialization takes around 10 to 15 seconds, which is not really normal)
Please Emmanuel it does not take this long for startup. You're going to
freak people out by saying stuff like this. Let's watch these kinds of
exaggerations.
It does take a little longer in 1.5 but really we have about 40% more
tests and initialization time is slower but no where near 10-15 seconds.
A single integration test case (with one test method) takes about 3.5
seconds to start, stop and cleanup on my old hardware (AMD 2800+).
Here's a run of just one empty test case. Committed here:
http://svn.apache.org/viewvc?view=rev&revision=503758
Here's how to run it in apacheds/core-unit
mvn -Dtest=OneEmptyITest -Dintegration test
Here's how much time it took for this ITest to run:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.directory.server.core.schema.OneEmptyITest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.538 sec
And as you can see 3.538 seconds is not anywhere near 10-15 seconds
exaggeration. Now if you're running this on a really slow machine (400
Mhz) with bad drives then I can see this being the case.
Now you said that this is 5x slower in 1.5 branch than it is in the 1.0
branch. Let's take a look at the same test case in the 1.0 branch.
Here's there result there:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.directory.server.core.schema.OneEmptyITest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.752 sec
So we're only 2x slower in the 1.5 branch than in the 1.0 branch. Again
that's way less than the 5 times figure you just threw out there.
Please let's be careful with what we toss out there next time. People
are listening and will run with negative news especially when it's not
true and is coming from the mouths of our own people.
Brotha man, I hope you understand my reasons for asking this and don't
think its me trying to bust your keyboard :).
...
May I suggest a third solution? We might use tools like jmeter which are
able to launch tests against a ldap server, playing full scenarios. The
good part of it is that you can prepare the scenarios with a GUI, and
save them as XML files to be replayed within maven (there is a
maven-jmeter-plugin). The good points about this scenario that you only
launch the server once, tests are easy to write and to maintain, and
there is no more relation with the actual code base, so no huge
refactoring to do in tests when modifying base classes like AttributeImpl.
This is a great idea but how do we get this to be part of the build
process? Basically how do we get maven to launch a test server on
startup that will be available for the entire run while tests operate on it?
This is a great idea if we can make maven do it.
Alex
begin:vcard
fn:Alex Karasulu
n:Karasulu;Alex
org:Apache Software Foundation;Apache Directory
adr:;;1005 N. Marsh Wind Way;Ponte Vedra ;FL;32082;USA
email;internet:[EMAIL PROTECTED]
title:Member, V.P.
tel;work:(904) 791-2766
tel;fax:(904) 808-4789
tel;home:(904) 808-4789
tel;cell:(904) 315-4901
note;quoted-printable:AIM: alexokarasulu=0D=0A=
MSN: [EMAIL PROTECTED]
Yahoo!: alexkarasulu=0D=0A=
IRC: aok=0D=0A=
PGP ID: 1024D/4E1370F8 BBCC E8D8 8756 2D51 C3D4 014A 3662 F96F 4E13 70F8=0D=0A=
x-mozilla-html:FALSE
url:http://people.apache.org/~akarasulu
version:2.1
end:vcard
