On Mon, Sep 17, 2012 at 5:10 AM, Justin Deoliveira <[email protected]>wrote:
> Hi all,
>
> A while back I sent an email about the state of GeoServer testing, and how
> at current the full unit test suite takes a lot of time to run. Here is the
> email for reference.
>
>
> http://www.mail-archive.com/[email protected]/msg17055.html
>
> I have been pushing on some of the ideas there and started a proposal
> which can be found here:
>
> http://geoserver.org/display/GEOS/GSIP+80+-+Testing+Overhaul
>
> By no means is the proposal in a complete state but I thought it would be
> beneficial to have something to help focus discussion around.
>
> Feedback welcome. I hope to bring it up as a topic of discussion at the
> bi-weekly skype meeting but understand it is pretty short notice for people
> to review before then.
>
Hi,
had a quick look, wow, long proposal.
Random remarks:
- generally speaking it's looking good
- even when running tests that do not alter the configuration during their
execution
we often add in the one time setup some extra layer, so I'm not sure how
these
would play against the idea of sharing tests in the same group
- how would MockTest work, would it just fill an in memory catalog without
any writing on disk?
- either JUnit4 and TestNG work for me, I'm happy to use their extra when
I need them
and grumpy that I have to write more code to use them in the normal case
(having to write the annotations instead of using name based conventions,
add the static import)
Wondering if we could have the base class extend Assert so that we don't
have to
statically import it all the time? :-p
- about tests modifying the data (as opposed to the configuration) un-doing
the changes may
be difficult, we would need some way to tell the mock setup to re-copy
the data over and
then force a "reset" of the resource pool
- similarly for tests that do modify the configuration, it would be handy
to have some way
to tell the test setup to copy over the original configuration and just
force a reload
- about figuring out the coverage structure without having to open it...
seems rather
hard, wondering if we can get more mileage by asking tests to declare
which data
they do want, since most tests only use a handful of layers?
There should still be a way to get the current default setup, just not by
default
About the tests having to manually undo what they modified I have some
concerns
in terms of broken builds, a developer might forget to do some of the
rollbacks and
if different test classes end up sharing the same setup we might see
OS/file system
dependent failures due to the different order in which the tests are
running.
An approach we could consider is to use catalog/configuration/transaction
listeners
to automatically mark resources that have been modified and roll them back
to their original state (not sure if it's 100% feasible, but at least part
of it could).
Btw, you have pretty long build times, on my almost 3 years old desktop
machine I can
run the release config in significant less time than what you reported:
mvn clean install -Prelease -o
--> [INFO] Total time: 24 minutes 42 seconds
(full build times added at the end of the mail)
Still, a long time regardless, the build should be way faster than this, in
2.1.x
the build is roughly taking half of the time.
Wondering if you have any disk indexing service running that slows down the
build?
Cheers
Andrea
PS: here is the full report with build times:
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
------------------------------------------------------------------------
[INFO] GeoServer ............................................. SUCCESS
[4.652s]
[INFO] Core Platform Module .................................. SUCCESS
[3.024s]
[INFO] Open Web Service Module ............................... SUCCESS
[6.283s]
[INFO] Main Module ........................................... SUCCESS
[2:00.991s]
[INFO] Web Feature Service Module ............................ SUCCESS
[2:33.199s]
[INFO] Web Coverage Service Module ........................... SUCCESS
[2.121s]
[INFO] Web Map Service Module ................................ SUCCESS
[2:36.512s]
[INFO] GeoServer Web Modules ................................. SUCCESS
[0.058s]
[INFO] Core UI Module ........................................ SUCCESS
[1:09.153s]
[INFO] Security UI Module .................................... SUCCESS
[1:52.482s]
[INFO] GeoServer Security Modules ............................ SUCCESS
[0.166s]
[INFO] GeoServer CAS Security Module ......................... SUCCESS
[13.768s]
[INFO] GeoServer JDBC Security Module ........................ SUCCESS
[2:07.383s]
[INFO] GeoServer LDAP Security Module ........................ SUCCESS
[2.674s]
[INFO] Web Coverage Service 1.0 Module ....................... SUCCESS
[32.144s]
[INFO] Web Coverage Service 1.1 Module ....................... SUCCESS
[34.731s]
[INFO] GeoWebCache (GWC) Module .............................. SUCCESS
[34.645s]
[INFO] REST Support Module ................................... SUCCESS
[12.459s]
[INFO] REST Configuration Service Module ..................... SUCCESS
[2:21.622s]
[INFO] WMS UI Module ......................................... SUCCESS
[32.537s]
[INFO] GWC UI Module ......................................... SUCCESS
[34.326s]
[INFO] WFS UI Module ......................................... SUCCESS
[13.353s]
[INFO] Demoes Module ......................................... SUCCESS
[24.569s]
[INFO] WCS UI Module ......................................... SUCCESS
[15.648s]
[INFO] GeoServer Web Application ............................. SUCCESS
[30.243s]
[INFO] Community Space ....................................... SUCCESS
[0.108s]
[INFO] GeoServer Extensions .................................. SUCCESS
[0.116s]
[INFO] Application Schema Support ............................ SUCCESS
[0.097s]
[INFO] Application Schema Integration Test ................... SUCCESS
[1:22.845s]
[INFO] Sample DataAccess Integration Test .................... SUCCESS
[11.009s]
[INFO] ArcSDE DataStore Extension ............................ SUCCESS
[13.157s]
[INFO] GeoSearch Index Module ................................ SUCCESS
[13.699s]
[INFO] H2 DataStore Extension ................................ SUCCESS
[7.568s]
[INFO] SQL Server DataStore Extension ........................ SUCCESS
[0.180s]
[INFO] Oracle DataStore Extension ............................ SUCCESS
[0.119s]
[INFO] MySQL DataStore Extension ............................. SUCCESS
[0.187s]
[INFO] DB2 DataStore Extension ............................... SUCCESS
[0.196s]
[INFO] ImageMap Output Format ................................ SUCCESS
[6.331s]
[INFO] ImageI/O-Ext GDAL Coverage Extension .................. SUCCESS
[0.702s]
[INFO] JP2K Coverage Extension ............................... SUCCESS
[0.213s]
[INFO] ogr2ogr Output Format ................................. SUCCESS
[13.169s]
[INFO] Excel Output Format ................................... SUCCESS
[15.366s]
[INFO] Validation Module ..................................... SUCCESS
[1.686s]
[INFO] Chart external graphics support ....................... SUCCESS
[0.293s]
[INFO] Feature Generalization Extension ...................... SUCCESS
[1.608s]
[INFO] Image Mosaic JDBC Extension ........................... SUCCESS
[0.188s]
[INFO] OWS request flow controller ........................... SUCCESS
[9.206s]
[INFO] Web Processing Service parent ......................... SUCCESS
[0.062s]
[INFO] Web Processing Service Module ......................... SUCCESS
[1:18.758s]
[INFO] Web Processing Service GUI ............................ SUCCESS
[17.917s]
[INFO] GeoServer Layer Querying filter functions ............. SUCCESS
[20.420s]
[INFO] Teradata DataStore Extension .......................... SUCCESS
[1.853s]
[INFO] GeoServer Release Module .............................. SUCCESS
[2.357s]
[INFO]
------------------------------------------------------------------------
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 24 minutes 42 seconds
[INFO] Finished at: Mon Sep 17 10:00:15 CEST 2012
[INFO] Final Memory: 93M/306M
[INFO]
------------------------------------------------------------------------
--
==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Geoserver-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
