Hi Andrea, Thanks for the investigation and fixes !
I put my tests results as comments in the PR ... here it improved the build times by 1 minutes and 34 seconds, i.e. from 5:52 minutes to 4:18 minutes (without documentation) using 'mvn clean install -T4 -Dall -nsu -fn'. I'm still not capable of building GeoTools documentation using Maven ... it always fails due to some ant error but it works when I invoke the ant command directly ... so documentation build was not take into account here. Cheers, Nuno Oliveira On 10/10/2017 05:29 PM, Andrea Aime wrote:
Hi, I just got myself a new desktop machine and I was testing its GeoTools build times. Having a 8 core CPU (AMD Ryzen 1700x) I set it free to use as many threads as possible (-T1C) and looked at the results. They were not bad, more than twice as fast as my old machine, still, the CPU consumption graph suggested there was room for improvement. Looking into it I've found a few parts that were network dependent (calling onto remote servers), sometimes without good reason, dependencies that were declared but not used, and sometimes used dependencies that were put there to save a handful of lines of code, but making the build a lot more serial than necessary. I've also backported from GeoServer a surefire test speedup trick that was already used there, and that after the fixes contributes significantly to the overall speed up. This pull request summarizes the work: https://github.com/geotools/geotools/pull/1716 One final note, the 4 minutes and a half time were obtained by building inside of "modules". The main reason is the documentation, which takes a lot of time at the very end, and most of the time it's "irrelevant" to the build success. That last statement needs qualifying... the documentation module contains actual example code that should be built, but also the sphinx docs that most of the commits are not touching at all, and yet we are spending a lot of time building them with sphinx. Imho that should be changed, so that unless someone is interested, the sphinx translation is not done, similar to how GeoServer works, when one builds the documents only if they are being worked on. Cheers Andrea PS: there is a NetCDF test that's using a lot of time, NetCdfCRSTest, like 38 seconds, but only when run from Maven, if I run it from IDE it completes in like 2.5 seconds. I've tried to look into it, but could not find why there is such a large difference. I've noticed that ignoring it brings the parallel build down by an overall 10 extra seconds, which is not bad. Anyone has a clue as to what's happening there? I've tried to look at the other NetCDF tests run before it, resetting the CRS subsystem, tried not setting up H2 in server mode, but all in vain... ==GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.==Ing. Andrea Aime @geowolfTechnical LeadGeoSolutions S.A.S.Via di Montramito 3/A55054 Massarosa (LU)phone: +39 0584 962313fax: +39 0584 1660272mob: +39 339 8844549http://www.geo-solutions.ithttp://twitter.com/geosolutions_it AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ GeoTools-Devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geotools-devel
-- Regards, Nuno Oliveira == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Nuno Miguel Carvalho Oliveira @nmcoliveira Software Engineer GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________ GeoTools-Devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geotools-devel
