Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )
Hi Andrea, sorry for this late answer, I was AFK on vacation for last 14 days. I would like to thank you for all this work you have done! I am really looking forward to testing this Thank you very much and have a nice Christmas =) Kind regards, Jaroslav On Sat, Dec 10, 2016 at 11:40 AM Andrea Aime <andrea.a...@geo-solutions.it> wrote: > Hi, > for those interested, I've prepared a couple of pull requests to deal with > these issues. Not sure > when they are going to land on 2.10 and 2.9, right now they are pending > review: > https://github.com/geoserver/geoserver/pull/1999 > https://github.com/geoserver/geoserver/pull/2008 > > Cheers > Andrea > > > On Tue, Dec 6, 2016 at 12:26 PM, Andrea Aime <andrea.a...@geo-solutions.it > > wrote: > > I'm afraid this won't help as the code I've discovered is in the GWC > integration and independent of > how the storage is setup. > The only way to get rid of it without changing the code would be to remove > all gwc jars from GeoServer (which is btw completely feasible, if one does > not need caching/WMTS) > > Cheers > Andrea > > On Mon, Dec 5, 2016 at 8:46 PM, kweis...@harris.com <kweis...@harris.com> > wrote: > > I haven’t tested this in a while, but we experienced this problem in the > past and discovered that using the JDBC-Config plugin (from community > modules) resolved the issue. There is an initial hit during the first run > with JDBC-Config enabled as it reads every resource file, validates it, and > then inserts into the database, but then on subsequent startups the system > is ready almost instantaneously. > > Kevin M. Weiss > Software Engineer > CRITICAL NETWORKS / HARRIS CORPORATION > harris.com / kevin.we...@harris.com > > > From: Jaroslav Urik [mailto:jarda.u...@gmail.com] > Sent: Thursday, December 01, 2016 8:30 AM > To: Julian Hollingbery <jul...@northtech.dk>; > geoserver-users@lists.sourceforge.net > Subject: Re: [Geoserver-users] GeoServer Slow to Start with Lots of > Namespaces ( > 3h ) > > Yes, that might solve it, but I do not know how to achieve this.. Also I > hope that geonetwork just check if file exist and does not read the whole > file.. > > On Thu, Dec 1, 2016 at 3:09 PM Julian Hollingbery <jul...@northtech.dk> > wrote: > … or possibly running these checks in parallel? > > Fra: Jaroslav Urik [mailto:jarda.u...@gmail.com] > Sendt: 1. december 2016 14:36 > Til: geoserver-users@lists.sourceforge.net > Emne: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > > 3h ) > > Hi, > I have the same problem as described in [1] -- I am running GeoServer > under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes > very long time to start - about 3-4 hours. > The problem seems to be in that GeoServer is checking all of those > resources one by one each time it starts.. > My question is: Is there any way to disable these checks? Or do them only > one a month or so? > Or is there any other solution? > It is quite annoying when server needs to be restarted and people are > waiting for it.. > > Thanks in advance > > Jaroslav Urik > > > [1] > http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-with-Lots-of-Namespaces-td5068101.html > > -- > ___ > Geoserver-users mailing list > Geoserver-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > > > > -- > == > GeoServer Professional Services from the experts! Visit > http://goo.gl/it488V for more information. > == > > Ing. Andrea Aime > @geowolf > Technical Lead > > GeoSolutions S.A.S. > Via di Montramito 3/A > 55054 Massarosa (LU) > phone: +39 0584 962313 <+39%200584%20962313> > fax: +39 0584 1660272 <+39%200584%20166%200272> > mob: +39 339 8844549 <+39%20339%20884%204549> > > http://www.geo-solutions.it > http://twitter.com/geosolutions_it > > *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* > > Le informazioni contenute in questo messaggio di posta elettronica e/o > nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il > loro utilizzo è consentito esclusivamente al destinatario del messaggio, > per le finalità indicate nel messaggio stesso. Qualora riceviate questo > messaggio senza esserne il destinatario, Vi preghiamo cortesemente di > darcene notizia via e-mail e di procedere alla distruzione del messaggio > stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, > divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od > utilizzarl
Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )
Yes, that might solve it, but I do not know how to achieve this.. Also I hope that geonetwork just check if file exist and does not read the whole file.. On Thu, Dec 1, 2016 at 3:09 PM Julian Hollingbery <jul...@northtech.dk> wrote: > … or possibly running these checks in parallel? > > > > *Fra:* Jaroslav Urik [mailto:jarda.u...@gmail.com] > *Sendt:* 1. december 2016 14:36 > *Til:* geoserver-users@lists.sourceforge.net > *Emne:* [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces > ( > 3h ) > > > > Hi, > > I have the same problem as described in [1] -- I am running GeoServer > under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes > very long time to start - about 3-4 hours. > > The problem seems to be in that GeoServer is checking all of those > resources one by one each time it starts.. > > My question is: Is there any way to disable these checks? Or do them only > one a month or so? > > Or is there any other solution? > > It is quite annoying when server needs to be restarted and people are > waiting for it.. > > > > Thanks in advance > > > > Jaroslav Urik > > > > > > [1] > http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-with-Lots-of-Namespaces-td5068101.html > -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
[Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )
Hi, I have the same problem as described in [1] -- I am running GeoServer under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes very long time to start - about 3-4 hours. The problem seems to be in that GeoServer is checking all of those resources one by one each time it starts.. My question is: Is there any way to disable these checks? Or do them only one a month or so? Or is there any other solution? It is quite annoying when server needs to be restarted and people are waiting for it.. Thanks in advance Jaroslav Urik [1] http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-with-Lots-of-Namespaces-td5068101.html -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
[Geoserver-users] GeoServer startup time (>2h)
Hi, I have been experiencing some very long GeoServer startup times - more than 2 hours. I looked into the logs and found out that geoserver is loading each layer, but at a pace of ~3/sec, which seems very slow, since I have ~14000 of these and will have more.. The rasters are stored on external server accessed via NFS. They are in JPEG2000 format, Geoserver is using Kakadu 6.4 library, has access to 1024MB of RAM (-XX:MaxPermSize=1024m), and the CPU load during startup is ~20%. So, is there any way to speed this up? Maybe to skip some checking of these layers? Thank you Jaroslav Urik -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] GeoServer startup time (>2h)
Hi Simone, I am running Geoserver 2.7.2. The images are mainly old maps in jp2, here is gdalinfo of one of them: >>> gdalinfo /storage/.../d01/00012_00010_300dpi.jp2 Driver: JP2KAK/JPEG-2000 (based on Kakadu v6.4) Files: /storage/.../d01/00012_00010_300dpi.jp2 /storage/.../d01/00012_00010_300dpi.wld Size is 2820, 16419 Coordinate System is `' Origin = (13.273995359042500,51.310171640957499) Pixel Size = (0.000343281915000,-0.000343281915000) Metadata: Corder=RPCL TIFFTAG_RESOLUTIONUNIT=3 (pixels/cm) TIFFTAG_XRESOLUTION=118.103 TIFFTAG_YRESOLUTION=118.103 Corner Coordinates: Upper Left ( 13.2739954, 51.3101716) Lower Left ( 13.2739954, 45.6738259) Upper Right ( 14.2420504, 51.3101716) Lower Right ( 14.2420504, 45.6738259) Center ( 13.7580229, 48.4919988) Band 1 Block=2048x128 Type=Byte, ColorInterp=Red Overviews: 1410x8210, 705x4105, 353x2053, 177x1027, 89x514 Image Structure Metadata: COMPRESSION=JP2000 Band 2 Block=2048x128 Type=Byte, ColorInterp=Green Overviews: 1410x8210, 705x4105, 353x2053, 177x1027, 89x514 Image Structure Metadata: COMPRESSION=JP2000 Band 3 Block=2048x128 Type=Byte, ColorInterp=Blue Overviews: 1410x8210, 705x4105, 353x2053, 177x1027, 89x514 Image Structure Metadata: COMPRESSION=JP2000 <<< Ad. -XX:MaxPermSize: Thank you, I will reconfigure this. The -Xmx is set to 2048M. Ad. layers: It is 50k (was 14k) of old maps, so I might have mistaken term "layer" for something else, the term "image" might suit it better? Anyway the Geoserver tells me, that I have 34 Workspaces, 49,721 Stores and 49,660 Layers. If I understand the system correctly, than yes, I need 50k of layers. I am using geoserver to view those maps in openLayers and via wms. Thank you Jaroslav On Mon, Oct 26, 2015 at 1:26 PM Simone Giannecchini < simone.giannecch...@geo-solutions.it> wrote: > Dear Jaroslav, > please, find my comments inline below > > Regards, > Simone Giannecchini > == > GeoServer Professional Services from the experts! > Visit http://goo.gl/it488V for more information. > == > Ing. Simone Giannecchini > @simogeo > Founder/Director > > GeoSolutions S.A.S. > Via Poggio alle Viti 1187 > 55054 Massarosa (LU) > Italy > phone: +39 0584 962313 > fax: +39 0584 1660272 > mob: +39 333 8128928 > > http://www.geo-solutions.it > http://twitter.com/geosolutions_it > > --- > AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 > Le informazioni contenute in questo messaggio di posta elettronica e/o > nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il > loro utilizzo è consentito esclusivamente al destinatario del messaggio, > per le finalità indicate nel messaggio stesso. Qualora riceviate questo > messaggio senza esserne il destinatario, Vi preghiamo cortesemente di > darcene notizia via e-mail e di procedere alla distruzione del messaggio > stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, > divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od > utilizzarlo per finalità diverse, costituisce comportamento contrario ai > principi dettati dal D.Lgs. 196/2003. > > The information in this message and/or attachments, is intended solely for > the attention and use of the named addressee(s) and may be confidential or > proprietary in nature or covered by the provisions of privacy act > (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection > Code).Any use not in accord with its purpose, any disclosure, reproduction, > copying, distribution, or either dissemination, either whole or partial, is > strictly forbidden except previous formal approval of the named > addressee(s). If you are not the intended recipient, please contact > immediately the sender by telephone, fax or e-mail and delete the > information in this message that has been received in error. The sender > does not give any warranty or accept liability as the content, accuracy or > completeness of sent messages and accepts no responsibility for changes > made after they were sent or for other risks which arise as a result of > e-mail transmission, viruses, etc. > > On Mon, Oct 26, 2015 at 8:34 AM, Jaroslav Urik <jarda.u...@gmail.com> > wrote: > >> Hi, >> I have been experiencing some very long GeoServer startup times - more >> than 2 hours. >> >> I looked into the logs and found out that geoserver is loading each >> layer, but at a pace of ~3/sec, which seems very slow, since I have ~14000 >> of these and will have more.. >> > > > This is quite strange. > Can you give us more info about the GeoServer version as well as the > output of gdalinfo on a single file? > > >> >> The rasters are stored on external server accessed via NFS. The
[Geoserver-users] GeoServer - kakadu 6.4 problem ( jp2000 )
Hi, I am trying to read my *.jp2 images using kakadu-6.4 library, but with no luck (yet). Until now I have managed to load and read them using the open source library for jpeg2000, but it was really slow, so I would like to use the kakadu library. My system: Ubuntu server 14.04 Tomcat 8.0.26 GeoServer 2.7.2 Kakadu 6.4 (in /usr/local/lib/kakadu-6.4/) I have compiled the kakadu sources and gdal with kakadu according to https://trac.osgeo.org/gdal/wiki/JP2KAK I have downloaded the jp2000 plugin for version 2.7.2 and placed it to webapps/geoserver/WEB-INF/lib/ I set the LD_LIBRARY_PATH in Tomcat, via ../tomcat/bin/setenv.sh by ... export LD_LIBRARY_PATH="/usr/local/lib" ... But when I open the loaded images (loaded programmatically as external.worldimage) in openlayers, it is still slow as it was. When I try to load new images (manually) via Stores/Add new store/JP2K (Direct) ... Where I input the image location, I get "Could not list layers for this store, an error occurred retrieving them: null" I tried the geoserver.log, but all I found mentioning jp2 was: >>> 2015-10-01 13:54:03,665 DEBUG [gdal.jp2ecw] - JP2ECWFormatFactory is not availaible. 2015-10-01 13:54:03,665 DEBUG [coverageio.jp2k] - JP2KFormatFactory is availaible. 2015-10-01 13:54:03,666 DEBUG [gdal.jp2k] - JP2KFormatFactory is not availaible. 2015-10-01 13:54:03,666 DEBUG [gdal.jp2mrisd] - JP2MrSIDFormatFactory is not availaible. 2015-10-01 13:54:03,666 DEBUG [coverageio.jp2k] - Creating a new JP2KFormat. <<< which seems ok to me.. Can anyone please point out where I could have made mistake? I would be grateful for any ideas.. Thank you Jaroslav Urik <http://mapy2.natur.cuni.cz:8080/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.store.NewDataPage#> -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
