Hi, thanks for your replies. We have not employed the image pyramid plugin. The current troublesome source material is an orthophoto delivered as a collection of rgb tiffs with .tfw reference files, 8cm resolution, 1000x1000m per tiff. Each tiff is 500MB, total size 324GB.
The tiffs were prepped thusly: for %%i in (*.tif) do gdal_translate -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -co TILED=YES %%i <folder>%%~ni.tif which reduced the size in MB of many tiffs by rather a lot. Then it was published via the Image Mosaic plugin in Geoserver. Would I need to gdal_merge into one gigantic tiff first in order to run gdal_retile? Or would you advice preparing this source material in some other way? (Eventually we might run a standalone gwc as some things should only be employed cached but we still need to learn how to prep and publish raster properly in Geoserver in order for gwc to have something to feed on :) ) (Haven't found a bug report about getMap requests that ignore the Geoserver WMS timeout set time, yet) Respectfully, David. On Mon, Nov 11, 2019 at 1:37 PM Simone Giannecchini < simone.giannecch...@geo-solutions.it> wrote: > Dear David, > I would take a different path and understand more about this "large > rasters": > > - tell us more about format and size > - tell us more about how you preprocessed them for performance > > The first step with serving (large) raster data is choosing the right > format then preprocessing, see here (slide 8 and onward): > https://www.slideshare.net/geosolutions/geoserver-in-production-we-do-it-here-is-how-foss4g-2016 > <https://gnldr.online/tracker/click?redirect=https%3A%2F%2Fwww.slideshare.net%2Fgeosolutions%2Fgeoserver-in-production-we-do-it-here-is-how-foss4g-2016&dID=1573475791127&linkName=https://www.slideshare.net/geosolutions/geoserver-in-production-we-do-it-here-is-how-foss4g-2016> > Everything else comes (way) afterwards. > > Regards, > Simone Giannecchini > == > GeoServer Professional Services from the experts! > Visit http://bit.ly/gs-services > <https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-services&dID=1573475791127&linkName=http://bit.ly/gs-services> > for more information. > == > Ing. Simone Giannecchini > @simogeo > Founder/Director > > GeoSolutions S.A.S. > Via di Montramito 3/A > 55054 Massarosa (LU) > Italy > phone: +39 0584 962313 > fax: +39 0584 1660272 > mob: +39 333 8128928 > > http://www.geo-solutions.it > <https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fwww.geo-solutions.it&dID=1573475791127&linkName=http://www.geo-solutions.it> > http://twitter.com/geosolutions_it > <https://gnldr.online/tracker/click?redirect=http%3A%2F%2Ftwitter.com%2Fgeosolutions_it&dID=1573475791127&linkName=http://twitter.com/geosolutions_it> > > ------------------------------------------------------- > Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE > 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si > precisa che ogni circostanza inerente alla presente email (il suo > contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è > riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il > messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra > operazione è illecita. Le sarei comunque grato se potesse darmene notizia. > > This email is intended only for the person or entity to which it is > addressed and may contain information that is privileged, confidential or > otherwise protected from disclosure. We remind that - as provided by > European Regulation 2016/679 “GDPR” - copying, dissemination or use of this > e-mail or the information herein by anyone other than the intended > recipient is prohibited. If you have received this email by mistake, please > notify us immediately by telephone or e-mail. > > > On Sun, Nov 10, 2019 at 3:47 AM David I <persso...@gmail.com> wrote: > >> Hi, >> >> We're currently running 2.11.5 (upgrade of the whole server scheduled for >> early next year) and are experiencing stability issues with the Jetty 9.3 >> hosting Geoserver (and a Geoexplorer). For instance large ortho rasters >> previewed in Geoserver try to render for a long time and eventually fail >> and >> if several of those heavy getMaps happen in parallell Jetty becomes slow >> to >> respond and will either end up in state where it doesn't appear to >> recover, >> at least for 10 minutes, or it recovers. Lesser rasters preview ok and do >> not cause >> >> (This is a problem when using these WMS layers in desktop applications, >> it's >> not that lots of people preview them from Geoserver's /web) >> >> >> Latest Jetty 9.3, latest Corretto JRE 8. >> >> Java options: >> Marlin 0.9.4.1, >> -XX:+UseParallelGC >> -XX:SoftRefLRUPolicyMSPerMB=36000 >> >> 6144MB initial heap and 8192 max, thread stack size 256KB. >> >> When Jetty becomes partially or totally unresponsive the heap via jmapheap >> -heap looks like so: >> >> Heap Configuration: >> MinHeapFreeRatio = 0 >> MaxHeapFreeRatio = 100 >> MaxHeapSize = 8589934592 (8192.0MB) >> NewSize = 2147483648 (2048.0MB) >> MaxNewSize = 2863136768 (2730.5MB) >> OldSize = 4294967296 (4096.0MB) >> NewRatio = 2 >> SurvivorRatio = 8 >> MetaspaceSize = 21807104 (20.796875MB) >> CompressedClassSpaceSize = 1073741824 (1024.0MB) >> MaxMetaspaceSize = 17592186044415 MB >> G1HeapRegionSize = 0 (0.0MB) >> >> Heap Usage: >> PS Young Generation >> Eden Space: >> capacity = 1982332928 (1890.5MB) >> used = 1982332928 (1890.5MB) >> free = 0 (0.0MB) >> 100.0% used >> From Space: >> capacity = 437256192 (417.0MB) >> used = 0 (0.0MB) >> free = 437256192 (417.0MB) >> 0.0% used >> To Space: >> capacity = 439353344 (419.0MB) >> used = 0 (0.0MB) >> free = 439353344 (419.0MB) >> 0.0% used >> PS Old Generation >> capacity = 5726797824 (5461.5MB) >> used = 5726435656 (5461.154609680176MB) >> free = 362168 (0.34539031982421875MB) >> 99.99367590735469% used >> >> In addition Geoserver doesn't appear to adhere to the max rendering time >> under WMS settings: 35s, this is what the Monitor plugin captured shortly >> before Jetty stopped responding earlier today: >> >> >> /<Request id="3,417,511"> >> <Operation>GetMap</Operation> >> <Resources>ortofoto:orto_tatort2018</Resources> >> <Path>/ows</Path> >> >> >> <QueryString>SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&LAYERS=ortofoto:orto_tatort2018&STYLES=&CRS=EPSG:3010&FORMAT=image/jpeg&TRANSPARENT=TRUE&WIDTH=502&HEIGHT=502&BBOX=6575000,141000,6579016,145016</QueryString> >> <Body> >> >> </Body> >> <StartTime>2019-11-07T07:06:21.469Z</StartTime> >> <EndTime>2019-11-07T07:13:26.694Z</EndTime> >> <TotalTime>425,225</TotalTime> >> <ResponseStatus>200</ResponseStatus> >> <Failed>false</Failed> >> </Request> / >> >> Geoserver's (production9 or jetty (dev) log do not mention >> java.lang.outofmemoryerror, but Geoserver will sometimes reply with just >> that : heap space, when in its unresponsive state and a wfs getFeature is >> sent. (logging typically stops when the servlet becomes unresponsive). >> >> Would a different garbage collector be of use? >> Setting MaxPermSize? >> Somehow optimizing the tifs of the mosaic stores? >> Is there a way to not allow these WMS services to be used except with >> tiled=true -> their cached variants? >> >> >> >> >> >> -- >> Sent from: >> http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html >> >> >> _______________________________________________ >> Geoserver-users mailing list >> >> Please make sure you read the following two resources before posting to >> this list: >> - Earning your support instead of buying it, but Ian Turton: >> http://www.ianturton.com/talks/foss4g.html#/ >> - The GeoServer user list posting guidelines: >> http://geoserver.org/comm/userlist-guidelines.html >> >> If you want to request a feature or an improvement, also see this: >> https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer >> >> >> Geoserver-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/geoserver-users >> >
_______________________________________________ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
