Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[Mats Elfström]

Hi Jukka!
Please see this thread, where reprocessing of all files was suggested.
https://sourceforge.net/p/geoserver/mailman/message/36782809/
Like I replied, I did not fancy that approach since it would duplicate an
already large number of files, and still pose the problem of mosaicking. I
contemplated merging the files four by four but ended up with the jpeg2000
solution. Adding an alpha band made the irregular pattern of tiles
transparent as well.

But there is one thing I cannot figure out: Is there any point in making a
Geowebcache for an ecw or jp2k file? My view is that these formats are kind
of tiled internally (pardon my lack of knowledge of the exact terms) and
that a geowebcache would be superfluous. My tests do not indicate any
performance gain by tiling. Only thing that happens is that the tiles
appear one by one, in roughly the same time it takes to wait for the
non-tiled service to fill the canvas in one request.

Regards, Mats.E


*GisKraft*, Geodatakonsult

Mats Elfström, Marsvinsholms stationsväg 80, 271 93 Ystad, Sweden
tel: +46 70 595 39 35 / web: www.giskraft.com


Den mån 25 nov. 2019 kl 14:26 skrev Rahkonen Jukka (MML) <
jukka.rahko...@maanmittauslaitos.fi>:

> Hi,
>
>
>
> The advice that we try to share on this list
>
>- tile and compress GeoTIFFs with compression that suits your data
>(deflate, jpeg)
>- don’t make image mosaic to open more than 20-30 images
>- for small scale situation (zoomed out) use image pyramid or create a
>scale dependent layer group
>
>
>
> If you have triad image mosaic and performance was horrible you most
> probably did not take care of the points above. Please give a link where
> you were asked to degrade the tiffs and compress them further. Advice
> feels odd but perhaps we can improve it even it may be too late for you now.
>
>
>
> Your solution of having one JPEG2000 file per year if a good as well. The
> mosaic is a bit heavy to create but once it is ready it will work fine. It
> is also possible to mosaic all images into one tiled, jpeg compressed
> BigTIFF which gives about the same speed, quality, and compression rate
> than ecw or JPEG2000 with moderate lossy compression. That applies to full
> resolution imagery, need to have separate overviews pay 30% extra for
> BigTIFF. When I do that I use naturally GDAL for the whole chain but there
> are indeed other alternatives.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Mats Elfström 
> *Lähetetty:* maanantai 25. marraskuuta 2019 9.52
> *Vastaanottaja:* David I 
> *Kopio:* geoserver-users@lists.sourceforge.net
> *Aihe:* Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet
> instability, JVM settings?
>
>
>
> Hi!
>
> I recently faced a similar task.
>
> Hundreds of high resolution geotiffs over a limited area that I wanted to
> serve on Geoserver. This is 2.16 but I do not think that matters. I also
> have several image sets from different years. In all, this amounts to
> several tens of gigabytes.
>
> I tried to use image mosaic, but performance was horrible. A suggestion
> from this list was to degrade the tiffs and compress them further. This
> would effectively duplicate an already large file storage and I hesitated
> to do that. Also, I do not want the Geoserver machine to double as a file
> storage.
>
> I solved the task by mosaicking the yearly file sets into jpeg200, one
> file per year. These I then serve, one store and layer per year. No need
> for tiling either. That is taken care of in the file format.
>
> Normally I use ecw for purposes like these, but there is a licensing issue
> with ecw in a server environment. JPEG2000 is true open source, and is
> almost as efficient as ecw. Files are slightly larger.
>
> Mosaicking, projection and file conversion was of course done with FME.
>
> *GisKraft*, Geodatakonsult
>
> Mats Elfström, Marsvinsholms stationsväg 80, 271 93 Ystad, Sweden
> tel: +46 70 595 39 35 / web: www.giskraft.com
>
>
>
>
>
> Den sön 24 nov. 2019 kl 16:14 skrev David I :
>
> Hello Simone,
>
> thanks for the concrete piece of advice (more than 30-40 files then
> imagemosaic might be unsuitable, find some other method, like using the
> pyramids plugin). In regards to whether one might feed our collection of
> 500MB tiffs (tiled for one scale, that is) to gdal_retile or if to try to
> produce something that isn't fantastically big with gdal_merge and then
> give
> that to gdal_retile or gdalbuildvrt on them and then gdal_translate to jpeg
> compress and produce and one large (but not super large) tif and feed that
> to gdal_retile - I shall tinker (the Geoserver docs give one big as the
> input in the example but the tool supposedly can handle multiple, in

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[David I]

Hi Andrea, 

It is a bit unclear whether overlays were produced for the currently
published raster but yes, trying to add them now to a bigtiff.

Thanks for clarifying what Simone wrote, I read it too fast.. anyways, it
would make sense to hide the ortho at smaller resolutions as it is of most
use in larger scales anyway, however currently our web map client has it as
a background layer and it would appear illogical for only white to appear
when switching to it at a smaller scale. Am currently laboratoring with
jpeg2k in fme and overlays with gdal, for imagemosaic or possibly
gdal_retile.py for imagepyramid. 

Thanks all for an interesting discussion.




geowolf wrote
> On Sun, Nov 24, 2019 at 4:23 PM David I 

> perssond9@

>  wrote:
> 
>> Hello Simone,
>>
>> thanks for the concrete piece of advice (more than 30-40 files then
>> imagemosaic might be unsuitable, find some other method, like using the
>> pyramids plugin).
> 
> 
> To clarify, one can have a mosaic with millions of images, as long as they
> are not opened
> all together. The file limit above is intended for the single GetMap or
> GetTile request,
> so as long as you set a scale denominator and have the mosaic image index
> with a spatial index
> on the bbox of the images, there is no trouble: just set a scale
> denominator limit in the style so that
> the images start showing up only when you are fairly zoomed in.
> 
> Also, a note, I don't see you adding overviews to the files, that's also
> going to limit the performance
> of the mosaic significantly.
> 
> Cheers
> Andrea
> 
> ==
> 
> GeoServer Professional Services from the experts! Visit
> http://goo.gl/it488V
> for more information. == Ing. Andrea Aime @geowolf Technical Lead
> GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39
> 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549
> http://www.geo-solutions.it http://twitter.com/geosolutions_it
> --- *Con riferimento
> alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 -
> Regolamento generale sulla protezione dei dati “GDPR”), si precisa che
> ogni
> circostanza inerente alla presente email (il suo contenuto, gli eventuali
> allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i
> destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per
> errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
> sarei comunque grato se potesse darmene notizia. This email is intended
> only for the person or entity to which it is addressed and may contain
> information that is privileged, confidential or otherwise protected from
> disclosure. We remind that - as provided by European Regulation 2016/679
> “GDPR” - copying, dissemination or use of this e-mail or the information
> herein by anyone other than the intended recipient is prohibited. If you
> have received this email by mistake, please notify us immediately by
> telephone or e-mail.*
> 
> 
> ___
> Geoserver-users mailing list
> 
> Please make sure you read the following two resources before posting to
> this list:
> - Earning your support instead of buying it, but Ian Turton:
> http://www.ianturton.com/talks/foss4g.html#/
> - The GeoServer user list posting guidelines:
> http://geoserver.org/comm/userlist-guidelines.html
> 
> If you want to request a feature or an improvement, also see this:
> https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
> 

> Geoserver-users@.sourceforge

> https://lists.sourceforge.net/lists/listinfo/geoserver-users





--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html


___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[David I]

Hello again Simone, do I understand correctly in that with a jpeg-compressed
bigtiff with overviews one might get away with using the ImageMosaic plugin?

I certainly have some testing to do (am currently adding overviews to the
~15GB bigtiff that was the result of gdal_translating the ~750 1km tiles
together and compressing them, it will take a bit of time; will try and use
the imagemosaic plugin with it and if not very successful then
gdal_retile.py for pyramid creation, sans compressing anew.

Rgds
David


Simone Giannecchini wrote
> Dear David,
> please, read below...
> 
> Regards,
> Simone Giannecchini
> ==
> GeoServer Professional Services from the experts!
> Visit http://bit.ly/gs-services
> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesdID=1574690204587linkName=http://bit.ly/gs-services;
> for more information.
> ==
> Ing. Simone Giannecchini
> @simogeo
> Founder/Director
> 
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob:   +39  333 8128928
> 
> http://www.geo-solutions.it
> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fwww.geo-solutions.itdID=1574690204587linkName=http://www.geo-solutions.it;
> http://twitter.com/geosolutions_it
> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Ftwitter.com%2Fgeosolutions_itdID=1574690204587linkName=http://twitter.com/geosolutions_it;
> 
> ---
> Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
> 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
> precisa che ogni circostanza inerente alla presente email (il suo
> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
> operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
> 
> This email is intended only for the person or entity to which it is
> addressed and may contain information that is privileged, confidential or
> otherwise protected from disclosure. We remind that - as provided by
> European Regulation 2016/679 “GDPR” - copying, dissemination or use of
> this
> e-mail or the information herein by anyone other than the intended
> recipient is prohibited. If you have received this email by mistake,
> please
> notify us immediately by telephone or e-mail.
> 
> 
> On Sun, Nov 24, 2019 at 4:18 PM David I 

> perssond9@

>  wrote:
> 
>> Hello Simone,
>>
>> thanks for the concrete piece of advice (more than 30-40 files then
>> imagemosaic might be unsuitable, find some other method, like using the
>> pyramids plugin). In regards to whether one might feed our collection of
>> 500MB tiffs (tiled for one scale, that is) to gdal_retile or if to try to
>> produce something that isn't fantastically big with gdal_merge and then
>> give
>> that to gdal_retile or gdalbuildvrt on them and then gdal_translate to
>> jpeg
>> compress and produce and one large (but not super large) tif and feed
>> that
>> to gdal_retile - I shall tinker (the Geoserver docs give one big as the
>> input in the example but the tool supposedly can handle multiple,
>> including
>> what's already tiled) (though if anyone has strong recommendations either
>> way then don't be afraid to holler :) )
>>
> 
> It is difficult to give you a concrete way forward as it need some
> testing.
> Generally speaking you can create a mosaic of a few large bigtiff jpeg
> compressed with overviews; tiles should be at least 512x512; overall size
> for a single file can be up to 50 to 60 gb.
> This means you can have a single mosaic of 1TB (more or less) of imagery
> which works decently.
> Notice that when you compress jpeg, even visually lossless, you still
> compress a lot so we are talking about _a lot_ of data (potentially the
> size of a country like Italy).
> 
> 
> 
>>
>> Won't open a ticket for the timeout as we are currently on an older
>> version
>> of Geoserver. An update is being planned but this tif-sink thing required
>> more immediate attention.
>>
>> Thanks again,
>>
>> Regards
>> David.
>> ==
>> GeoServer Professional Services from the experts!
>> Visit http://bit.ly/gs-services
>> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesdID=1574690204587linkName=http://bit.ly/gs-services;
>> 
>> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesamp;dID=1574517951672amp;linkName=http://bit.ly/gs-servicesgt;
>> https://gnldr.online/track
>> https://gnldr.online/tracker/click?redirect=https%3A%2F%2Fgnldr.online%2FtrackdID=1574690204587linkName=https://gnldr.online/track;
>> er/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesdID=1574517951672linkName=
>> http://bit.ly/gs-services;
>> for more information.
>> ==
>> Ing. Simone Giannecchini
>> @simogeo
>> Founder/Director
>>
>> GeoSolutions S.A.S.
>> 

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[David I]

Hi there, thanks for your reply!
Just so that I understand you correctly, did you after creating one jpeg2k
file per store set them up as imagemosaic stores in Geoserver? As
ImageMosaic can be used with jpeg2k files after a GDAL extension is
installed for Geoserver and I didn't see you explicitly mentioning another
type of data store and the description for the ImageMosaic type is that it
mosaics 'two or more' images. Anyways, have currently ordered FME to 322GB
(~750) tifs -> 1 jpeg2k, will be interesting to see what it produces : ) 

Regards,
David



matself wrote
> Hi!
> I recently faced a similar task.
> Hundreds of high resolution geotiffs over a limited area that I wanted to
> serve on Geoserver. This is 2.16 but I do not think that matters. I also
> have several image sets from different years. In all, this amounts to
> several tens of gigabytes.
> I tried to use image mosaic, but performance was horrible. A suggestion
> from this list was to degrade the tiffs and compress them further. This
> would effectively duplicate an already large file storage and I hesitated
> to do that. Also, I do not want the Geoserver machine to double as a file
> storage.
> I solved the task by mosaicking the yearly file sets into jpeg200, one
> file
> per year. These I then serve, one store and layer per year. No need for
> tiling either. That is taken care of in the file format.
> Normally I use ecw for purposes like these, but there is a licensing issue
> with ecw in a server environment. JPEG2000 is true open source, and is
> almost as efficient as ecw. Files are slightly larger.
> Mosaicking, projection and file conversion was of course done with FME.
> 
> 
> *GisKraft*, Geodatakonsult
> 
> Mats Elfström, Marsvinsholms stationsväg 80, 271 93 Ystad, Sweden
> tel: +46 70 595 39 35 / web: www.giskraft.com
> 
> 
> Den sön 24 nov. 2019 kl 16:14 skrev David I 

> perssond9@

> :
> 
>> Hello Simone,
>>
>> thanks for the concrete piece of advice (more than 30-40 files then
>> imagemosaic might be unsuitable, find some other method, like using the
>> pyramids plugin). In regards to whether one might feed our collection of
>> 500MB tiffs (tiled for one scale, that is) to gdal_retile or if to try to
>> produce something that isn't fantastically big with gdal_merge and then
>> give
>> that to gdal_retile or gdalbuildvrt on them and then gdal_translate to
>> jpeg
>> compress and produce and one large (but not super large) tif and feed
>> that
>> to gdal_retile - I shall tinker (the Geoserver docs give one big as the
>> input in the example but the tool supposedly can handle multiple,
>> including
>> what's already tiled) (though if anyone has strong recommendations either
>> way then don't be afraid to holler :) )
>>
>> Won't open a ticket for the timeout as we are currently on an older
>> version
>> of Geoserver. An update is being planned but this tif-sink thing required
>> more immediate attention.
>>
>> Thanks again,
>>
>> Regards
>> David.
>> ==
>> GeoServer Professional Services from the experts!
>> Visit http://bit.ly/gs-services
>> 
>> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesdID=1574517951672linkName=http://bit.ly/gs-services
>> ;
>> for more information.
>> ==
>> Ing. Simone Giannecchini
>> @simogeo
>> Founder/Director
>>
>> GeoSolutions S.A.S.
>> Via di Montramito 3/A
>> 55054  Massarosa (LU)
>> Italy
>> phone: +39 0584 962313
>> fax: +39 0584 1660272
>> mob:   +39  333 8128928
>>
>> http://www.geo-solutions.it
>> 
>> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fwww.geo-solutions.itdID=1574517951672linkName=http://www.geo-solutions.it
>> ;
>> http://twitter.com/geosolutions_it
>> 
>> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Ftwitter.com%2Fgeosolutions_itdID=1574517951672linkName=http://twitter.com/geosolutions_it
>> ;
>>
>> ---
>> Con riferimento alla normativa sul trattamento dei dati personali (Reg.
>> UE
>> 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
>> precisa che ogni circostanza inerente alla presente email (il suo
>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
>> operazione è illecita. Le sarei comunque grato se potesse darmene
>> notizia.
>>
>> This email is intended only for the person or entity to which it is
>> addressed and may contain information that is privileged, confidential or
>> otherwise protected from disclosure. We remind that - as provided by
>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of
>> this
>> e-mail or the information herein by anyone other than the intended
>> recipient is prohibited. If you have received this email by mistake,
>> please
>> notify us immediately by telephone or e-mail.
>>
>>
>> On Sat, Nov 23, 2019 at 2:26 PM David Persson 

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[Simone Giannecchini]

Dear David,
please, read below...

Regards,
Simone Giannecchini
==
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services

for more information.
==
Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob:   +39  333 8128928

http://www.geo-solutions.it

http://twitter.com/geosolutions_it


---
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
precisa che ogni circostanza inerente alla presente email (il suo
contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential or
otherwise protected from disclosure. We remind that - as provided by
European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
e-mail or the information herein by anyone other than the intended
recipient is prohibited. If you have received this email by mistake, please
notify us immediately by telephone or e-mail.


On Sun, Nov 24, 2019 at 4:18 PM David I  wrote:

> Hello Simone,
>
> thanks for the concrete piece of advice (more than 30-40 files then
> imagemosaic might be unsuitable, find some other method, like using the
> pyramids plugin). In regards to whether one might feed our collection of
> 500MB tiffs (tiled for one scale, that is) to gdal_retile or if to try to
> produce something that isn't fantastically big with gdal_merge and then
> give
> that to gdal_retile or gdalbuildvrt on them and then gdal_translate to jpeg
> compress and produce and one large (but not super large) tif and feed that
> to gdal_retile - I shall tinker (the Geoserver docs give one big as the
> input in the example but the tool supposedly can handle multiple, including
> what's already tiled) (though if anyone has strong recommendations either
> way then don't be afraid to holler :) )
>

It is difficult to give you a concrete way forward as it need some testing.
Generally speaking you can create a mosaic of a few large bigtiff jpeg
compressed with overviews; tiles should be at least 512x512; overall size
for a single file can be up to 50 to 60 gb.
This means you can have a single mosaic of 1TB (more or less) of imagery
which works decently.
Notice that when you compress jpeg, even visually lossless, you still
compress a lot so we are talking about _a lot_ of data (potentially the
size of a country like Italy).



>
> Won't open a ticket for the timeout as we are currently on an older version
> of Geoserver. An update is being planned but this tif-sink thing required
> more immediate attention.
>
> Thanks again,
>
> Regards
> David.
> ==
> GeoServer Professional Services from the experts!
> Visit http://bit.ly/gs-services
> 
> 
> 
> https://gnldr.online/track
> 
> er/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesdID=1574517951672linkName=
> http://bit.ly/gs-services;
> for more information.
> ==
> Ing. Simone Giannecchini
> @simogeo
> Founder/Director
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob:   +39  333 8128928
>
> http://www.geo-solutions.it
> 
> 
> 
> https://gnldr.online/track
> 
> er/click?redirect=http%3A%2F%2Fwww.geo-solutions.it
> dID=1574517951672linkName=http://www.geo-solutions.it;
> 

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[Rahkonen Jukka (MML)]

Hi,

The advice that we try to share on this list

  *   tile and compress GeoTIFFs with compression that suits your data 
(deflate, jpeg)
  *   don’t make image mosaic to open more than 20-30 images
  *   for small scale situation (zoomed out) use image pyramid or create a 
scale dependent layer group

If you have triad image mosaic and performance was horrible you most probably 
did not take care of the points above. Please give a link where you were asked 
to degrade the tiffs and compress them further. Advice feels odd but perhaps we 
can improve it even it may be too late for you now.

Your solution of having one JPEG2000 file per year if a good as well. The 
mosaic is a bit heavy to create but once it is ready it will work fine. It is 
also possible to mosaic all images into one tiled, jpeg compressed BigTIFF 
which gives about the same speed, quality, and compression rate than ecw or 
JPEG2000 with moderate lossy compression. That applies to full resolution 
imagery, need to have separate overviews pay 30% extra for BigTIFF. When I do 
that I use naturally GDAL for the whole chain but there are indeed other 
alternatives.

-Jukka Rahkonen-

Lähettäjä: Mats Elfström 
Lähetetty: maanantai 25. marraskuuta 2019 9.52
Vastaanottaja: David I 
Kopio: geoserver-users@lists.sourceforge.net
Aihe: Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet 
instability, JVM settings?

Hi!
I recently faced a similar task.
Hundreds of high resolution geotiffs over a limited area that I wanted to serve 
on Geoserver. This is 2.16 but I do not think that matters. I also have several 
image sets from different years. In all, this amounts to several tens of 
gigabytes.
I tried to use image mosaic, but performance was horrible. A suggestion from 
this list was to degrade the tiffs and compress them further. This would 
effectively duplicate an already large file storage and I hesitated to do that. 
Also, I do not want the Geoserver machine to double as a file storage.
I solved the task by mosaicking the yearly file sets into jpeg200, one file per 
year. These I then serve, one store and layer per year. No need for tiling 
either. That is taken care of in the file format.
Normally I use ecw for purposes like these, but there is a licensing issue with 
ecw in a server environment. JPEG2000 is true open source, and is almost as 
efficient as ecw. Files are slightly larger.
Mosaicking, projection and file conversion was of course done with FME.

[https://docs.google.com/uc?export=download=0B6X-OqqLVTjPM2pwU05zTXNZWkU=0B6X-OqqLVTjPRzhtZUZyMDg3eFBoejRubGlkaGx3bVhNTHJrPQ]

GisKraft, Geodatakonsult

Mats Elfström, Marsvinsholms stationsväg 80, 271 93 Ystad, Sweden
tel: +46 70 595 39 35 / web: www.giskraft.com<http://www.giskraft.com>


Den sön 24 nov. 2019 kl 16:14 skrev David I 
mailto:persso...@gmail.com>>:
Hello Simone,

thanks for the concrete piece of advice (more than 30-40 files then
imagemosaic might be unsuitable, find some other method, like using the
pyramids plugin). In regards to whether one might feed our collection of
500MB tiffs (tiled for one scale, that is) to gdal_retile or if to try to
produce something that isn't fantastically big with gdal_merge and then give
that to gdal_retile or gdalbuildvrt on them and then gdal_translate to jpeg
compress and produce and one large (but not super large) tif and feed that
to gdal_retile - I shall tinker (the Geoserver docs give one big as the
input in the example but the tool supposedly can handle multiple, including
what's already tiled) (though if anyone has strong recommendations either
way then don't be afraid to holler :) )

Won't open a ticket for the timeout as we are currently on an older version
of Geoserver. An update is being planned but this tif-sink thing required
more immediate attention.

Thanks again,

Regards
David.
==
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services
https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesdID=1574517951672linkName=http://bit.ly/gs-services;
for more information.
==
Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob:   +39  333 8128928

http://www.geo-solutions.it
https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fwww.geo-solutions.itdID=1574517951672linkName=http://www.geo-solutions.it;
http://twitter.com/geosolutions_it
https://gnldr.online/tracker/click?redirect=http%3A%2F%2Ftwitter.com%2Fgeosolutions_itdID=1574517951672linkName=http://twitter.com/geosolutions_it;

---
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
precisa che ogni circostanza inerente alla presente email (il suo
contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
riservata al/i solo

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[Andrea Aime]

On Sun, Nov 24, 2019 at 4:23 PM David I  wrote:

> Hello Simone,
>
> thanks for the concrete piece of advice (more than 30-40 files then
> imagemosaic might be unsuitable, find some other method, like using the
> pyramids plugin).


To clarify, one can have a mosaic with millions of images, as long as they
are not opened
all together. The file limit above is intended for the single GetMap or
GetTile request,
so as long as you set a scale denominator and have the mosaic image index
with a spatial index
on the bbox of the images, there is no trouble: just set a scale
denominator limit in the style so that
the images start showing up only when you are fairly zoomed in.

Also, a note, I don't see you adding overviews to the files, that's also
going to limit the performance
of the mosaic significantly.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information. == Ing. Andrea Aime @geowolf Technical Lead
GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39
0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549
http://www.geo-solutions.it http://twitter.com/geosolutions_it
--- *Con riferimento
alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 -
Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni
circostanza inerente alla presente email (il suo contenuto, gli eventuali
allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i
destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per
errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
sarei comunque grato se potesse darmene notizia. This email is intended
only for the person or entity to which it is addressed and may contain
information that is privileged, confidential or otherwise protected from
disclosure. We remind that - as provided by European Regulation 2016/679
“GDPR” - copying, dissemination or use of this e-mail or the information
herein by anyone other than the intended recipient is prohibited. If you
have received this email by mistake, please notify us immediately by
telephone or e-mail.*
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[Mats Elfström]

Hi!
I recently faced a similar task.
Hundreds of high resolution geotiffs over a limited area that I wanted to
serve on Geoserver. This is 2.16 but I do not think that matters. I also
have several image sets from different years. In all, this amounts to
several tens of gigabytes.
I tried to use image mosaic, but performance was horrible. A suggestion
from this list was to degrade the tiffs and compress them further. This
would effectively duplicate an already large file storage and I hesitated
to do that. Also, I do not want the Geoserver machine to double as a file
storage.
I solved the task by mosaicking the yearly file sets into jpeg200, one file
per year. These I then serve, one store and layer per year. No need for
tiling either. That is taken care of in the file format.
Normally I use ecw for purposes like these, but there is a licensing issue
with ecw in a server environment. JPEG2000 is true open source, and is
almost as efficient as ecw. Files are slightly larger.
Mosaicking, projection and file conversion was of course done with FME.


*GisKraft*, Geodatakonsult

Mats Elfström, Marsvinsholms stationsväg 80, 271 93 Ystad, Sweden
tel: +46 70 595 39 35 / web: www.giskraft.com


Den sön 24 nov. 2019 kl 16:14 skrev David I :

> Hello Simone,
>
> thanks for the concrete piece of advice (more than 30-40 files then
> imagemosaic might be unsuitable, find some other method, like using the
> pyramids plugin). In regards to whether one might feed our collection of
> 500MB tiffs (tiled for one scale, that is) to gdal_retile or if to try to
> produce something that isn't fantastically big with gdal_merge and then
> give
> that to gdal_retile or gdalbuildvrt on them and then gdal_translate to jpeg
> compress and produce and one large (but not super large) tif and feed that
> to gdal_retile - I shall tinker (the Geoserver docs give one big as the
> input in the example but the tool supposedly can handle multiple, including
> what's already tiled) (though if anyone has strong recommendations either
> way then don't be afraid to holler :) )
>
> Won't open a ticket for the timeout as we are currently on an older version
> of Geoserver. An update is being planned but this tif-sink thing required
> more immediate attention.
>
> Thanks again,
>
> Regards
> David.
> ==
> GeoServer Professional Services from the experts!
> Visit http://bit.ly/gs-services
> 
> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesdID=1574517951672linkName=http://bit.ly/gs-services
> ;
> for more information.
> ==
> Ing. Simone Giannecchini
> @simogeo
> Founder/Director
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob:   +39  333 8128928
>
> http://www.geo-solutions.it
> 
> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fwww.geo-solutions.itdID=1574517951672linkName=http://www.geo-solutions.it
> ;
> http://twitter.com/geosolutions_it
> 
> https://gnldr.online/tracker/click?redirect=http%3A%2F%2Ftwitter.com%2Fgeosolutions_itdID=1574517951672linkName=http://twitter.com/geosolutions_it
> ;
>
> ---
> Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
> 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
> precisa che ogni circostanza inerente alla presente email (il suo
> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
> operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
>
> This email is intended only for the person or entity to which it is
> addressed and may contain information that is privileged, confidential or
> otherwise protected from disclosure. We remind that - as provided by
> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
> e-mail or the information herein by anyone other than the intended
> recipient is prohibited. If you have received this email by mistake, please
> notify us immediately by telephone or e-mail.
>
>
> On Sat, Nov 23, 2019 at 2:26 PM David Persson perssond9@ wrote:
>
> > Hi, thanks for your replies.
> >
> > We have not employed the image pyramid plugin. The current troublesome
> > source material is an orthophoto delivered as a collection of rgb tiffs
> > with .tfw reference files, 8cm resolution, 1000x1000m per tiff. Each tiff
> > is 500MB, total size 324GB.
> >
> > The tiffs were prepped thusly: for %%i in (*.tif) do gdal_translate -co
> > COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -co TILED=YES %%i %%~ni.tif
> > which reduced the size in MB of many tiffs by rather a lot. Then it was
> > published via the Image Mosaic plugin in Geoserver.
> > Would I need to gdal_merge into one gigantic tiff first in order to run
> > gdal_retile? Or would you advice preparing this source material in some
> 

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[David I]

Hello Simone,

thanks for the concrete piece of advice (more than 30-40 files then
imagemosaic might be unsuitable, find some other method, like using the
pyramids plugin). In regards to whether one might feed our collection of
500MB tiffs (tiled for one scale, that is) to gdal_retile or if to try to
produce something that isn't fantastically big with gdal_merge and then give
that to gdal_retile or gdalbuildvrt on them and then gdal_translate to jpeg
compress and produce and one large (but not super large) tif and feed that
to gdal_retile - I shall tinker (the Geoserver docs give one big as the
input in the example but the tool supposedly can handle multiple, including
what's already tiled) (though if anyone has strong recommendations either
way then don't be afraid to holler :) )

Won't open a ticket for the timeout as we are currently on an older version
of Geoserver. An update is being planned but this tif-sink thing required
more immediate attention.

Thanks again,

Regards
David.
==
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services
https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fbit.ly%2Fgs-servicesdID=1574517951672linkName=http://bit.ly/gs-services;
for more information.
==
Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob:   +39  333 8128928

http://www.geo-solutions.it
https://gnldr.online/tracker/click?redirect=http%3A%2F%2Fwww.geo-solutions.itdID=1574517951672linkName=http://www.geo-solutions.it;
http://twitter.com/geosolutions_it
https://gnldr.online/tracker/click?redirect=http%3A%2F%2Ftwitter.com%2Fgeosolutions_itdID=1574517951672linkName=http://twitter.com/geosolutions_it;

---
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
precisa che ogni circostanza inerente alla presente email (il suo
contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential or
otherwise protected from disclosure. We remind that - as provided by
European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
e-mail or the information herein by anyone other than the intended
recipient is prohibited. If you have received this email by mistake, please
notify us immediately by telephone or e-mail.


On Sat, Nov 23, 2019 at 2:26 PM David Persson perssond9@ wrote:

> Hi, thanks for your replies.
>
> We have not employed the image pyramid plugin. The current troublesome
> source material is an orthophoto delivered as a collection of rgb tiffs
> with .tfw reference files, 8cm resolution, 1000x1000m per tiff. Each tiff
> is 500MB, total size 324GB.
>
> The tiffs were prepped thusly: for %%i in (*.tif) do gdal_translate -co
> COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -co TILED=YES %%i %%~ni.tif
> which reduced the size in MB of many tiffs by rather a lot. Then it was
> published via the Image Mosaic plugin in Geoserver.
> Would I need to gdal_merge into one gigantic tiff first in order to run
> gdal_retile? Or would you advice preparing this source material in some
> other way?
>

The key point for the imagemosaic is the number of files it has to open to
answer a request.
If you mosaic is comprised of more than 30 to 40 files as you zoom out
GeoServer will suffer a lot to answer, up to a point where it will probably
die for one reason or the other.

So we usually use pyramids or layergroup with different layers plus styling
rules based on the scale to exclude some layers.


>
> (Eventually we might run a standalone gwc as some things should only be
> employed cached but we still need to learn how to prep and publish raster
> properly in Geoserver in order for gwc to have something to feed on :) )
>
> (Haven't found a bug report about getMap requests that ignore the
> Geoserver WMS timeout set time, yet)
>

You can open that, this is true only for raster data atm.


>
> Respectfully,
> David.
>
> On Mon, Nov 11, 2019 at 1:37 PM Simone Giannecchini <
> simone.giannecchini@> wrote:
>
>> Dear David,
>> I would take a different path and understand more about this "large
>> rasters":
>>
>> - tell us more about format and size
>> - tell us more about how you preprocessed them for performance
>>
>> The first step with serving (large) raster data is choosing the right
>> format then preprocessing, see here (slide 8 and onward):
>> 

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[Simone Giannecchini]

Dear David,
please, read below...

Regards,
Simone Giannecchini
==
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services

for more information.
==
Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob:   +39  333 8128928

http://www.geo-solutions.it

http://twitter.com/geosolutions_it


---
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
precisa che ogni circostanza inerente alla presente email (il suo
contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential or
otherwise protected from disclosure. We remind that - as provided by
European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
e-mail or the information herein by anyone other than the intended
recipient is prohibited. If you have received this email by mistake, please
notify us immediately by telephone or e-mail.


On Sat, Nov 23, 2019 at 2:26 PM David Persson  wrote:

> Hi, thanks for your replies.
>
> We have not employed the image pyramid plugin. The current troublesome
> source material is an orthophoto delivered as a collection of rgb tiffs
> with .tfw reference files, 8cm resolution, 1000x1000m per tiff. Each tiff
> is 500MB, total size 324GB.
>
> The tiffs were prepped thusly: for %%i in (*.tif) do gdal_translate -co
> COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -co TILED=YES %%i %%~ni.tif
> which reduced the size in MB of many tiffs by rather a lot. Then it was
> published via the Image Mosaic plugin in Geoserver.
> Would I need to gdal_merge into one gigantic tiff first in order to run
> gdal_retile? Or would you advice preparing this source material in some
> other way?
>

The key point for the imagemosaic is the number of files it has to open to
answer a request.
If you mosaic is comprised of more than 30 to 40 files as you zoom out
GeoServer will suffer a lot to answer, up to a point where it will probably
die for one reason or the other.

So we usually use pyramids or layergroup with different layers plus styling
rules based on the scale to exclude some layers.


>
> (Eventually we might run a standalone gwc as some things should only be
> employed cached but we still need to learn how to prep and publish raster
> properly in Geoserver in order for gwc to have something to feed on :) )
>
> (Haven't found a bug report about getMap requests that ignore the
> Geoserver WMS timeout set time, yet)
>

You can open that, this is true only for raster data atm.


>
> Respectfully,
> David.
>
> On Mon, Nov 11, 2019 at 1:37 PM Simone Giannecchini <
> simone.giannecch...@geo-solutions.it> wrote:
>
>> Dear David,
>> I would take a different path and understand more about this "large
>> rasters":
>>
>> - tell us more about format and size
>> - tell us more about how you preprocessed them for performance
>>
>> The first step with serving (large) raster data is choosing the right
>> format then preprocessing, see here (slide 8 and onward):
>> 
>> https://www
>> 
>> .slideshare.net/geosolutions/
>> 
>> geoserver-in-production-we-do-it-here-is-how-foss4g-2016
>> Everything else comes (way) afterwards.
>>
>> Regards,
>> Simone Giannecchini
>> ==
>> GeoServer Professional Services from the experts!
>> Visit http://bit.ly/gs-services
>> 
>> for more information.
>> ==
>> Ing. Simone Giannecchini
>> @simogeo
>> Founder/Director
>>
>> GeoSolutions S.A.S.
>> Via di Montramito 3/A
>> 55054  Massarosa (LU)
>> Italy
>> phone: +39 0584 

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[David Persson]

Hi, thanks for your replies.

We have not employed the image pyramid plugin. The current troublesome
source material is an orthophoto delivered as a collection of rgb tiffs
with .tfw reference files, 8cm resolution, 1000x1000m per tiff. Each tiff
is 500MB, total size 324GB.

The tiffs were prepped thusly: for %%i in (*.tif) do gdal_translate -co
COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -co TILED=YES %%i %%~ni.tif
which reduced the size in MB of many tiffs by rather a lot. Then it was
published via the Image Mosaic plugin in Geoserver.
Would I need to gdal_merge into one gigantic tiff first in order to run
gdal_retile? Or would you advice preparing this source material in some
other way?

(Eventually we might run a standalone gwc as some things should only be
employed cached but we still need to learn how to prep and publish raster
properly in Geoserver in order for gwc to have something to feed on :) )

(Haven't found a bug report about getMap requests that ignore the Geoserver
WMS timeout set time, yet)

Respectfully,
David.

On Mon, Nov 11, 2019 at 1:37 PM Simone Giannecchini <
simone.giannecch...@geo-solutions.it> wrote:

> Dear David,
> I would take a different path and understand more about this "large
> rasters":
>
> - tell us more about format and size
> - tell us more about how you preprocessed them for performance
>
> The first step with serving (large) raster data is choosing the right
> format then preprocessing, see here (slide 8 and onward):
> https://www.slideshare.net/geosolutions/geoserver-in-production-we-do-it-here-is-how-foss4g-2016
> 
> Everything else comes (way) afterwards.
>
> Regards,
> Simone Giannecchini
> ==
> GeoServer Professional Services from the experts!
> Visit http://bit.ly/gs-services
> 
> for more information.
> ==
> Ing. Simone Giannecchini
> @simogeo
> Founder/Director
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob:   +39  333 8128928
>
> http://www.geo-solutions.it
> 
> http://twitter.com/geosolutions_it
> 
>
> ---
> Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
> 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
> precisa che ogni circostanza inerente alla presente email (il suo
> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
> operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
>
> This email is intended only for the person or entity to which it is
> addressed and may contain information that is privileged, confidential or
> otherwise protected from disclosure. We remind that - as provided by
> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
> e-mail or the information herein by anyone other than the intended
> recipient is prohibited. If you have received this email by mistake, please
> notify us immediately by telephone or e-mail.
>
>
> On Sun, Nov 10, 2019 at 3:47 AM David I  wrote:
>
>> Hi,
>>
>> We're currently running 2.11.5 (upgrade of the whole server scheduled for
>> early next year) and are experiencing stability issues with the Jetty 9.3
>> hosting Geoserver (and a Geoexplorer). For instance large ortho rasters
>> previewed in Geoserver try to render for a long time and eventually fail
>> and
>> if several of those heavy getMaps happen in parallell Jetty becomes slow
>> to
>> respond and will either end up in state where it doesn't appear to
>> recover,
>> at least for 10 minutes, or it recovers. Lesser rasters preview ok and do
>> not cause
>>
>> (This is a problem when using these WMS layers in desktop applications,
>> it's
>> not that lots of people preview them from Geoserver's /web)
>>
>>
>> Latest Jetty 9.3, latest Corretto JRE 8.
>>
>> Java options:
>> Marlin 0.9.4.1,
>> -XX:+UseParallelGC
>> -XX:SoftRefLRUPolicyMSPerMB=36000
>>
>> 6144MB initial heap and 8192 max, thread stack size 256KB.
>>
>> When Jetty becomes partially or totally unresponsive the heap via jmapheap
>> -heap looks like so:
>>
>> Heap Configuration:
>>MinHeapFreeRatio = 0
>>MaxHeapFreeRatio = 100
>>MaxHeapSize  

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[Simone Giannecchini]

Dear David,
I would take a different path and understand more about this "large
rasters":

- tell us more about format and size
- tell us more about how you preprocessed them for performance

The first step with serving (large) raster data is choosing the right
format then preprocessing, see here (slide 8 and onward):
https://www.slideshare.net/geosolutions/geoserver-in-production-we-do-it-here-is-how-foss4g-2016

Everything else comes (way) afterwards.

Regards,
Simone Giannecchini
==
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services

for more information.
==
Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob:   +39  333 8128928

http://www.geo-solutions.it

http://twitter.com/geosolutions_it


---
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
precisa che ogni circostanza inerente alla presente email (il suo
contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential or
otherwise protected from disclosure. We remind that - as provided by
European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
e-mail or the information herein by anyone other than the intended
recipient is prohibited. If you have received this email by mistake, please
notify us immediately by telephone or e-mail.


On Sun, Nov 10, 2019 at 3:47 AM David I  wrote:

> Hi,
>
> We're currently running 2.11.5 (upgrade of the whole server scheduled for
> early next year) and are experiencing stability issues with the Jetty 9.3
> hosting Geoserver (and a Geoexplorer). For instance large ortho rasters
> previewed in Geoserver try to render for a long time and eventually fail
> and
> if several of those heavy getMaps happen in parallell Jetty becomes slow to
> respond and will either end up in state where it doesn't appear to recover,
> at least for 10 minutes, or it recovers. Lesser rasters preview ok and do
> not cause
>
> (This is a problem when using these WMS layers in desktop applications,
> it's
> not that lots of people preview them from Geoserver's /web)
>
>
> Latest Jetty 9.3, latest Corretto JRE 8.
>
> Java options:
> Marlin 0.9.4.1,
> -XX:+UseParallelGC
> -XX:SoftRefLRUPolicyMSPerMB=36000
>
> 6144MB initial heap and 8192 max, thread stack size 256KB.
>
> When Jetty becomes partially or totally unresponsive the heap via jmapheap
> -heap looks like so:
>
> Heap Configuration:
>MinHeapFreeRatio = 0
>MaxHeapFreeRatio = 100
>MaxHeapSize  = 8589934592 (8192.0MB)
>NewSize  = 2147483648 (2048.0MB)
>MaxNewSize   = 2863136768 (2730.5MB)
>OldSize  = 4294967296 (4096.0MB)
>NewRatio = 2
>SurvivorRatio= 8
>MetaspaceSize= 21807104 (20.796875MB)
>CompressedClassSpaceSize = 1073741824 (1024.0MB)
>MaxMetaspaceSize = 17592186044415 MB
>G1HeapRegionSize = 0 (0.0MB)
>
> Heap Usage:
> PS Young Generation
> Eden Space:
>capacity = 1982332928 (1890.5MB)
>used = 1982332928 (1890.5MB)
>free = 0 (0.0MB)
>100.0% used
> From Space:
>capacity = 437256192 (417.0MB)
>used = 0 (0.0MB)
>free = 437256192 (417.0MB)
>0.0% used
> To Space:
>capacity = 439353344 (419.0MB)
>used = 0 (0.0MB)
>free = 439353344 (419.0MB)
>0.0% used
> PS Old Generation
>capacity = 5726797824 (5461.5MB)
>used = 5726435656 (5461.154609680176MB)
>free = 362168 (0.34539031982421875MB)
>99.99367590735469% used
>
> In addition Geoserver doesn't appear to adhere to the max rendering time
> under WMS settings:  35s, this is what the Monitor plugin captured shortly
> before Jetty stopped responding earlier today:
>
>
> /
>GetMap

Re: [Geoserver-users] 2.11.5 large rasters -> wms cause servlet instability, JVM settings?

[dracic]

Do you use ImagePyramid plugin?



--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html


___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users