Hi Andrea, it's Lambert conformal conic to WGS84 but regardless of fast or slow path for the reprojection, I don't think it should take 70x more time than the native one. Does it all happen within memory or is it saving the intermediate results somewhere? As Jukka mentioned, this might be rather server issue but understanding the processes within Geoserver would really help me figure out which component is stalling the request.
Thanks again, Paul P. On th 20. 10. 2022 at 18:36 Rahkonen Jukka < jukka.rahko...@maanmittauslaitos.fi sent>: > Hi, > > > > I wonder if the developers have missed the essential, that the same > process is fast on a physical server but very slow on the virtual one. Or > is there something in the virtualization that has an effect that the > virtual server must take some extra slow route? > > If doing so simple rendering takes 7000 ms I believe that about 6900 ms is > used for waiting something to happen and not for anything useful. > Unfortunately I have no idea about what could be the reason for a sluggish > performance with Windows server running on Hyper V and I fear that the main > developers do not have that kind of environment up for testing. > > > > -Jukka Rahkonen- > > > > *Lähettäjä:* Pa Pu <ppprim...@gmail.com> > *Lähetetty:* torstai 20. lokakuuta 2022 17.35 > *Vastaanottaja:* Jody Garnett <jody.garn...@gmail.com> > *Kopio:* Geoserver-users@lists.sourceforge.net > *Aihe:* Re: [Geoserver-users] Geoserver bottleneck when writing response > image > > > > Hi Jody, > > > > not sure how to identify if the "fast-path" was taken or not, I don't see > anything suggesting it did, but I could be looking for wrong key words. > > Thank you for explaining the components. What confused me was that I saw > the image chain and reprojection information logged in but didn't realize > it's just the course of action to be taken later. > > > > I had a look at image processing chains of both requests, JAI operations > used are probably as expected: > > 1) Native: ImageRead -> Crop -> Scale -> RasterClassifier -> Mosaic > > 2) Reprojected: ImageRead -> Crop -> Warp -> Crop -> RasterClassifier -> > Mosaic > > > > Here is the full image processing chain when requesting reprojected layer > - any idea why could the additional warp operation slow down the request so > much? > > > > 2022-10-20 13:35:43,589 DEBUG [org.geoserver.wms] - setting up map > 2022-10-20 13:35:43,590 DEBUG [org.geoserver.wms] - setting up 1385x912 > image > 2022-10-20 13:35:43,642 DEBUG [org.geoserver.wms.map] - Direct rendering > path produced the following image chain: > JAI op: Mosaic(it.geosolutions.jaiext.mosaic.MosaicOpImage) at Level: 0, > offset:0, 0, size:1385 x 912, tile size:895 x 407 > Params. Parameter 1:MOSAIC_TYPE_OVERLAY; Parameter 2:null; Parameter > 3:[javax.media.jai.ROI@2b4e135f]; Parameter 4:[[0.0]]; Parameter 5:[1.0]; > Parameter 6:[RangeDouble[15.0, 15.0]]; > Bands: 1, type: Byte; Color model:class java.awt.image.IndexColorModel, > transparency: Bitmask > Tile cache: > it.geosolutions.concurrent.ConcurrentTileCacheMultiMap@411b2638 > Tile scheduler: com.sun.media.jai.util.SunTileScheduler@7482407<global>, > parallelism 7, priority 5 > Number of sources: 1 > JAI op: > RasterClassifier(it.geosolutions.jaiext.classifier.RasterClassifierOpImage) > at Level: 1, offset:-11, -11, size:1407 x 934, tile size:895 x 407 > Params. Parameter 1:[Domain description: > name= Label > input range=RangeDouble[1.0, 1.0] > output range=RangeDouble[0.0, 0.0] > colors=java.awt.Color[r=62,g=192,b=224]]; Parameter 2:-1; Parameter > 3:javax.media.jai.ROI@c91b673; Parameter 4:RangeDouble[15.0, 15.0]; > Bands: 1, type: Byte; Color model:class java.awt.image.IndexColorModel, > transparency: Bitmask > Tile cache: > it.geosolutions.concurrent.ConcurrentTileCacheMultiMap@411b2638 > Tile scheduler: com.sun.media.jai.util.SunTileScheduler@7482407<global>, > parallelism 7, priority 5 > Number of sources: 1 > JAI op: Crop(it.geosolutions.jaiext.mosaic.MosaicOpImage) at Level: > 2, offset:-11, -11, size:1407 x 934, tile size:895 x 407 > Params. Parameter 1:-11.0; Parameter 2:-11.0; Parameter 3:1407.0; > Parameter 4:934.0; Parameter 5:null; Parameter 6:RangeDouble[15.0, 15.0]; > Parameter 7:[15.0]; > Bands: 1, type: Byte; Color model:class > java.awt.image.ComponentColorModel, transparency: Opaque > Tile cache: > it.geosolutions.concurrent.ConcurrentTileCacheMultiMap@411b2638 > Tile scheduler: > com.sun.media.jai.util.SunTileScheduler@7482407<global>, parallelism 7, > priority 5 > Number of sources: 1 > JAI op: Warp(it.geosolutions.jaiext.warp.WarpNearestOpImage) at > Level: 3, offset:-12, -1313, size:1514 x 3516, tile size:895 x 407 > Params. Parameter > 1:org.geotools.referencing.operation.transform.WarpAdapter@1b6a660; > Parameter 2:InterpolationNearest; Parameter 3:[15.0]; Parameter 4:null; > Parameter 5:RangeDouble[15.0, 15.0]; > Bands: 1, type: Byte; Color model:class > java.awt.image.ComponentColorModel, transparency: Opaque > Tile cache: > it.geosolutions.concurrent.ConcurrentTileCacheMultiMap@411b2638 > Tile scheduler: > com.sun.media.jai.util.SunTileScheduler@7482407<global>, parallelism 7, > priority 5 > Number of sources: 1 > JAI op: Crop(it.geosolutions.jaiext.mosaic.MosaicOpImage) at > Level: 4, offset:0, 33, size:1628 x 895, tile size:512 x 512 > Params. Parameter 1:0.0; Parameter 2:33.0; Parameter 3:1628.0; > Parameter 4:895.0; Parameter 5:null; Parameter 6:RangeDouble[15.0, 15.0]; > Parameter 7:[15.0]; > Bands: 1, type: Byte; Color model:class > java.awt.image.ComponentColorModel, transparency: Opaque > Tile cache: > it.geosolutions.concurrent.ConcurrentTileCacheMultiMap@411b2638 > Tile scheduler: > com.sun.media.jai.util.SunTileScheduler@7482407<global>, parallelism 7, > priority 5 > Number of sources: 1 > JAI op: > ImageRead(com.sun.media.jai.imageioimpl.ImageReadOpImage) at Level: 5, > offset:0, 0, size:1628 x 977, tile size:512 x 512 > Params. Parameter 1:FileImageInputStreamExtImpl which > points to PathToFile\RasterFileName.tif.ovr; Parameter 2:5; Parameter > 3:false; Parameter 4:false; Parameter 5:false; Parameter 6:null; Parameter > 7:null; Parameter 8:[class javax.imageio.ImageReadParam > SourceSubsampling[ssx:1, ssy:1] DestinationOffset(Point)[x:0, y:0]]; > Parameter > 9:it.geosolutions.imageioimpl.plugins.tiff.TIFFImageReader@7fbf999a; > Bands: 1, type: Byte; Color model:class > java.awt.image.ComponentColorModel, transparency: Opaque > Tile cache: > it.geosolutions.concurrent.ConcurrentTileCacheMultiMap@411b2638 > Tile scheduler: > com.sun.media.jai.util.SunTileScheduler@7482407<global>, parallelism 7, > priority 5 > Number of sources: 0 > > > > Thank you for your help, > > Paul > > > > On 19. 10. 2022 at 18:11 Jody Garnett <jody.garn...@gmail.com> sent: > > Components: > > - GeoServer uses the geotools library for the heavy lifting (gis data > formats, referencing, data structures) > > - image processing provided by the java advanced imaging library; with > JAI-EXT providing GIS operations (to respect concepts like no-data) > > - image formats provided by image-is library, with imageio-ext for GIS > formats > > > > A goal of the image processing is to be lazy and only do working when > output is required ... for example when writing content out. > > > > So the native and reproject requests are taking the same time to setup; > but when starting to write the result out the data will be pulled through > an image processing chain - and the work required for reprojecting is much > greater. > > > > There is a fast-path that engages when output and input line up; allowing > the output to be written out directly from sampling the input. In other > cases (say when requesting several layers as a single getmap everything > must be drawn; and then the result encoded into an output image. > > > > I hope this helps - your developer logging may indicate if the "fast-path" > was taken? > > > > Jody > > > > On Wed, Oct 19, 2022 at 2:45 AM Pa Pu <ppprim...@gmail.com> wrote: > > Hi all, > > > > I posted this issue on stackoverflow > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F74114071%2Fgeoserver-bottleneck-when-writing-response-image&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C49e13c77061a4641440908dab2a8962e%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638018734430075374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Wv%2Bpk5cLMcyFbpJsYsLXhXvP0nYt%2BeLWwnYbRj9rCFE%3D&reserved=0> > but it was recommended to me to ask rather here: > > > > I've come across a bottleneck that occurs when writing the final png file > from a WMS request. I'm rendering a geotiff (tiled, LZW compressed, with > external overviews, 90 MB file), reprojecting the layer from native CRS to > EPSG:4326. When requesting an image of the same size in native CRS, the > request takes around 100 ms, when reprojected, the request takes on average > around 7000 ms. > > I had a look in logs with GEOSERVER_DEVELOPER_LOGGING options and both > requests (native/reprojected) are taking similar amount of time except when > writing the final png output image, where the request for reprojected image > gets stucked: > > 2022-10-18 14:12:02,059 DEBUG [org.geoserver.wms.map] - Writing png image > ... > 2022-10-18 14:12:09,139 DEBUG [org.geoserver.wms.map] - Writing png image > ... done! > > Using these log messages, I was able to look up following function in java: > > public void formatImageOutputStream( > RenderedImage image, OutputStream outStream, WMSMapContent > mapContent) > > As I'm not sure what this function does, could anyone advise me where to > look next? The server is a virtual machine with Windows Server running in > HyperV, multiple cores and plenty of memory allocated to java. The issue > seems to be affecting all layers, however same rasters and layers are > working fine in other Geoserver instances that I run on physical machines. > I understand this might not be a Geoserver issue but I would still love to > hear your suggestions on where I could investigate. > > Thanks a lot. > > Geoserver version: 2.20.2 Container: Tomcat 9.0 (jre1.8.0_321) > > _______________________________________________ > Geoserver-users mailing list > > Please make sure you read the following two resources before posting to > this list: > - Earning your support instead of buying it, but Ian Turton: > http://www.ianturton.com/talks/foss4g.html#/ > <https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ianturton.com%2Ftalks%2Ffoss4g.html%23%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C49e13c77061a4641440908dab2a8962e%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638018734430075374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XnuJDJZxHsXVaTPdHFgY3IldBCosatukasxS2eKtOP0%3D&reserved=0> > - The GeoServer user list posting guidelines: > http://geoserver.org/comm/userlist-guidelines.html > <https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgeoserver.org%2Fcomm%2Fuserlist-guidelines.html&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C49e13c77061a4641440908dab2a8962e%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638018734430075374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UgYsIdiPBNKHpM5xpbJ6qNsmYY3Txyu9nmoAaSh2iB8%3D&reserved=0> > > If you want to request a feature or an improvement, also see this: > https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgeoserver%2Fgeoserver%2Fwiki%2FSuccessfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C49e13c77061a4641440908dab2a8962e%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638018734430231579%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BDGWG5lgIG8AEt1DKbuVngVdKRZJzNq%2BU5iyuTaEelQ%3D&reserved=0> > > > Geoserver-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-users > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fgeoserver-users&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C49e13c77061a4641440908dab2a8962e%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638018734430231579%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=f4DzSgee0XG6EPByjAuqBjd1v3M9X7wtvwub83iela8%3D&reserved=0> > > -- > > -- > > Jody Garnett > >
_______________________________________________ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
