Hi Paul,
From my reading of the memory page I linked earlier, and my fuzzy memories
of having dealt with this in the past, it doesn't matter what the *actual* data
is.
The memory sizing is a simplistic pre-run-time guestimation that can be summed
up as: "Do I think the image will be too large based on dimensions, pixel
depth, and number of featureStyle things." It doesn't factor in actual layer
content or real-world memory usage. If if thinks you'll be fine and /then/ you
hit a memory issue, that's an OutOfMemory error I believe, which is a very
different (and typically much worse!) problem.
So for some reason GeoServer _thinks_ your image will take a huge amount of
processing. Whether it does or not.... maybe you should (temporarily!) increase
the value to 1500000 (1.5 gigabytes - not a typo) and see what happens. Does it
use that much RAM for rendering? Does it render it? What does it look like? Etc.
I'm assuming your reply when you say 60 - 80 "GB", you actually mean MB. :-)
Cheers,
Jonathan
---- On Mon, 27 Mar 2017 15:41:22 +0100 Paul
Wittle<[email protected]> wrote ----
Hi,
In theory, assuming as we are that the MasterMap level is not in that request
then the answer was a worrying small amount of layers. It should have just been
a basemap layer (raster with single FeatureTypeStyle) and a single overlay.
I’m thinking it must be the raw data which I believe is just geotif but may
not have pyramids.
Anyway, I’ll take a look at that again at some point; thank you for pointing
me in the right direction and most importantly confirming that I’m not going
mad on the limit I set. It was 60GB previously and I had pushed it to 80GB; I
just wanted to confirm that the maximum was reasonable or too high.
Thanks,
Paul
From: Jonathan Moules [mailto:[email protected]]
Sent: 27 March 2017 15:32
To: Paul Wittle <[email protected]>
Cc: '[email protected]'
<[email protected]>
Subject: RE: [Geoserver-users] Rendering size of layer groups
Hi Paul,
Ah ok. That's part of it then. I'm guessing these are WMS requests - if you
look in the logs (may have to put on geoserver (I think rather than geotools)
debugging) I think it tells you how large in pixels the request is.
But even at 300dpi, if you're doing A4 paper, the request should be about 8 *
300 = 2400 wide, by about 3300 tall. Reading the config page below, that should
easily fit into your 80Mb of RAM.
We were printing A3 at... 300dpi - so four times the number of pixels, and I
think we set the RAM limit to 128Mb and didn't have any real problems. So
whatever is going on at your end, it's not just a result of large images.
Do your SLD files each only have one FeatureTypeStyle? And do how many laryers
will be visible during this 1GB-in-RAM request?
Cheers,
Jonathan
---- On Mon, 27 Mar 2017 15:10:29 +0100 Paul
Wittle<[email protected]> wrote ----
Hi,
I should point out that the requests are print to scale using the printing
module at 300dpi so you shouldn’t compare them to standard tile requests.
That said, it is still not happy.
Cheers,
Paul
From: Jonathan Moules [mailto:[email protected]]
Sent: 27 March 2017 15:02
To: Paul Wittle <[email protected]>
Cc: '[email protected]'
<[email protected]>
Subject: RE: [Geoserver-users] Rendering size of layer groups
Hi Paul,
I'm 99% certain that a rendering request that's going to use 1GB of RAM is a
*very* bad thing and that's likely your problem. 80Mb should be fine in most
cases.
This page may be of help, as I'm guessing it's maxRequestMemory that's causing
your problem:
http://docs.geoserver.org/stable/en/user/services/wms/configuration.html
Reading that, I can only guess you have a huge number of "FeatureTypeStyle" in
your styles (you should typically only have one per SLD), or you're requesting
a huge area.
One of the devs may have more idea as to what's causing this if you're not
doing either of those.
The later two are probably ancillary to the problem, but yes, you likely have
to do some tweaking with your source datasets.
Cheers,
Jonathan
---- On Mon, 27 Mar 2017 14:49:11 +0100 Paul
Wittle<[email protected]> wrote ----
Hi Jonathan,
I’ve used the GeoTools logging and I’m now pretty convinced that it is not
calling the database; which is great.
The main errors I’m getting are:
· Layer not found (this is to be expected on the DEV server)
· Rendering request would use 1003480KB, whilst the maximum memory
allowed is 80000KB
· [org.geotools.gce.imagepyramid] - D:\data_dir\Raster\10k\so96ne.prj
(The system cannot find the file specified)
· java.io.FileNotFoundException: D:\data_dir\Raster\10k\so96ne.prj
(The system cannot find the file specified)
I think the last two may be further evidence that our raw data is not
pre-processed well enough but I’m already working on a solution for that.
Is 80,000KB big enough in your opinion?
Cheers,
Paul
From: Jonathan Moules [mailto:[email protected]]
Sent: 27 March 2017 13:27
To: Paul Wittle <[email protected]>
Cc: '[email protected]'
<[email protected]>
Subject: Re: [Geoserver-users] Rendering size of layer groups
Hi Paul,
In my experience GeoServer is smart enough to know when to ask for a given
layer (and layergroup) based on the styles and min/max denominator. At
Warwickshire I had a basemap that consisted of:
* All Ordnance Survey Layers (layergroup) which in turn contained:
** OS Strategi (a Layergroup)
** OS Meridian (a Layergroup)
** OS VMD (a Layergroup)
** OS VML (a Layergroup)
** OS MasterMap (a Layergroup)
** OS MasterMap 3D effect (a Layergroup)
I forget if the bottom two were part of another nested layergroup or not.
But it worked well and only requested data at the required level.
It's usually worthwhile manually confirming each layergroup only displays at
the level desired/required as a standalone, and testing the performance there.
You can use Geotools Debug level logging to see what's being requested from
the database. This way you can confirm the requests are what you think they are.
That was then cached with the integrated GWC which went fairly quickly (well,
as quickly as the database would let it be!).
What is the processing memory setting for your control flow set at? And what
do the other related portions of the config for that plugin look like. You may
just have it set too low.
Thanks,
Jonathan
---- On Mon, 27 Mar 2017 12:42:37 +0100 Paul
Wittle<[email protected]> wrote ----
Hi,
We have been finding that some of our layergroups are consistently failing to
load as the processing memory exceeds the control flow setting.
I just made a layer group referencing some other layergroups and the styles
mean that only one of the groups should be displaying at a given scale.
I’ve had situations where the child layergroup would draw (I think) but the
group of layergroups will not. How does the scale dependency work?
The layers do have min/max scales but does GeoServer still request the
underlying data even if there is no style to display it or does it check the
style first and then decide not to run the query at all?
I ask because it would be a big overhead if it was requesting the data just to
throw it away.
Thanks,
Paul
"This e-mail is intended for the named addressee(s) only and may contain
information about individuals or other sensitive information and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this email in error, kindly disregard the
content of the message and notify the sender immediately. Please be aware that
all email may be subject to recording and/or monitoring in accordance with
relevant legislation."
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org!
http://sdm.link/slashdot_______________________________________________
Geoserver-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
"This e-mail is intended for the named addressee(s) only and may contain
information about individuals or other sensitive information and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this email in error, kindly disregard the
content of the message and notify the sender immediately. Please be aware that
all email may be subject to recording and/or monitoring in accordance with
relevant legislation."
"This e-mail is intended for the named addressee(s) only and may contain
information about individuals or other sensitive information and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this email in error, kindly disregard the
content of the message and notify the sender immediately. Please be aware that
all email may be subject to recording and/or monitoring in accordance with
relevant legislation."
"This e-mail is intended for the named addressee(s) only and may contain
information about individuals or other sensitive information and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this email in error, kindly disregard the
content of the message and notify the sender immediately. Please be aware that
all email may be subject to recording and/or monitoring in accordance with
relevant legislation."
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
