Hi,
Absolutely do not increase RAM, CPU or anything on that line. Instead, install
all your data on a small slow laptop and make the system fast on that.
- Analyze your situaltion and think what you want to give for your customers
- Optimise the data for that purpose
- Final step is to acquire as much computing power as needed
Let's assume that you are going to publish your data through WMS. You said you
have 4 000 000 features and it takes a long time to load them. I believe it is
slow but fortunately you do not need to do it. The computer screen with
1152x864 resolution has only 955 328 pixels so your full screen view is trying
to illustrate on average four features per pixel. If someone orders you to draw
a map by hand with a pen and paper in that way you would say it is lunatic and
that's correct. The key to draw faster is to draw less.
This link shows a course example of speeding vector renderign by making
Mapserver to draw less.
http://www.paikkatietoikkuna.fi/web/en/map-window?zoomLevel=6&coord=291620_6798042&mapLayers=base_35+100+!default!,34+100+&showMarker=false&forceCache=true
There is a scale dependent group layer "Lan Parcel Register 2010". The group
consists or the original data layer and two reduced sets. Original data is
shown at scale 1:100000 or bigger (big scale - small number in scale
denominator) and it comes from a shapefile with 1.2 million polygons. In the
application the switch to original data happens between named zoom levels
"Town" and "Part of town". When zoomed out the data comes from teh first
reduced data set (10% sample) and finally from the second reduced data set (1%
sample).
In this case I made the ten percent sample and one percent sample by selecting
parcel IDs ending to one zero or two zeros, respectively. This way I can
broadcast the selected parcels uniformly on the whole map area for giving a
hint about the places having parcels. There is also a maximum limit for the
number of parcels which are rendered (MAXFEATURES 10000). I considered that ten
thousand parcels per screen is always enough.
For other data some other basis could be better for reducing the number of
features to render. For example it is usually enough to render at maximum a
few hundred biggest lakes an a map and that can be done by sorting data to
descending order according to area and then applying MAXFEATURES.
Another good way for making Mapserver to draw less and thus draw faster is to
generalize the geometries to have less vertises. If you have a hundred pixels
on a screen for showing a polygon then it will look just the same for the user
with 20 or 20 000 vertises.
The mapfile used in the linked example is attached. It is a little bit modified
and does not necesarily work straight ahead but it should show the idea.
Here is my ordered list about making vector rendering fast. Other people can
suggest re-ordering or adding new items. I suppose there is something to think
about when using classified layers and selecting classes with classitems and
expressions.
1. Check the spatial indexes for shapefiles and in the database
2. Check other database indexes. In 98 percent of cases when user says that
database is slow it just does not have a proper index for the query.
3. Draw less - less features, less vertices
==============================
4. Draw simple - multiple styles and advanced labels have some cost
5. Re-project data beforehand instead of on-the-fly
-Jukka Rahkonen-
________________________________
Peter Maes wrote:
Hi Steve,
Thanks for your reply. However, most of my datasources are vector data. I've
created a spatial index on the shapefiles, but with 4 000 000 features it still
takes a long time to load everything.
Should I increase ram, or cpu? or Both? the machines have 8 GB ram atm. I'm not
sure what mapserver needs in heavy configurations.
Besides the page on mapserver.org<http://mapserver.org> i dont find much on
improving performance for load intensive systems.
2011/12/9 Lime, Steve D (DNR)
<[email protected]<mailto:[email protected]>>
Apache should be plenty quick. Start with using fastcgi and optimizing your
data: e.g. create spatial indexes, use optimized raster formats and then
optimizing your configuration (scale dependent layers w/things like reduced
resolution layers where possible). -Steve
From:
[email protected]<mailto:[email protected]>
[mailto:[email protected]<mailto:[email protected]>]
On Behalf Of Peter Maes
Sent: Friday, December 09, 2011 6:31 AM
To: [email protected]<mailto:[email protected]>
Subject: [mapserver-users] Mapserver performance on apache - heavy load systems
Hi all,
We are planning to launch a major mapserver production site where i'm required
to launch many mapserver instances in some form of cluster.
So I basically have some questions related to this:
- Does anyone have good tips on performance for running mapserver on apache?
- Any specific build parameters I should pay attention to?
- Does it run as a native apache module, or is it required to work through cgi?
In which case: will i get better performance by using fastcgi? (we are not
using databases but shapefiles).
- Should I consider lighttpd or nginx instead?
- Any other tips related to clustering mapserver instances? :).
Note that we are using live data, so we cannot cache them, except for a minute
or so.
Thanks for your time.
LAYER
GROUP "parcels"
NAME "all_parcels"
TYPE POLYGON
STATUS ON
DATA "all_parcels.shp"
UNITS METERS
#DEBUG 5
SIZEUNITS PIXELS
TOLERANCE 3
TOLERANCEUNITS PIXELS
MAXFEATURES 10000
MAXSCALEDENOM 100000
CLASS
NAME "all_parcels"
STYLE
OUTLINECOLOR 250 0 180
WIDTH 2
END
END
METADATA
"ows_include_items" "all"
"wms_title" "all_parcels"
"wms_srs" "EPSG:2393 EPSG:3067 EPSG:4326 EPSG:900913"
END
PROJECTION
"init=epsg:2393"
END
END
LAYER
GROUP "parcels"
NAME "parsels_ten_percent"
TYPE POLYGON
STATUS ON
DATA "parcels_ten_percent.shp"
UNITS METERS
# DEBUG 5
SIZEUNITS PIXELS
TOLERANCE 3
TOLERANCEUNITS PIXELS
MAXFEATURES 10000
MAXSCALEDENOM 1000000
MINSCALEDENOM 100000
CLASS
NAME "parcels_ten_percent"
STYLE
OUTLINECOLOR 250 0 180
WIDTH 3
END
END
METADATA
"ows_include_items" "all"
"wms_title" "parcels_ten_percent"
"wms_srs" "EPSG:2393 EPSG:3067 EPSG:4326 EPSG:900913"
END
PROJECTION
"init=epsg:2393"
END
END
LAYER
GROUP "parcels"
NAME "parcels_one_percent"
TYPE POLYGON
STATUS ON
DATA "parcels_one_percent.shp"
UNITS METERS
#DEBUG 5
SIZEUNITS PIXELS
TOLERANCE 3
TOLERANCEUNITS PIXELS
MAXFEATURES 100000
MINSCALEDENOM 1000000
CLASS
NAME "parcels_one_percent"
STYLE
OUTLINECOLOR 250 0 180
WIDTH 3
END
END
METADATA
"ows_include_items" "all"
"wms_title" "parcels_one_percent"
"wms_srs" "EPSG:2393 EPSG:3067 EPSG:4326 EPSG:900913"
END
PROJECTION
"init=epsg:2393"
END
END
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
