Many good questions. I will see if I can catch them all. A bit on the nature of the application.
* We are only using vector data (we assumed that the raster would slow it
down).
* We are serving many different maps stored separately with the same
composition of layers. Each map has 6 layers (4 polygon and 2 point layers for
labeling).
* Each of the different map sets has differing quantities of features.
For the most significant layers they average around 10,000 features but may be
as high as 100,000.
* A majority of the requests are to display a small area of one of the
maps so the rendering focuses in to a few features. The user will query a
database which will allow for viewing the map that is zoomed to the area of
interest.
* No layer reprojecting (we assumed this would also slow it down).
* The output map is PNG with dimensions 419 X 403.
* We are using PHP_mapscript to generate the requests. The parameters
for the map generation come from a database and the user requested location.
So there are a few lines of code to find the location on the map and generate
the images.
* The mapfile contains about 12 layers. Several layers to display the
primary polygon layer thematically and a couple extra to show the polygons with
outlines.
* Data is stored in Shapefiles
I made attempts to stream-line the use of extra features to ensure the speed.
I certainly may be using items that hurt instead of help. Here is the mapfile.
Now that I look at the mapfile there may be a couple items that I originally
had intended to use but are now just relics and time wasters.
<<postforwebforum.map>>
Now as for the users. We are assuming 5000 simultaneous - all at the same
instant. This would assume a substantially larger group of users accessing the
site at the same time. We assume this to be the peak stress for Stage 1 of
the app.
Bruce -
My channeling sensors went off when Frank rang <g>.
It is certainly true that a bit of experience and contemplation can help you
discover optimization opportunities that aren't immediately self-evident. Can
you describe the nature of your map application? Are you using raster data,
vector data, or both? What size is your data, in numbers of features and/or
files? What kind of disk subsystem is being used? Is there layer reprojection
going on?
Generalizations are rarely helpful (except for this one). It's like being told
the average man is 5' 7" tall - it tells you nothing about how tall I am.
MapServer performance depends on a number of factors, but the best place to
start is a detailed understanding of what exactly you're trying to do with
MapServer.
It would be most helpful to us if you could post your map file and a sample URL
request, preferably one that is externally (publicly) visible. And can you
define what you mean by "simultaneous" users? Do you mean 5,000 map requests
all being generated at exactly the same time? Or do you mean 5,000 human users
asking for a new map every X seconds or so? And if the latter, what value are
you using for X?
- Ed
Ed McNierney
Chief Mapmaker
Demand Media / TopoZone.com
73 Princeton Street, Suite 305
North Chelmsford, MA 01863
Phone: 978-251-4242, Fax: 978-251-1396
[EMAIL PROTECTED]
-----Original Message-----
From: UMN MapServer Users List [mailto:[EMAIL PROTECTED] On Behalf Of Frank
Warmerdam
Sent: Wednesday, November 07, 2007 8:26 PM
To: [email protected]
Subject: Re: [UMN_MAPSERVER-USERS] System Configuration
Bruce Cheney wrote:
> We have been given a requirement to support 5000 simultaneous users.
> What we are finding is that MapServer bogs down around 400
> simultaneous users on a test machine. It looks like it is likely
> slowing because of the threading issue. We haven't tested on a
> production machine but are estimating that it should support double
> what are test machine could handle (double the processor and RAM). So
> at least 800 simultaneous users. Divide that out with the 5000 and we
> need a minimum of 6-7 web servers supporting MapServer. We will
> certainly scale this as is needed but I do need some idea going in as to what
> is going to be required.
Bruce,
I'm curious how many map requests per minute you expect 800 simultaneous users
to generate.
> Does this sound like results that others expect or is this quantity
> above what others have tested? Also Does anyone know of a solution in
> the works to run make mapserver thread safe and/or up the overall
> speed? I am not complaining about the speed just wondering what is in
> the works.
In various aspects MapServer is already thread safe though there are also known
"unsafe" components, and some components are wrapped by big locks that
significantly reduce the value of multiple threads.
Progress occurs by fits and starts, largely based on support from user
organizations depending on multi-threading. For instance, in 5.0 I implement
locking around OGR for a client of mine in Australia.
(This is a subtle way of suggesting you hire someone to make this happen if it
is what you want!)
All this aside, by default MapServer is *massively multi-threaded*.
I say this since the default operation is to start a new cgi instance for each
request - each is essentially an independent thread.
Of course, the downside of whole-process cgi style multithreading is that very
little context is preserved from request to request. Map files, data file
headers, etc all need to be reparsed for each request.
My point here is that you need to think carefully about the application flow to
take much advantage of multiple threading within a single process.
Also, if I may channel Ed, if you wanted to squeeze more performance out of
mapserver, you really need to start by figuring out what it is spending it's
time doing. Where is it spending it's time?
o waiting for disk? (perhaps you are reading more data than you need?)
o rendering (perhaps your data is overdense, or you are using expensive
rendering options?)
o parsing mapfiles (perhaps you mapfile has too many unused layers?) etc.
Best regards,
--
---------------------------------------+--------------------------------
---------------------------------------+------
I set the clouds in motion - turn up | Frank Warmerdam, [EMAIL PROTECTED]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org
postforwebforum.map
Description: postforwebforum.map
