Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
A couple of issues / tips we have found in this sort of scenario. 1) Edit your epsg file to contain only the projections you want - this can make a big difference - presumably because its doind a slow string search each time 2) If using OpenLayers (or google maps etc) - the tiling sends multiple requests to the WMS server. If you are using IE in particular - this restricts you to 2 concurent threads to any one server - mean9ing that your (say) 16 tile requests can only be processed 2 at a time. Firefox has a similar restriction by default - but you can override it. You can see this at work using Firefox with a tool like firebug. We foun it was much much quicker if you can change yout application to request a single WMS tile for the whol map rather than individual 256x256 tiles. Obviously this may have implications if you were trying to cache tiles etc - but if not its massively quicker HTH James On Wed, Sep 17, 2008 at 4:40 PM, John Westwood [EMAIL PROTECTED] wrote: Hi, I am trying to use fast_cgi with MapServer, specifically mod_fcgid on SLES 10.2. I have mod_fcgid correctly set up within Apache because it works with another FCGI application (IIPImage). I have successfully compiled MapServer 5.2 with fast_cgi support. I used this page to help me: http://mapserver.gis.umn.edu/docs/howto/fastcgi However, when I try to use MapServer with mod_fcgid I receive the following error: The server encountered an internal error and was unable to complete your request Error message: Premature end of script headers: mapserv I am really lost as to what the problem could be; the non fast_cgi version of MapServer that I compiled worked fine. Perhaps it is something to do with setting the LD_LIBRARY_PATH in the mod_fcgid.conf file? At the moment I have it set as follows: DefaultInitEnv LD_LIBRARY_PATH /usr/lib:/usr/local/lib:/usr/lib/postgresql The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? I will also be using Tilecache, but my experiments have also found this to be slow. It is currently set up to use mod_python. The hardware we are running on is a dual quad core (eight cores in total) Xeon . So it shouldn't be slow! I would greatly appreciate any help on getting MapServer to work with fast_cgi . I would also welcome any comments regarding performance issues with OpenLayers / MapServer / Tilecache . Thanks very much, John Westwood Great Britain Historical GIS Project ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users -- Dr James Perrins exeGesIS SDM Ltd Great House Barn New Street, Talgarth Powys LD3 0AH. Direct Line: 01646 686650 Tel: 01874 711145 Fax: 01874 711156 Email: [EMAIL PROTECTED] This E-mail and any files transmitted with it are private and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, the E-mail and any files have been transmitted to you in error and any copying, distribution or other use of the information contained in them is strictly prohibited. Nothing in this E-mail message amounts to a contractual or other legal commitment on the part of exeGesIS SDM Ltd unless confirmed by a signed communication. exeGesIS SDM Ltd will make every effort to keep its network free of viruses. However, the recipient of this message will need to scan this message, and any attachments, for viruses, as exeGesIS SDM Ltd can take no responsibility for any computer virus that might be transferred by this e-mail. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
On Fri, Sep 19, 2008 at 03:34:42PM +0100, John Westwood wrote: Hi Everybody, I am attaching a cut down version of my html, javascript and map files. On our server the EmbeddedMap.js lives in the OpenLayers directory and is used by gbhgis_openlayers.html. The test address http://148.197.8.119/gbhgis_openlayers.html . This is just a test server so please don't tell the whole World about it! :) There are four layers, 2 x raster (europe 1940s, WMS and MapServer) and 2 x vector / raster (Digital Chart of the World, WMS and MapServer) . Okay, so to cut OpenLayers out of the loop: http://148.197.8.119/cgi-bin/mapserv?map=/data/map-files/gbhgis.mapservice=WMSLAYERS=europeFORMAT=image%2FpngVERSION=1.1.1REQUEST=GetMapSTYLES=EXCEPTIONS=application%2Fvnd.ogc.se_inimageSRS=epsg%3A3034BBOX=2097152,2097152,4194304,4194304WIDTH=256HEIGHT=256 http://148.197.8.119/cgi-bin/mapserv?map=/data/map-files/gbhgis.maplayers=europeformat=image%2Fpngmode=mapmap_imagetype=pngmapext=2097152+2097152+4194304+4194304imgext=2097152+2097152+4194304+4194304map_size=256+256imgx=128imgy=128imgxy=256+256 The former is WMS, the latter is not-WMS. There is a very obvious speed difference between the two. The reason the MapServer image has its colors wrong, for the record, is because it is using 8bit pngs instead of 24 bit pngs. My first impression was that was the reason for the difference, but unfortunately, 8bit gifs don't seem to fare any better. Time for the URLs: MapServer: real0m0.660s WMS: real 0m2.707s These may help someone who knows MapServer investigate more. Regards, -- Christopher Schmidt MetaCarta ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
I guess that wms makes a reprojection job as the request comes with an EPSG code and the map projection is set by a proj string. Even if they are equivalent, can MapServer know that and forget about reprojection ? The cgi is rather straight forward as no EPSG is required. As the layers don't have PROJECTION blocks, they are drawn straight forward. my 2 cents. Guillaume Christopher Schmidt a écrit : On Fri, Sep 19, 2008 at 03:34:42PM +0100, John Westwood wrote: Hi Everybody, I am attaching a cut down version of my html, javascript and map files. On our server the EmbeddedMap.js lives in the OpenLayers directory and is used by gbhgis_openlayers.html. The test address http://148.197.8.119/gbhgis_openlayers.html . This is just a test server so please don't tell the whole World about it! :) There are four layers, 2 x raster (europe 1940s, WMS and MapServer) and 2 x vector / raster (Digital Chart of the World, WMS and MapServer) . Okay, so to cut OpenLayers out of the loop: http://148.197.8.119/cgi-bin/mapserv?map=/data/map-files/gbhgis.mapservice=WMSLAYERS=europeFORMAT=image%2FpngVERSION=1.1.1REQUEST=GetMapSTYLES=EXCEPTIONS=application%2Fvnd.ogc.se_inimageSRS=epsg%3A3034BBOX=2097152,2097152,4194304,4194304WIDTH=256HEIGHT=256 http://148.197.8.119/cgi-bin/mapserv?map=/data/map-files/gbhgis.maplayers=europeformat=image%2Fpngmode=mapmap_imagetype=pngmapext=2097152+2097152+4194304+4194304imgext=2097152+2097152+4194304+4194304map_size=256+256imgx=128imgy=128imgxy=256+256 The former is WMS, the latter is not-WMS. There is a very obvious speed difference between the two. The reason the MapServer image has its colors wrong, for the record, is because it is using 8bit pngs instead of 24 bit pngs. My first impression was that was the reason for the difference, but unfortunately, 8bit gifs don't seem to fare any better. Time for the URLs: MapServer: real0m0.660s WMS: real 0m2.707s These may help someone who knows MapServer investigate more. Regards, ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
Hi John, Could you try to set your map projection with its EPSG code instead of the proj string, and see what happens ? Regards, Guillaume John Westwood a écrit : Hi Everybody, I am attaching a cut down version of my html, javascript and map files. On our server the EmbeddedMap.js lives in the OpenLayers directory and is used by gbhgis_openlayers.html. The test address http://148.197.8.119/gbhgis_openlayers.html . This is just a test server so please don't tell the whole World about it! :) There are four layers, 2 x raster (europe 1940s, WMS and MapServer) and 2 x vector / raster (Digital Chart of the World, WMS and MapServer) . A problem with using the OpenLayers.MapServer method is that the (1) The raster colours are messed up and (2) Anti-aliasing is not being done on the vector layers. I don't know why this is. Regards, John Westwood Stephen Woodbridge [EMAIL PROTECTED] 18/09/08 9:08 PM John Westwood wrote: Hi Paul and other MapServer Gurus, I have discovered a huge speed increase by using OpenLayers.MapServer (native) layers instead of OpenLayers.WMS layers. It is about ten times faster, I do not exaggerate. Why is WMS so slow? I have read that MapServer is a fast WMS server, have I done something wrong for it not to be? What do you think could be the problem? You might want to check that you are comparing apples and apples. I would expect WMS to be a little slower because there is some overhead to query the server before actually making the image request. Do you have ratio set the same for both layers? What versions of Mapserver and OpenLayers are you using? Can you post a URL that has the two layers present in the layer switch so we can look at the requests and how your have it configured? Or post the HTML so we can look at. In fact you might want to post the HTML to the openlayers list and as the question there first, to make sure you have equivalent requests between WNS and Mapserver. -Steve W I will give it some thought tomorrow. John Paul Ramsey [EMAIL PROTECTED] 17/09/08 5:15 PM John, The idea that CGI is naturally a much slower situation than a long-running process is a bit of a red herring in the case of Mapserver, and I say that as someone who is anal retentive about these things. Unless your Mapserver installation has some naturally latent components (database connections, primarily) you'll find that moving from CGI to FastCGI is worth about 15ms per request. On Wed, Sep 17, 2008 at 8:40 AM, John Westwood [EMAIL PROTECTED] wrote: The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? Yes and no. If you are experiencing noticeably poor performance (you can actually *see* it being slow) the only place that the CGI overhead could be the cause is if you're connecting to Oracle or SDE for some of your layers. If that's not the case, look elsewhere first, the very small gains you will receive from moving to FastCGI will not change your underlying problem. Paul ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users Map Information ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users - ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
Hi Paul and other MapServer Gurus, I have discovered a huge speed increase by using OpenLayers.MapServer (native) layers instead of OpenLayers.WMS layers. It is about ten times faster, I do not exaggerate. Why is WMS so slow? I have read that MapServer is a fast WMS server, have I done something wrong for it not to be? What do you think could be the problem? I will give it some thought tomorrow. John Paul Ramsey [EMAIL PROTECTED] 17/09/08 5:15 PM John, The idea that CGI is naturally a much slower situation than a long-running process is a bit of a red herring in the case of Mapserver, and I say that as someone who is anal retentive about these things. Unless your Mapserver installation has some naturally latent components (database connections, primarily) you'll find that moving from CGI to FastCGI is worth about 15ms per request. On Wed, Sep 17, 2008 at 8:40 AM, John Westwood [EMAIL PROTECTED] wrote: The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? Yes and no. If you are experiencing noticeably poor performance (you can actually *see* it being slow) the only place that the CGI overhead could be the cause is if you're connecting to Oracle or SDE for some of your layers. If that's not the case, look elsewhere first, the very small gains you will receive from moving to FastCGI will not change your underlying problem. Paul ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
Okay, maybe not ten times faster - but at least five times as fast - if not more. Anyway, ALOT faster than WMS :) Paul Ramsey [EMAIL PROTECTED] 17/09/08 5:15 PM John, The idea that CGI is naturally a much slower situation than a long-running process is a bit of a red herring in the case of Mapserver, and I say that as someone who is anal retentive about these things. Unless your Mapserver installation has some naturally latent components (database connections, primarily) you'll find that moving from CGI to FastCGI is worth about 15ms per request. On Wed, Sep 17, 2008 at 8:40 AM, John Westwood [EMAIL PROTECTED] wrote: The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? Yes and no. If you are experiencing noticeably poor performance (you can actually *see* it being slow) the only place that the CGI overhead could be the cause is if you're connecting to Oracle or SDE for some of your layers. If that's not the case, look elsewhere first, the very small gains you will receive from moving to FastCGI will not change your underlying problem. Paul ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
John Westwood wrote: Hi Paul and other MapServer Gurus, I have discovered a huge speed increase by using OpenLayers.MapServer (native) layers instead of OpenLayers.WMS layers. It is about ten times faster, I do not exaggerate. Why is WMS so slow? I have read that MapServer is a fast WMS server, have I done something wrong for it not to be? What do you think could be the problem? You might want to check that you are comparing apples and apples. I would expect WMS to be a little slower because there is some overhead to query the server before actually making the image request. Do you have ratio set the same for both layers? What versions of Mapserver and OpenLayers are you using? Can you post a URL that has the two layers present in the layer switch so we can look at the requests and how your have it configured? Or post the HTML so we can look at. In fact you might want to post the HTML to the openlayers list and as the question there first, to make sure you have equivalent requests between WNS and Mapserver. -Steve W I will give it some thought tomorrow. John Paul Ramsey [EMAIL PROTECTED] 17/09/08 5:15 PM John, The idea that CGI is naturally a much slower situation than a long-running process is a bit of a red herring in the case of Mapserver, and I say that as someone who is anal retentive about these things. Unless your Mapserver installation has some naturally latent components (database connections, primarily) you'll find that moving from CGI to FastCGI is worth about 15ms per request. On Wed, Sep 17, 2008 at 8:40 AM, John Westwood [EMAIL PROTECTED] wrote: The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? Yes and no. If you are experiencing noticeably poor performance (you can actually *see* it being slow) the only place that the CGI overhead could be the cause is if you're connecting to Oracle or SDE for some of your layers. If that's not the case, look elsewhere first, the very small gains you will receive from moving to FastCGI will not change your underlying problem. Paul ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
On Thu, Sep 18, 2008 at 08:59:51PM +0100, John Westwood wrote: Hi Paul and other MapServer Gurus, I have discovered a huge speed increase by using OpenLayers.MapServer (native) layers instead of OpenLayers.WMS layers. It is about ten times faster, I do not exaggerate. Why is WMS so slow? I have read that MapServer is a fast WMS server, have I done something wrong for it not to be? What do you think could be the problem? I will give it some thought tomorrow. Is it possible the difference is 'projections'? The Layer.MapServer in OpenLayers does not include any projection information, so if you have your OpenLayers Map configured in a way that would (via WMS) cause MapServer to reproject on the fly, it will have significantly different performance. (There are other side effects if your projections are actually different, but they might not be noticable at a first pass.) I agree with Steve that it probably makes sense to generate: * A WMS-served URL * A MapServer-served URL * Your mapfile configuration Those three things will probably help point towards the misconfiguration in your current setup. Regards, -- Christopher Schmidt MetaCarta ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
Hi, I am trying to use fast_cgi with MapServer, specifically mod_fcgid on SLES 10.2. I have mod_fcgid correctly set up within Apache because it works with another FCGI application (IIPImage). I have successfully compiled MapServer 5.2 with fast_cgi support. I used this page to help me: http://mapserver.gis.umn.edu/docs/howto/fastcgi However, when I try to use MapServer with mod_fcgid I receive the following error: The server encountered an internal error and was unable to complete your request Error message: Premature end of script headers: mapserv I am really lost as to what the problem could be; the non fast_cgi version of MapServer that I compiled worked fine. Perhaps it is something to do with setting the LD_LIBRARY_PATH in the mod_fcgid.conf file? At the moment I have it set as follows: DefaultInitEnv LD_LIBRARY_PATH /usr/lib:/usr/local/lib:/usr/lib/postgresql The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? I will also be using Tilecache, but my experiments have also found this to be slow. It is currently set up to use mod_python. The hardware we are running on is a dual quad core (eight cores in total) Xeon . So it shouldn't be slow! I would greatly appreciate any help on getting MapServer to work with fast_cgi . I would also welcome any comments regarding performance issues with OpenLayers / MapServer / Tilecache . Thanks very much, John Westwood Great Britain Historical GIS Project ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
John, The idea that CGI is naturally a much slower situation than a long-running process is a bit of a red herring in the case of Mapserver, and I say that as someone who is anal retentive about these things. Unless your Mapserver installation has some naturally latent components (database connections, primarily) you'll find that moving from CGI to FastCGI is worth about 15ms per request. On Wed, Sep 17, 2008 at 8:40 AM, John Westwood [EMAIL PROTECTED] wrote: The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? Yes and no. If you are experiencing noticeably poor performance (you can actually *see* it being slow) the only place that the CGI overhead could be the cause is if you're connecting to Oracle or SDE for some of your layers. If that's not the case, look elsewhere first, the very small gains you will receive from moving to FastCGI will not change your underlying problem. Paul ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
Hi Paul, Thanks for the useful information. The performance is pretty terrible, so I will have to do some more investigating. The MapServer executable almost completely maxes out two cores and it can take 30 seconds or more to fetch all the tiles for a particular view. I do connect to Postgres to fetch map metadata and I pull the info from MapServer using WFS. Perhaps this is causing a problem. One large problem. I tried going back to my old MapServer without the FastCGI and I commented out my MapServer configuration lines in the mod_fcgid.conf file (and restarted the server). Unfortunately, this too now no longer works and gives the same error: The server encountered an internal error and was unable to complete your request Error message: Premature end of script headers: mapserv I really need to fix this! Aarg! Thanks, John Paul Ramsey [EMAIL PROTECTED] 17/09/08 5:15 PM John, The idea that CGI is naturally a much slower situation than a long-running process is a bit of a red herring in the case of Mapserver, and I say that as someone who is anal retentive about these things. Unless your Mapserver installation has some naturally latent components (database connections, primarily) you'll find that moving from CGI to FastCGI is worth about 15ms per request. On Wed, Sep 17, 2008 at 8:40 AM, John Westwood [EMAIL PROTECTED] wrote: The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? Yes and no. If you are experiencing noticeably poor performance (you can actually *see* it being slow) the only place that the CGI overhead could be the cause is if you're connecting to Oracle or SDE for some of your layers. If that's not the case, look elsewhere first, the very small gains you will receive from moving to FastCGI will not change your underlying problem. Paul ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Using Fast CGI with MapServer to alleviate poor performance
Pull out your command-line, you probably have a linking library problem, and mapserv is just dying unceremoniously. So first: can you just run mapserv on the command line. If not, what's the error? is it instructive? Second, if just a plain mapserv runs, what happens when you feed it your workload? the trick is to use the mysterious QUERY_STRING commandline parameter, thusly: ./mapserv QUERY_STRING=map=/your/map.maplayers=yourlayersetc This runs mapserv in pretend CGI mode, without having a web server in the way, very handy for debugging. P. PS - I just realized, usually you can see the error in the apache error_log, skipping all this tedious command-line stuff. On Wed, Sep 17, 2008 at 3:00 PM, John Westwood [EMAIL PROTECTED] wrote: Hi Paul, Thanks for the useful information. The performance is pretty terrible, so I will have to do some more investigating. The MapServer executable almost completely maxes out two cores and it can take 30 seconds or more to fetch all the tiles for a particular view. I do connect to Postgres to fetch map metadata and I pull the info from MapServer using WFS. Perhaps this is causing a problem. One large problem. I tried going back to my old MapServer without the FastCGI and I commented out my MapServer configuration lines in the mod_fcgid.conf file (and restarted the server). Unfortunately, this too now no longer works and gives the same error: The server encountered an internal error and was unable to complete your request Error message: Premature end of script headers: mapserv I really need to fix this! Aarg! Thanks, John Paul Ramsey [EMAIL PROTECTED] 17/09/08 5:15 PM John, The idea that CGI is naturally a much slower situation than a long-running process is a bit of a red herring in the case of Mapserver, and I say that as someone who is anal retentive about these things. Unless your Mapserver installation has some naturally latent components (database connections, primarily) you'll find that moving from CGI to FastCGI is worth about 15ms per request. On Wed, Sep 17, 2008 at 8:40 AM, John Westwood [EMAIL PROTECTED] wrote: The reason I am trying to get MapServer to work with fast_cgi is because I am experiencing poor performance with OpenLayers. I believe that OpenLayers starts a new MapServer instance for each tile request, thus causing an unnecessary overhead. Am I correct? Yes and no. If you are experiencing noticeably poor performance (you can actually *see* it being slow) the only place that the CGI overhead could be the cause is if you're connecting to Oracle or SDE for some of your layers. If that's not the case, look elsewhere first, the very small gains you will receive from moving to FastCGI will not change your underlying problem. Paul ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
