[mapserver-users] Re: gradient display of raster DEM data (John Callahan)
John, I was able to get datarange and classrange to work. However I found a couple issues: 1) I had to include an expression for the raster range. - This may have been fixed in newer versions of MS. 2) I was getting unwanted black areas on my map. - I was using PC256 (to make the image smaller), I had to revert to RGB, which has more colors. Mapserver did not report an error when I was using PC256. CLASS EXPRESSION ([pixel] = 1030 AND [pixel] 1133) STYLE COLORRANGE 176 240 232 255 255 179 DATARANGE 1030 1133 END END #class Mark Volz GIS Specialist -Original Message- From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of mapserver-users-requ...@lists.osgeo.org Sent: Sunday, November 20, 2011 11:00 AM To: mapserver-users@lists.osgeo.org Subject: mapserver-users Digest, Vol 46, Issue 53 Send mapserver-users mailing list submissions to mapserver-users@lists.osgeo.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.osgeo.org/mailman/listinfo/mapserver-users or, via email, send a message with subject or body 'help' to mapserver-users-requ...@lists.osgeo.org You can reach the person managing the list at mapserver-users-ow...@lists.osgeo.org When replying, please edit your Subject line so it is more specific than Re: Contents of mapserver-users digest... Today's Topics: 1. Re: gradient display of raster DEM data (John Callahan) -- Message: 1 Date: Sat, 19 Nov 2011 13:51:38 -0500 From: John Callahan john.calla...@udel.edu Subject: Re: [mapserver-users] gradient display of raster DEM data To: mapserver-users@lists.osgeo.org mapserver-users@lists.osgeo.org Message-ID: CAL=axxkvjpjkkby+2fff0r6cf8yarp7zgmlgvc21gkdnxwm...@mail.gmail.com Content-Type: text/plain; charset=iso-8859-1 Thanks again for the idea of using multiple classes but I can't seem to get it to work. Has anyone else had success with this method? It seems as though it's applying only the first defined COLORRANGE to full range of the values in the dataset. I've tried using the below syntax (only COLORRANGE and DATARANGE in each CLASS) as well as the (proposed?) block syntax as defined at http://mapserver.org/development/rfc/ms-rfc-6.html (which gives me errors for RANGEITEM and invalid hex color for MINCOLOR.) I am using MapServer 6.0.1 as part of MSW 3.0.3. Incorrect image being displayed: http://i44.tinypic.com/2hdwuuu.jpg Correct image, created through QGIS: http://i42.tinypic.com/21d2x5h.jpg Here is the LAYER portion of my map file. Is there anything obvious I should change? Thanks for any suggestions. *** LAYER NAME 'de_dem2m' TYPE RASTER DATA 'C:\data\delaware2m.img' PROCESSING CLOSE_CONNECTION=DEFER EXTENT 166773.496 49314.974 233217.496 204730.974 METADATA 'ows_title' 'Delaware DEM' INCLUDE ogc_projections.map 'wms_include_items' 'all' END STATUS ON DUMP TRUE TRANSPARENCY 100 PROJECTION init=epsg:26957 END CLASS STYLE COLORRANGE 170 230 209 255 255 179 DATARANGE 0.0 7.0 END END CLASS STYLE COLORRANGE 255 255 179 0 128 64 DATARANGE 7.01 11.0 END END CLASS STYLE COLORRANGE 0 128 64 252 186 3 DATARANGE 11.01 14.0 END END CLASS STYLE COLORRANGE 252 186 3 199 15 2 DATARANGE 14.01 20.0 END END CLASS STYLE COLORRANGE 199 15 2 105 48 13 DATARANGE 20.01 40.0 END END CLASS STYLE COLORRANGE 105 48 13 171 171 171 DATARANGE 40.01 100.0 END END CLASS STYLE COLORRANGE 171 171 171 232 230 232 DATARANGE 100.01 140.0 END END END * - John *** John Callahan, Research Scientist Delaware Geological Survey, University of Delaware URL: http://www.dgs.udel.edu * On Mon, Oct 17, 2011 at 3:05 PM, Smith, Michael ERDC-CRREL-NH michael.sm...@usace.army.mil wrote: John, I got a larger color range by using multiple classes LAYER NAME hillshade TYPE raster TILEINDEX /benchmarking/wms/2011/data/raster/DEM/bil_16int/dem_bil16int.shp PROCESSING CLOSE_CONNECTION=DEFER PROJECTION +init=epsg:4326 END CLASS STYLE COLORRANGE 46 154 88 251 255 128 DATARANGE 1003 1800 END END CLASS STYLE COLORRANGE 251 255 128 224 108 31 DATARANGE 1800 2800 END END CLASS STYLE COLORRANGE 224 108 31 200 55 55 DATARANGE2800 3500 END END CLASS STYLE COLORRANGE 200 55 55 215 244 244 DATARANGE 3500 4397 END END END From: John Callahan john.calla...@udel.edu Date: Mon, 17 Oct 2011 14:54:13 -0400
Re: [mapserver-users] mapcache in a cluster
Thomas, We have been running into some performance issues mapcache and nfs. We feel the issue may be related to how nfs locks files/directories compared to smb. We are trying a few thing on our end (disable locking / nfs4 etc). Do you have any ideas? Regards On 20 October 2011 12:19, thomas bonfort thomas.bonf...@gmail.com wrote: So, this discussion inspired me to completely rework the locking mechanism in mapcache, to stop relying on file locks which have their quirks on network filesystems. I have tried using multiple apache instances configured to used a SMB-mounted lock directory and hammered both instances on unseeded identical area to force locking, and ended up with absolutely no duplicate wms requests or failed requests for the clients. The code is committed in trunk. Thanks for bringing this up, this allowed me to really simplify the locking code and remove a lot of unneeded stuff :) -- thomas On Thu, Oct 20, 2011 at 17:08, Travis Kirstine traviskirst...@gmail.com wrote: Andreas and Thomas Thanks for you responses, I have discussed this with some of our IT staff and they had similar solution as Andreas using gfs. Their comments are below: I suspect this scheme is not reliable over NFS. The problem is the directory updates are not synchronized across multiple nodes. I had a similar issue with the IMAP E-mail protocol. Our workaround currently is to force each user to leverage a single server. Ref: http://wiki.dovecot.org/NFS Seems like there's some tweaks to disable directory attribute caching but this can trigger slower performance. Only workaround is to use GFS which I found to have it's own issues. Regards On 20 October 2011 05:32, Eichner, Andreas - SID-NLKM andreas.eich...@sid.sachsen.de wrote: We use TileCache.py on two servers with the cache on an OCFS2 on a shared LUN in the SAN. No known issues with that for now. Note: Spurious stale lock files occurred already on a single machine. There seemed to be issues with lots of requests and a very slow upstream server. I used a cron job to delete lock files older than 5 minutes or so. As Thomas noted, if the lock files are created on a shared filesystem and you make sure the filesystem you use is able to lock files properly (read the docs carefully!) there's no reason why it should not work. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Re: gradient display of raster DEM data (John Callahan)
On Nov 21, 2011, at 8:09 AM, Mark Volz wrote: John, I was able to get datarange and classrange to work. However I found a couple issues: 1) I had to include an expression for the raster range. - This may have been fixed in newer versions of MS. At least with ms 6.0.1 I didn't have to use any expression directive. Just colorrange and datarange did the trick. 2) I was getting unwanted black areas on my map. - I was using PC256 (to make the image smaller), I had to revert to RGB, which has more colors. Mapserver did not report an error when I was using PC256. CLASS EXPRESSION ([pixel] = 1030 AND [pixel] 1133) STYLE COLORRANGE 176 240 232 255 255 179 DATARANGE 1030 1133 END END #class Mark Volz GIS Specialist ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] mapcache in a cluster
What kind of performance issues? The current locking code only uses the presence/absence of a file for it's locking functions, and does not rely on flock/fcntl. -- thomas On Mon, Nov 21, 2011 at 15:16, Travis Kirstine traviskirst...@gmail.com wrote: Thomas, We have been running into some performance issues mapcache and nfs. We feel the issue may be related to how nfs locks files/directories compared to smb. We are trying a few thing on our end (disable locking / nfs4 etc). Do you have any ideas? Regards On 20 October 2011 12:19, thomas bonfort thomas.bonf...@gmail.com wrote: So, this discussion inspired me to completely rework the locking mechanism in mapcache, to stop relying on file locks which have their quirks on network filesystems. I have tried using multiple apache instances configured to used a SMB-mounted lock directory and hammered both instances on unseeded identical area to force locking, and ended up with absolutely no duplicate wms requests or failed requests for the clients. The code is committed in trunk. Thanks for bringing this up, this allowed me to really simplify the locking code and remove a lot of unneeded stuff :) -- thomas On Thu, Oct 20, 2011 at 17:08, Travis Kirstine traviskirst...@gmail.com wrote: Andreas and Thomas Thanks for you responses, I have discussed this with some of our IT staff and they had similar solution as Andreas using gfs. Their comments are below: I suspect this scheme is not reliable over NFS. The problem is the directory updates are not synchronized across multiple nodes. I had a similar issue with the IMAP E-mail protocol. Our workaround currently is to force each user to leverage a single server. Ref: http://wiki.dovecot.org/NFS Seems like there's some tweaks to disable directory attribute caching but this can trigger slower performance. Only workaround is to use GFS which I found to have it's own issues. Regards On 20 October 2011 05:32, Eichner, Andreas - SID-NLKM andreas.eich...@sid.sachsen.de wrote: We use TileCache.py on two servers with the cache on an OCFS2 on a shared LUN in the SAN. No known issues with that for now. Note: Spurious stale lock files occurred already on a single machine. There seemed to be issues with lots of requests and a very slow upstream server. I used a cron job to delete lock files older than 5 minutes or so. As Thomas noted, if the lock files are created on a shared filesystem and you make sure the filesystem you use is able to lock files properly (read the docs carefully!) there's no reason why it should not work. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] mapcache in a cluster
When running the seeder (I haven't tested on the fly generation) the process is running extremely slow. The seeder is launched using all available cpus (24) on the server. We are running MapServer as the WMS source. When I launch the seeder using a cache on the same file system the seeder will spawn 18-24 concurrent WMS request and mapcache_seed will be running at 250-350 % CPU. If I launch the same seeding process using a NFS mounted cache the performce will drop - The WMS requests will drop to only one request (if that) and map_cache will be running at less than 60% CPU. The server is basically idle. I rechecked the mapcache configuration script and we are writing the lock files to the same mounted directory as the cache, I'm not sure if this could be a issue. I will try writing the lock files locally to see if this has any effect. Regards On 21 November 2011 09:39, thomas bonfort thomas.bonf...@gmail.com wrote: What kind of performance issues? The current locking code only uses the presence/absence of a file for it's locking functions, and does not rely on flock/fcntl. -- thomas On Mon, Nov 21, 2011 at 15:16, Travis Kirstine traviskirst...@gmail.com wrote: Thomas, We have been running into some performance issues mapcache and nfs. We feel the issue may be related to how nfs locks files/directories compared to smb. We are trying a few thing on our end (disable locking / nfs4 etc). Do you have any ideas? Regards On 20 October 2011 12:19, thomas bonfort thomas.bonf...@gmail.com wrote: So, this discussion inspired me to completely rework the locking mechanism in mapcache, to stop relying on file locks which have their quirks on network filesystems. I have tried using multiple apache instances configured to used a SMB-mounted lock directory and hammered both instances on unseeded identical area to force locking, and ended up with absolutely no duplicate wms requests or failed requests for the clients. The code is committed in trunk. Thanks for bringing this up, this allowed me to really simplify the locking code and remove a lot of unneeded stuff :) -- thomas On Thu, Oct 20, 2011 at 17:08, Travis Kirstine traviskirst...@gmail.com wrote: Andreas and Thomas Thanks for you responses, I have discussed this with some of our IT staff and they had similar solution as Andreas using gfs. Their comments are below: I suspect this scheme is not reliable over NFS. The problem is the directory updates are not synchronized across multiple nodes. I had a similar issue with the IMAP E-mail protocol. Our workaround currently is to force each user to leverage a single server. Ref: http://wiki.dovecot.org/NFS Seems like there's some tweaks to disable directory attribute caching but this can trigger slower performance. Only workaround is to use GFS which I found to have it's own issues. Regards On 20 October 2011 05:32, Eichner, Andreas - SID-NLKM andreas.eich...@sid.sachsen.de wrote: We use TileCache.py on two servers with the cache on an OCFS2 on a shared LUN in the SAN. No known issues with that for now. Note: Spurious stale lock files occurred already on a single machine. There seemed to be issues with lots of requests and a very slow upstream server. I used a cron job to delete lock files older than 5 minutes or so. As Thomas noted, if the lock files are created on a shared filesystem and you make sure the filesystem you use is able to lock files properly (read the docs carefully!) there's no reason why it should not work. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Re: gradient display of raster DEM data (John Callahan)
Thanks guys. Adding an EXPRESSION for each class worked great. When using only one class, the EXPRESSION was not necessary. I'm using RGB and everything looks great. Any thoughts about the Legend Graphic? I saw some brief discussions on the doc page. - John On Mon, Nov 21, 2011 at 9:09 AM, Mark Volz markv...@co.lyon.mn.us wrote: John, I was able to get datarange and classrange to work. However I found a couple issues: 1) I had to include an expression for the raster range. - This may have been fixed in newer versions of MS. 2) I was getting unwanted black areas on my map. - I was using PC256 (to make the image smaller), I had to revert to RGB, which has more colors. Mapserver did not report an error when I was using PC256. CLASS EXPRESSION ([pixel] = 1030 AND [pixel] 1133) STYLE COLORRANGE 176 240 232 255 255 179 DATARANGE 1030 1133 END END #class Mark Volz GIS Specialist -Original Message- From: mapserver-users-boun...@lists.osgeo.org [mailto: mapserver-users-boun...@lists.osgeo.org] On Behalf Of mapserver-users-requ...@lists.osgeo.org Sent: Sunday, November 20, 2011 11:00 AM To: mapserver-users@lists.osgeo.org Subject: mapserver-users Digest, Vol 46, Issue 53 Send mapserver-users mailing list submissions to mapserver-users@lists.osgeo.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.osgeo.org/mailman/listinfo/mapserver-users or, via email, send a message with subject or body 'help' to mapserver-users-requ...@lists.osgeo.org You can reach the person managing the list at mapserver-users-ow...@lists.osgeo.org When replying, please edit your Subject line so it is more specific than Re: Contents of mapserver-users digest... Today's Topics: 1. Re: gradient display of raster DEM data (John Callahan) -- Message: 1 Date: Sat, 19 Nov 2011 13:51:38 -0500 From: John Callahan john.calla...@udel.edu Subject: Re: [mapserver-users] gradient display of raster DEM data To: mapserver-users@lists.osgeo.org mapserver-users@lists.osgeo.org Message-ID: CAL=axxkvjpjkkby+2fff0r6cf8yarp7zgmlgvc21gkdnxwm...@mail.gmail.com Content-Type: text/plain; charset=iso-8859-1 Thanks again for the idea of using multiple classes but I can't seem to get it to work. Has anyone else had success with this method? It seems as though it's applying only the first defined COLORRANGE to full range of the values in the dataset. I've tried using the below syntax (only COLORRANGE and DATARANGE in each CLASS) as well as the (proposed?) block syntax as defined at http://mapserver.org/development/rfc/ms-rfc-6.html (which gives me errors for RANGEITEM and invalid hex color for MINCOLOR.) I am using MapServer 6.0.1 as part of MSW 3.0.3. Incorrect image being displayed: http://i44.tinypic.com/2hdwuuu.jpg Correct image, created through QGIS: http://i42.tinypic.com/21d2x5h.jpg Here is the LAYER portion of my map file. Is there anything obvious I should change? Thanks for any suggestions. *** LAYER NAME 'de_dem2m' TYPE RASTER DATA 'C:\data\delaware2m.img' PROCESSING CLOSE_CONNECTION=DEFER EXTENT 166773.496 49314.974 233217.496 204730.974 METADATA 'ows_title' 'Delaware DEM' INCLUDE ogc_projections.map 'wms_include_items' 'all' END STATUS ON DUMP TRUE TRANSPARENCY 100 PROJECTION init=epsg:26957 END CLASS STYLE COLORRANGE 170 230 209 255 255 179 DATARANGE 0.0 7.0 END END CLASS STYLE COLORRANGE 255 255 179 0 128 64 DATARANGE 7.01 11.0 END END CLASS STYLE COLORRANGE 0 128 64 252 186 3 DATARANGE 11.01 14.0 END END CLASS STYLE COLORRANGE 252 186 3 199 15 2 DATARANGE 14.01 20.0 END END CLASS STYLE COLORRANGE 199 15 2 105 48 13 DATARANGE 20.01 40.0 END END CLASS STYLE COLORRANGE 105 48 13 171 171 171 DATARANGE 40.01 100.0 END END CLASS STYLE COLORRANGE 171 171 171 232 230 232 DATARANGE 100.01 140.0 END END END * - John *** John Callahan, Research Scientist Delaware Geological Survey, University of Delaware URL: http://www.dgs.udel.edu * On Mon, Oct 17, 2011 at 3:05 PM, Smith, Michael ERDC-CRREL-NH michael.sm...@usace.army.mil wrote: John, I got a larger color range by using multiple classes LAYER NAME hillshade TYPE raster TILEINDEX /benchmarking/wms/2011/data/raster/DEM/bil_16int/dem_bil16int.shp PROCESSING CLOSE_CONNECTION=DEFER PROJECTION +init=epsg:4326 END CLASS STYLE COLORRANGE 46 154 88 251 255 128 DATARANGE 1003 1800 END
Re: [mapserver-users] tileindex query
Thanks Jukka, Below is a list of the MapServer specs. Please let me know if you notice something out of the ordinary. MapServer version 5.6.5 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=ICONV SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=GEOS INPUT=EPPL7 INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE Thanks EW On 11/19/2011 at 1:39 AM, in message 84446def76453c439e9e97e438e13a6303e...@suutari.haapa.mmm.fi, Rahkonen Jukka jukka.rahko...@mmmtike.fi wrote: Hi, Polygon layer defined like this works for me TILEINDEX D:\data\tindex\tindex_1.shp,0 Also in my mapfile I have a commented line #CONNECTIONTYPE OGR #Don't use, breaks classification and GetFeatureInfo I do not remember exactly what was the Mapserver version which gave to troubles with GetFeatureInfo and tileindex layer. Data referred by tileindex are shapefiles and they do have .qix indexes. -Jukka Rahkonen- Lähettäjä: mapserver-users-boun...@lists.osgeo.org [mapserver-users-boun...@lists.osgeo.org] käyttäjän Eric Weisbender [weisb...@wapa.gov] puolesta Lähetetty: 18. marraskuuta 2011 22:55 Vastaanottaja: mapserver-users@lists.osgeo.org Aihe: [mapserver-users] tileindex query Hello, After two weeks of working on querying vector layers displayed by a tileindex I have still not made any progress. I have tried every variation of layer definition including connectiontype ogr. I have updated my gdal, ogr libraries and have tested on MapServer 565 and 601 both. I have tried different shapefiles and different shapefile combinations both with and with .qix files. I've done this in the past with great success on MapServer 410 running on a red hat box. We are now running in production, MapServer 565 on SUSE 10 and I'm starting to wonder if that isn't the problem. As I said before the query results seem to be random. Sometimes successful but most of the time I get an Internal Server Error. Below is the MapServer Error Log file for both. I am out of ideas and any help or insight is much appreciated. Internal Server Error [Tue Nov 15 15:21:23 2011].248360 msDrawMap(): Layer 0 (fotest), 0.007s [Tue Nov 15 15:21:23 2011].248891 msDrawMap(): Layer 1 (fotest), 0.000s [Tue Nov 15 15:21:23 2011].270075 msDrawMap(): Drawing Label Cache, 0.003s [Tue Nov 15 15:21:23 2011].270088 msDrawMap() total time: 0.030s [Tue Nov 15 15:21:23 2011].331671 msSaveImage() total time: 0.062s [Tue Nov 15 15:21:23 2011].335688 msSaveImage() total time: 0.002s [Tue Nov 15 15:21:23 2011].360078 msSaveImage() total time: 0.021s [Tue Nov 15 15:21:23 2011].361651 mapserv request processing time (loadmap not incl.): 0.122s [Tue Nov 15 15:21:23 2011].361661 msFreeMap(): freeing map at 0x668900. [Tue Nov 15 15:21:23 2011].362122 freeLayer(): freeing layer at 0x670e10. [Tue Nov 15 15:21:28 2011].104416 freeLayer(): freeing layer at 0x7fff030e0ad0. Successful Query [Tue Nov 15 15:22:22 2011].104410 msDrawMap(): Layer 0 (fotest), 0.007s [Tue Nov 15 15:22:22 2011].104949 msDrawMap(): Layer 1 (fotest), 0.000s [Tue Nov 15 15:22:22 2011].125857 msDrawMap(): Drawing Label Cache, 0.003s [Tue Nov 15 15:22:22 2011].125869 msDrawMap() total time: 0.031s [Tue Nov 15 15:22:22 2011].184217 msSaveImage() total time: 0.058s [Tue Nov 15 15:22:22 2011].188075 msSaveImage() total time: 0.002s [Tue Nov 15 15:22:22 2011].212534 msSaveImage() total time: 0.022s [Tue Nov 15 15:22:22 2011].214087 mapserv request processing time (loadmap not incl.): 0.119s [Tue Nov 15 15:22:22 2011].214097 msFreeMap(): freeing map at 0x668900. [Tue Nov 15 15:22:22 2011].214668 freeLayer(): freeing layer at 0x670e10. [Tue Nov 15 15:22:25 2011].736672 freeLayer(): freeing layer at 0x7fff3e289d90. [Tue Nov 15 15:22:25 2011].737327 msDrawMap(): Layer 0 (fotest), 0.005s [Tue Nov 15 15:22:25 2011].737607 msDrawMap(): Layer 1 (fote st), 0.000s [Tue Nov 15 15:22:25 2011].758983 msDrawMap(): Drawing Label Cache, 0.003s [Tue Nov 15 15:22:25 2011].758996 msDrawMap() total time: 0.028s [Tue Nov 15 15:22:25 2011].778785 msSaveImage() total time: 0.020s [Tue Nov 15 15:22:25 2011].782604 msSaveImage() total time: 0.002s [Tue Nov 15 15:22:25 2011].806905 msSaveImage() total time: 0.021s [Tue Nov 15 15:22:25 2011].807766 mapserv request processing time (loadmap not incl.): 0.077s [Tue Nov 15 15:22:25 2011].80 msFreeMap(): freeing map at 0x668e00. [Tue Nov 15 15:22:25 2011].808246 freeLayer(): freeing layer at 0x671310. Thanks EW ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] mapcache in a cluster
Travis, In the past I have used NFS in a custer and in general the performance was very bad when compared to SMB. Now this was a few years ago, ie: not using mapcache, but my experience was that we ran into a few issues with NFS: 1. NFS CPU overhead was significantly more 2. connection recovery was poor if a server was pulled from the cluster and added back again (ie: recovery after an error condition) 3. the protocol seems to have a lot more over head -Steve W On 11/21/2011 9:39 AM, thomas bonfort wrote: What kind of performance issues? The current locking code only uses the presence/absence of a file for it's locking functions, and does not rely on flock/fcntl. -- thomas On Mon, Nov 21, 2011 at 15:16, Travis Kirstinetraviskirst...@gmail.com wrote: Thomas, We have been running into some performance issues mapcache and nfs. We feel the issue may be related to how nfs locks files/directories compared to smb. We are trying a few thing on our end (disable locking / nfs4 etc). Do you have any ideas? Regards On 20 October 2011 12:19, thomas bonfortthomas.bonf...@gmail.com wrote: So, this discussion inspired me to completely rework the locking mechanism in mapcache, to stop relying on file locks which have their quirks on network filesystems. I have tried using multiple apache instances configured to used a SMB-mounted lock directory and hammered both instances on unseeded identical area to force locking, and ended up with absolutely no duplicate wms requests or failed requests for the clients. The code is committed in trunk. Thanks for bringing this up, this allowed me to really simplify the locking code and remove a lot of unneeded stuff :) -- thomas On Thu, Oct 20, 2011 at 17:08, Travis Kirstinetraviskirst...@gmail.com wrote: Andreas and Thomas Thanks for you responses, I have discussed this with some of our IT staff and they had similar solution as Andreas using gfs. Their comments are below: I suspect this scheme is not reliable over NFS. The problem is the directory updates are not synchronized across multiple nodes. I had a similar issue with the IMAP E-mail protocol. Our workaround currently is to force each user to leverage a single server. Ref: http://wiki.dovecot.org/NFS Seems like there's some tweaks to disable directory attribute caching but this can trigger slower performance. Only workaround is to use GFS which I found to have it's own issues. Regards On 20 October 2011 05:32, Eichner, Andreas - SID-NLKM andreas.eich...@sid.sachsen.de wrote: We use TileCache.py on two servers with the cache on an OCFS2 on a shared LUN in the SAN. No known issues with that for now. Note: Spurious stale lock files occurred already on a single machine. There seemed to be issues with lots of requests and a very slow upstream server. I used a cron job to delete lock files older than 5 minutes or so. As Thomas noted, if the lock files are created on a shared filesystem and you make sure the filesystem you use is able to lock files properly (read the docs carefully!) there's no reason why it should not work. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] QGis 'Zoom to best scale' Not Working
I'm not sure if this is a QGis problem or a problem with my Mapserver-generated WMS layer. When I load a WMS image in QGis, everything works as expected except when I select 'Zoom to best scale' I get a Mapserver error stating that the bounding box is illegal. And it is - the bounding box corner points in the getmap request are identical. This selections is supposed to zoom in so the pixels displayed are 1:1 or 100% A look at the capabilities request returned by Mapserver and the mapfile and images involved don't show anything obviously wrong. Is it possible that QGis is sending the wrong request? Or is something missing from either the capabilities document or the mapfile layer? I'm not really sure where to start looking to isolate the problem. Note that if I load images directly instead of through Mapserver WMS I am able to zoom to scale without any problem, which suggests a problem with Mapserver, likely in the mapfile. I am using VRT files as intermediaries between the Mapfile and the underlying NITF files; it seems that Mapserver should be using the affine transform contained in the VRT to determine proper scaling, but I'm not sure if that is sufficient. Are there arguments that need to be placed in the Mapfile metadata to help WMS generation along? Or something that would cause items in the capabilities document to be left out? Using QGis 1.7.0 on Linux. -- View this message in context: http://osgeo-org.1803224.n2.nabble.com/QGis-Zoom-to-best-scale-Not-Working-tp7017776p7017776.html Sent from the Mapserver - User mailing list archive at Nabble.com. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] QGis 'Zoom to best scale' Not Working
Hi, WMS does not advertise by default the native resolution of the data. I have never tried but perhaps adding wms_resx, wms_resy metadata into layer metadata would add that information into GetCapabilities http://mapserver.org/ogc/wms_server.html. I do not know if QGis can utilise that information even if it exists. -Jukka Rahkonen- SixDegrees wrote: I'm not sure if this is a QGis problem or a problem with my Mapserver-generated WMS layer. When I load a WMS image in QGis, everything works as expected except when I select 'Zoom to best scale' I get a Mapserver error stating that the bounding box is illegal. And it is - the bounding box corner points in the getmap request are identical. This selections is supposed to zoom in so the pixels displayed are 1:1 or 100% A look at the capabilities request returned by Mapserver and the mapfile and images involved don't show anything obviously wrong. Is it possible that QGis is sending the wrong request? Or is something missing from either the capabilities document or the mapfile layer? I'm not really sure where to start looking to isolate the problem. Note that if I load images directly instead of through Mapserver WMS I am able to zoom to scale without any problem, which suggests a problem with Mapserver, likely in the mapfile. I am using VRT files as intermediaries between the Mapfile and the underlying NITF files; it seems that Mapserver should be using the affine transform contained in the VRT to determine proper scaling, but I'm not sure if that is sufficient. Are there arguments that need to be placed in the Mapfile metadata to help WMS generation along? Or something that would cause items in the capabilities document to be left out? Using QGis 1.7.0 on Linux. -- View this message in context: http://osgeo-org.1803224.n2.nabble.com/QGis-Zoom-to-best-scale-Not-Working-tp7017776p7017776.html Sent from the Mapserver - User mailing list archive at Nabble.com. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
