Re: [mapserver-users] TopoZone, MapServer, and me
Steve Lime wrote: We'll have to find a new performance, data optimization, projection and raster guru. ;-) You forgot to write that all this was on Windows servers... making it even harder to find a replacement for Ed's advice and Topozone as a showcase for large MapServer/Windows deployments. Best of luck in your new job Ed... that will be an interesting one for sure! Daniel -- Daniel Morissette http://www.mapgears.com/ ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] mapserver web site {Scanned}
Hi, I can't connect to the mapserver web site (mapserver.gis.umn.edu) for two days. Is the server down or the site has been moved? Thanks Zoltan ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] mapserver web site {Scanned}
Siki, Network issues. I think the machine is still up, but it is not reachable from the outside world. Working on it... Wayback or Google Cache as a stopgap for now. Howard On Sep 15, 2008, at 2:51 PM, Siki Zoltan wrote: Hi, I can't connect to the mapserver web site (mapserver.gis.umn.edu) for two days. Is the server down or the site has been moved? Thanks Zoltan ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] TopoZone, MapServer, and me
Hey Ed, Now we're really gonna be behind schedule on that graticule improvement stuff... :) Many, many thanks for your years of support, and best wishes on your new venture (I expect the OLPC s/w will have awesome performance!). Brent Fraser Ed McNierney wrote: Steve - I think you and Brent will never let me REALLY retire until that graticule (and title blocks, and neatlines, etc., etc.) are done! - Ed On 9/13/08 9:19 PM, Steve Lime [EMAIL PROTECTED] wrote: Congrats Ed, sounds like you've moved on to an equally interesting topic from TopoZone. Glad to hear TopoZone lives on in some shape or form. I'm sure I speak for all community members when I say thanks for all your help over the years. We'll have to find a new performance, data optimization, projection and raster guru. ;-) I hope you'll be back since you still owe me real graticule support... Best of luck! Steve Ed McNierney [EMAIL PROTECTED] 09/13/08 7:45 PM Folks - Old-timers may have noticed that I've been relatively quiet on this list for the last few months. Newer folks might only have seen the improvement g. I am about to sign off - at least temporarily - as I will no longer be an active MapServer user or developer. My TopoZone business was acquired by Demand Media, the parent company of trails.com, in June, 2007. I worked with trails.com to integrate our technology into theirs (they were already using a very similar MapServer-based system) and completed that work last April. As of April, 2008 we closed down the former TopoZone servers and office. It was great to work with the trails.com team, but it made little sense to keep a teeny satellite team out in Massachusetts (they're based in Seattle). I took a little time off to do a few things on my to-do list. As of last Monday I've started a new position as VP of Software Development for the One Laptop Per Child Foundation in Cambridge, MA, building low-cost laptops for children around the world with lots of cool applications on them. This is keeping me pretty busy with a rather massive open source development project with a team of great people scattered around the globe. I'll be busy doing that work and won't really have the time to help out here (until I start needing more maps...) Thanks to all the present and former members of this list who've helped me a lot with MapServer (and GDAL, and PROJ, etc.) and to those of you I've actually met in person at our conferences both pre- and post-OSGeo. If I can be of any assistance to any of you, please let me know - thanks! - Ed Ed McNierney 205 Indian Hill Road Groton, MA 01450 [EMAIL PROTECTED] +1 (978) 761-0049 ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Ed's Rules for the Best Raster Performance
Better yet, Add your comments to: http://mapserver.gis.umn.edu/docs/howto/optimizeraster and http://mapserver.gis.umn.edu/docs/howto/optimizevector I had always thought that all we needed to do to make these pages great was to grok the list for all of Ed's posts... David. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brent Fraser Sent: Monday, September 15, 2008 12:55 PM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Ed's Rules for the Best Raster Performance In honor of Ed's imminent retirement from the Mapserver Support Group, I've put together Ed's List for the Best Raster Performance: #1. Pyramid the data - use MAXSCALE and MINSCALE in the LAYER object. #2. Tile the data (and merge your upper levels of the pyramid for fewer files). - see the TILEINDEX object #3. Don't compress your data - avoid jpg, ecw, and mrsid formats. #4. Don't re-project your data on-the-fly. #5. Get the fastest disks you can afford. (Ed, feel free to edit...) Brent Fraser ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Ed's Rules for the Best Raster Performance
Jeff McKenna wrote: my quick comments: - adding overviews with GDAL's 'gdaladdo' utility is very important In some cases. It depends on your data. As Ed once posted, it may be a good idea to switch to external overviews and merge some of the files to limit the number of file-opens Mapserver must do when the view is zoomed way out. - I find your use of the word pyramid confusing (this seems to be a word that Arc* users are familiar with but has no real meaning in the MapServer world...I guess I've been on the good side for too long ha) Not being an Arc* user I can't comment on it's origins. Overview is a good alternative, but it doesn't seem to convey the same multi-levelness as pyramids. For example to be able to display the Global Landsat Mosaic, I created seven levels of a pyramid (each an external overview of the higher-resolution below it). Hmmm, may be the plural of overview is pyramid... :) Hey Steve L., maybe we should have a PYRAMID Layer type, to replace a set of scale-sensitive TILEINDEX Layers (this would help my every-layer-is-exposed-in-WMS problem too: http://trac.osgeo.org/mapserver/ticket/300). Brent ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Ed's Rules for the Best Raster Performance
Brent Fraser wrote: Jeff McKenna wrote: my quick comments: - adding overviews with GDAL's 'gdaladdo' utility is very important In some cases. It depends on your data. As Ed once posted, it may be a good idea to switch to external overviews and merge some of the files to limit the number of file-opens Mapserver must do when the view is zoomed way out. I'm not here to argue with you, I am only pointing out the importance of the utility, which you did not mention in your notes. -- Jeff McKenna FOSS4G Consulting and Training Services http://www.gatewaygeomatics.com/ ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] WMS loadparams weird behavior on Ubuntu
Answering my own question, here's what I found: At first I had installed the php5 apache module, then quickly remembered CGI would be better and installed that. I thought I had uninstalled the php module, but I was mistaken. So all this time PHP was running as the Apache module. I got PHP CGI running and now the WMS request loadparams() works. Apparently WMS doesn't work on PHP as Apache module (my OSX system uses PHP CGI). Or maybe there is a way to make it work? (though I don't really care now that I have PHP CGI running) On Sep 15, 2008, at 11:32 AM, William Kyngesburye wrote: I testing/debugging some WMS PHP on Ubuntu 8.04 that works on OSX. On OSX I have MapServer 5.0.0. Same on Ubuntu, from the repositories. When my PHP script gets to the loadparams() statement it immediately sends data back to the browser, in some unknown form (Firefox pops up the open in/save dialog). If I save it, it's 0 bytes. The PHP magic_quotes_gpc was on, so I turned it off, but that didn't help. The GET params appear to be properly escaped and the request has all the required parameters (I took the URL from the request OpenLayers was sending). - William Kyngesburye kyngchaos*at*kyngchaos*dot*com http://www.kyngchaos.com/ First Pogril: Why is life like sticking your head in a bucket filled with hyena offal? Second Pogril: I don't know. Why IS life like sticking your head in a bucket filled with hyena offal? First Pogril: I don't know either. Wretched, isn't it? -HitchHiker's Guide to the Galaxy ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Ed's Rules for the Best Raster Performance
Jeff, Very good point (sometimes I get stuck on just one way of doing things). As David Fawcett pointed out, I should add those comments to the doc. Thanks! Brent Jeff McKenna wrote: Brent Fraser wrote: Jeff McKenna wrote: my quick comments: - adding overviews with GDAL's 'gdaladdo' utility is very important In some cases. It depends on your data. As Ed once posted, it may be a good idea to switch to external overviews and merge some of the files to limit the number of file-opens Mapserver must do when the view is zoomed way out. I'm not here to argue with you, I am only pointing out the importance of the utility, which you did not mention in your notes. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] How to improve raster imagery display ?
On Mon, Sep 15, 2008 at 12:28:54PM -0600, Mike Flannigan wrote:
Good luck with your new assignment. You did a great
job on TopoZone.
A question for everybody on the list. When I go to
the Archives at
http://lists.osgeo.org/pipermail/mapserver-users/
and download the GZiped text file, I don't get
good text even after converting from UNIX to
DOS format. I get garbage like this:
?Ô⢲?|¼?öò¥b!#²E¨ø??ÊðÝ(?JùÂüZ`C¸uÍËã«?{D
Can anybody suggest a method of obtaining the archives
in text format for easy searching?
Er, it looks like you didn't use gzip to unpack it yet? gzipped is
like zipped... You'll need to unzip... (doing that works for me.)
Regards,
--
Christopher Schmidt
MetaCarta
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Ed's Rules for the Best Raster Performance
I love it! I had some half-baked convoluted list-of-tileindexes idea; this is much better. We may have to allow PROJECTION=AUTO for the tiles (in the case where the tiles are in UTM several zones), and allow the tile index to be in a different SRS (e.g geographic) than the tiles (I can't recall if this already implemented; I know it caused me a problem some years ago). A elegant enhancement with great potential... Brent Steve Lime wrote: Interesting idea. This could take the form of a tile index with a couple of additonal columns, minscale and maxscale. Tiles would be grouped together by using common with those values. You could do interesting things like have high resolution data in some areas with other areas covered with lower resolution data over a broader range of scales. The whole layer could have it's own floor/ceiling but tiles would be handled individually. I wouldn't handle this as a new layer type, but rather by introducing parameters to indicate which columns to use, kinda like TILEITEM. Your pyramids would be defined in the tile index... I think the gdal tools would already support this nicely since they add to an index if it already exists so you could run those tools over mutliple datasets. Vector layers could also be handled this way, no reason you couldn't have 1 tile per scale range. Steve On 9/15/2008 at 2:18 PM, in message [EMAIL PROTECTED], Brent Fraser [EMAIL PROTECTED] wrote: Jeff McKenna wrote: my quick comments: - adding overviews with GDAL's 'gdaladdo' utility is very important In some cases. It depends on your data. As Ed once posted, it may be a good idea to switch to external overviews and merge some of the files to limit the number of file-opens Mapserver must do when the view is zoomed way out. - I find your use of the word pyramid confusing (this seems to be a word that Arc* users are familiar with but has no real meaning in the MapServer world...I guess I've been on the good side for too long ha) Not being an Arc* user I can't comment on it's origins. Overview is a good alternative, but it doesn't seem to convey the same multi-levelness as pyramids. For example to be able to display the Global Landsat Mosaic, I created seven levels of a pyramid (each an external overview of the higher-resolution below it). Hmmm, may be the plural of overview is pyramid... :) Hey Steve L., maybe we should have a PYRAMID Layer type, to replace a set of scale-sensitive TILEINDEX Layers (this would help my every-layer-is-exposed-in-WMS problem too: http://trac.osgeo.org/mapserver/ticket/300). Brent ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Different color for each row and it'schild
Sorry LAYER NAME crews GROUP data CONNECTIONTYPE postgis CONNECTION db DATA the_geom from (select prikey, the_geom, crewname || ' ' || last_name as name, color as symcolor from crews) as foo using srid=2236, using unique prikey PROCESSING CLOSE_CONNECTION=DEFER METADATA wms_title Crews wms_group_title data wms_layer_title Crews END PROJECTION init=epsg:2236 END STATUS ON TYPE POINT # MAXSCALE 25 LABELMAXSCALE 25 LABELITEM name CLASS NAME 'crews' # MAXSCALE 25 COLOR [symcolor] BACKGROUNDCOLOR 255 255 255 OUTLINECOLOR 255 255 255 SYMBOL 'crewsymbol' SIZE 25 LABEL COLOR [symcolor] OUTLINECOLOR 255 255 255 TYPE TRUETYPE FONT arial SIZE 9 MAXSIZE 9 MINSIZE 8 ANTIALIAS TRUE POSITION UC PARTIALS FALSE FORCE TRUE END END END -Original Message- From: Steve Lime [mailto:[EMAIL PROTECTED] Sent: Monday, September 15, 2008 3:39 PM To: mapserver-users@lists.osgeo.org; David Fawcett; Lee Keel Subject: RE: [mapserver-users] Different color for each row and it'schild Sharing the snippet of mapfile would help... Steve On 9/15/2008 at 3:35 PM, in message [EMAIL PROTECTED] et, Lee Keel [EMAIL PROTECTED] wrote: David, Thanks for the reply, but I can't seem to get this to work. When I change my mapfile to use this new attribute column, that layer (and most all my others) go away and I get this as the response in FF: HTML HEADTITLEMapServer Message/TITLE/HEAD !-- MapServer version 5.0.0 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP OUTPUT=PDF OUTPUT=SWF OUTPUT =SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS SUPPORTS =GEOS INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE -- BODY BGCOLOR=#FF getSymbol(): Symbol definition error. Parsing error near (symcolor):(line 1684) /BODY/HTML I added the column to my query and add that column name, in [] to my class item. Is there something that I am missing? Please note: I have looked at the http://mapserver.gis.umn.edu/development/rfc/ms-rfc-19/ page and I think I understand what is required. But now I am thinking that this is not in MS4W 2.2.6. Can anyone answer if that is the case? And if so, is there another solution like this around that version? Thanks in advance, Lee From: Fawcett, David [mailto:[EMAIL PROTECTED] Sent: Monday, September 15, 2008 9:51 AM To: Lee Keel; mapserver-users@lists.osgeo.org Subject: RE: [mapserver-users] Different color for each row and it's child I would suggest creating a column in table A to store a color value and pre-populating it. You would want to store the RGB triplet (e.g. '255 0 0'). I assume that you could have some sort of a stored proc that gets triggered when a new record gets added. Using the new attribute binding in MapServer 5, you could then just set the feature color by specifying the column name in the style in your map file layer definition. Something like: STYLE COLOR [myColorColumn] OUTLINECOLOR 0 0 0 END As soon as someone kicks the MapServer site server and it restarts, I suggest looking at the map file reference document under style. Also, make sure that your color column is specified in the query in your data statement. David. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Lee Keel Sent: Monday, September 15, 2008 9:33 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Different color for each row and it's child I had a client request something that sounds really cool, but I don't know how to do it (or if it is even possible in mapserver). First, my setup: I am running MS4W 2.6 on windows xp against postgis 8.2 database. I have table A that has anywhere from 4 to 35 rows. And table B which could have several hundred rows each relating back to one of the rows from table A. For Example: TableA IDName
RE: [mapserver-users] Different color for each row and it'schild
It could just be the documentation, but it looks like the attribute binding may not work at the class level. Wrap your styling info in a STYLE and see if that works. http://mapserver.gis.umn.edu/docs/reference/mapfile/style CLASS NAME 'crews' # MAXSCALE 25 STYLE COLOR [symcolor] BACKGROUNDCOLOR 255 255 255 OUTLINECOLOR 255 255 255 SYMBOL 'crewsymbol' SIZE 25 END # end of style LABEL COLOR [symcolor] OUTLINECOLOR 255 255 255 TYPE TRUETYPE FONT arial SIZE 9 MAXSIZE 9 MINSIZE 8 ANTIALIAS TRUE POSITION UC PARTIALS FALSE FORCE TRUE END END -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Lee Keel Sent: Monday, September 15, 2008 3:43 PM To: Steve Lime; mapserver-users@lists.osgeo.org; David Fawcett Subject: RE: [mapserver-users] Different color for each row and it'schild Sorry LAYER NAME crews GROUP data CONNECTIONTYPE postgis CONNECTION db DATA the_geom from (select prikey, the_geom, crewname || ' ' || last_name as name, color as symcolor from crews) as foo using srid=2236, using unique prikey PROCESSING CLOSE_CONNECTION=DEFER METADATA wms_title Crews wms_group_title data wms_layer_title Crews END PROJECTION init=epsg:2236 END STATUS ON TYPE POINT # MAXSCALE 25 LABELMAXSCALE 25 LABELITEM name CLASS NAME 'crews' # MAXSCALE 25 COLOR [symcolor] BACKGROUNDCOLOR 255 255 255 OUTLINECOLOR 255 255 255 SYMBOL 'crewsymbol' SIZE 25 LABEL COLOR [symcolor] OUTLINECOLOR 255 255 255 TYPE TRUETYPE FONT arial SIZE 9 MAXSIZE 9 MINSIZE 8 ANTIALIAS TRUE POSITION UC PARTIALS FALSE FORCE TRUE END END END -Original Message- From: Steve Lime [mailto:[EMAIL PROTECTED] Sent: Monday, September 15, 2008 3:39 PM To: mapserver-users@lists.osgeo.org; David Fawcett; Lee Keel Subject: RE: [mapserver-users] Different color for each row and it'schild Sharing the snippet of mapfile would help... Steve On 9/15/2008 at 3:35 PM, in message [EMAIL PROTECTED] et, Lee Keel [EMAIL PROTECTED] wrote: David, Thanks for the reply, but I can't seem to get this to work. When I change my mapfile to use this new attribute column, that layer (and most all my others) go away and I get this as the response in FF: HTML HEADTITLEMapServer Message/TITLE/HEAD !-- MapServer version 5.0.0 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP OUTPUT=PDF OUTPUT=SWF OUTPUT =SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS SUPPORTS =GEOS INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE -- BODY BGCOLOR=#FF getSymbol(): Symbol definition error. Parsing error near (symcolor):(line 1684) /BODY/HTML I added the column to my query and add that column name, in [] to my class item. Is there something that I am missing? Please note: I have looked at the http://mapserver.gis.umn.edu/development/rfc/ms-rfc-19/ page and I think I understand what is required. But now I am thinking that this is not in MS4W 2.2.6. Can anyone answer if that is the case? And if so, is there another solution like this around that version? Thanks in advance, Lee From: Fawcett, David [mailto:[EMAIL PROTECTED] Sent: Monday, September 15, 2008 9:51 AM To: Lee Keel; mapserver-users@lists.osgeo.org Subject: RE: [mapserver-users] Different color for each row and it's child I would suggest creating a column in table A to store a color value and pre-populating it. You would want to store the RGB triplet (e.g. '255 0 0'). I assume that you could have some sort of a stored proc that gets triggered when a new record gets added. Using the new attribute binding in MapServer 5, you could then just set the feature color by specifying the column name in the style in your
RE: [mapserver-users] Different color for each row and it'schild
David, Thanks all of your help! That was the trick. Guess I was just making assumptions and you know what they say about that. Sorry for the mix up. -Lee ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] WMS loadparams weird behavior on Ubuntu
William Kyngesburye wrote: I got PHP CGI running and now the WMS request loadparams() works. Apparently WMS doesn't work on PHP as Apache module (my OSX system uses PHP CGI). Or maybe there is a way to make it work? (though I don't really care now that I have PHP CGI running) Seems that it's a known issue related to the missing CGI env vars in the module environment: http://trac.osgeo.org/mapserver/ticket/1975#comment:13 Daniel -- Daniel Morissette http://www.mapgears.com/ ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] ZUM
I will give an extent within or as part of a Layer??? thanks sorry for the English ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Run-time Substitution vs. Variable Substitutionand what are the Parameters Supported?
Variable substitution refers to those parameters that can have part of their value changed remotely. The supported parameters are: LAYER: data, tileindex, connection and filter CLASS: expression Setting up parameter validation is STRONGLY encouraged. In your layer metadata you'd define keys based on variable names to do validation. For example, if your variable was called 'foo' and you wanted to allow a single digit integer from 1 to 9 as a value you'd do: METADATA ... 'foo_validation_pattern' '^[1-9]$' ... END The value of 'foo' would have to pass the supplied regex before the parameter would be altered. It is up to you to author the correct validation pattern. MapServer also supports some URL-based configuration. This is different than substitution in that you are completely changing values of object parameters or even creating new objects. That is, you can change a class color or create a new point feature. Support here (at the moment) is limited to parameters that are validated as part of mapfile parsing. For example, MapServer checks to make sure a color is of the correct format, that a double is a double and so on. The only two exceptions are layer DATA and TEMPLATE properties and they are required to validate against DATAPATTERN or TEMPLATEPATTERN before being used. This is all going to be generalized and improved in 5.4. Steve On 9/9/2008 at 2:42 PM, in message [EMAIL PROTECTED], John Mitchell [EMAIL PROTECTED] wrote: Hi, What is the difference between Run-time Substitution vs. Variable Substitution within a map file? They look like they are the same thing. Also what are the Parameters Supported? From documentation that is almost 3 years old it lists the following parametes (listed below) is this list complete? - LAYER: DATA (must validate against DATAPATTERN) - LAYER: TILEINDEX - LAYER: CONNECTION - LAYER: FILTER - CLASS EXPRESSION Thanks, ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] GRID or not to GRID (USNG question)
All, in the following interface: http://gis.ci.stpaul.mn.us/gis/gismo_public/html/?mapbook=/datasets/CONFIGS/SAINT_PAUL/PUBLIC_WORKS/MAPBOOKS/TS/usng_1st.xml The last two layers in the US National Grid Layers folder show a layer with lines Easting and Northing, that I want to label. They label just fine, but I would like to align them along an edge of the view automatically, the ideal situation would be to have two labels on each end of the lines along the edges of the view. We've tried using the GRID object, even going so modifying the source for GRID, you can see an example in the third to last layer above, USNG via MapServer GRID (Wait for it, it's kinda slow right now) And just can't seem to get the output in any sort of usable result for printing. Question, anyone got any ideas about how to align those labels along the edges? Thanks bobb ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Ed's Rules for the Best Raster Performance
Just out of curiosity, has anyone tested the performance of Jpegs vs. GeoTiffs? I would expect at some point the additional disk access time required for GeoTiffs (of the same pixel count) as Jpegs would outweigh the additional processor time required to decompress the Jpegs. (Also the number of Jpegs that can fit in disk cache is greater than for similar GeoTiffs.) For reference we use 1000px by 1000px Jpeg tiles (with world files). We store multiple resolutions of the dataset, each in its own directory. We start at the native dataset resolution, and half that for each step, stopping when there are less than 10 tiles produced at that particular resolution. (I.e for one of our county wide datasets 6in/px, 1ft/px, 2ft/px, ... 32ft/px). A tileindex is then created for each resolution (using gdaltindex followed by shptree) and a layer is created in the mapfile for each tileindex and appropriate min/maxscales are set. The outputformat in the mapfile is set to jpeg. Our typical tile size is 200KB. There are about 20k tiles in the 6in/px dataset, 80k tiles in the 3in/px dataset (actually 4in data, but stored in 3in so it fits with the rest of the datasets well). I have tested and this large number of files in a directory doesn't seem to effect performance on our system. Average access time for a 500x500px request to mapserver is 300ms measured at the client using perl/LWP and about 220ms with shp2img. Machine is mapserver 5.2.0/x86-64/2.8GHz Xeon/Linux 2.6.16/ext3 filesystem. Jim Klassen City of Saint Paul Fawcett, David [EMAIL PROTECTED] 09/15/08 1:10 PM Better yet, Add your comments to: http://mapserver.gis.umn.edu/docs/howto/optimizeraster and http://mapserver.gis.umn.edu/docs/howto/optimizevector I had always thought that all we needed to do to make these pages great was to grok the list for all of Ed's posts... David. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brent Fraser Sent: Monday, September 15, 2008 12:55 PM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Ed's Rules for the Best Raster Performance In honor of Ed's imminent retirement from the Mapserver Support Group, I've put together Ed's List for the Best Raster Performance: #1. Pyramid the data - use MAXSCALE and MINSCALE in the LAYER object. #2. Tile the data (and merge your upper levels of the pyramid for fewer files). - see the TILEINDEX object #3. Don't compress your data - avoid jpg, ecw, and mrsid formats. #4. Don't re-project your data on-the-fly. #5. Get the fastest disks you can afford. (Ed, feel free to edit...) Brent Fraser ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Ed's Rules for the Best Raster Performance
Jim Klassen wrote: Just out of curiosity, has anyone tested the performance of Jpegs vs. GeoTiffs? Yep. In my tests, GeoTIFF was the fastest format by some margin, even up to 2 GB filesizes. That was on 8-CPU machines, too. If you check the mailing list archive, you'll likely find the papers I posted to the list putting real numbers to it. -- Gregor Mosheh / Greg Allensworth, BS, A+ System Administrator HostGIS cartographic development hosting services http://www.HostGIS.com/ Remember that no one cares if you can back up, only if you can restore. - AMANDA ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Ed's Rules for the Best Raster Performance
Has anyone tried Erdas Imagine .img files? Robert Sanson, BVSc, MACVSc, PhD Geospatial Services AsureQuality Limited PO Box 585, Palmerston North NEW ZEALAND Phone: +64 6 351-7990 Fax: +64 6 351-7919 Mobile: 021 448-472 E-mail: [EMAIL PROTECTED] Gregor Mosheh [EMAIL PROTECTED] 16/09/2008 10:57 a.m. Jim Klassen wrote: Just out of curiosity, has anyone tested the performance of Jpegs vs. GeoTiffs? Yep. In my tests, GeoTIFF was the fastest format by some margin, even up to 2 GB filesizes. That was on 8-CPU machines, too. If you check the mailing list archive, you'll likely find the papers I posted to the list putting real numbers to it. -- Gregor Mosheh / Greg Allensworth, BS, A+ System Administrator HostGIS cartographic development hosting services http://www.HostGIS.com/ Remember that no one cares if you can back up, only if you can restore. - AMANDA ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users -- The contents of this email are confidential to AsureQuality. If you have received this communication in error please notify the sender immediately and delete the message and any attachments. The opinions expressed in this email are not necessarily those of AsureQuality. This message has been scanned for known viruses before delivery. AsureQuality supports the Unsolicited Electronic Messages Act 2007. If you do not wish to receive similar communications in future, please notify the sender of this message. -- This message has been scanned for malware by SurfControl plc. www.surfcontrol.com ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] How to improve raster imagery display ?
Christopher Schmidt wrote: Er, it looks like you didn't use gzip to unpack it yet? gzipped is like zipped... You'll need to unzip... (doing that works for me.) Regards, Thanks for the reply. I'm stymied by this. I was using WinZip, which uncompressed with no error messages, but the file was apparently corrupted. Now I tried gzip32.exe in a command prompt and get the exact same results. I am using the -d switch. Very peculiar because I know I have successfully done this before with other gzip files. Oh well, if it works every time - it isn't a computer :-) Mike This is the beginning of the file I get: ‹Þº’Hÿ/var/lib/mailman/archives/private/mapserver-users/2008-July.txt ì½yâF¶0üwëS Ôxò´ñ v¦ [EMAIL PROTECTED], ™éÜÛNÇ©ö:uêìçÔuFâÜœºÖØ×Û¦õ»0}ѳ ¬¡3™X#xšê M]³3´ô�3}Ðû®·SKü0 ‘™l5_®²ÂÈdÊÚ)4VÝ°µÄaýÃíiª\ÿ©~híFó?ïdw‡ÆÉÝqó¶ywqxœ;lTz¶o¹î¡Ñ4N [EMAIL PROTECTED] ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Ed's Rules for the Best Raster Performance
Jim, you would think that ;) However, in practice I wouldn't expect the disk access time for geotiffs to be significantly different from jpeg if you have properly optimized your geotiffs using gdal_translate -co TILED=YES - the internal structure is efficiently indexed so that gdal only has to read the minimum number of 256x256 blocks to cover the requested extent. And using gdaladdo to generate overviews just makes it that much more efficient. Even if you are reading less physical data from the disk to get the equivalent coverage from jpeg, the decompression overhead is enough to negate the difference in IO time based on Ed's oft quoted advice (and other's experience too I think). The rules that apply in this case seem to be 'tile your data', 'do not compress it' and 'buy the fastest disk you can afford'. Compression is useful and probably necessary if you hit disk space limits. Cheers Paul On 15-Sep-08, at 5:48 PM, Jim Klassen wrote: Just out of curiosity, has anyone tested the performance of Jpegs vs. GeoTiffs? I would expect at some point the additional disk access time required for GeoTiffs (of the same pixel count) as Jpegs would outweigh the additional processor time required to decompress the Jpegs. (Also the number of Jpegs that can fit in disk cache is greater than for similar GeoTiffs.) For reference we use 1000px by 1000px Jpeg tiles (with world files). We store multiple resolutions of the dataset, each in its own directory. We start at the native dataset resolution, and half that for each step, stopping when there are less than 10 tiles produced at that particular resolution. (I.e for one of our county wide datasets 6in/px, 1ft/px, 2ft/px, ... 32ft/px). A tileindex is then created for each resolution (using gdaltindex followed by shptree) and a layer is created in the mapfile for each tileindex and appropriate min/maxscales are set. The outputformat in the mapfile is set to jpeg. Our typical tile size is 200KB. There are about 20k tiles in the 6in/ px dataset, 80k tiles in the 3in/px dataset (actually 4in data, but stored in 3in so it fits with the rest of the datasets well). I have tested and this large number of files in a directory doesn't seem to effect performance on our system. Average access time for a 500x500px request to mapserver is 300ms measured at the client using perl/LWP and about 220ms with shp2img. Machine is mapserver 5.2.0/x86-64/2.8GHz Xeon/Linux 2.6.16/ext3 filesystem. Jim Klassen City of Saint Paul Fawcett, David [EMAIL PROTECTED] 09/15/08 1:10 PM Better yet, Add your comments to: http://mapserver.gis.umn.edu/docs/howto/optimizeraster and http://mapserver.gis.umn.edu/docs/howto/optimizevector I had always thought that all we needed to do to make these pages great was to grok the list for all of Ed's posts... David. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brent Fraser Sent: Monday, September 15, 2008 12:55 PM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Ed's Rules for the Best Raster Performance In honor of Ed's imminent retirement from the Mapserver Support Group, I've put together Ed's List for the Best Raster Performance: #1. Pyramid the data - use MAXSCALE and MINSCALE in the LAYER object. #2. Tile the data (and merge your upper levels of the pyramid for fewer files). - see the TILEINDEX object #3. Don't compress your data - avoid jpg, ecw, and mrsid formats. #4. Don't re-project your data on-the-fly. #5. Get the fastest disks you can afford. (Ed, feel free to edit...) Brent Fraser ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users __ Paul Spencer Chief Technology Officer DM Solutions Group Inc http://www.dmsolutions.ca/ ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Ed's Rules for the Best Raster Performance
Damn, I'm going to have to get around to unsubscribing soon so I can shut myself up! Jim, please remember that your disk subsystem does not read only the precise amount of data you request. The most expensive step is telling the disk head to seek to a random location to start reading the data. The actual reading takes much less time in almost every case. Let's invent an example so we don't have to do too much hard research g. A 7,200-RPM IDE drive has about a 9 ms average read seek time, and most are able to really transfer real data at around 60 MB/s or so (these are very rough approximations). So to read 256KB of sequential data, you spend 9 ms seeking to the right track and then 4 ms reading the data - that's 13 ms. Doubling the read size to 512KB will only take 4 ms (or 30%) longer, not 100% longer. But even that's likely to be an exaggeration, because your disk drive - knowing that seeks are expensive - will typically read a LOT of data after doing a seek. Remember that 16MB buffer on the package? The drive will likely read far more than you need, so the improvement you get by cutting the amount of data read in a given seek in half is likely to be nothing at all. There are limits, of course. The larger your data read is, the more likely it is to be split up into more than one location on disk. That would mean another seek, which would definitely hurt. But in general if you're already reading modest amounts of data in each shot, reducing the amount of data read by compression is likely to save you almost nothing in read time and cost you something in decompression time (CPUs are fast, so it might not cost much, but it will very likely require more RAM, boosting your per-request footprint, which means you're more at risk of starting to swap, etc.). And remember that not all formats are created equal. In order to decompress ANY portion of a JPEG image, you must read the WHOLE file. If I have a 4,000x4,000 pixel 24-bit TIFF image that's 48 megabytes, and I want to read a 256x256 piece of it, I may only need to read one megabyte or less of that file. But if I convert it to a JPEG and compress it to only 10% of the TIFF's size, I'll have a 4.8 megabyte JPEG but I will need to read the whole 4.8 megabytes (and expand it into that RAM you're trying to conserve) in order to get that 256x256 piece! Paul is right - sometimes compression is necessary when you run out of disk (but disks are pretty darn cheap - the cost per megabyte of the first hard drive I ever purchased (a Maynard Electronics 10 MB drive for my IBM PC) is approximately 450,000 times higher than it is today). If you are inclined toward JPEG compression, read about and think about using tiled TIFFs with JPEG compression in the tiles; it's a reasonable compromise that saves space while reducing the whole-file-read overhead of JPEG. Where the heck is that unsubscribe button? - Ed On 9/15/08 9:23 PM, Paul Spencer [EMAIL PROTECTED] wrote: Jim, you would think that ;) However, in practice I wouldn't expect the disk access time for geotiffs to be significantly different from jpeg if you have properly optimized your geotiffs using gdal_translate -co TILED=YES - the internal structure is efficiently indexed so that gdal only has to read the minimum number of 256x256 blocks to cover the requested extent. And using gdaladdo to generate overviews just makes it that much more efficient. Even if you are reading less physical data from the disk to get the equivalent coverage from jpeg, the decompression overhead is enough to negate the difference in IO time based on Ed's oft quoted advice (and other's experience too I think). The rules that apply in this case seem to be 'tile your data', 'do not compress it' and 'buy the fastest disk you can afford'. Compression is useful and probably necessary if you hit disk space limits. Cheers Paul On 15-Sep-08, at 5:48 PM, Jim Klassen wrote: Just out of curiosity, has anyone tested the performance of Jpegs vs. GeoTiffs? I would expect at some point the additional disk access time required for GeoTiffs (of the same pixel count) as Jpegs would outweigh the additional processor time required to decompress the Jpegs. (Also the number of Jpegs that can fit in disk cache is greater than for similar GeoTiffs.) For reference we use 1000px by 1000px Jpeg tiles (with world files). We store multiple resolutions of the dataset, each in its own directory. We start at the native dataset resolution, and half that for each step, stopping when there are less than 10 tiles produced at that particular resolution. (I.e for one of our county wide datasets 6in/px, 1ft/px, 2ft/px, ... 32ft/px). A tileindex is then created for each resolution (using gdaltindex followed by shptree) and a layer is created in the mapfile for each tileindex and appropriate min/maxscales are set. The outputformat in the mapfile is set to jpeg. Our typical tile size is 200KB.
