[mapserver-users] Validation example wanted
Hi, Could somebody tell how can I make my WMS requests to validate? I am sending WMS GetMaps with two extra parameters tag and value by adding them to the end of the request like tag=amenityvalue=supermarket. Tag and value can be anything and they should be used inside the following LAYER-DATA DATA way from (select (way),osm_id,tags from osm_point where (tags @'%tag%=%value%')) as foo using unique osm_id using srid=3067 I have tried to put a validation block in LAYER VALIDATION tag /./ value /./ END I have also tried to add these two lines into WEB-METADATA tag_validation_pattern /./ value_validation_pattern /./ Because I am here you can guess that my trials were not successful. I am using MS 6.0 beta7. Queries with tag and value worked with 5.6.1 version. -Jukka Rahkonen- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Validation example wanted
have you tried without the surrounding slashes? i.e. VALIDATION tag . value . END On Fri, Apr 29, 2011 at 10:37, Rahkonen Jukka jukka.rahko...@mmmtike.fi wrote: Hi, Could somebody tell how can I make my WMS requests to validate? I am sending WMS GetMaps with two extra parameters tag and value by adding them to the end of the request like tag=amenityvalue=supermarket. Tag and value can be anything and they should be used inside the following LAYER-DATA DATA way from (select (way),osm_id,tags from osm_point where (tags @'%tag%=%value%')) as foo using unique osm_id using srid=3067 I have tried to put a validation block in LAYER VALIDATION tag /./ value /./ END I have also tried to add these two lines into WEB-METADATA tag_validation_pattern /./ value_validation_pattern /./ Because I am here you can guess that my trials were not successful. I am using MS 6.0 beta7. Queries with tag and value worked with 5.6.1 version. -Jukka Rahkonen- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Validation example wanted
Hi, Great, it works with plain . I was just reading a mail from yesterday by Stephen Woodbridge telling I do not think there is a global OFF switch for validation, but where validation is required you can include the regex validation string of /./ which means match anything except a null string, or to also accept a null string then use /.*/ You can see the supermarkets here: http://188.64.1.61/cgi-bin/mapserv?map=/usr/map/osm-mapserver_i.mapREQUEST=GetMapSERVICE=WMSVERSION=1.1.1WIDTH=1059HEIGHT=687LAYERS=sea,borders,POI_free_queryTRANSPARENT=TRUEFORMAT=image%2FpngBBOX=-574187,6558020,1615375,7978444SRS=EPSG:3067STYLES=tag=shopvalue=supermarket -Jukka- thomas bonfort wrote: have you tried without the surrounding slashes? i.e. VALIDATION tag . value . END On Fri, Apr 29, 2011 at 10:37, Rahkonen Jukka jukka.rahko...@mmmtike.fi wrote: Hi, Could somebody tell how can I make my WMS requests to validate? I am sending WMS GetMaps with two extra parameters tag and value by adding them to the end of the request like tag=amenityvalue=supermarket. Tag and value can be anything and they should be used inside the following LAYER-DATA DATA way from (select (way),osm_id,tags from osm_point where (tags @'%tag%=%value%')) as foo using unique osm_id using srid=3067 I have tried to put a validation block in LAYER VALIDATION tag /./ value /./ END I have also tried to add these two lines into WEB-METADATA tag_validation_pattern /./ value_validation_pattern /./ Because I am here you can guess that my trials were not successful. I am using MS 6.0 beta7. Queries with tag and value worked with 5.6.1 version. -Jukka Rahkonen- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] RE: Validation beyond [A-z]
Thanks Steve. I don't understand the syntax: in the regex versions I use, . means one and just one character, not any string. Any string excluding the null-string would be .+ or ..* More generally, I still have problems with validation as a concept: it's too difficult, and perhaps that's why it isn't documented. I have postings about this subject in my personal mapserver-dev mail-archive dating from 2002 (couldn't find them on the official site any more), and there still isn't a clear solution almost ten years later. IMHO the major security risk of MapServer CGI is that it gives access to the filesystem outside the web-root. Wouldn't it be better to keep security at that level, i.e. only let MapServer access explicitly defined parts of the filesystem? Within these parts, it's up to the web-site builder to put only those things that should be visible and nothing else. You don't put an ultra-secret document on the web and afterward restrict access to portions only, you just put there what you want to show to the world. Same goes for validations on extent or styles: just make your selections of what you want to show before you let MapServer loose on it. The same story can be told for database access and restrictions on SQL queries: IMHO that is a matter for the database system. It's easy enough to put everything behind barriers with user privileges and views. Why should mapserver double all that security? Any competent database administrator should know how to prevent SQL injects, and MapServer should not be there to protect those who are unable to. I've been working with Cloud VM's for about a year now, and in that environment many security problems disappear: just make small, dedicated servers and interconnect them, e.g. with cascading services. So my view would be: let the Operating System and the Database do everything needed to secure files and databases, and put in your web services only afterwards. It all gets too complex with all those interconnected securities at all levels of the system (my main problem with Apache). In the last ressort: KISS (Keep It Simple Steve (whoever)) :-) Jan - On 04/28/11 18:48, Steve Woodbridge wrote Hi Jan, I do not think there is a global OFF switch for validation, but where validation is required you can include the regex validation string of /./ which means match anything except a null string, or to also accept a null string then use /.*/ You still need to be aware of when you should put a validation in place even if it accepts any string. Regards, -Steve W On 04/28/11 18:48, Jan Hartmann wrote: I find the whole validation issue difficult and not well documented (http://trac.osgeo.org/mapserver/ticket/3754, last updated four hours ago). How do I put all validation off? I really don need that much security. Jan On 04/28/11 18:19, Lime, Steve D (DNR) wrote: I see the problem, just not sure how to fix it. Steve W. provided some possibilities but that's probably not the only approach. It would be helpful if some interested person(s) got together and drafted an RFC. I think the devs would be in a position to help define implementation details if the problem is well defined along with a proposed solution. Steve *From:*mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] *On Behalf Of *Rahkonen Jukka *Sent:* Thursday, April 28, 2011 4:21 AM *To:* mapserver-users@lists.osgeo.org *Subject:* [mapserver-users] Validation beyond [A-z] Hi, Validation is nowadays needed in quite a many places in a mapfile. However, we who live outside the English speaking world tend to have more characters in the alphabet than A to Z. This makes the mapfile validation idea only half effective because for making things to work at all with the native data we must accept almost everything that is non-numeric with wildcards. Are there others who think that this is a problem? Stephen Woodbidge commented slightly this topic in another thead a month ago (Mar 29, 2011) http://lists.osgeo.org/pipermail/mapserver-users/2011-March/068307.html -Jukka Rahkonen- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] RE: Validation beyond [A-z]
Hi, I am not so sure that we all are competent database administrators. Lots of us has started with tutorials like http://www.duif.net/postgis/ and they do not tell much about fighting against SQL injection. I consider it is good to have security settings in Mapserver. I admit they should be easy to use and well documented. -Jukka Rahkonen- -Alkuperäinen viesti- Lähettäjä: mapserver-users-boun...@lists.osgeo.org puolesta: Jan Hartmann Lähetetty: pe 29.4.2011 13:37 Vastaanottaja: wood...@swoodbridge.com Kopio: Rahkonen Jukka; mapserver-users@lists.osgeo.org Aihe: Re: [mapserver-users] RE: Validation beyond [A-z] Thanks Steve. I don't understand the syntax: in the regex versions I use, . means one and just one character, not any string. Any string excluding the null-string would be .+ or ..* More generally, I still have problems with validation as a concept: it's too difficult, and perhaps that's why it isn't documented. I have postings about this subject in my personal mapserver-dev mail-archive dating from 2002 (couldn't find them on the official site any more), and there still isn't a clear solution almost ten years later. IMHO the major security risk of MapServer CGI is that it gives access to the filesystem outside the web-root. Wouldn't it be better to keep security at that level, i.e. only let MapServer access explicitly defined parts of the filesystem? Within these parts, it's up to the web-site builder to put only those things that should be visible and nothing else. You don't put an ultra-secret document on the web and afterward restrict access to portions only, you just put there what you want to show to the world. Same goes for validations on extent or styles: just make your selections of what you want to show before you let MapServer loose on it. The same story can be told for database access and restrictions on SQL queries: IMHO that is a matter for the database system. It's easy enough to put everything behind barriers with user privileges and views. Why should mapserver double all that security? Any competent database administrator should know how to prevent SQL injects, and MapServer should not be there to protect those who are unable to. I've been working with Cloud VM's for about a year now, and in that environment many security problems disappear: just make small, dedicated servers and interconnect them, e.g. with cascading services. So my view would be: let the Operating System and the Database do everything needed to secure files and databases, and put in your web services only afterwards. It all gets too complex with all those interconnected securities at all levels of the system (my main problem with Apache). In the last ressort: KISS (Keep It Simple Steve (whoever)) :-) Jan - On 04/28/11 18:48, Steve Woodbridge wrote Hi Jan, I do not think there is a global OFF switch for validation, but where validation is required you can include the regex validation string of /./ which means match anything except a null string, or to also accept a null string then use /.*/ You still need to be aware of when you should put a validation in place even if it accepts any string. Regards, -Steve W On 04/28/11 18:48, Jan Hartmann wrote: I find the whole validation issue difficult and not well documented (http://trac.osgeo.org/mapserver/ticket/3754, last updated four hours ago). How do I put all validation off? I really don need that much security. Jan On 04/28/11 18:19, Lime, Steve D (DNR) wrote: I see the problem, just not sure how to fix it. Steve W. provided some possibilities but that's probably not the only approach. It would be helpful if some interested person(s) got together and drafted an RFC. I think the devs would be in a position to help define implementation details if the problem is well defined along with a proposed solution. Steve *From:*mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] *On Behalf Of *Rahkonen Jukka *Sent:* Thursday, April 28, 2011 4:21 AM *To:* mapserver-users@lists.osgeo.org *Subject:* [mapserver-users] Validation beyond [A-z] Hi, Validation is nowadays needed in quite a many places in a mapfile. However, we who live outside the English speaking world tend to have more characters in the alphabet than A to Z. This makes the mapfile validation idea only half effective because for making things to work at all with the native data we must accept almost everything that is non-numeric with wildcards. Are there others who think that this is a problem? Stephen Woodbidge commented slightly this topic in another thead a month ago (Mar 29, 2011) http://lists.osgeo.org/pipermail/mapserver-users/2011-March/068307.html -Jukka Rahkonen- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org
Re: [mapserver-users] free() methods in MapScript, compatibility MS 5.x and 6
Hi Armin, Although I'm not sure if the free call was useful... since it was only removing the php object from a global list but the object was not really freed until the end of the script... I would suggest to simply add a such condition if ms_version 6 ... obj-free() rather than removing them to be compatible 5.6/6.0 and avoid any issue. You can also do a quick test in your script with some echo of the memory used before the end of the script and see if the free() removal has any effect. Alan On April 27, 2011 01:25:54 pm Armin Burger wrote: Hi everybody in MapServer/MapScript 6 the various class methods free() have been removed. If I want to have PHP/MapScript code compatible with both MapServer 5.x and 6, the easiest would be to completely remove these method calls from the code. My question is just if these free() methods are still somehow required or recommended for MapServer/MapScript 5.x or if I can silently drop them without any drawbacks. Cheers Armin ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users -- Alan Boudreault Mapgears http://www.mapgears.com ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Force Mapserver into rendering 8bpp png images (instead of 4bpp)?
Dear All, former subject of this thread was Re: NoDATA value, transparency for paletted IMG file - I'm having difficulties displaying 4bit-per-pixel PNG images served via WMS to a GoogleEarth-API-Application (the NoData-Values aren't transparent but shaded black, whereas this worls well for 8bpp PNG images- so I assume the GE-API can't display transparency for 4bpp PNGs. Is there a way to force mapserver into the creation of 8bpp PNG files, even if the corresponding color table only has 9 entries (I think that's the reason for this)? Thanks for your help and greetings! Chris On Thu, Apr 28, 2011 at 12:42 PM, ChrisFern [via OSGeo.org] ml-node+6312790-588073264-302...@n2.nabble.com wrote: Dear Frank, dear all, thanks for your reply. I think I have narrowed down the problem and it isn't really a mapserver issue - the 8bit paletted img gets rendered to a 4bit-per-pixel png whereas the correctly transparent RGB img gets rendered to an 8bit-per-pixel png. Seems that the Google Earth API can't detect/deal with the NoData value in this 4bpp png and renders this value sort of shaded black (which I would like to get ridd of). GIMP for example can deal with the transparency in this 4bpp png. If I switch to RGBA in the output format, then I have the transparency however the image is much bigger: OUTPUTFORMAT NAME png DRIVER GD/PNG MIMETYPE image/png # IMAGEMODE RGBA IMAGEMODE PC256 EXTENSION png TRANSPARENT ON FORMATOPTION transparent=on END Is there a way I can force the mapserver to produce 8bpp pngs? THANKS for your help - highly appreciated - and greetings, Chris PS here's the gdalinfo output for the 4bpp png: Driver: PNG/Portable Network Graphics Files: lb_usa.png Size is 1024, 768 Coordinate System is `' Corner Coordinates: Upper Left ( 0.0, 0.0) Lower Left ( 0.0, 768.0) Upper Right ( 1024.0, 0.0) Lower Right ( 1024.0, 768.0) Center ( 512.0, 384.0) Band 1 Block=1024x1 Type=Byte, ColorInterp=Palette NoData Value=0 Image Structure Metadata: NBITS=4 Color Table (RGB with 9 entries) 0: 1,1,1,0 1: 0,0,0,255 2: 12,16,120,255 3: 31,62,140,255 4: 33,110,158,255 5: 42,156,154,255 6: 61,184,104,255 7: 59,214,45,255 8: 113,235,47,255 If you reply to this email, your message will be added to the discussion below: http://osgeo-org.1803224.n2.nabble.com/NoDATA-value-transparency-for-paletted-IMG-file-tp6305135p6312790.html To unsubscribe from NoDATA value, transparency for paletted IMG file, click here. ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Mapserver usage with commercial vendors data
Hi, I work for an LBS based company, we have our own proprietary rendering engine for producing maps, and we work mainly with data from Navteq and Teleatlas. Presently our rendering engine is behind the competition in terms of visual quality (we have a bad support for antialiasing, label names with both native and transliterate names are missing, and so on). Introducing new features in our current rendering architecture would require quite a lot of coding and re-engineering, so we are looking for alternative renderers, possibly open sourced. During this research project I came across Mapserver, and it seems it would suit our needs in terms of high quality rendering and customization. However, there are still some open issues, mainly questions, I'd like to ask. In regards to rendering: 1) Is there any way to align labels in different encondings for the same city? I mean something like writing ??? and /Moskvá /vertically aligned, like on Google Maps. 2) Is there any plan to support 2.5D rendering for buildings? In regards to working with high loads of data: 1) We render our maps from data provided by vendors like Navteq, and they have a lot of details and features. Is there anyone working in the same field, who could share some of his experience? 2) Is it more efficient to work with PostGis/Oracle Spatial or with shapefiles? I suppose the former would be faster, since shapefiles provided by Navteq would require about 100gigs for Europe only, just to store the data. 3) In regards to the hardware, I reckon we would need at least one workstation dedicated to rendering. Currently we are hosting our rendering service on dual Xeon (quad core), with 16G of ram and SAS arrays of hard disks, would one server like this be ok or would it be better to have more machines, especially if planning to use RDBMS to hold the data? I'm asking this because with currently work with detailed data from Europe, North and South America, so it's quite a lot of stuff :) Thanks in advance for any feedback. Regards, Paolo Crosato -- Paolo Crosato Ubiest SPA http://www.ubiest.com ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Mapserver usage with commercial vendors data
Paolo,
On Fri, Apr 29, 2011 at 15:19, Paolo Crosato paolo.cros...@ubiest.com wrote:
Hi,
I work for an LBS based company, we have our own proprietary rendering
engine for producing maps, and we work mainly with data from Navteq and
Teleatlas. Presently our rendering engine is behind the competition in terms
of visual quality (we have a bad support for antialiasing, label names with
both native and transliterate names are missing, and so on). Introducing new
features in our current rendering architecture would require quite a lot of
coding and re-engineering, so we are looking for alternative renderers,
possibly open sourced. During this research project I came across Mapserver,
and it seems it would suit our needs in terms of high quality rendering and
customization.
However, there are still some open issues, mainly questions, I'd like to
ask.
In regards to rendering:
1) Is there any way to align labels in different encondings for the same
city? I mean something like writing Москва́ and Moskvá vertically aligned,
like on Google Maps.
MapServer itself currently only supports a single label per feature,
but it does support label wrapping on specific characters so you can
pass it both Москва́ and Moskvá with a bit of database scripting.
You'd probably have to use utf8 encoding and have a font that supports
all international characters, but that shouldn't be a problem.
Extending mapserver so it supports multiple labels per feature could
also be a possible solution.
for the wrapping method, supposing you have a label_int (Москва́) and
label_en (Moscow) fields, you'd use something like select id,
the_geom, label_int||'#('||label_en||')' as label from mytable in
your data statement, and then
LABELITEM label
LABEL
ENCODING UTF8
WRAP '#'
ALIGN CENTER
END
2) Is there any plan to support 2.5D rendering for buildings?
There's a bug open for that in the bug tracker, with no concrete
follow-up recently due to lack of funding and/or developper interest.
Adding such support would be feasible.
In regards to working with high loads of data:
1) We render our maps from data provided by vendors like Navteq, and they
have a lot of details and features. Is there anyone working in the same
field, who could share some of his experience?
building a relatively complete mapfile for navteq data isn't a
daunting task, although getting everything to display correctly at all
scales can be time-consuming.
2) Is it more efficient to work with PostGis/Oracle Spatial or with
shapefiles? I suppose the former would be faster, since shapefiles provided
by Navteq would require about 100gigs for Europe only, just to store the
data.
Indexed shapefiles will be slightly faster than postgis, but the
flexibility you gain by being able to do complex queries with
postgis (like ordering the data to get most important ones to show up
first, etc...) is largely worth the slight overhead. I'd say that if
you want high quality map rendering, postgis is going to be a must,
otherwise shapefiles will do. (I have no experience with oracle)
3) In regards to the hardware, I reckon we would need at least one
workstation dedicated to rendering. Currently we are hosting our rendering
service on dual Xeon (quad core), with 16G of ram and SAS arrays of hard
disks, would one server like this be ok or would it be better to have more
machines, especially if planning to use RDBMS to hold the data? I'm asking
this because with currently work with detailed data from Europe, North and
South America, so it's quite a lot of stuff :)
From my tests in these scenarios, the db backend is the bottleneck, so
you can beaf that one up as much as you can (i.e. lots of ram and ssd
disks). A fast cpu and a reasonable amount of memory for the rendering
host can do no harm, although that is a less important factor from my
testing.
regards,
thomas
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Mapserver usage with commercial vendors data
Paolo,
You can also do labeling via the TEXT directive with something like
TEXT [LABEL_INT]#[LABEL_EN] #MapServer 6 syntax
WRAP #
Performance on Oracle Spatial (which we use for national and international
scale datasets, including NavTeq and TomTom) is quite good.
Mike
--
Michael Smith
Remote Sensing/GIS Center
US Army Corps of Engineers
Hanover, NH
On 4/29/11 10:03 AM, thomas bonfort thomas.bonf...@gmail.com wrote:
Paolo,
On Fri, Apr 29, 2011 at 15:19, Paolo Crosato paolo.cros...@ubiest.com wrote:
Hi,
I work for an LBS based company, we have our own proprietary rendering
engine for producing maps, and we work mainly with data from Navteq and
Teleatlas. Presently our rendering engine is behind the competition in terms
of visual quality (we have a bad support for antialiasing, label names with
both native and transliterate names are missing, and so on). Introducing new
features in our current rendering architecture would require quite a lot of
coding and re-engineering, so we are looking for alternative renderers,
possibly open sourced. During this research project I came across Mapserver,
and it seems it would suit our needs in terms of high quality rendering and
customization.
However, there are still some open issues, mainly questions, I'd like to
ask.
In regards to rendering:
1) Is there any way to align labels in different encondings for the same
city? I mean something like writing Москва́ and Moskvá vertically aligned,
like on Google Maps.
MapServer itself currently only supports a single label per feature,
but it does support label wrapping on specific characters so you can
pass it both Москва́ and Moskvá with a bit of database scripting.
You'd probably have to use utf8 encoding and have a font that supports
all international characters, but that shouldn't be a problem.
Extending mapserver so it supports multiple labels per feature could
also be a possible solution.
for the wrapping method, supposing you have a label_int (Москва́) and
label_en (Moscow) fields, you'd use something like select id,
the_geom, label_int||'#('||label_en||')' as label from mytable in
your data statement, and then
LABELITEM label
LABEL
ENCODING UTF8
WRAP '#'
ALIGN CENTER
END
2) Is there any plan to support 2.5D rendering for buildings?
There's a bug open for that in the bug tracker, with no concrete
follow-up recently due to lack of funding and/or developper interest.
Adding such support would be feasible.
In regards to working with high loads of data:
1) We render our maps from data provided by vendors like Navteq, and they
have a lot of details and features. Is there anyone working in the same
field, who could share some of his experience?
building a relatively complete mapfile for navteq data isn't a
daunting task, although getting everything to display correctly at all
scales can be time-consuming.
2) Is it more efficient to work with PostGis/Oracle Spatial or with
shapefiles? I suppose the former would be faster, since shapefiles provided
by Navteq would require about 100gigs for Europe only, just to store the
data.
Indexed shapefiles will be slightly faster than postgis, but the
flexibility you gain by being able to do complex queries with
postgis (like ordering the data to get most important ones to show up
first, etc...) is largely worth the slight overhead. I'd say that if
you want high quality map rendering, postgis is going to be a must,
otherwise shapefiles will do. (I have no experience with oracle)
3) In regards to the hardware, I reckon we would need at least one
workstation dedicated to rendering. Currently we are hosting our rendering
service on dual Xeon (quad core), with 16G of ram and SAS arrays of hard
disks, would one server like this be ok or would it be better to have more
machines, especially if planning to use RDBMS to hold the data? I'm asking
this because with currently work with detailed data from Europe, North and
South America, so it's quite a lot of stuff :)
From my tests in these scenarios, the db backend is the bottleneck, so
you can beaf that one up as much as you can (i.e. lots of ram and ssd
disks). A fast cpu and a reasonable amount of memory for the rendering
host can do no harm, although that is a less important factor from my
testing.
regards,
thomas
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Parameterized MapFile using CGI variable
Dear All, I have 2 MapServer servers. One for Production use, and the other for Validation use. The MAP files placed on those servers are similar except that each server has a different Oracle database as datasource. For now, I have some different CONNECTION values depending if the map file is on the Production or on the Validation server. So finally I must have different MAP files. Is there a way to have exactly the same map files on the 2 servers, and to use a variable to parameterize the CONNECTION. For example, I would like to set the CONNECTION of my layers to %ORACLECONNECTIONSTRING% and to set a CGI Environment variable named ORACLECONNECTIONSTRING to the correct value on each server. Nicolas ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Parameterized MapFile using CGI variable
One potential way to do it would be with the use of 'include's. http://mapserver.org/mapfile/Include.html The primary mapfile wouldn't have a connection object, but it would contain: INCLUDE connect_info.map Then on each machine, you would have a different version of connect_info.map that contained the correct connection object for that machine. This wouldn't give you the flexibility of changing it through the CGI interface, but would allow you to use the same exact mapfile on both machines. David. From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Nicolas BOUTET Sent: Friday, April 29, 2011 8:57 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Parameterized MapFile using CGI variable Dear All, I have 2 MapServer servers. One for Production use, and the other for Validation use. The MAP files placed on those servers are similar except that each server has a different Oracle database as datasource. For now, I have some different CONNECTION values depending if the map file is on the Production or on the Validation server. So finally I must have different MAP files. Is there a way to have exactly the same map files on the 2 servers, and to use a variable to parameterize the CONNECTION. For example, I would like to set the CONNECTION of my layers to %ORACLECONNECTIONSTRING% and to set a CGI Environment variable named ORACLECONNECTIONSTRING to the correct value on each server. Nicolas ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Parameterized MapFile using CGI variable
Nicolas, I didn't have a mapfile that utilizes connection handy, but I did test using INCLUDE to substitute the value for DATA in an individual layer. Here is part of the layer definition: LAYER NAME watersheds STATUS DEFAULT #DATA 'majclip' INCLUDE inc_data_majclip.map TYPE POLYGON The file inc_data_majclip.map contains: DATA majclip Based on this, I would assume that you should be able to do the same with the CONNECTION for each of your layers. David. From: Nicolas BOUTET [mailto:nicolas.bou...@geosys.com] Sent: Friday, April 29, 2011 9:49 AM To: Fawcett, David (MPCA); mapserver-users@lists.osgeo.org Subject: RE: [mapserver-users] Parameterized MapFile using CGI variable Thank you David, Yes, using INCLUDE should be a good solution. But as I know in MapServer 5.6, CONNECTION is a parameter of my LAYER objects, and not an object itself. So I cannot have a CONNECTION object declared in a separate map file... And I prefer to not have to duplicate the layers in different map files. Nicolas De : Fawcett, David (MPCA) [mailto:david.fawc...@state.mn.us] Envoyé : vendredi 29 avril 2011 16:37 À : nicolas.bou...@geosys.com; mapserver-users@lists.osgeo.org Objet : RE: [mapserver-users] Parameterized MapFile using CGI variable One potential way to do it would be with the use of 'include's. http://mapserver.org/mapfile/Include.html The primary mapfile wouldn't have a connection object, but it would contain: INCLUDE connect_info.map Then on each machine, you would have a different version of connect_info.map that contained the correct connection object for that machine. This wouldn't give you the flexibility of changing it through the CGI interface, but would allow you to use the same exact mapfile on both machines. David. From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Nicolas BOUTET Sent: Friday, April 29, 2011 8:57 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Parameterized MapFile using CGI variable Dear All, I have 2 MapServer servers. One for Production use, and the other for Validation use. The MAP files placed on those servers are similar except that each server has a different Oracle database as datasource. For now, I have some different CONNECTION values depending if the map file is on the Production or on the Validation server. So finally I must have different MAP files. Is there a way to have exactly the same map files on the 2 servers, and to use a variable to parameterize the CONNECTION. For example, I would like to set the CONNECTION of my layers to %ORACLECONNECTIONSTRING% and to set a CGI Environment variable named ORACLECONNECTIONSTRING to the correct value on each server. Nicolas ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Parameterized MapFile using CGI variable
Thank you David, Yes, using INCLUDE should be a good solution. But as I know in MapServer 5.6, CONNECTION is a parameter of my LAYER objects, and not an object itself. So I cannot have a CONNECTION object declared in a separate map file And I prefer to not have to duplicate the layers in different map files. Nicolas De : Fawcett, David (MPCA) [mailto:david.fawc...@state.mn.us] Envoyé : vendredi 29 avril 2011 16:37 À : nicolas.bou...@geosys.com; mapserver-users@lists.osgeo.org Objet : RE: [mapserver-users] Parameterized MapFile using CGI variable One potential way to do it would be with the use of includes. http://mapserver.org/mapfile/Include.html The primary mapfile wouldnt have a connection object, but it would contain: INCLUDE connect_info.map Then on each machine, you would have a different version of connect_info.map that contained the correct connection object for that machine. This wouldnt give you the flexibility of changing it through the CGI interface, but would allow you to use the same exact mapfile on both machines. David. From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Nicolas BOUTET Sent: Friday, April 29, 2011 8:57 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Parameterized MapFile using CGI variable Dear All, I have 2 MapServer servers. One for Production use, and the other for Validation use. The MAP files placed on those servers are similar except that each server has a different Oracle database as datasource. For now, I have some different CONNECTION values depending if the map file is on the Production or on the Validation server. So finally I must have different MAP files. Is there a way to have exactly the same map files on the 2 servers, and to use a variable to parameterize the CONNECTION. For example, I would like to set the CONNECTION of my layers to %ORACLECONNECTIONSTRING% and to set a CGI Environment variable named ORACLECONNECTIONSTRING to the correct value on each server. Nicolas ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Parameterized MapFile using CGI variable
As David stated, the INCLUDE keyword does not have to be a full mapserver object, it can be any valid mapserver keywords. -- thomas On Fri, Apr 29, 2011 at 16:49, Nicolas BOUTET nicolas.bou...@geosys.com wrote: Thank you David, Yes, using INCLUDE should be a good solution. But as I know in MapServer 5.6, CONNECTION is a parameter of my LAYER objects, and not an object itself. So I cannot have a CONNECTION object declared in a separate map file… And I prefer to not have to duplicate the layers in different map files. Nicolas De : Fawcett, David (MPCA) [mailto:david.fawc...@state.mn.us] Envoyé : vendredi 29 avril 2011 16:37 À : nicolas.bou...@geosys.com; mapserver-users@lists.osgeo.org Objet : RE: [mapserver-users] Parameterized MapFile using CGI variable One potential way to do it would be with the use of ‘include’s. http://mapserver.org/mapfile/Include.html The primary mapfile wouldn’t have a connection object, but it would contain: INCLUDE “connect_info.map” Then on each machine, you would have a different version of connect_info.map that contained the correct connection object for that machine. This wouldn’t give you the flexibility of changing it through the CGI interface, but would allow you to use the same exact mapfile on both machines. David. From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Nicolas BOUTET Sent: Friday, April 29, 2011 8:57 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Parameterized MapFile using CGI variable Dear All, I have 2 MapServer servers. One for Production use, and the other for Validation use. The MAP files placed on those servers are similar except that each server has a different Oracle database as datasource. For now, I have some different CONNECTION values depending if the map file is on the Production or on the Validation server. So finally I must have different MAP files. Is there a way to have exactly the same map files on the 2 servers, and to use a variable to parameterize the CONNECTION. For example, I would like to set the CONNECTION of my layers to “%ORACLECONNECTIONSTRING%” and to set a CGI Environment variable named ORACLECONNECTIONSTRING to the correct value on each server. Nicolas ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Parameterized MapFile using CGI variable
Hi, No problem at all. I have it this way For each layer LAYER STATUS ON GROUP default PROJECTION init=epsg:3067 END INCLUDE /usr/map/connection_osm_i.map maxscaledenom 500010 minscaledenom 250010 TYPE LINE NAME roadsclose DATA way from (select way, osm_id ,highway,ref, name, tunnel from osm_line where highway is not null order by priority desc) as foo using unique osm_id using srid=3067 ... ... And the /usr/map/connection_osm_i.map file is CONNECTIONTYPE postgis CONNECTION host=localhost dbname=xxx user=yyy password=zzz port=5432 PROCESSING CLOSE_CONNECTION=DEFER Helps a lot when changing something on the database side also because there is only one place to update in the mapfiles. -Jukka Rahkonen- -Alkuperäi nen viesti- Lähettäjä: mapserver-users-boun...@lists.osgeo.org puolesta: Nicolas BOUTET Lähetetty: pe 29.4.2011 17:49 Vastaanottaja: 'Fawcett, David (MPCA)'; mapserver-users@lists.osgeo.org Aihe: RE: [mapserver-users] Parameterized MapFile using CGI variable Thank you David, Yes, using INCLUDE should be a good solution. But as I know in MapServer 5.6, CONNECTION is a parameter of my LAYER objects, and not an object itself. So I cannot have a CONNECTION object declared in a separate map file. And I prefer to not have to duplicate the layers in different map files. Nicolas De : Fawcett, David (MPCA) [mailto:david.fawc...@state.mn.us] Envoyé : vendredi 29 avril 2011 16:37 À : nicolas.bou...@geosys.com; mapserver-users@lists.osgeo.org Objet : RE: [mapserver-users] Parameterized MapFile using CGI variable One potential way to do it would be with the use of 'include's. http://mapserver.org/mapfile/Include.html The primary mapfile wouldn't have a connection object, but it would contain: INCLUDE connect_info.map Then on each machine, you would have a different version of connect_info.map that contained the correct connection object for that machine. This wouldn't give you the flexibility of changing it through the CGI interface, but would allow you to use the same exact mapfile on both machines. David. From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Nicolas BOUTET Sent: Friday, April 29, 2011 8:57 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Parameterized MapFile using CGI variable Dear All, I have 2 MapServer servers. One for Production use, and the other for Validation use. The MAP files placed on those servers are similar except that each server has a different Oracle database as datasource. For now, I have some different CONNECTION values depending if the map file is on the Production or on the Validation server. So finally I must have different MAP files. Is there a way to have exactly the same map files on the 2 servers, and to use a variable to parameterize the CONNECTION. For example, I would like to set the CONNECTION of my layers to %ORACLECONNECTIONSTRING% and to set a CGI Environment variable named ORACLECONNECTIONSTRING to the correct value on each server. Nicolas ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] error and temp file handling with 6.0.0
Hi, I'm a bit confused about how MapServer log errors and where it saves temporary files. In my mapfile I have declared CONFIG MS_ERRORFILE file.err #and DEBUG is not set LOG file.log #in the WEB block TEMPPATH ... #in the WEB block Playing with SLD and polygon fill with exernal graphic doesn't work, but creates nevertheless some output. 1. error output will be written in file.err and file.log, but the messages not the same. file.log Fri Apr 29 14:20:31 2011,2492,127.0.0.1,simple_continents_shp_4326,-1,86.660156 -93.339844 183.339844 3.339844,-1.00 -1.00,,msDrawMap(): Image handling error. Failed to draw layer named 'continents'. br msOGRFileNextShape(): OGR error. IllegalArgumentException: Points of LinearRing do not form a closed linestring br file.err [Fri Apr 29 14:27:03 2011].453000 msOGRFileNextShape(): OGR error. IllegalArgumentException: Points of LinearRing do not form a closed linestring [Fri Apr 29 14:27:03 2011].453000 msDrawMap(): Image handling error. Failed to draw layer named 'continents'. Why do MapServer uses two files to log errors? And it seems, one have to look in both, because the messages are not equal. 2. TEMPPATH If declared, this path is used to save the temporary mapfile and the temporary sldfiles. But the external graphic is saved in the System Temp Directory C:\WINDOWS\TEMP. Should the graphic not saved at the same place? Also there is an error message that speaks of TMPPATH instead of TEMPPATH: [Fri Apr 29 14:10:42 2011].937000 msSLDApplySLDURL: WMS server error. Could not open SLD http://localhost/.../sld_polygon_fill_symbol_external.xml and save it in temporary file C:\WINDOWS\TEMP\4dbaaac2_b34_0.sld.xml. Please make sure that the sld url is valid and that the temporary path is set. The temporary path can be defined for example by setting TMPPATH in the map file. Please check the MapServer documentation on temporary path settings. Arnd ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] error and temp file handling with 6.0.0
On 11-04-29 1:59 PM, Arnd Wippermann wrote: Hi, I'm a bit confused about how MapServer log errors and where it saves temporary files. In my mapfile I have declared CONFIG MS_ERRORFILE file.err #and DEBUG is not set LOG file.log #in the WEB block TEMPPATH ... #in the WEB block The recommending steps to log errors in MapServer are documented at http://www.mapserver.org/optimization/debugging.html If I was in your situation I would follow those 7 steps listed in that document. Have a nice weekend :) -jeff -- Jeff McKenna MapServer Consulting and Training Services http://www.gatewaygeomatics.com/ ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] free() methods in MapScript, compatibility MS 5.x and 6
Alan I used a script creating 3 different type of objects without applying the free method, put it in a loop with 1M cycles and opened it in a browser. Checking the memory usage of the Apache process while running the script showed no increase of memory at all. So I would guess the free() is not needed any more. Test was made with MS 5.6 Armin On 29/04/2011 15:13, Alan Boudreault wrote: Hi Armin, Although I'm not sure if the free call was useful... since it was only removing the php object from a global list but the object was not really freed until the end of the script... I would suggest to simply add a such condition if ms_version 6 ... obj-free() rather than removing them to be compatible 5.6/6.0 and avoid any issue. You can also do a quick test in your script with some echo of the memory used before the end of the script and see if the free() removal has any effect. Alan On April 27, 2011 01:25:54 pm Armin Burger wrote: Hi everybody in MapServer/MapScript 6 the various class methods free() have been removed. If I want to have PHP/MapScript code compatible with both MapServer 5.x and 6, the easiest would be to completely remove these method calls from the code. My question is just if these free() methods are still somehow required or recommended for MapServer/MapScript 5.x or if I can silently drop them without any drawbacks. Cheers Armin ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Some issues with SLD and MapServer 6.0.0 Beta RC1
On 28/04/2011 3:12 PM, Yewondwossen Assefa wrote: Sorry I hit send with finishing the e-mail. I will through each items tomorrow and report. On 28/04/2011 3:10 PM, Yewondwossen Assefa wrote: On 28/04/2011 9:34 AM, Arnd Wippermann wrote: Hi, I'm interested in map styling with SLD and have tested it locally with MapServer 6.0.0 Beta RC1 windows binaries from www.gisinternals.com. I have created an online application to play with SLD and the different MapServer versions. http://gis.ibbeck.de/ginfo/apps/OLExamples/OL210/MapServer_SLD_World/MapServ er_SLD_World.asp There the below issues can be seen. For the online application I have only 6.0.0 Beta 7 available, but it is the same with RC1. Most things work great and create a good looking output. But some things doesn't work as expected. 1. It seems, that the output directory for debug is set static to C:\Windows\Temp. msApplySLD(): Map file after SLD was applied C:\WINDOWS\TEMP\4db963c5_ab0_1._sld.map IMGPATH or IMGURL is not take in account. Temporary files are now written into the tmp directory location. It depends on the platform but should respect the environment variables set on your platform regarding temporary paths. It is not hardcoded in Mapserver. Ticket http://trac.osgeo.org/mapserver/ticket/3354 has notes on this. 2. fill external symbol Polygon fill with external graphic is erratic. If I zoom in one level i get one tile (South America). It creates a file http%3A%2F%2Fgis.ibbeck.de%2FOLClient%2FSLD%2Fsmileyface.png in C:\Windows\Temp. I am not sure I could reproduce this. With local test file, I was able to get a fill polygon with external symbol working. If you have a simple map/data I can use, you can create a trac bug and attach it to them and assign it to me (osgeoid: assefa) 3. Continent, Continent_Else Filter PropertyIsLike doesn't work. With 5.6.1 it is ok. Expression from generated mapfile : EXPRESSION ([Continent] =~ /^Africa/) 4. or_like_abcdef Filter Or and PropertyIsLike doesn't work. With 5.6.1 only the last entry of PropertyIsLike are taken. With Or and PropertyIsEqualTo all entries are taken in both versions. Expression from generated mapfile (6 beta): EXPRESSION ( ([country] =~ /^A/) Or ( ([country] =~ /^B/) Or ( ([country] =~ /^C/) Or ( ([country] =~ /^D/) Or ( ([country] =~ /^E/) Or ([country] =~ /^F/)) ) ) ) ) msEvalExpression: Expression parser error. Failed to parse expression: ([country] =~ /^A/) Or ... With changing =* to ~* and / to the mapfile works : EXPRESSION ( ([country] ~* ^A) Or ( ([country] ~* ^B) Or ... It seems, that in the generated mapfile is a wrong expression. I was able to reproduce this. I have added a bug (http://trac.osgeo.org/mapserver/ticket/3855) on it and a patch that should correct the issue. Is it possible for you to test the patch and report through the bug? Thx 5. symbol fill rotated Polygon fill with rotated square looks interesting, but not as expected. Try with 5.6.1. Here again I tried vectir synbols (weel-know sld symbols like star) as well as the graphic symbols but could not see a problem. The roration parameter should translate into an angle parameter in the style. I have tried this using agg output. Here again if you have test data that demonstrate it please open a trac bug. 6. fill solid Polygon with solid fill without stroke shows thin lines of the borders. Same as above. Some other issues are likely not supported by MapServer, but would be nice to have them. A. Changing the hatch width with sld. see group3, hatch as it is possible with mapfile class. B. UserStyles from applied SLD not supported in WMS Styles C. WMS request GetStyles creates only one userstyle with rules from the different styles described in the mapfile. Should it not create different userstyles for each style described in the mapfile? Regards, Arnd ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users For new features/enhancements, It would also be better to add them in trac and mark them as enhancements. They can be addressed for future versions 6.0. best regards, -- Assefa Yewondwossen Software Analyst Email: yass...@dmsolutions.ca http://www.dmsolutions.ca/ Phone: (613) 565-5056 (ext 14) Fax: (613) 565-0925 ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] free() methods in MapScript, compatibility MS 5.x and 6
That's what I thought. Thanks for testing. Alan On April 29, 2011 01:40:42 pm Armin Burger wrote: Alan I used a script creating 3 different type of objects without applying the free method, put it in a loop with 1M cycles and opened it in a browser. Checking the memory usage of the Apache process while running the script showed no increase of memory at all. So I would guess the free() is not needed any more. Test was made with MS 5.6 Armin On 29/04/2011 15:13, Alan Boudreault wrote: Hi Armin, Although I'm not sure if the free call was useful... since it was only removing the php object from a global list but the object was not really freed until the end of the script... I would suggest to simply add a such condition if ms_version 6 ... obj-free() rather than removing them to be compatible 5.6/6.0 and avoid any issue. You can also do a quick test in your script with some echo of the memory used before the end of the script and see if the free() removal has any effect. Alan On April 27, 2011 01:25:54 pm Armin Burger wrote: Hi everybody in MapServer/MapScript 6 the various class methods free() have been removed. If I want to have PHP/MapScript code compatible with both MapServer 5.x and 6, the easiest would be to completely remove these method calls from the code. My question is just if these free() methods are still somehow required or recommended for MapServer/MapScript 5.x or if I can silently drop them without any drawbacks. Cheers Armin ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users -- Alan Boudreault Mapgears http://www.mapgears.com ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] RE: Validation beyond [A-z]
. is a pattern that says the string contains at least one character, it's a wild card so that's why anything matches. Granted, regex's aren't easy to use but you can accomplish a lot with very simple patterns, plus I've not seen or been presented with a better idea. Validation is (and will be) pushed for a couple of reasons: - To keep users from making poor decisions by forcing them to explicitly make (and think about) them - To try and limit unintended uses (we could go much further here) Not everyone is risk aware and I feel a responsibility to try and help users limit risk. I suppose the lack of documentation doesn't support that last statement but we can fix that. Without validation efforts there's potentially an inherent assumption that the rest of the codebase is insecure. We've made mistakes and take the topic seriously. That said, I'm not adverse to a no validation switch as long as it's off by default, file a ticket... Steve From: Jan Hartmann [mailto:j.l.h.hartm...@uva.nl] Sent: Friday, April 29, 2011 5:38 AM To: wood...@swoodbridge.com Cc: Lime, Steve D (DNR); mapserver-users@lists.osgeo.org; Rahkonen Jukka Subject: Re: [mapserver-users] RE: Validation beyond [A-z] Thanks Steve. I don't understand the syntax: in the regex versions I use, . means one and just one character, not any string. Any string excluding the null-string would be .+ or ..* More generally, I still have problems with validation as a concept: it's too difficult, and perhaps that's why it isn't documented. I have postings about this subject in my personal mapserver-dev mail-archive dating from 2002 (couldn't find them on the official site any more), and there still isn't a clear solution almost ten years later. IMHO the major security risk of MapServer CGI is that it gives access to the filesystem outside the web-root. Wouldn't it be better to keep security at that level, i.e. only let MapServer access explicitly defined parts of the filesystem? Within these parts, it's up to the web-site builder to put only those things that should be visible and nothing else. You don't put an ultra-secret document on the web and afterward restrict access to portions only, you just put there what you want to show to the world. Same goes for validations on extent or styles: just make your selections of what you want to show before you let MapServer loose on it. The same story can be told for database access and restrictions on SQL queries: IMHO that is a matter for the database system. It's easy enough to put everything behind barriers with user privileges and views. Why should mapserver double all that security? Any competent database administrator should know how to prevent SQL injects, and MapServer should not be there to protect those who are unable to. I've been working with Cloud VM's for about a year now, and in that environment many security problems disappear: just make small, dedicated servers and interconnect them, e.g. with cascading services. So my view would be: let the Operating System and the Database do everything needed to secure files and databases, and put in your web services only afterwards. It all gets too complex with all those interconnected securities at all levels of the system (my main problem with Apache). In the last ressort: KISS (Keep It Simple Steve (whoever)) :-) Jan - On 04/28/11 18:48, Steve Woodbridge wrote Hi Jan, I do not think there is a global OFF switch for validation, but where validation is required you can include the regex validation string of /./ which means match anything except a null string, or to also accept a null string then use /.*/ You still need to be aware of when you should put a validation in place even if it accepts any string. Regards, -Steve W On 04/28/11 18:48, Jan Hartmann wrote: I find the whole validation issue difficult and not well documented (http://trac.osgeo.org/mapserver/ticket/3754, last updated four hours ago). How do I put all validation off? I really don need that much security. Jan On 04/28/11 18:19, Lime, Steve D (DNR) wrote: I see the problem, just not sure how to fix it. Steve W. provided some possibilities but that's probably not the only approach. It would be helpful if some interested person(s) got together and drafted an RFC. I think the devs would be in a position to help define implementation details if the problem is well defined along with a proposed solution. Steve From: mapserver-users-boun...@lists.osgeo.orgmailto:mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Rahkonen Jukka Sent: Thursday, April 28, 2011 4:21 AM To: mapserver-users@lists.osgeo.orgmailto:mapserver-users@lists.osgeo.org Subject: [mapserver-users] Validation beyond [A-z] Hi, Validation is nowadays needed in quite a many
Re: [mapserver-users] Efficiency - Postgres/gis view or Mapserver data definition join
Hi Dana, Usually it does not matter much, as mapserver passes the query to Postgis anyway, as you noted. However, mapserever does do some parsing rewriting of the query before issuing it, and this can cause issues. Also note that when working in the db, it is easier to use explain, check indexes, etc to optimise your query, so if it is anything but a simple join, I'd suggest creating it in the database is the tidier way to do this. I guess I approach it with the premise that mapserver is the rendering engine, Postgis is the data management/query engine. If your dba is going to be able to see the views/queries/joins which the db implements, manage/optimise the db for these, it is much easier if they are inplemented in the db than scattered around in mapfiles. This said, I use where clauses all the time in mapfiles, for layers based on data driven categories within a single table, for example, map LAYER DTIS = select ... where category='DTIS';, etc. I'd suggest views in this case are just clutter :-) It is a case-by-case basis... for informed advice, seeing the actual query would be useful . Cheers, Brent Wood --- On Fri, 4/29/11, Dara Olson dol...@glifwc.org wrote: From: Dara Olson dol...@glifwc.org Subject: [mapserver-users] Efficiency - Postgres/gis view or Mapserver data definition join To: mapserver-users@lists.osgeo.org Date: Friday, April 29, 2011, 6:34 AM Greetings. I am hoping that I have posted this on the most appropriate list, please let me know if I should be posting to a different list. In our Mapserver application, we join a lot of tables together - generally one table with geometry to a flat table with tabular data. My question is - is it more efficient/faster to create a view in PostgreSQL doing the join and link to the view table from Mapserver or to define the join (SQL statement) with Mapserver in the data definition within the layer definition? Does it even matter because Mapserver sends the request to PostgreSQL/GIS? Also, we have joins that take a field within the flat table to determine which table to join to get geometry. For example, if the flat table column join value is equal to 1 then it joins to table1 to get geometry or if column join value is equal to 2 then it joins to table2 to get geometry. I read something that you were not supposed to use WHERE clauses in the Mapserver data definition. Would this be better to do as a view and link to the view table from the data definition? Any advice would be greatly appreciated. Thanks in advance! Dara -Inline Attachment Follows- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
AW: [mapserver-users] Some issues with SLD and MapServer 6.0.0 Beta RC1
Hi Here you can online see external graphic mostly not showing http://gis.ibbeck.de/ginfo/apps/OLExamples/OL210/MapServer_SLD_World/MapServ er_SLD_World.asp?SLDsrcIdx=51mapservVersion=6.0.0 Perhaps it has something to do with the complexity of the polygons. Locally test with simplier polygons works. But also with the simplier polygons it doesn't show the graphic if i apply the sld file from http://demo.mapserver.org/ogc-demos/map/sld/sld_polygon_fill_symbol_external .xml (gif image). Rotated square not like 5.6.1 http://gis.ibbeck.de/ginfo/apps/OLExamples/OL210/MapServer_SLD_World/MapServ er_SLD_World.asp?SLDsrcIdx=51mapservVersion=6.0.0 borders as thin lines without stroke set http://gis.ibbeck.de/ginfo/apps/OLExamples/OL210/MapServer_SLD_World/MapServ er_SLD_World.asp?SLDsrcIdx=52mapservVersion=6.0.0 The patch I cannot test. I'm not able to compile MapServer by myself. Thank for watching the points Regards, Arnd -Ursprüngliche Nachricht- Von: Yewondwossen Assefa [mailto:yass...@dmsolutions.ca] Gesendet: Freitag, 29. April 2011 19:48 An: Arnd Wippermann Cc: mapserver-users@lists.osgeo.org Betreff: Re: [mapserver-users] Some issues with SLD and MapServer 6.0.0 Beta RC1 On 28/04/2011 3:12 PM, Yewondwossen Assefa wrote: Sorry I hit send with finishing the e-mail. I will through each items tomorrow and report. On 28/04/2011 3:10 PM, Yewondwossen Assefa wrote: On 28/04/2011 9:34 AM, Arnd Wippermann wrote: Hi, I'm interested in map styling with SLD and have tested it locally with MapServer 6.0.0 Beta RC1 windows binaries from www.gisinternals.com. I have created an online application to play with SLD and the different MapServer versions. http://gis.ibbeck.de/ginfo/apps/OLExamples/OL210/MapServer_SLD_World /MapServ er_SLD_World.asp There the below issues can be seen. For the online application I have only 6.0.0 Beta 7 available, but it is the same with RC1. Most things work great and create a good looking output. But some things doesn't work as expected. 1. It seems, that the output directory for debug is set static to C:\Windows\Temp. msApplySLD(): Map file after SLD was applied C:\WINDOWS\TEMP\4db963c5_ab0_1._sld.map IMGPATH or IMGURL is not take in account. Temporary files are now written into the tmp directory location. It depends on the platform but should respect the environment variables set on your platform regarding temporary paths. It is not hardcoded in Mapserver. Ticket http://trac.osgeo.org/mapserver/ticket/3354 has notes on this. 2. fill external symbol Polygon fill with external graphic is erratic. If I zoom in one level i get one tile (South America). It creates a file http%3A%2F%2Fgis.ibbeck.de%2FOLClient%2FSLD%2Fsmileyface.png in C:\Windows\Temp. I am not sure I could reproduce this. With local test file, I was able to get a fill polygon with external symbol working. If you have a simple map/data I can use, you can create a trac bug and attach it to them and assign it to me (osgeoid: assefa) 3. Continent, Continent_Else Filter PropertyIsLike doesn't work. With 5.6.1 it is ok. Expression from generated mapfile : EXPRESSION ([Continent] =~ /^Africa/) 4. or_like_abcdef Filter Or and PropertyIsLike doesn't work. With 5.6.1 only the last entry of PropertyIsLike are taken. With Or and PropertyIsEqualTo all entries are taken in both versions. Expression from generated mapfile (6 beta): EXPRESSION ( ([country] =~ /^A/) Or ( ([country] =~ /^B/) Or ( ([country] =~ /^C/) Or ( ([country] =~ /^D/) Or ( ([country] =~ /^E/) Or ([country] =~ /^F/)) ) ) ) ) msEvalExpression: Expression parser error. Failed to parse expression: ([country] =~ /^A/) Or ... With changing =* to ~* and / to the mapfile works : EXPRESSION ( ([country] ~* ^A) Or ( ([country] ~* ^B) Or ... It seems, that in the generated mapfile is a wrong expression. I was able to reproduce this. I have added a bug (http://trac.osgeo.org/mapserver/ticket/3855) on it and a patch that should correct the issue. Is it possible for you to test the patch and report through the bug? Thx 5. symbol fill rotated Polygon fill with rotated square looks interesting, but not as expected. Try with 5.6.1. Here again I tried vectir synbols (weel-know sld symbols like star) as well as the graphic symbols but could not see a problem. The roration parameter should translate into an angle parameter in the style. I have tried this using agg output. Here again if you have test data that demonstrate it please open a trac bug. 6. fill solid Polygon with solid fill without stroke shows thin lines of the borders. Same as above. Some other issues are likely not supported by MapServer, but would be nice to have them. A. Changing the hatch width with sld. see group3, hatch as it is possible with mapfile class. B. UserStyles from applied SLD not supported in WMS Styles C. WMS request GetStyles creates only one userstyle with rules
Re: [mapserver-users] RE: Validation beyond [A-z]
On 4/29/2011 6:37 AM, Jan Hartmann wrote: Thanks Steve. I don't understand the syntax: in the regex versions I use,. meansone and just one character, not any string. Any string excluding the null-string would be.+ or..* Not . means match any single character in the string. It does not mean match a string of just one character. Basically it means match any string the is not a null string. . - match any character in a string the is not a null string .* - match any string the is 0 or more characters ^.$ - match a string the is exactly one character long ^$ - match only an empty/null string .+ - match any string that is one of more characters ..* - match any character followed by 0 or more characters More generally, I still have problems with validation as a concept: it's too difficult, and perhaps that's why it isn't documented. I The doc team has done a great job with the documentation, but there are always areas for improvement and this is one of those areas that we need to improve the documentation. have postings about this subject in my personal mapserver-dev mail-archive dating from 2002 (couldn't find them on the official site any more), and there still isn't a clear solution almost ten years later. IMHO the major security risk of MapServer CGI is that it gives access to the filesystem outside the web-root. Wouldn't it be better to keep security at that level, i.e. only let MapServer access explicitly defined parts of the filesystem? Within these parts, it's up to the Definitely not, in my opinion, I do not want my valuable data files which are sometimes licensed data in my document root tree where someone can download them. Also, mapserver does not have any idea of what the webserver's document root is. In fact under Apache you can map any arbitrary directory into the a serviceable directory by the webserver. web-site builder to put only those things that should be visible and nothing else. You don't put an ultra-secret document on the web and afterward restrict access to portions only, you just put there what you want to show to the world. Same goes for validations on extent or styles: just make your selections of what you want to showbefore you let MapServer loose on it. Mapserver serves imagery that is rendered in a context, it does not serve the underlining raw data files. The same story can be told for database access and restrictions on SQL queries: IMHO that is a matter for the database system. It's easy enough to put everything behind barriers with user privileges and views. Why should mapserver double all that security? Any competent database administrator should know how to prevent SQL injects, and MapServer should not be there to protect those who are unable to. I've been working with Cloud VM's for about a year now, and in that environment many security problems disappear: just make small, dedicated servers and interconnect them, e.g. with cascading services. So my view would be: let the Operating System and the Database do everything needed to secure files and databases, and put in your web services only afterwards. It all gets too complex with all those interconnected securities at all levels of the system (my main problem with Apache). In the last ressort: KISS (Keep It Simple Steve (whoever)) :-) This all sounds great, but I don't have a Cloud, or all the services to interconnect and cascade, and I'm not sure how I would set that up for all the applications I have running. Maybe this needs some documentation on how to do this better. Regardless, mapserver has to support multiple environments. I do not think it is reasonable to tell all mapserver users that they have to re-architect there production environments because we do not want to provide some basic tools to help with security if they are not building systems like you are. If you know what you are doing then there is not problem defeating the tools mapserver provides by effectively making them a NOP. As Steve L said, we can add a switch to turn it off if you want to create a bug and we should do a better job on this documentation. Personally, I would be interested in learning more about how you setup and deal with the database, and filesystem issues and specifically how to deal with SQL injection attacks in postgresql. Thanks for your interesting and thoughtful use case. -Steve W Jan - On 04/28/11 18:48, Steve Woodbridge wrote Hi Jan, I do not think there is a global OFF switch for validation, but where validation is required you can include the regex validation string of /./ which means match anything except a null string, or to also accept a null string then use /.*/ You still need to be aware of when you should put a validation in place even if it accepts any string. Regards, -Steve W On 04/28/11 18:48, Jan Hartmann wrote: I find the whole validation issue
