Re: [mapserver-users] MapServer Layer Filter not applying to WFS
Hi, Thanks for the help. Using filter in the CLASS did nothing more than adding FILTER to the layer tag. I tried this definition: LAYER NAME Traseer GROUP TelMe TYPE LINE DATA TM_Nett/TM_Traces minscaledenom 0 maxscaledenom 550 CLASS EXPRESSION ('[type]' != LUFT) TEMPLATE 'void' NAME TelMe_Traseer STYLE COLOR 255 0 0 WIDTH 2 END END # Layer It does not display the data, nor queryable in MapScript, but I can get the feature geometry with WFS. Ive until now used CONNECTIONTYPE OGR with CONNECTION, and it has worked flawlessly. Recently, when upgrading to PHP 5.4, it started UTF-8 encoding my tables. Ive addressed the issue here: http://comments.gmane.org/gmane.comp.gis.mapserver.user/49756 . I have some more research to do regarding the issue, but Id be very thankful for any hints. Håkon From: Lime, Steve D (DNR) [mailto:steve.l...@state.mn.us] Sent: 5. november 2012 18:30 To: Hawk AA; mapserver-users@lists.osgeo.org Subject: RE: [mapserver-users] MapServer Layer Filter not applying to WFS One other idea. I believe you can avoid using OGR by adding your filter to class expressions and setting a template (which enables queries) at the class level. Class expressions are always checked. If you have one class its easy: LAYER NAME Traseer GROUP TelMe TYPE LINE DATA TM_Nett/TM_Traces CLASS EXPRESSION'[type]' != Bru) # Styling and more etc TEMPLATE void END # No template set at the layer level END Might consider WMS-only vs. WFS-only layer defs to simplify life if you have multiple classes. Steve From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Hawk AA Sent: Monday, November 05, 2012 8:26 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] MapServer Layer Filter not applying to WFS Hi there, mailing list, We have a customer sending us data with sensitive information. The most convenient way for us is to receive the complete data set and applying a filter in the mapfile. The layer definition looks like this: LAYER NAME Traseer GROUP TelMe TYPE LINE DATA TM_Nett/TM_Traces FILTER('[type]' != Bru) #Styling and more etc END This works flawlessly using PHP Mapscript and the mapserver WMS service. The data is not accessible. The problem occurs when querying by WFS. The Mapserver WFS service seems to omit the FILTER information and opens up for selecting items with the type Bru, which is a serious security flaw. Id consider this as a bug, although Im not certain. If anyone please can confirm this, or show me how to make mapserver filter data in WFS as well, Id be much obliged. Best Regards, Håkon ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] MapServer Layer Filter not applying to WFS
Hi there, mailing list, We have a customer sending us data with sensitive information. The most convenient way for us is to receive the complete data set and applying a filter in the mapfile. The layer definition looks like this: LAYER NAME Traseer GROUP TelMe TYPE LINE DATA TM_Nett/TM_Traces FILTER('[type]' != Bru) #Styling and more etc END This works flawlessly using PHP Mapscript and the mapserver WMS service. The data is not accessible. The problem occurs when querying by WFS. The Mapserver WFS service seems to omit the FILTER information and opens up for selecting items with the type Bru, which is a serious security flaw. Id consider this as a bug, although Im not certain. If anyone please can confirm this, or show me how to make mapserver filter data in WFS as well, Id be much obliged. Best Regards, Håkon ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] MapServer Layer Filter not applying to WFS
This is known issue, but it's a bigger deal with shapefiles. The WFS filters essentially replace the defined filter, they are not additive. This is true for all drivers. This would need to be an enhancement, but It would be tricky to do so I think given the variety of filter types. With RDBMS you'd just encode the filter in the data/connection information. The workaround in this case would be to use an OGR layer to access the shapefile instead which will allow you to apply SQL-like syntax at the driver level. See http://mapserver.org/input/vector/ogr.html. Steve From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Hawk AA Sent: Monday, November 05, 2012 8:26 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] MapServer Layer Filter not applying to WFS Hi there, mailing list, We have a customer sending us data with sensitive information. The most convenient way for us is to receive the complete data set and applying a filter in the mapfile. The layer definition looks like this: LAYER NAME Traseer GROUP TelMe TYPE LINE DATA TM_Nett/TM_Traces FILTER('[type]' != Bru) #Styling and more etc END This works flawlessly using PHP Mapscript and the mapserver WMS service. The data is not accessible. The problem occurs when querying by WFS. The Mapserver WFS service seems to omit the FILTER information and opens up for selecting items with the type Bru, which is a serious security flaw. I'd consider this as a bug, although I'm not certain. If anyone please can confirm this, or show me how to make mapserver filter data in WFS as well, I'd be much obliged. Best Regards, Håkon ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] MapServer Layer Filter not applying to WFS
One other idea. I believe you can avoid using OGR by adding your filter to class expressions and setting a template (which enables queries) at the class level. Class expressions are always checked. If you have one class it's easy: LAYER NAME Traseer GROUP TelMe TYPE LINE DATA TM_Nett/TM_Traces CLASS EXPRESSION'[type]' != Bru) # Styling and more etc TEMPLATE 'void' END # No template set at the layer level END Might consider WMS-only vs. WFS-only layer defs to simplify life if you have multiple classes. Steve From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Hawk AA Sent: Monday, November 05, 2012 8:26 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] MapServer Layer Filter not applying to WFS Hi there, mailing list, We have a customer sending us data with sensitive information. The most convenient way for us is to receive the complete data set and applying a filter in the mapfile. The layer definition looks like this: LAYER NAME Traseer GROUP TelMe TYPE LINE DATA TM_Nett/TM_Traces FILTER('[type]' != Bru) #Styling and more etc END This works flawlessly using PHP Mapscript and the mapserver WMS service. The data is not accessible. The problem occurs when querying by WFS. The Mapserver WFS service seems to omit the FILTER information and opens up for selecting items with the type Bru, which is a serious security flaw. I'd consider this as a bug, although I'm not certain. If anyone please can confirm this, or show me how to make mapserver filter data in WFS as well, I'd be much obliged. Best Regards, Håkon ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Mapserver Layer Filter
Hi all, Thank you for splendid support. I solved the problem by coincidence. By using the OGR driver to view the shape-file and using FILTER, the WFS would not be able to fetch the hidden data. CONNECTIONTYPE OGR CONNECTION TM_Nett/TM_Points.shp FILTER('[Type]' != SKAP) Thanks anyway, Hawk ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Mapserver Layer Filter
Hi there, Mailing List, We have a customer that wants their maps to be published online, but some of the features is confidential and should not be displayed. We have added following to the layer definition: FILTER ('[type]' != GRØFT ) It works, the features with type GRØFT are not showed in the map. However, we have a window that let the user browse the data, and it will be possible to access the features of type GRØFT. When you press show in map, the JavaScript clients triggers a WFS request with a filter asking for elements with the current ID. By doing this, WFS omits the LAYER FILTER definition, so the element can be showed in the map, even if it is of type GRØFT. I need a way to make sure the WFS cant deliver any data of type GRØFT. Ive seen three possible solutions: 1. We have a service that automatically converts uploaded MapInfo *.TAB-files to SHP. If I am correct, I could use the -where-argument on the ogr2ogr to filter data and only get features which does not have type GRØFT. I hope I do not have to do this, since debugging and developing a windows service is quite tricky. 2. Add additional filter values to the WFS query. I could also define the types not to show in the JavaScript, and let the application automatically add PropertyIsNotEqualTo-tags to the AJAX query. This would be less secure, and I need to define types not to show at two places. 3. The best solution in my eyes is to let the magic happen in the MAP-file, and that is mostly why I am posting here. Is there a way to add a query to the Layer Data definition, like you do if you are querying a MSSQL database? I can imagine a syntax like this: CONNECTIONTYPE OGR CONNECTION TM_Nett/TM_Points DATA SELECT * FROM tm_points WHERE type!=GRØFT It does not seem to work. Any suggestions, especially regarding the third solution? Best regards, Håkon Åmdal ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Mapserver Layer Filter
We do something similar here is how we implement your solution 3 CONNECTIONTYPE PLUGIN PLUGIN .\msplugin_mssql2008.dll CONNECTION server=SERVER\MSSQL;uid=USERID;pwd=PASSWORD;database=DATABASE;Integrated Security=false DATA gExtent FROM (SELECT * FROM tm_points WHERE type!=GRØFT)AS FOO USING UNIQUE [iId] USING SRID=4326 Note the connection string needs to access your MSSQL Instance and the plugin dll must be available the format of the data statement lets you put quite a complex SQL statement inside the ()s. Gabe Codina From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Hawk AA Sent: Friday, 13 January 2012 3:24 AM To: mapserver-users@lists.osgeo.org Subject: [mapserver-users] Mapserver Layer Filter Hi there, Mailing List, We have a customer that wants their maps to be published online, but some of the features is confidential and should not be displayed. We have added following to the layer definition: FILTER ('[type]' != GRØFT ) It works, the features with type GRØFT are not showed in the map. However, we have a window that let the user browse the data, and it will be possible to access the features of type GRØFT. When you press show in map, the JavaScript clients triggers a WFS request with a filter asking for elements with the current ID. By doing this, WFS omits the LAYER FILTER definition, so the element can be showed in the map, even if it is of type GRØFT. I need a way to make sure the WFS cant deliver any data of type GRØFT. Ive seen three possible solutions: 1. We have a service that automatically converts uploaded MapInfo *.TAB-files to SHP. If I am correct, I could use the -where-argument on the ogr2ogr to filter data and only get features which does not have type GRØFT. I hope I do not have to do this, since debugging and developing a windows service is quite tricky. 2. Add additional filter values to the WFS query. I could also define the types not to show in the JavaScript, and let the application automatically add PropertyIsNotEqualTo-tags to the AJAX query. This would be less secure, and I need to define types not to show at two places. 3. The best solution in my eyes is to let the magic happen in the MAP-file, and that is mostly why I am posting here. Is there a way to add a query to the Layer Data definition, like you do if you are querying a MSSQL database? I can imagine a syntax like this: CONNECTIONTYPE OGR CONNECTION TM_Nett/TM_Points DATA SELECT * FROM tm_points WHERE type!=GRØFT It does not seem to work. Any suggestions, especially regarding the third solution? Best regards, Håkon Åmdal ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
AW: [mapserver-users] Mapserver Layer Filter
You might try OGR Virtual Data Driver: http://www.gdal.org/ogr/drv_vrt.html. -Ursprüngliche Nachricht- Von: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] Im Auftrag von Hawk AA Gesendet: Donnerstag, 12. Januar 2012 17:24 An: mapserver-users@lists.osgeo.org Betreff: [mapserver-users] Mapserver Layer Filter Hi there, Mailing List, We have a customer that wants their maps to be published online, but some of the features is confidential and should not be displayed. We have added following to the layer definition: FILTER ('[type]' != GRØFT ) It works, the features with type GRØFT are not showed in the map. However, we have a window that let the user browse the data, and it will be possible to access the features of type GRØFT. When you press show in map, the JavaScript clients triggers a WFS request with a filter asking for elements with the current ID. By doing this, WFS omits the LAYER FILTER definition, so the element can be showed in the map, even if it is of type GRØFT. I need a way to make sure the WFS can't deliver any data of type GRØFT. I've seen three possible solutions: 1. We have a service that automatically converts uploaded MapInfo *.TAB-files to SHP. If I am correct, I could use the -where-argument on the ogr2ogr to filter data and only get features which does not have type GRØFT. I hope I do not have to do this, since debugging and developing a windows service is quite tricky. 2. Add additional filter values to the WFS query. I could also define the types not to show in the JavaScript, and let the application automatically add PropertyIsNotEqualTo-tags to the AJAX query. This would be less secure, and I need to define types not to show at two places. 3. The best solution in my eyes is to let the magic happen in the MAP-file, and that is mostly why I am posting here. Is there a way to add a query to the Layer Data definition, like you do if you are querying a MSSQL database? I can imagine a syntax like this: CONNECTIONTYPE OGR CONNECTION TM_Nett/TM_Points DATA SELECT * FROM tm_points WHERE type!='GRØFT' It does not seem to work. Any suggestions, especially regarding the third solution? Best regards, Håkon Åmdal ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Mapserver Layer Filter
Even better: use solution 3, but create a view on the database and query the view instead. I'm not sure if it can be done it but I guess that a malicious user could trick mapserver and add a filter like ' OR 1=1 ' and in that case ALL features (including GROFT) will be shown. HTH, Umberto On Fri, Jan 13, 2012 at 1:09 AM, Gabe Codina g...@agtrix.com wrote: We do something similar here is how we implement your solution 3 ** ** CONNECTIONTYPE PLUGIN PLUGIN .\msplugin_mssql2008.dll CONNECTION server=SERVER\MSSQL;uid=USERID;pwd=PASSWORD;database=DATABASE;Integrated Security=false DATA gExtent FROM (SELECT * FROM tm_points WHERE type!=’GRØFT’)AS FOO USING UNIQUE [iId] USING SRID=4326 ** ** ** ** Note the connection string needs to access your MSSQL Instance and the plugin dll must be available the format of the data statement lets you put quite a complex SQL statement inside the ()s. ** ** Gabe Codina ** ** ** ** *From:* mapserver-users-boun...@lists.osgeo.org [mailto: mapserver-users-boun...@lists.osgeo.org] *On Behalf Of *Hawk AA *Sent:* Friday, 13 January 2012 3:24 AM *To:* mapserver-users@lists.osgeo.org *Subject:* [mapserver-users] Mapserver Layer Filter ** ** Hi there, Mailing List, ** ** We have a customer that wants their maps to be published online, but some of the features is confidential and should not be displayed. ** ** We have added following to the layer definition: FILTER ('[type]' != GRØFT ) It works, the features with type GRØFT are not showed in the map. ** ** However, we have a window that let the user browse the data, and it will be possible to access the features of type GRØFT. When you press “show in map”, the JavaScript clients triggers a WFS request with a filter asking for elements with the current ID. By doing this, WFS omits the LAYER FILTER definition, so the element can be showed in the map, even if it is of type GRØFT. I need a way to make sure the WFS can’t deliver any data of type GRØFT. ** ** I’ve seen three possible solutions: **1. **We have a service that automatically converts uploaded MapInfo *.TAB-files to SHP. If I am correct, I could use the “-where”-argument on the ogr2ogr to filter data and only get features which does not have type GRØFT. I hope I do not have to do this, since debugging and developing a windows service is quite tricky. **2. **Add additional filter values to the WFS query. I could also define the types not to show in the JavaScript, and let the application automatically add PropertyIsNotEqualTo-tags to the AJAX query. This would be less secure, and I need to define types not to show at two places. **3. **The best solution in my eyes is to let the magic happen in the MAP-file, and that is mostly why I am posting here. Is there a way to add a query to the Layer Data definition, like you do if you are querying a MSSQL database? I can imagine a syntax like this: CONNECTIONTYPE OGR CONNECTION TM_Nett/TM_Points DATA SELECT * FROM tm_points WHERE type!=’GRØFT’” It does not seem to work. ** ** Any suggestions, especially regarding the third solution? ** ** Best regards, Håkon Åmdal ** ** ** ** ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Mapserver Layer Filter
Dear Håkon, I am not quite sure about MSSQL databases, but for postgres I would suggest creating a view on the datatable and using the view as datasource. This gives you higher flexibility, although a WHERE-clause in the datarequest in the mapfile is possible at all (see this postgis examples here: http://mapserver.org/input/vector/postgis.html). You get the highest flexibility, if you use a kind of variable substitution as filter by passing a vendor specific parameter to the map-request. (http://mapserver.org/mapfile/variable_sub.html), use this in combination with the WHERE clause from above. In order to make sure, that unauthorized users could not access the forbidden attributes, you have to use something like a proxy between the requesting client and the mapserver, which adds this parameter for you and in the same state deletes all other possibly added parameters and also checks for authorization. Regards, Till On 13.01.2012 08:24, Eichner, Andreas - SID-NLKM wrote: You might try OGR Virtual Data Driver: http://www.gdal.org/ogr/drv_vrt.html. -Ursprüngliche Nachricht- Von: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] Im Auftrag von Hawk AA Gesendet: Donnerstag, 12. Januar 2012 17:24 An: mapserver-users@lists.osgeo.org Betreff: [mapserver-users] Mapserver Layer Filter Hi there, Mailing List, We have a customer that wants their maps to be published online, but some of the features is confidential and should not be displayed. We have added following to the layer definition: FILTER ('[type]' != GRØFT ) It works, the features with type GRØFT are not showed in the map. However, we have a window that let the user browse the data, and it will be possible to access the features of type GRØFT. When you press show in map, the JavaScript clients triggers a WFS request with a filter asking for elements with the current ID. By doing this, WFS omits the LAYER FILTER definition, so the element can be showed in the map, even if it is of type GRØFT. I need a way to make sure the WFS can't deliver any data of type GRØFT. I've seen three possible solutions: 1. We have a service that automatically converts uploaded MapInfo *.TAB-files to SHP. If I am correct, I could use the -where-argument on the ogr2ogr to filter data and only get features which does not have type GRØFT. I hope I do not have to do this, since debugging and developing a windows service is quite tricky. 2. Add additional filter values to the WFS query. I could also define the types not to show in the JavaScript, and let the application automatically add PropertyIsNotEqualTo-tags to the AJAX query. This would be less secure, and I need to define types not to show at two places. 3. The best solution in my eyes is to let the magic happen in the MAP-file, and that is mostly why I am posting here. Is there a way to add a query to the Layer Data definition, like you do if you are querying a MSSQL database? I can imagine a syntax like this: CONNECTIONTYPE OGR CONNECTION TM_Nett/TM_Points DATA SELECT * FROM tm_points WHERE type!='GRØFT' It does not seem to work. Any suggestions, especially regarding the third solution? Best regards, Håkon Åmdal ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users -- GO Mobile - make WebGIS on Smartphones -- ! Achtung - terrestris hat eine neue Adresse ! - terrestris GmbH Co. KG Puetzchens Chaussee 56 53227 Bonn Germany Till Adams Geschaeftsfuehrung Tel:+49 (0)228 / 962 899-52 Mobile: +49 (0)151 / 25394429 Fax:+49 (0)228 / 962 899-57 ad...@terrestris.de http://www.terrestris.de Amtsgericht Bonn, HRA 6835 - Komplementaerin: terrestris Verwaltungs GmbH vertreten durch: Hinrich Paulsen, Till Adams ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Mapserver Layer Filter
Hi, I made a test with a shapefile with ogrinfo as ogrinfo -al roads.shp -sql select * from roads where type!='motorway' It gives a correct result (everything that is not motorways). I would check first what happens with a type value which has just ASCII characters. Perhaps OGR just has troubles with GRØFT. -Jukka Rahkonen- Eichner, Andreas wrote: You might try OGR Virtual Data Driver: http://www.gdal.org/ogr/drv_vrt.html. -Ursprüngliche Nachricht- Von: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] Im Auftrag von Hawk AA Gesendet: Donnerstag, 12. Januar 2012 17:24 An: mapserver-users@lists.osgeo.org Betreff: [mapserver-users] Mapserver Layer Filter Hi there, Mailing List, We have a customer that wants their maps to be published online, but some of the features is confidential and should not be displayed. We have added following to the layer definition: FILTER ('[type]' != GRØFT ) It works, the features with type GRØFT are not showed in the map. However, we have a window that let the user browse the data, and it will be possible to access the features of type GRØFT. When you press show in map, the JavaScript clients triggers a WFS request with a filter asking for elements with the current ID. By doing this, WFS omits the LAYER FILTER definition, so the element can be showed in the map, even if it is of type GRØFT. I need a way to make sure the WFS can't deliver any data of type GRØFT. I've seen three possible solutions: 1. We have a service that automatically converts uploaded MapInfo *.TAB-files to SHP. If I am correct, I could use the -where-argument on the ogr2ogr to filter data and only get features which does not have type GRØFT. I hope I do not have to do this, since debugging and developing a windows service is quite tricky. 2. Add additional filter values to the WFS query. I could also define the types not to show in the JavaScript, and let the application automatically add PropertyIsNotEqualTo-tags to the AJAX query. This would be less secure, and I need to define types not to show at two places. 3. The best solution in my eyes is to let the magic happen in the MAP-file, and that is mostly why I am posting here. Is there a way to add a query to the Layer Data definition, like you do if you are querying a MSSQL database? I can imagine a syntax like this: CONNECTIONTYPE OGR CONNECTION TM_Nett/TM_Points DATA SELECT * FROM tm_points WHERE type!='GRØFT' It does not seem to work. Any suggestions, especially regarding the third solution? Best regards, Håkon Åmdal ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users