Jukka, Thanks for replying and what I would hope to get is "Rio&Paulo" as you say.
However I do not understand your explanation. Regards Ian -----Original Message----- From: Rahkonen Jukka (MML) [mailto:[email protected]] Sent: Friday, April 22, 2016 8:14 AM To: Ian Walberg <[email protected]>; [email protected] Subject: Re: [mapserver-users] WFS layer - XML/JSON safe chars Hi Ian, I can see that those names are used as a part of gml:id: <ms:DATA_TEST gml:id="DATA_TEST.Rio and Paulo" and I suppose that you would like to see something like <ms:DATA_TEST gml:id="DATA_TEST.Rio&Paulo" but I believe that it will never happen because gml:id http://www.datypic.com/sc/niem20/a-gml_id.html is of type xsd:ID http://www.datypic.com/sc/xsd/t-xsd_ID.html which must be NCName, which in turn means that "it must start with a letter or underscore, and can only contain letters, digits, underscores, hyphens, and periods." Et-character is denied there both as raw as escaped. I am not sure how you can control how Mapserver is building the gml:id. However, I guess that if you use an integer field as primary key in your Spatialite database then it would be utilized and gml:id would look like <ms:DATA_TEST gml:id="DATA_TEST.1" -Jukka Rahkonen- -----Alkuperäinen viesti----- Lähettäjä: mapserver-users [mailto:[email protected]] Puolesta Ian Walberg Lähetetty: 22. huhtikuuta 2016 17:10 Vastaanottaja: [email protected] Aihe: Re: [mapserver-users] WFS layer - XML/JSON safe chars Folks, I think we may be taking about different things, the wfs requests below show the problem we are seeing. There are 2 cities in each of the sqlite databases and the only difference is on has the text 'and' and the other has the symbol '&' Layer DATA_TEST - INSERT INTO "test" VALUES('-14.235004','-51.92528','Rio and Paulo'); http://wfs-dev.airborne.aero/cgi-bin/mapserv?map=/data/wfs/WFS_TEST.map&SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=DATA_TEST Layer DATA_TEST2 - INSERT INTO "test" VALUES('-14.235004','-51.92528','Rio & Paulo'); http://wfs-dev.airborne.aero/cgi-bin/mapserv?map=/data/wfs/WFS_TEST.map&SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=DATA_TEST2 So our question is do the strings in database need to be xml safe or does mapserver check that the strings are safe before serving. Thanks Ian -----Original Message----- From: Even Rouault [mailto:[email protected]] Sent: Friday, April 15, 2016 9:15 AM To: Ian Walberg <[email protected]> Cc: [email protected] Subject: Re: [mapserver-users] WFS layer - XML/JSON safe chars Le vendredi 15 avril 2016 17:32:36, Ian Walberg a écrit : > If I understand this we would not get the issue if we use a WFS > version > 1.0.0 request. > > We have just tested that and it appears that changing the version does > not change the behavior. > > This is with mapserver version 7.0.1. We are probably talking about different things. In https://github.com/mapserver/mapserver/commit/105877468590e99401d234e1bbac13d7645b77d0 , I have just fixed issues in GetCapabilities response generation, but your issues might be different. I've also just tested reading from a CSV (the input datastore type shouldn't matter) file a field with & and it is properly escaped as & in the GML output of WFS 1.0, 1.1 and 2.0 Without a mapfile + data + request you issue, it is difficult to be sure we're talking about the same thing. > > Thanks > > Ian > > -----Original Message----- > From: mapserver-users [mailto:[email protected]] > On Behalf Of Even Rouault Sent: Friday, April 15, 2016 7:50 AM > To: [email protected] > Subject: Re: [mapserver-users] WFS layer - XML/JSON safe chars > > Le vendredi 15 avril 2016 16:41:28, Richard Watson a écrit : > > I am using Map Server 6.4.1. I have a map project file which has > > the > > > > following title: > > WEB > > > > METADATA > > > > [ text omitted] > > "wms_title" "(*#(@#*$_)@#&\\\///" > > > > END # METADATA > > > > END # WEB > > > > When a GetCapabilties is done on the project then this is the source > > XML which is returned: > > > > <Service> > > <Name>WMS</Name> > > <Title>(*#(@#*$_)@#&\\///</Title> > > It seems to be escaping as required. > > WMS and WFS 1.0 use MapServer manual XML composition with its own XML > escaping routines. Some other services (such as WFS 1.1 & 2.0) use > libxml (sometimes not in an appropriate way). > > > On Fri, Apr 15, 2016 at 9:21 AM, Rahkonen Jukka (MML) < > > > > [email protected]> wrote: > > > Hi, > > > > > > > > > > > > I was thinking that the output of GetCapabilities is XML and & > > > must be escaped in XML. So if I write & into the abstract and > > > Mapserver works correctly it should write my test abstract “test ä > > > &” escaped perhaps as > > > > > > test ä & > > > > > > That is the output from the online tool at > > > http://www.freeformatter.com/xml-escape.html#ad-output > > > > > > > > > > > > So my conclusion was that because Mapserver does not do it right > > > here it won’t do it right elsewhere. > > > > > > > > > > > > But now I made another test by editing the Geoserver test file > > > “states.shp” in Spatialite as > > > > > > UPDATE states set STATE_NAME=’Colo&rado’ WHERE > > > STATE_NAME=’Colorado’; > > > > > > > > > > > > Output of GetFeature from WFS 1.1.0 from Mapserver 7.0.0 includes > > > > > > <ms:STATE_NAME>Colo&rado</ms:STATE_NAME> > > > > > > So my current conclusion is that my Mapserver does not escape “&” > > > correctly when it is used inside a mapfile and when I do > > > GetCapabilities request. However, when data come from Spatialite > > > database and they are consumed by GetFeature then escaping happens > > > correctly. > > > > > > > > > > > > We are still comparing different things because you have PostGIS > > > and I have Spatialite. > > > > > > > > > > > > -Jukka Rahkonen- > > > > > > > > > > > > > > > > > > Ian Walberg wrote: > > > > > > > > > > > > Jukka, > > > > > > > > > > > > I am not sure I understand the test you did. > > > > > > > > > > > > The data we are referring to is from the database field for the > > > individual features name. > > > > > > > > > > > > Regards > > > > > > > > > > > > Ian > > > > > > > > > > > > *From:* Rahkonen Jukka (MML) > > > [mailto:[email protected] > > > <[email protected]>] > > > *Sent:* Thursday, April 14, 2016 10:19 PM > > > *To:* Ian Walberg <[email protected]>; MapserverList OSGEO > > > < [email protected]> > > > *Subject:* Re: WFS layer - XML/JSON safe chars > > > > > > > > > > > > Hi Ian, > > > > > > > > > > > > Thanks for the clarification. > > > > > > I made a simple test by adding an abstract into layer metadata. > > > This shows “test ä” as an abstract > > > > > > "wfs_abstract" "test ä” > > > > > > > > > > > > This one shows an empty abstract > > > > > > "wfs_abstract" "test ä &" > > > > > > > > > > > > So it looks like Mapserver doesn’t know to convert “&” into “&” > > > for XML. > > > > > > > > > > > > -Jukka Rahkonen- > > > > > > > > > > > > Ian Walberg wrote: > > > > > > > > > > > > Jukka, > > > > > > > > > > > > Our question was for the data/text read from the database and then > > > published as XML by the WFS service. > > > > > > > > > > > > So a pub with the name ‘Dog & bone’ created illegal xml as the & > > > is not converted to & > > > > > > > > > > > > Regards > > > > > > > > > > > > Ian > > > > > > > > > > > > *From:* Rahkonen Jukka (MML) > > > [mailto:[email protected] > > > <[email protected]>] > > > *Sent:* Thursday, April 14, 2016 3:10 PM > > > *To:* Ian Walberg <[email protected]>; MapserverList OSGEO > > > < [email protected]> > > > *Subject:* Re: WFS layer - XML/JSON safe chars > > > > > > > > > > > > Hi, > > > > > > > > > > > > You can’t have & in feature type names or attribute names in WFS > > > because they are defined in the XML schema to be “QName”. The > > > prefix and LocalPart are of type NCName > > > https://en.wikipedia.org/wiki/QName > > > and from > > > http://stackoverflow.com/questions/1631396/what-is-an-xsncname-typ > > > e- > > > and-w > > > hen-should-it-be-used > > > > > > > > > > > > The practical restrictions of NCName are that it cannot contain > > > several symbol characters like :, @, $, %, &, /, +, ,, ;, > > > whitespace characters or different parenthesis. Furthermore an > > > NCName cannot begin with a number, dot or minus character although > > > they can appear later in an NCName. > > > > > > > > > > > > -Jukka Rahkonen- > > > > > > > > > > > > *Lähettäjä:* mapserver-users [ > > > mailto:[email protected] > > > <[email protected]>] *Puolesta *Ian Walberg > > > *Lähetetty:* 14. huhtikuuta 2016 19:23 > > > *Vastaanottaja:* MapserverList OSGEO > > > *Aihe:* [mapserver-users] WFS layer - XML/JSON safe chars > > > > > > > > > > > > Folks, > > > > > > > > > > > > Can mapserver produce XML or JSON safe characters for WFS served > > > data or do we need to ensure the source data has the escaped values. > > > > > > > > > > > > We got tripped up today as there were some ‘&’ in our names which > > > we had not considered. > > > > > > > > > > > > Thanks > > > > > > > > > > > > Ian > > > > > > _______________________________________________ > > > mapserver-users mailing list > > > [email protected] > > > http://lists.osgeo.org/mailman/listinfo/mapserver-users > > -- > Spatialys - Geospatial professional services http://www.spatialys.com > _______________________________________________ mapserver-users > mailing list [email protected] > http://lists.osgeo.org/mailman/listinfo/mapserver-users -- Spatialys - Geospatial professional services http://www.spatialys.com _______________________________________________ mapserver-users mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/mapserver-users _______________________________________________ mapserver-users mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/mapserver-users
