Jez, Even,
there are actually several issues relating to using GDAL/OGR to read
Ordnance Survey of Great Britain (OSGB) GML files distributed as their Mastermap
product. One of these I reported as bug #1604 - I now find that this was
against GDAL-1.4.0 - which concerns handling 'duplicate' tokens: OGR ignores
Namespace and so treats <osgb:point> as the starter and fouls on the following
<gml:point> which contains the geometry. There is a similar problem with
polygon objects.
The data Jez describes below is simpler than much of the data in the file -
perhaps this next point is not an issue for him. Several of the tokens are
described in the schema snippet as 'unbounded': this means that there can be
several instances
<osgb:changeHistory>
<osgb:changeDate>2004-12-19</osgb:changeDate>
<osgb:reasonForChange>Revised</osgb:reasonForChange>
<osgb:changeDate>2002-09-07</osgb:changeDate>
<osgb:reasonForChange>Revised</osgb:reasonForChange>
<osgb:changeDate>2001-03-12</osgb:changeDate>
<osgb:reasonForChange>New</osgb:reasonForChange>
</osgb:changeHistory>
Here it is not only GDAL/OGR that has a problem! Currently, I know of no
importer that can handle this construct, other than the tool (from Snowflake)
used by OSGB to generate it - and there is also the question of onwards storage.
OSGB use Oracle internally and so I presume that these data are stored as
VARRAYs or 'Nested Tables' - complex objects that have no equivalent in many
other data structures. GDAL/OGR has support in the OCI driver for VARRAYs, but
only in the context of the Geometry column; GDAL/OGR has a data structure for a
string array, but this is not supported in the OCI driver. Whilst many of us
may be happy to ignore the change history and descriptive term data in the
example Jez provides, according to the OSGB Mastermap User Guide (Reference
Section) OSGB use unbounded multiple instances for:
theme )
changeHistory )
descriptiveGroup ) present in all Mastermap layers
descriptiveTerm )
referenceToFeature CartographicSymbol layer
networkMember )
referenceToTopgraphicArea )
referenceToNetwork )
dateTimeQualifier ) ITN layer; the qualifiers contain critical data
vehicleQualifier )
environmentQualifier )
subsectionDistance )
subsectionPoint )
Also in the ITN layer, directedNode is a two-part structure.
Even notes that XML permits the specification of a hierarchical dataset,
whilst GDAL/OGR's approach is linear. Surely to comply with the OGC
specification, a gml reader must be able to handle an hierarchical structure?
GML appears to be defined in that way (see Lake, et al, 2004, Geography Markup
Language, Chichester: John Wiley & Sons). I do not know whether the GDAL/OGR
GML driver was designed primarily for writing gml rather than for reading:
maybe.
Where does this leave us? As I mentioned, there are also problems with most
other gml readers: this is not solely an issue with GDAL/OGR. I have an
immediate need for the ITN data and have written my own parser to extract the
information from the gml source: so far, so good. However, as I mentioned,
there is now the problem of how to store these data: shapefiles use the dBaseIV
format and have no structure for handling these multiple attributes. In a
sample dataset, I have a record with 49 changeHistory records, for example; some
other multiple constructs have several thousand entries. I happen to have
access to Oracle, although to use GDAL/OGR to write to it requires that I do
some significant work on the oci driver: I've been trying to understand the code
of that to assess what I can reasonably do. Alternatively, I could use oci
directly and bypass GDAL/OGR entirely. All this, however, is non-trivial and
holding me back from doing what I am supposed to be doing ... but does seem to
be the only way forward, having exhausted FME, etc.
Please do not take this as a rant or criticism: that is far from my
intention. Frank and team do an extremely good job but, with the best will in
the world, they cannot be expected to do everything. What I am trying to do is
report the problem that I raised a while back (but had not then realised how big
the task was) and which has now been raised by Jez. If GDAL/OGR is to be
extended to handle these OSGB data, then it is going to require a group of us to
work together to design and deliver: OSGB must be a low priority for the core
team as it can only be of national interest.
Best wishes,
Peter
(sorry that's so long, but there is a lot of information there ...)
Jez Walters wrote:
Even,
This is precisely the fix I came up with yesterday, but I was wondering if there was a
more "official" solution instead of editing the GML?
Warm regards,
Jez
-----Original Message-----
From: Even Rouault [mailto:[email protected]]
Sent: Wednesday 30 June 2010 19:54
To: [email protected]
Cc: Jez Walters; 'Frank Warmerdam'
Subject: Re: [gdal-dev] ogr2ogr problem converting UK Ordnance Survey MasterMap
Jez,
the reason for the orientation field to be recognize in the CartographicSymbol
layer and not in the CartographicText layer lies in the structure of the GML
file and limitations of the OGR GML driver and/or OGR data model.
Namely OGR only supports a "flat" organization of fields whereas GML (being
XML based) supports a tree-like organization. In the CartographicSymbol
layer, the orientation element is at the first level, whereas in the
CartographicText, it is included inside the <textRendering> element, and then
is ignored (see below examples)
<osgb:CartographicSymbol fid='osgb1000000729439973'>
<osgb:featureCode>10165</osgb:featureCode>
<osgb:version>1</osgb:version>
<osgb:versionDate>2001-11-11</osgb:versionDate>
<osgb:theme>Rail</osgb:theme>
<osgb:changeHistory>
<osgb:changeDate>2001-03-12</osgb:changeDate>
<osgb:reasonForChange>New</osgb:reasonForChange>
</osgb:changeHistory>
<osgb:descriptiveGroup>Rail</osgb:descriptiveGroup>
<osgb:descriptiveTerm>Switch</osgb:descriptiveTerm>
<osgb:orientation>3009</osgb:orientation>
<osgb:physicalLevel>50</osgb:physicalLevel>
<osgb:point>
<gml:Point srsName='osgb:BNG'>
<gml:coordinates>291781.470,92943.570</gml:coordinates>
</gml:Point>
</osgb:point>
</osgb:CartographicSymbol>
<osgb:CartographicText fid='osgb1000000729439996'>
<osgb:featureCode>10026</osgb:featureCode>
<osgb:version>1</osgb:version>
<osgb:versionDate>2001-11-11</osgb:versionDate>
<osgb:theme>Buildings</osgb:theme>
<osgb:anchorPoint>
<gml:Point srsName='osgb:BNG'>
<gml:coordinates>291636.850,92668.150</gml:coordinates>
</gml:Point>
</osgb:anchorPoint>
<osgb:changeHistory>
<osgb:changeDate>1987-12-01</osgb:changeDate>
<osgb:reasonForChange>New</osgb:reasonForChange>
</osgb:changeHistory>
<osgb:descriptiveGroup>Buildings Or Structure</osgb:descriptiveGroup>
<osgb:make>Manmade</osgb:make>
<osgb:physicalLevel>50</osgb:physicalLevel>
<osgb:textRendering>
<osgb:anchorPosition>3</osgb:anchorPosition>
<osgb:font>2</osgb:font>
<osgb:height>1.500</osgb:height>
<osgb:orientation>545</osgb:orientation>
</osgb:textRendering>
<osgb:textString>44</osgb:textString>
</osgb:CartographicText>
We could imagine an enhancement of the GML driver to try to "flatten" the
attributes. For example to return textRendering_anchorPosition,
textRendering_font, .... But it doesn't look trivial to add.
To workaround this, you can open the GML with your favorite text editor (or
sed ;-)) and replace <osgb:textRendering> and </osgb:textRendering> by
nothing
This works nicely ;-)
Best regards,
Even
Le Wednesday 30 June 2010 17:17:12 Jez Walters, vous avez écrit :
Frank,
Thanks for looking in to this! I've just repeated the problem with the
Ordnance Survey sample MasterMap, which is available from here:
http://www.ordnancesurvey.co.uk/oswebsite/products/innovations/sampledata/O
SMasterMap_Topo/58116-SX9192-2c1.gz
I tried converting the above GML to ESRI shapefile using the following
command:
ogr2ogr SX9192 58116-SX9192-2c1
One thing I've noticed is that the schema referenced in the GML
(http://www.ordnancesurvey.co.uk/xml/schema/v7/OSDNFFeatures.xsd) defines a
number of elements called "complexType". The "complexType" element with a
name of "CartographicTextType" is defined like this:
<complexType name="CartographicTextType">
<complexContent>
<extension base="osgb:AbstractFeatureType">
<sequence>
<element name="anchorPoint"
type="gml:PointPropertyType"/>
<element name="changeHistory"
type="osgb:changeHistoryType"
maxOccurs="unbounded"/> <element name="descriptiveGroup"
type="osgb:descriptiveGroupType" minOccurs="0" maxOccurs="unbounded"/>
<element name="descriptiveTerm" type="string" minOccurs="0"
maxOccurs="unbounded"/> <element name="make" type="osgb:makeType"
minOccurs="0"/>
<element name="physicalLevel"
type="osgb:physicalLevelType"/>
<element name="physicalPresence"
type="osgb:physicalPresenceType"
minOccurs="0"/> <element name="textRendering"
type="osgb:textRenderingType"/>
<element name="textString" type="string"/>
</sequence>
</extension>
</complexContent>
</complexType>
>From which I conclude that the orientation is actually part of the
"textRendering" element. I'm sure I'm on to something with this, but I'm
not sure how to proceed from here!
Warm regards,
Jez
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Frank
Warmerdam Sent: Wednesday 30 June 2010 15:38
To: Jez Walters
Cc: [email protected]
Subject: Re: [gdal-dev] ogr2ogr problem converting UK Ordnance Survey
MasterMap
Jez,
Can you provide a mastermap file I can test with?
Best regards,
On Wed, Jun 30, 2010 at 4:31 PM, Jez Walters <[email protected]> wrote:
Not quite sure if I've got the right forum, but I'm having trouble using
the 'ogr2ogr' utility (part of GDAL 1.7.2) to convert Ordnance Survey GML
into ESRI shapefiles.
The 'orientation' field is being ignored when I convert the
CartographicText layer, although this field gets successfully translated
for the
CartographicSymbol layer. All the other CartographicText layer fields
appear just fine in the generated '.dbf' file.
Is there a way of forcing ogr2ogr to recognise the orientation field?
Can anyone suggest an alternative Open Source conversion utility? I'm
fairly new to GIS systems, so I might be making an obvious mistake, but
any advice is very gratefully received! :-)
--
View this message in context:
http://osgeo-org.1803224.n2.nabble.com/ogr2ogr-problem-converting-UK-Ordn
ance-Survey-MasterMap-tp5237796p5237796.html Sent from the GDAL - Dev
mailing list archive at Nabble.com.
_______________________________________________
gdal-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
The information transmitted is intended only for the person
or entity to which it is addressed and may contain
confidential and/or privileged material. If you are not the
addressee, any disclosure, reproduction, copying,
distribution, or other dissemination or use of this
communication is strictly prohibited. If you have received
this transmission in error please notify the sender
immediately and then delete this email.
Any representations or commitments expressed in this email
are subject to contract.
This message has been scanned for viruses and dangerous
content. However, it is essential that the recipient also
checks this message using commercially available mail
scanning and anti-virus software. IPL Information Processing
Limited accepts no liability for any loss or damage resulting
from any virus or other dangerous content in this message.
IPL Information Processing Limited is registered in England
and Wales under company registration number 1418818.
Registration took place at Cardiff on 10 May 1979. IPL
Information Processing Limited's registered office and
normal place of business is Eveleigh House, Grove Street,
Bath, BA1 5LR, United Kingdom. IPL is also registered for
Value Added Tax (VAT) under registration number GB 601 2931 83.
_______________________________________________
gdal-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
--------------------------------------------------------------------------------
Peter J Halls, GIS Advisor, University of York
Telephone: 01904 433806 Fax: 01904 433740
Snail mail: Computing Service, University of York, Heslington, York YO10 5DD
This message has the status of a private and personal communication
--------------------------------------------------------------------------------
_______________________________________________
gdal-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
