Re: [mapserver-users] Expression segmentation fault
Frank, can you try without including the hstore tags in your select? -- thomas On Thu, Sep 5, 2013 at 3:49 PM, Frank Broniewski b...@metrico.lu wrote: Ok, this is weird. It is, as I know now, data source related. I was poking in the layer configuration to find out why the core dump happens. Reinstalling didn't solve the issue, btw. So first I converted the data to shapefile with ogr2ogr (as always very handy!) and changed the layer accordingly. And no core dump happened! Then I tried several versions of the data and filter statements without success. Since this is a osm2pgsql database I swapped finally the table from planet_osm_polygon to planet_osm_line (same table schema apart from the geometry) - and voilà - no core dump. So I can relate the problem to the planet_osm_polygon table - but I'm not sure what difference actually is responsible for the crash ... The most apparent difference is of course the geometry type: planet_osm_line is linestring, planet_osm_polygon is geometry ... I will investigate the attributes further tomorrow, but now I need to earn some money. It's always the deadlines ... Frank Am 2013-09-05 09:47, schrieb thomas bonfort: Frank, This is such a simple use-case that I doubt it's a bug in the code per-se. Can you make sure that you are using a clean build (make clean make make install) as I suspect this could be happening due to a compilation problem (as a rule of thumb, with mapserver 6.4, always run make clean after re-running configure) -- thomas On Wed, Sep 4, 2013 at 2:39 PM, Frank Broniewski b...@metrico.lu wrote: Am 2013-09-04 14:33, schrieb thomas bonfort: Frank, please supply a backtrace of the crash (`bt` in gdb once it has halted at the segfault) -- thomas On Wed, Sep 4, 2013 at 2:25 PM, Frank Broniewski b...@metrico.lu wrote: Hi, I just updated my mapserver installation to the latest version (6.2.1). I'm running FreeBSD 9.1 and I'm getting a segmentation fault related to expression usage in my mapfile. I've already compiled mapserver with debug symbols and I loaded the core file into gdb: gdb /usr/local/www/apache22/cgi-bin/mapserv mapserv.core snip Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/libintl.so.9...done. Loaded symbols for /usr/local/lib/libintl.so.9 Reading symbols from /usr/lib/libsupc++.so.1...done. Loaded symbols for /usr/lib/libsupc++.so.1 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0008008e2520 in yylex (lvalp=0x7fffd250, p=0x7fffd3b0) at mapparser.y:649 649 mapparser.y: No such file or directory. in mapparser.y [New Thread 809007400 (LWP 101134/mapserv)] So apparently there seems to be a file missing!? Looking at the source code (git), line 649 is a blank line, so I'm not sure what's missing exactly. I've got yacc, bison and flex installed. For testing purposes I'm using the mapserver cgi on the command line: /usr/local/www/apache22/cgi-bin/mapserv -nh QUERY_STRING=map=/data/web/mapserver/cnra/cnra.mapmode=maplayer=boundaries The boundaries layer: Layer #Classitem level Connection host=10.0.0.2 dbname=osm user=user password=guessme Connectiontype Postgis Data geom FROM (SELECT osm_id, way AS geom, name, admin_level AS level, tags FROM planet_osm_polygon) AS foo USING UNIQUE osm_id USING SRID=3857 Filter tags ? ARRAY['boundary', 'admin_level'] Name boundaries Processing CLOSE_CONNECTION=DEFER Status on Type line Units meters Metadata ows_title Boundary Map ows_abstract Boundary map - data from OpenStreetMap, ODbl licensed End Projection init=epsg:3857 End Class Expression ([level] = 6) #Expression 6 Name communes Style Color 10 10 10 Opacity 50 Width 2 End End End The classitem / simple expression (Expression 6) works with the mapserver cgi, but python mapscript throws an error: _mapscript.MapServerError: msEvalExpression(): General error message. Invalid item index. I've read the mapserver expressions documentation [1] and the note that says something about the working environment might be linked to more than one expression library. But my operating system skills are not high enough to turn this paragraph into something useful for me. So any help is greatly appreciated. Frank Frank BRONIEWSKI METRICO s.à r.l. géomètres technologies d'information géographique rue des Romains 36 L-5433 NIEDERDONVEN tél.: +352 26 74 94 - 28 fax.: +352 26 74 94 99 http://www.metrico.lu ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Expression segmentation fault
Sorry, but it's not that simple :-(: My current layer configuration: I will write a PostGIS query and see what differences turn up between the line and polygon tables and if it is really the geometry as Jukka Rahkonen and Steve W suggest ... Layer Connection host=10.0.0.2 dbname=osm user=me password=guessit Connectiontype Postgis Data way from (select osm_id, way, admin_level from planet_osm_polygon) as foo using unique osm_id using srid=2169 #Filter tags ? ARRAY['boundary', 'admin_level'] name boundaries status on type line units meters Projection init=epsg:3857 End Class expression ([admin_level] = 6) name communes Style color 10 10 10 width 2 End End End Am 2013-09-06 10:16, schrieb thomas bonfort: Frank, can you try without including the hstore tags in your select? -- thomas On Thu, Sep 5, 2013 at 3:49 PM, Frank Broniewski b...@metrico.lu wrote: Ok, this is weird. It is, as I know now, data source related. I was poking in the layer configuration to find out why the core dump happens. Reinstalling didn't solve the issue, btw. So first I converted the data to shapefile with ogr2ogr (as always very handy!) and changed the layer accordingly. And no core dump happened! Then I tried several versions of the data and filter statements without success. Since this is a osm2pgsql database I swapped finally the table from planet_osm_polygon to planet_osm_line (same table schema apart from the geometry) - and voilà - no core dump. So I can relate the problem to the planet_osm_polygon table - but I'm not sure what difference actually is responsible for the crash ... The most apparent difference is of course the geometry type: planet_osm_line is linestring, planet_osm_polygon is geometry ... I will investigate the attributes further tomorrow, but now I need to earn some money. It's always the deadlines ... Frank Am 2013-09-05 09:47, schrieb thomas bonfort: Frank, This is such a simple use-case that I doubt it's a bug in the code per-se. Can you make sure that you are using a clean build (make clean make make install) as I suspect this could be happening due to a compilation problem (as a rule of thumb, with mapserver 6.4, always run make clean after re-running configure) -- thomas On Wed, Sep 4, 2013 at 2:39 PM, Frank Broniewski b...@metrico.lu wrote: Am 2013-09-04 14:33, schrieb thomas bonfort: Frank, please supply a backtrace of the crash (`bt` in gdb once it has halted at the segfault) -- thomas On Wed, Sep 4, 2013 at 2:25 PM, Frank Broniewski b...@metrico.lu wrote: Hi, I just updated my mapserver installation to the latest version (6.2.1). I'm running FreeBSD 9.1 and I'm getting a segmentation fault related to expression usage in my mapfile. I've already compiled mapserver with debug symbols and I loaded the core file into gdb: gdb /usr/local/www/apache22/cgi-bin/mapserv mapserv.core snip Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/libintl.so.9...done. Loaded symbols for /usr/local/lib/libintl.so.9 Reading symbols from /usr/lib/libsupc++.so.1...done. Loaded symbols for /usr/lib/libsupc++.so.1 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0008008e2520 in yylex (lvalp=0x7fffd250, p=0x7fffd3b0) at mapparser.y:649 649 mapparser.y: No such file or directory. in mapparser.y [New Thread 809007400 (LWP 101134/mapserv)] So apparently there seems to be a file missing!? Looking at the source code (git), line 649 is a blank line, so I'm not sure what's missing exactly. I've got yacc, bison and flex installed. For testing purposes I'm using the mapserver cgi on the command line: /usr/local/www/apache22/cgi-bin/mapserv -nh QUERY_STRING=map=/data/web/mapserver/cnra/cnra.mapmode=maplayer=boundaries The boundaries layer: Layer #Classitem level Connection host=10.0.0.2 dbname=osm user=user password=guessme Connectiontype Postgis Data geom FROM (SELECT osm_id, way AS geom, name, admin_level AS level, tags FROM planet_osm_polygon) AS foo USING UNIQUE osm_id USING SRID=3857 Filter tags ? ARRAY['boundary', 'admin_level'] Name boundaries Processing CLOSE_CONNECTION=DEFER Status on Type line Units meters Metadata ows_title Boundary Map ows_abstract Boundary map - data from OpenStreetMap, ODbl licensed End Projection init=epsg:3857 End Class Expression ([level] = 6) #Expression 6 Name communes Style Color 10 10 10 Opacity 50 Width 2 End End End The classitem / simple expression (Expression 6) works with the mapserver cgi, but python mapscript throws an error:
Re: [mapserver-users] Expression segmentation fault
Ok, so I did a little bit of geometry testing. Don't know why that
should cause an expression crash, but well ...
here's the query I ran on planet_osm_line and planet_osm_polygon in PostGIS:
select
osm_id, boundary, admin_level, tags,
st_isvalid(way) as isvalid,
st_isvalidreason(way) as reason,
st_isclosed(way) as isclosed,
st_isempty(way) as isempty,
st_geometrytype(way) as geometrytype,
st_length(way) as length,
st_perimeter(way) as perimeter,
st_numgeometries(way) as numgeometries,
st_numinteriorrings(way) as numinteriorrings,
st_astext(way) as wkt
from planet_osm_polygon
where way st_transform(st_setsrid('BOX3D(0 0, 15 20)'::box3d,
2169), 3857)
and tags ? ARRAY['boundary', 'admin_level']
order by geometrytype, admin_level, name
There's one record that differs from the rest in the planet_osm_polygon
table. I've pasted the result below. It has a geometrytype of
GEOMETRYCOLLECTION and is empty, so that may cause a crash ...
229382895;administrative;8;area=yes, name=RW 10,
boundary=administrative, admin_level=8, flood_prone=no,
is_in:hamlet=MENTENG DALAM, is_in:district=Jakarta Selatan,
is_in:province=DKI Jakarta, is_in:subdistrict=TEBET;t;Valid
Geometry;;t;ST_GeometryCollection;0;0;0;;GEOMETRYCOLLECTION EMPTY
The attributes confirm, that this record could be a remain of
http://www.openstreetmap.org/browse/way/229382895 , but I really wonder
why it gets caught in the bbox which should center around Luxembourg
with parts of the surrounding countries. But when the geometry is a
black hole you never know :-)
Am 2013-09-05 16:30, schrieb Stephen Woodbridge:
Hi,
This is a great idea. Regardless we should try to identify the case that
is causing the crash and trap that. Mapserver should never crash as a
general rule so finding the specific case is important so Thomas can
reproduce it.
If you have (or can load) the data in a database table, then it should
be fairly easy to do a manual binary search to find the offending object
by adding where gid between lower and upper to your query and
adjusting the value of lower and upper to narrow the search to the
problem object.
Thanks,
-Steve W
On 9/5/2013 10:11 AM, Rahkonen Jukka wrote:
Hi,
Osm2pgsql can create invalid polygons with self-intersections and/or
three-vertex A-B-A rings. It might be good to run MakeValid or simply
find possibly invalid features with IsValid and delete them and then
see if Mapserver gets happy. Actually, run IsValid and save the
faulty geometries somewhere before correcting or deleting them. It
would be nice to catch them if they happen to be the reason for the
crash.
-Jukka Rahkonen-
Frank Broniewski wrote:
Ok, this is weird. It is, as I know now, data source related. I was
poking in the layer configuration to find out why the core dump
happens. Reinstalling didn't solve the issue, btw.
So first I converted the data to shapefile with ogr2ogr (as always
very handy!) and changed the layer accordingly. And no core dump
happened! Then I tried several versions of the data and filter
statements without success. Since this is a osm2pgsql database I
swapped finally the table from planet_osm_polygon to
planet_osm_line (same table schema apart from the geometry) - and
voilà - no core dump.
So I can relate the problem to the planet_osm_polygon table - but
I'm not sure what difference actually is responsible for the crash
...
The most apparent difference is of course the geometry type:
planet_osm_line is linestring, planet_osm_polygon is geometry ...
I will investigate the attributes further tomorrow, but now I need
to earn some money. It's always the deadlines ...
Frank
Am 2013-09-05 09:47, schrieb thomas bonfort:
Frank, This is such a simple use-case that I doubt it's a bug in
the code per-se. Can you make sure that you are using a clean
build (make clean make make install) as I suspect this
could be happening due to a compilation problem (as a rule of
thumb, with mapserver 6.4, always run make clean after
re-running configure)
-- thomas
On Wed, Sep 4, 2013 at 2:39 PM, Frank Broniewski
b...@metrico.lu wrote:
Am 2013-09-04 14:33, schrieb thomas bonfort:
Frank, please supply a backtrace of the crash (`bt` in gdb
once it has halted at the segfault)
-- thomas
On Wed, Sep 4, 2013 at 2:25 PM, Frank Broniewski
b...@metrico.lu
wrote:
Hi,
I just updated my mapserver installation to the latest
version (6.2.1). I'm running FreeBSD 9.1 and I'm getting a
segmentation fault related to expression usage in my
mapfile. I've already compiled mapserver with debug symbols
and I loaded the core file into gdb:
gdb /usr/local/www/apache22/cgi-bin/mapserv mapserv.core
snip Reading symbols from /lib/libc.so.7...done. Loaded
symbols for /lib/libc.so.7 Reading symbols from
/usr/local/lib/libintl.so.9...done. Loaded symbols for
/usr/local/lib/libintl.so.9 Reading symbols from
/usr/lib/libsupc++.so.1...done. Loaded symbols for
/usr/lib/libsupc++.so.1 Reading symbols from
Re: [mapserver-users] Expression segmentation fault
Does this feature in particular crash mapserver?
On Fri, Sep 6, 2013 at 2:20 PM, Frank Broniewski b...@metrico.lu wrote:
Ok, so I did a little bit of geometry testing. Don't know why that should
cause an expression crash, but well ...
here's the query I ran on planet_osm_line and planet_osm_polygon in PostGIS:
select
osm_id, boundary, admin_level, tags,
st_isvalid(way) as isvalid,
st_isvalidreason(way) as reason,
st_isclosed(way) as isclosed,
st_isempty(way) as isempty,
st_geometrytype(way) as geometrytype,
st_length(way) as length,
st_perimeter(way) as perimeter,
st_numgeometries(way) as numgeometries,
st_numinteriorrings(way) as numinteriorrings,
st_astext(way) as wkt
from planet_osm_polygon
where way st_transform(st_setsrid('BOX3D(0 0, 15 20)'::box3d,
2169), 3857)
and tags ? ARRAY['boundary', 'admin_level']
order by geometrytype, admin_level, name
There's one record that differs from the rest in the planet_osm_polygon
table. I've pasted the result below. It has a geometrytype of
GEOMETRYCOLLECTION and is empty, so that may cause a crash ...
229382895;administrative;8;area=yes, name=RW 10,
boundary=administrative, admin_level=8, flood_prone=no,
is_in:hamlet=MENTENG DALAM, is_in:district=Jakarta Selatan,
is_in:province=DKI Jakarta, is_in:subdistrict=TEBET;t;Valid
Geometry;;t;ST_GeometryCollection;0;0;0;;GEOMETRYCOLLECTION EMPTY
The attributes confirm, that this record could be a remain of
http://www.openstreetmap.org/browse/way/229382895 , but I really wonder why
it gets caught in the bbox which should center around Luxembourg with parts
of the surrounding countries. But when the geometry is a black hole you
never know :-)
Am 2013-09-05 16:30, schrieb Stephen Woodbridge:
Hi,
This is a great idea. Regardless we should try to identify the case that
is causing the crash and trap that. Mapserver should never crash as a
general rule so finding the specific case is important so Thomas can
reproduce it.
If you have (or can load) the data in a database table, then it should
be fairly easy to do a manual binary search to find the offending object
by adding where gid between lower and upper to your query and
adjusting the value of lower and upper to narrow the search to the
problem object.
Thanks,
-Steve W
On 9/5/2013 10:11 AM, Rahkonen Jukka wrote:
Hi,
Osm2pgsql can create invalid polygons with self-intersections and/or
three-vertex A-B-A rings. It might be good to run MakeValid or simply
find possibly invalid features with IsValid and delete them and then
see if Mapserver gets happy. Actually, run IsValid and save the
faulty geometries somewhere before correcting or deleting them. It
would be nice to catch them if they happen to be the reason for the
crash.
-Jukka Rahkonen-
Frank Broniewski wrote:
Ok, this is weird. It is, as I know now, data source related. I was
poking in the layer configuration to find out why the core dump
happens. Reinstalling didn't solve the issue, btw.
So first I converted the data to shapefile with ogr2ogr (as always
very handy!) and changed the layer accordingly. And no core dump
happened! Then I tried several versions of the data and filter
statements without success. Since this is a osm2pgsql database I
swapped finally the table from planet_osm_polygon to
planet_osm_line (same table schema apart from the geometry) - and
voilà - no core dump.
So I can relate the problem to the planet_osm_polygon table - but
I'm not sure what difference actually is responsible for the crash
...
The most apparent difference is of course the geometry type:
planet_osm_line is linestring, planet_osm_polygon is geometry ...
I will investigate the attributes further tomorrow, but now I need
to earn some money. It's always the deadlines ...
Frank
Am 2013-09-05 09:47, schrieb thomas bonfort:
Frank, This is such a simple use-case that I doubt it's a bug in
the code per-se. Can you make sure that you are using a clean
build (make clean make make install) as I suspect this
could be happening due to a compilation problem (as a rule of
thumb, with mapserver 6.4, always run make clean after
re-running configure)
-- thomas
On Wed, Sep 4, 2013 at 2:39 PM, Frank Broniewski
b...@metrico.lu wrote:
Am 2013-09-04 14:33, schrieb thomas bonfort:
Frank, please supply a backtrace of the crash (`bt` in gdb
once it has halted at the segfault)
-- thomas
On Wed, Sep 4, 2013 at 2:25 PM, Frank Broniewski
b...@metrico.lu
wrote:
Hi,
I just updated my mapserver installation to the latest
version (6.2.1). I'm running FreeBSD 9.1 and I'm getting a
segmentation fault related to expression usage in my
mapfile. I've already compiled mapserver with debug symbols
and I loaded the core file into gdb:
gdb /usr/local/www/apache22/cgi-bin/mapserv mapserv.core
snip Reading symbols from /lib/libc.so.7...done. Loaded
symbols for /lib/libc.so.7 Reading symbols from
Re: [mapserver-users] Expression segmentation fault
No, it doesn't. Tried right now the mapfile on my desktop machine
(ubuntu) and it crashes here also (same error msg, more or less):
Segfault happened at: 0x7fb2bb94895c yylex+764: mov (%rax,%rdx,8),%rdi
PC (0x7fb2bb94895c) ok
source (%rax,%rdx,8) (0x) not located in a known VMA region
(needed readable region)!
destination %rdi ok
I will investigate further ...
Frank
Am 2013-09-06 14:24, schrieb thomas bonfort:
Does this feature in particular crash mapserver?
On Fri, Sep 6, 2013 at 2:20 PM, Frank Broniewski b...@metrico.lu wrote:
Ok, so I did a little bit of geometry testing. Don't know why that should
cause an expression crash, but well ...
here's the query I ran on planet_osm_line and planet_osm_polygon in PostGIS:
select
osm_id, boundary, admin_level, tags,
st_isvalid(way) as isvalid,
st_isvalidreason(way) as reason,
st_isclosed(way) as isclosed,
st_isempty(way) as isempty,
st_geometrytype(way) as geometrytype,
st_length(way) as length,
st_perimeter(way) as perimeter,
st_numgeometries(way) as numgeometries,
st_numinteriorrings(way) as numinteriorrings,
st_astext(way) as wkt
from planet_osm_polygon
where way st_transform(st_setsrid('BOX3D(0 0, 15 20)'::box3d,
2169), 3857)
and tags ? ARRAY['boundary', 'admin_level']
order by geometrytype, admin_level, name
There's one record that differs from the rest in the planet_osm_polygon
table. I've pasted the result below. It has a geometrytype of
GEOMETRYCOLLECTION and is empty, so that may cause a crash ...
229382895;administrative;8;area=yes, name=RW 10,
boundary=administrative, admin_level=8, flood_prone=no,
is_in:hamlet=MENTENG DALAM, is_in:district=Jakarta Selatan,
is_in:province=DKI Jakarta, is_in:subdistrict=TEBET;t;Valid
Geometry;;t;ST_GeometryCollection;0;0;0;;GEOMETRYCOLLECTION EMPTY
The attributes confirm, that this record could be a remain of
http://www.openstreetmap.org/browse/way/229382895 , but I really wonder why
it gets caught in the bbox which should center around Luxembourg with parts
of the surrounding countries. But when the geometry is a black hole you
never know :-)
Am 2013-09-05 16:30, schrieb Stephen Woodbridge:
Hi,
This is a great idea. Regardless we should try to identify the case that
is causing the crash and trap that. Mapserver should never crash as a
general rule so finding the specific case is important so Thomas can
reproduce it.
If you have (or can load) the data in a database table, then it should
be fairly easy to do a manual binary search to find the offending object
by adding where gid between lower and upper to your query and
adjusting the value of lower and upper to narrow the search to the
problem object.
Thanks,
-Steve W
On 9/5/2013 10:11 AM, Rahkonen Jukka wrote:
Hi,
Osm2pgsql can create invalid polygons with self-intersections and/or
three-vertex A-B-A rings. It might be good to run MakeValid or simply
find possibly invalid features with IsValid and delete them and then
see if Mapserver gets happy. Actually, run IsValid and save the
faulty geometries somewhere before correcting or deleting them. It
would be nice to catch them if they happen to be the reason for the
crash.
-Jukka Rahkonen-
Frank Broniewski wrote:
Ok, this is weird. It is, as I know now, data source related. I was
poking in the layer configuration to find out why the core dump
happens. Reinstalling didn't solve the issue, btw.
So first I converted the data to shapefile with ogr2ogr (as always
very handy!) and changed the layer accordingly. And no core dump
happened! Then I tried several versions of the data and filter
statements without success. Since this is a osm2pgsql database I
swapped finally the table from planet_osm_polygon to
planet_osm_line (same table schema apart from the geometry) - and
voilà - no core dump.
So I can relate the problem to the planet_osm_polygon table - but
I'm not sure what difference actually is responsible for the crash
...
The most apparent difference is of course the geometry type:
planet_osm_line is linestring, planet_osm_polygon is geometry ...
I will investigate the attributes further tomorrow, but now I need
to earn some money. It's always the deadlines ...
Frank
Am 2013-09-05 09:47, schrieb thomas bonfort:
Frank, This is such a simple use-case that I doubt it's a bug in
the code per-se. Can you make sure that you are using a clean
build (make clean make make install) as I suspect this
could be happening due to a compilation problem (as a rule of
thumb, with mapserver 6.4, always run make clean after
re-running configure)
-- thomas
On Wed, Sep 4, 2013 at 2:39 PM, Frank Broniewski
b...@metrico.lu wrote:
Am 2013-09-04 14:33, schrieb thomas bonfort:
Frank, please supply a backtrace of the crash (`bt` in gdb
once it has halted at the segfault)
-- thomas
On Wed, Sep 4, 2013 at 2:25 PM, Frank Broniewski
b...@metrico.lu
wrote:
Hi,
I just updated my mapserver installation to the latest
version (6.2.1). I'm running FreeBSD 9.1 and I'm
Re: [mapserver-users] Expression segmentation fault
Ok, maybe it does it when called in a resultset?
I ran the mapfile with -all_debug 5 and the last two
msPostGISLayerNextShape calls (without the geometry) are below:
msPostGISLayerNextShape called.
msPostGISReadShape called.
msPostGISReadShape: PQgetlength = 0
msPostGISReadShape: [admin_level]
msPostGISReadShape: Setting shape-index = -2371818
msPostGISReadShape: Setting shape-resultindex = 5640
msPostGISReadShape: [index] -2371818
msPostGISLayerNextShape called.
msPostGISReadShape called.
msPostGISReadShape: [shape] (null)
The last msPostGISLayerNextShape stalls without further output and the
the segmentation fault happens. Maybe I can find out if the [shape]
(null) record in the DB is the same GEOMETRYCOLLECTION as already
reported ...
Dubious at least,
Frank
Am 2013-09-06 14:24, schrieb thomas bonfort:
Does this feature in particular crash mapserver?
On Fri, Sep 6, 2013 at 2:20 PM, Frank Broniewski b...@metrico.lu wrote:
Ok, so I did a little bit of geometry testing. Don't know why that should
cause an expression crash, but well ...
here's the query I ran on planet_osm_line and planet_osm_polygon in PostGIS:
select
osm_id, boundary, admin_level, tags,
st_isvalid(way) as isvalid,
st_isvalidreason(way) as reason,
st_isclosed(way) as isclosed,
st_isempty(way) as isempty,
st_geometrytype(way) as geometrytype,
st_length(way) as length,
st_perimeter(way) as perimeter,
st_numgeometries(way) as numgeometries,
st_numinteriorrings(way) as numinteriorrings,
st_astext(way) as wkt
from planet_osm_polygon
where way st_transform(st_setsrid('BOX3D(0 0, 15 20)'::box3d,
2169), 3857)
and tags ? ARRAY['boundary', 'admin_level']
order by geometrytype, admin_level, name
There's one record that differs from the rest in the planet_osm_polygon
table. I've pasted the result below. It has a geometrytype of
GEOMETRYCOLLECTION and is empty, so that may cause a crash ...
229382895;administrative;8;area=yes, name=RW 10,
boundary=administrative, admin_level=8, flood_prone=no,
is_in:hamlet=MENTENG DALAM, is_in:district=Jakarta Selatan,
is_in:province=DKI Jakarta, is_in:subdistrict=TEBET;t;Valid
Geometry;;t;ST_GeometryCollection;0;0;0;;GEOMETRYCOLLECTION EMPTY
The attributes confirm, that this record could be a remain of
http://www.openstreetmap.org/browse/way/229382895 , but I really wonder why
it gets caught in the bbox which should center around Luxembourg with parts
of the surrounding countries. But when the geometry is a black hole you
never know :-)
Am 2013-09-05 16:30, schrieb Stephen Woodbridge:
Hi,
This is a great idea. Regardless we should try to identify the case that
is causing the crash and trap that. Mapserver should never crash as a
general rule so finding the specific case is important so Thomas can
reproduce it.
If you have (or can load) the data in a database table, then it should
be fairly easy to do a manual binary search to find the offending object
by adding where gid between lower and upper to your query and
adjusting the value of lower and upper to narrow the search to the
problem object.
Thanks,
-Steve W
On 9/5/2013 10:11 AM, Rahkonen Jukka wrote:
Hi,
Osm2pgsql can create invalid polygons with self-intersections and/or
three-vertex A-B-A rings. It might be good to run MakeValid or simply
find possibly invalid features with IsValid and delete them and then
see if Mapserver gets happy. Actually, run IsValid and save the
faulty geometries somewhere before correcting or deleting them. It
would be nice to catch them if they happen to be the reason for the
crash.
-Jukka Rahkonen-
Frank Broniewski wrote:
Ok, this is weird. It is, as I know now, data source related. I was
poking in the layer configuration to find out why the core dump
happens. Reinstalling didn't solve the issue, btw.
So first I converted the data to shapefile with ogr2ogr (as always
very handy!) and changed the layer accordingly. And no core dump
happened! Then I tried several versions of the data and filter
statements without success. Since this is a osm2pgsql database I
swapped finally the table from planet_osm_polygon to
planet_osm_line (same table schema apart from the geometry) - and
voilà - no core dump.
So I can relate the problem to the planet_osm_polygon table - but
I'm not sure what difference actually is responsible for the crash
...
The most apparent difference is of course the geometry type:
planet_osm_line is linestring, planet_osm_polygon is geometry ...
I will investigate the attributes further tomorrow, but now I need
to earn some money. It's always the deadlines ...
Frank
Am 2013-09-05 09:47, schrieb thomas bonfort:
Frank, This is such a simple use-case that I doubt it's a bug in
the code per-se. Can you make sure that you are using a clean
build (make clean make make install) as I suspect this
could be happening due to a compilation problem (as a rule of
thumb, with mapserver 6.4, always run make clean after
re-running configure)
-- thomas
On Wed, Sep 4,
[mapserver-users] MapServer 6.4 Release Candidate released
The MapServer Team is pleased to announce the release of MapServer 6.4.0 RC1. This is a release candiate for the 6.4 release, and contains only minor fixes compared to beta2. We currently have no blocking issues, so depending on this feedback phase our next release will hopefully be a final 6.4.0. The changelog since beta2 can be consulted here: http://mapserver.org/development/changelog/changelog-6-4.html#changelog-6-4-rc1 The source for this release can be downloaded at: http://mapserver.org/download.html or http://download.osgeo.org/mapserver/mapserver-6.4.0-rc1.tar.gz Once again we need your help to ensure a high quality product, so please help out by testing your applications with this new code base. Thanks! - The MapServer Team ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Expression segmentation fault
Frank,
I'm able to reproduce this with a geometrycollection empty. What's
strange is I was obliged to force postgis to return the feature, as
the bounding box where clause was filtering it out and it was
therefore never returned to mapserver.
I'll come up with a patch soon, would you mind opening an issue for this please?
thanks,
thomas
On Fri, Sep 6, 2013 at 2:20 PM, Frank Broniewski b...@metrico.lu wrote:
Ok, so I did a little bit of geometry testing. Don't know why that should
cause an expression crash, but well ...
here's the query I ran on planet_osm_line and planet_osm_polygon in PostGIS:
select
osm_id, boundary, admin_level, tags,
st_isvalid(way) as isvalid,
st_isvalidreason(way) as reason,
st_isclosed(way) as isclosed,
st_isempty(way) as isempty,
st_geometrytype(way) as geometrytype,
st_length(way) as length,
st_perimeter(way) as perimeter,
st_numgeometries(way) as numgeometries,
st_numinteriorrings(way) as numinteriorrings,
st_astext(way) as wkt
from planet_osm_polygon
where way st_transform(st_setsrid('BOX3D(0 0, 15 20)'::box3d,
2169), 3857)
and tags ? ARRAY['boundary', 'admin_level']
order by geometrytype, admin_level, name
There's one record that differs from the rest in the planet_osm_polygon
table. I've pasted the result below. It has a geometrytype of
GEOMETRYCOLLECTION and is empty, so that may cause a crash ...
229382895;administrative;8;area=yes, name=RW 10,
boundary=administrative, admin_level=8, flood_prone=no,
is_in:hamlet=MENTENG DALAM, is_in:district=Jakarta Selatan,
is_in:province=DKI Jakarta, is_in:subdistrict=TEBET;t;Valid
Geometry;;t;ST_GeometryCollection;0;0;0;;GEOMETRYCOLLECTION EMPTY
The attributes confirm, that this record could be a remain of
http://www.openstreetmap.org/browse/way/229382895 , but I really wonder why
it gets caught in the bbox which should center around Luxembourg with parts
of the surrounding countries. But when the geometry is a black hole you
never know :-)
Am 2013-09-05 16:30, schrieb Stephen Woodbridge:
Hi,
This is a great idea. Regardless we should try to identify the case that
is causing the crash and trap that. Mapserver should never crash as a
general rule so finding the specific case is important so Thomas can
reproduce it.
If you have (or can load) the data in a database table, then it should
be fairly easy to do a manual binary search to find the offending object
by adding where gid between lower and upper to your query and
adjusting the value of lower and upper to narrow the search to the
problem object.
Thanks,
-Steve W
On 9/5/2013 10:11 AM, Rahkonen Jukka wrote:
Hi,
Osm2pgsql can create invalid polygons with self-intersections and/or
three-vertex A-B-A rings. It might be good to run MakeValid or simply
find possibly invalid features with IsValid and delete them and then
see if Mapserver gets happy. Actually, run IsValid and save the
faulty geometries somewhere before correcting or deleting them. It
would be nice to catch them if they happen to be the reason for the
crash.
-Jukka Rahkonen-
Frank Broniewski wrote:
Ok, this is weird. It is, as I know now, data source related. I was
poking in the layer configuration to find out why the core dump
happens. Reinstalling didn't solve the issue, btw.
So first I converted the data to shapefile with ogr2ogr (as always
very handy!) and changed the layer accordingly. And no core dump
happened! Then I tried several versions of the data and filter
statements without success. Since this is a osm2pgsql database I
swapped finally the table from planet_osm_polygon to
planet_osm_line (same table schema apart from the geometry) - and
voilà - no core dump.
So I can relate the problem to the planet_osm_polygon table - but
I'm not sure what difference actually is responsible for the crash
...
The most apparent difference is of course the geometry type:
planet_osm_line is linestring, planet_osm_polygon is geometry ...
I will investigate the attributes further tomorrow, but now I need
to earn some money. It's always the deadlines ...
Frank
Am 2013-09-05 09:47, schrieb thomas bonfort:
Frank, This is such a simple use-case that I doubt it's a bug in
the code per-se. Can you make sure that you are using a clean
build (make clean make make install) as I suspect this
could be happening due to a compilation problem (as a rule of
thumb, with mapserver 6.4, always run make clean after
re-running configure)
-- thomas
On Wed, Sep 4, 2013 at 2:39 PM, Frank Broniewski
b...@metrico.lu wrote:
Am 2013-09-04 14:33, schrieb thomas bonfort:
Frank, please supply a backtrace of the crash (`bt` in gdb
once it has halted at the segfault)
-- thomas
On Wed, Sep 4, 2013 at 2:25 PM, Frank Broniewski
b...@metrico.lu
wrote:
Hi,
I just updated my mapserver installation to the latest
version (6.2.1). I'm running FreeBSD 9.1 and I'm getting a
segmentation fault related to expression usage in
Re: [mapserver-users] Expression segmentation fault
and here is a patch for mappostgis.c. it will apply cleanly to master,
you may have to massage it a bit for 6.2 but that should be simple:
diff --git a/mappostgis.c b/mappostgis.c
index 0631b79..de90766 100644
--- a/mappostgis.c
+++ b/mappostgis.c
@@ -586,9 +586,9 @@ wkbConvCollectionToShape(wkbObj *w, shapeObj *shape)
wkbSkipGeometry(w);
failures++;
}
}
- if ( failures == ncomponents )
+ if ( failures == ncomponents || ncomponents == 0)
return MS_FAILURE;
else
return MS_SUCCESS;
}
@@ -2195,8 +2195,10 @@ int msPostGISReadShape(layerObj *layer, shapeObj *shape)
shape-numvalues = layer-numitems;
msComputeBounds(shape);
+ } else {
+shape-type = MS_SHAPE_NULL;
}
if( layer-debug 2 ) {
char *tmp = msShapeToWKT(shape);
On Fri, Sep 6, 2013 at 4:40 PM, thomas bonfort thomas.bonf...@gmail.com wrote:
Frank,
I'm able to reproduce this with a geometrycollection empty. What's
strange is I was obliged to force postgis to return the feature, as
the bounding box where clause was filtering it out and it was
therefore never returned to mapserver.
I'll come up with a patch soon, would you mind opening an issue for this
please?
thanks,
thomas
On Fri, Sep 6, 2013 at 2:20 PM, Frank Broniewski b...@metrico.lu wrote:
Ok, so I did a little bit of geometry testing. Don't know why that should
cause an expression crash, but well ...
here's the query I ran on planet_osm_line and planet_osm_polygon in PostGIS:
select
osm_id, boundary, admin_level, tags,
st_isvalid(way) as isvalid,
st_isvalidreason(way) as reason,
st_isclosed(way) as isclosed,
st_isempty(way) as isempty,
st_geometrytype(way) as geometrytype,
st_length(way) as length,
st_perimeter(way) as perimeter,
st_numgeometries(way) as numgeometries,
st_numinteriorrings(way) as numinteriorrings,
st_astext(way) as wkt
from planet_osm_polygon
where way st_transform(st_setsrid('BOX3D(0 0, 15 20)'::box3d,
2169), 3857)
and tags ? ARRAY['boundary', 'admin_level']
order by geometrytype, admin_level, name
There's one record that differs from the rest in the planet_osm_polygon
table. I've pasted the result below. It has a geometrytype of
GEOMETRYCOLLECTION and is empty, so that may cause a crash ...
229382895;administrative;8;area=yes, name=RW 10,
boundary=administrative, admin_level=8, flood_prone=no,
is_in:hamlet=MENTENG DALAM, is_in:district=Jakarta Selatan,
is_in:province=DKI Jakarta, is_in:subdistrict=TEBET;t;Valid
Geometry;;t;ST_GeometryCollection;0;0;0;;GEOMETRYCOLLECTION EMPTY
The attributes confirm, that this record could be a remain of
http://www.openstreetmap.org/browse/way/229382895 , but I really wonder why
it gets caught in the bbox which should center around Luxembourg with parts
of the surrounding countries. But when the geometry is a black hole you
never know :-)
Am 2013-09-05 16:30, schrieb Stephen Woodbridge:
Hi,
This is a great idea. Regardless we should try to identify the case that
is causing the crash and trap that. Mapserver should never crash as a
general rule so finding the specific case is important so Thomas can
reproduce it.
If you have (or can load) the data in a database table, then it should
be fairly easy to do a manual binary search to find the offending object
by adding where gid between lower and upper to your query and
adjusting the value of lower and upper to narrow the search to the
problem object.
Thanks,
-Steve W
On 9/5/2013 10:11 AM, Rahkonen Jukka wrote:
Hi,
Osm2pgsql can create invalid polygons with self-intersections and/or
three-vertex A-B-A rings. It might be good to run MakeValid or simply
find possibly invalid features with IsValid and delete them and then
see if Mapserver gets happy. Actually, run IsValid and save the
faulty geometries somewhere before correcting or deleting them. It
would be nice to catch them if they happen to be the reason for the
crash.
-Jukka Rahkonen-
Frank Broniewski wrote:
Ok, this is weird. It is, as I know now, data source related. I was
poking in the layer configuration to find out why the core dump
happens. Reinstalling didn't solve the issue, btw.
So first I converted the data to shapefile with ogr2ogr (as always
very handy!) and changed the layer accordingly. And no core dump
happened! Then I tried several versions of the data and filter
statements without success. Since this is a osm2pgsql database I
swapped finally the table from planet_osm_polygon to
planet_osm_line (same table schema apart from the geometry) - and
voilà - no core dump.
So I can relate the problem to the planet_osm_polygon table - but
I'm not sure what difference actually is responsible for the crash
...
The most apparent difference is of course the geometry type:
planet_osm_line is linestring, planet_osm_polygon is geometry ...
I will investigate the attributes further tomorrow, but now I need
to earn some money. It's always the deadlines ...
[mapserver-users] New ScribeUI 0.3 Release
Hi List, ScribeUI is a tool meant to help the editing and management of mapfiles, supporting Scribe and Standard mapfile syntax. I am currently developing this tool as a Google Summer of Code project. The Google Summer of Code ending very soon, the goal of this new release is bugfixes, and the beginning of the new plugin system! Please keep in mind that a lot of bugs remain, any feedback is welcome, should it be new feature ideas, general comments or bug reports. *What's new ?* - Fixed issue #11. This bug caused the client not to keep up with maps deleted in the backend. - Added the current zoom level on the map! ( issue #12 ) This makes it much easier to define your Scribe scale ranges. - Fixed font size issue #14, the browser tab is now much more comfortable to use. *Work In Progress* Still in progress is a simple plugin system. Some functions were added on the client side to make it easy to add buttons and tabs to the interface. The back-end side of plugins is not supported yet. *Download* ScribeUI and its installation instructions are available on github: https://github.com/mapgears/scribeui To get ScribeUI 0.3, you can clone the repo on github and checkout v0.3 or you can download an archive: https://github.com/mapgears/scribeui/releases/tag/v0.3* https://github.com/mapgears/scribeui/releases/tag/v0.3* Bug reports can be made on github: https://github.com/mapgears/scribeui/issues Thanks a lot! ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] mapserver minimum installation
what does installing mapserver mean (on windows, so no apt-get)? i don't mean also installing server, database or language. not even mapcache if that's possible. no optional libraries either and no changing environment variables. just plain vanilla mapserver. does it mean simply dropping mapserv.exe in apache's cgi-bin folder along with its dlls? http://mapserver.org/mapcache/install.html jzs ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] using MASK in context of WCS 1.0.0 GetCoverage
Hi, i try to use a MASK layer to clip a WCS layer (5 bands a 16bit) in a GetCoverage request. It works fine when i use it in context of WMS GetMap requests. But the GetCoverage requests just ignores the MASK and i get my result unclipped. I would expect that the requested Geotiff contains NoData values where the mask was applied but it does'nt. Any idea why this is not working as expected? requested outputformat is OUTPUTFORMAT NAME GEOTIFF_INT16 DRIVER GDAL/GTiff IMAGEMODE INT16 EXTENSION tif FORMATOPTION COMPRESS=DEFLATE FORMATOPTION ZLEVEL=8 FORMATOPTION TILED=YES FORMATOPTION PREDICTOR=2 FORMATOPTION NULLVALUE=0 END Thanks for any hint on that topic. I am using Mapserver 6.4beta2 with latest GDAL from trunk. Kind regards, Peter ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] [mapserver-dev] New ScribeUI 0.3 Release
Jessica, would you mind sharing some screenshots? On Sep 6, 2013 10:31 PM, Jessica Lapointe jlapoi...@mapgears.com wrote: Hi List, ScribeUI is a tool meant to help the editing and management of mapfiles, supporting Scribe and Standard mapfile syntax. I am currently developing this tool as a Google Summer of Code project. The Google Summer of Code ending very soon, the goal of this new release is bugfixes, and the beginning of the new plugin system! Please keep in mind that a lot of bugs remain, any feedback is welcome, should it be new feature ideas, general comments or bug reports. *What's new ?* - Fixed issue #11. This bug caused the client not to keep up with maps deleted in the backend. - Added the current zoom level on the map! ( issue #12 ) This makes it much easier to define your Scribe scale ranges. - Fixed font size issue #14, the browser tab is now much more comfortable to use. *Work In Progress* Still in progress is a simple plugin system. Some functions were added on the client side to make it easy to add buttons and tabs to the interface. The back-end side of plugins is not supported yet. *Download* ScribeUI and its installation instructions are available on github: https://github.com/mapgears/scribeui To get ScribeUI 0.3, you can clone the repo on github and checkout v0.3 or you can download an archive: https://github.com/mapgears/scribeui/releases/tag/v0.3* https://github.com/mapgears/scribeui/releases/tag/v0.3* Bug reports can be made on github: https://github.com/mapgears/scribeui/issues Thanks a lot! ___ mapserver-dev mailing list mapserver-...@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-dev ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
