Hi Steve,
Thanks for asking. I put the details in:
https://github.com/mapserver/mapserver/issues/5120
I don't know what changed from 6.2 to 6.4 other than it seems that
all(many?) default values changed which changed the behavior significantly.
I was able to get things working by using a combination of setting
explicitly various tags in the layer.
My solution works and it turns out that I was able to fit all the
queries into two classes of problems
1. one solved with TOLERANCE 0
2. one solved with PROCESSING "RASTER_QUERY_MAX_RESULT=1" and change the
ajax handler
So now I can get to the real work I need to do. Upgrading the site was
only a side show. I always underestimate that "sure that can be done as
part of ..." always takes more time and has gotchas :)
I trace a lot of code through gdb, but there have been a lot of changes
to mapserver since I last did a deep dive into the source. I think
someone needs to look at this and probably just add a note to the
MIGRATION-GUIDE and maybe reference the issue above for details. Obvious
it has not been a problem for other people ot they found a way around it.
Thanks,
-Steve W
On 7/16/2015 2:15 PM, Lime, Steve D (MNIT) wrote:
Steve: Did you ever track down what changed between versions? Was is
just a default value or something more significant in the query
processing. --Steve
-----Original Message----- From: Stephen Woodbridge
[mailto:[email protected]] Sent: Wednesday, July 15, 2015 8:43
AM To: Lime, Steve D (MNIT); thomas bonfort Cc: MapserverList OSGEO
Subject: Re: [mapserver-users] Very strange problem doing a query and
hanging on MS 6.4.1
On 7/14/2015 4:09 PM, Lime, Steve D (MNIT) wrote:
I think the map->query.maxfeatures is pretty much settable only
through WFS. If you're using the template driver output format you
can limit the number of features that way. You'd use the limit
property of the feature tag.
That said, it seems to me that you'd want to use TOLERANCE 0 with
whatever units you like. That means just grabbing the value of the
pixel the "click" falls in so you'd only ever get one value back.
Steve,
Thanks! From my limited test cases, setting TOLERANCE 0 seems to give
me the old behavior. I updated #5129 to reflect this and this should
save me a bunch of time tweaking scripts and mapfiles.
Thanks again, -Steve W
Steve
-----Original Message----- From:
[email protected]
[mailto:[email protected]] On Behalf Of
Stephen Woodbridge Sent: Tuesday, July 14, 2015 2:49 PM To: thomas
bonfort Cc: MapserverList OSGEO Subject: Re: [mapserver-users] Very
strange problem doing a query and hanging on MS 6.4.1
Ans not related to apache, I can still reproduce this at the
commandline, although it is harder to find a case that fails now
that I have set TOLERANCE and TOLERANCEUNITS.
Is there a way to limit the number of features like MAXFEATURES
for drawing, only for queries?
I see in the code map->query.maxfeatures but I'm not finding a way
to set that via CGI or mapfile?
-Steve
On 7/14/2015 11:20 AM, Stephen Woodbridge wrote:
OK, not totally solved! Things are better by explicitly setting
TOLERANCE and TOLERANCEUNITS but I am still getting random
hanging of processes doing queries.
I'm wondering if this is related to apache 2.4 and the way it is
setting up cgi-bin processes. What is the best practice for
configuring apache 2.4 (things changed in apache 2.4 from 2.2)
cgi for use with mapserver?
I'm running Ubuntu 14.04 LTS and have these packages installed:
apache2:amd64/trusty-security 2.4.7-1ubuntu4.4 uptodate
apache2-bin:amd64/trusty-security 2.4.7-1ubuntu4.4 uptodate
apache2-data:all/trusty-security 2.4.7-1ubuntu4.4 uptodate
apache2-mpm-prefork:amd64/trusty-security 2.4.7-1ubuntu4.4
uptodate apache2-utils:amd64/trusty-security 2.4.7-1ubuntu4.4
uptodate libapache2-mod-geoip:amd64/trusty 1.2.8-2 uptodate
libapache2-mod-mapcache:amd64/trusty 1.2.1-2 uptodate
libapache2-mod-php5:amd64/trusty-security 5.5.9+dfsg-1ubuntu4.11
uptodate libapache2-mod-python:amd64/trusty 3.3.1-11ubuntu2
uptodate
cgi-mapserver:amd64/trusty 6.4.1-2 uptodate
libmapserver1:amd64/trusty 6.4.1-2 uptodate
mapserver-bin:amd64/trusty 6.4.1-2 uptodate
ls /etc/apache2/mods-enabled/ access_compat.load autoindex.load
geoip.load php5.conf alias.load cgi.load
mapcache.conf php5.load auth_basic.load deflate.conf
mapcache.load python.load authn_core.load deflate.load
mime.conf setenvif.conf authn_file.load dir.conf
mime.load setenvif.load authz_core.load dir.load
mpm_prefork.conf status.conf authz_host.load env.load
mpm_prefork.load status.load authz_user.load filter.load
negotiation.conf autoindex.conf geoip.conf negotiation.load
On 7/13/2015 11:48 AM, Stephen Woodbridge wrote:
SOLVED: see below ...
According to the docs, the defaults are:
TOLERANCE 3 TOLERANCEUNITS pixels
and with:
QUERY_STRING='map=/maps/wms/bathymetry.map&mode=query&layers=model_query&mapxy=-100.98125+41.58125'
How is zoom calculated?
LAYER NAME "model_query" STATUS on TYPE raster
TILEINDEX "NOAA_Coastal_Relief_Model/index.shp" TILEITEM
"LOCATION" CLASS TEMPLATE "/maps/wms/pixel.value.html" END END
Did this behavior change from 6.2.1 to 6.4.1? because it is
not noted in the MIGRATION-GUIDE if it did. On 6.2.1 I get the
following result:
Content-Type: text/html <HTML <HEAD><TITLE>MapServer
Message</TITLE></HEAD <!-- MapServer version 6.2.1 OUTPUT=GIF
OUTPUT=PNG OUTPUT=JPEG SUPPORTS=PROJ SUPPORTS=GD SUPPORTS=AGG
SUPPORTS=FREETYPE SUPPORTS=CAIRO SUPPORTS=SVG_SYMBOLS
SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT
SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI
SUPPORTS=THREADS SUPPORTS=GEOS INPUT=JPEG INPUT=POSTGIS
INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE --> <BODY
BGCOLOR="#FFFFFF" msQueryByPoint(): Search returned no results.
No matching record(s) found. </BODY></HTML>
which returns very quickly.
I rsynced the data and the mapfile for Ubuntu 12.04 to 14.04
and installed the mapserver package. Everything else appears to
be working without a problem except this issue.
SOLVED:
I explicitly added:
TOLERANCE 1 TOLERANCEUNITS pixels
to my LAYER definition and now it works.
d# /usr/lib/cgi-bin/mapserv
QUERY_STRING='map=/maps/wms/bathymetry.map&mode=query&layers=model_query&mapxy=-100.98125+41.58125'
Content-Type: text/html
<HTML> <HEAD><TITLE>MapServer Message</TITLE></HEAD> <!--
MapServer version 6.4.1 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG
OUTPUT=KML SUPPORTS=PROJ SUPPORTS=GD SUPPORTS=AGG
SUPPORTS=FREETYPE SUPPORTS=CAIRO SUPPORTS=SVG_SYMBOLS
SUPPORTS=RSVG SUPPORTS=ICONV SUPPORTS=FRIBIDI
SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER
SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER
SUPPORTS=FASTCGI SUPPORTS=THREADS SUPPORTS=GEOS INPUT=JPEG
INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE --> <BODY
BGCOLOR="#FFFFFF"> msQueryByPoint(): Search returned no
results. No matching record(s) found. </BODY></HTML>
So it looks like maybe the defaults are not getting
initialized.
Opened: https://github.com/mapserver/mapserver/issues/5120
Thanks, -Steve
On 7/13/2015 11:05 AM, thomas bonfort wrote:
Or that could just be the number of pixels under the query
point if you are zoomed out far or with a large tolerance...
On 13 July 2015 at 17:01, Stephen Woodbridge
<[email protected]> wrote:
Thomas,
More information:
/u/software/mapserver-rel-6-4-2/maptemplate.c:4366
4366 for(j=0; j<lp->resultcache->numresults; j++) {
(gdb) p lp->resultcache->numresults $7 = 2884991
this is clearly a bad value my guess is that it is not
getting initialized to zero and assumes it will get set
when some number of results are found, but if no results
are found it is not getting set to zero. (Just a
hypothesis)
-Steve W
On 7/13/2015 10:41 AM, Stephen Woodbridge wrote:
Thomas,
Here is the initial backtrace afte starting it and and
^C while it is printing -99999 lines:
#0 processLine (mapserv=0x609050, instr=0x7fffffffcc60
"[value_0]\n", stream=0x688460, mode=9) at
/u/software/mapserver-rel-6-4-2/maptemplate.c:3697 #1
0x00007ffff7a2e9cc in msReturnPage (mapserv=0x609050,
html=0x6612e0 "/maps/wms/pixel.value.html", mode=9,
papszBuffer=0x0) at
/u/software/mapserver-rel-6-4-2/maptemplate.c:4159 #2
0x00007ffff7a2f7ed in msReturnNestedTemplateQuery
(mapserv=0x609050, pszMimeType=0x60c8f0 "text/html",
papszBuffer=0x0) at
/u/software/mapserver-rel-6-4-2/maptemplate.c:4385 #3
0x00007ffff7a1f586 in msReturnTemplateQuery
(mapserv=0x609050, queryFormat=0x60c8f0 "text/html",
papszBuffer=0x0) at
/u/software/mapserver-rel-6-4-2/maptemplate.c:315 #4
0x00007ffff79f1230 in msCGIDispatchQueryRequest
(mapserv=0x609050) at
/u/software/mapserver-rel-6-4-2/mapservutil.c:1476 #5
0x00007ffff79f2205 in msCGIDispatchRequest
(mapserv=0x609050) at
/u/software/mapserver-rel-6-4-2/mapservutil.c:1784 #6
0x000000000040115b in main (argc=2, argv=0x7fffffffdf98)
at /u/software/mapserver-rel-6-4-2/mapserv.c:259
I'll poke around a little more in gdb and see if I can
figure out anything more.
Thanks, -Steve
On 7/13/2015 4:12 AM, thomas bonfort wrote:
Steve, please include a backtrace from gdb where you
believe the infinite loop is happening.
-- thomas
On 13 July 2015 at 01:22, Stephen Woodbridge
<[email protected]> wrote:
I added gdalinfo on an actual raster file below. It
turns out the -99999 is the NoData Value. I would be
happy to just get this value back (once), but I
think something is broken and putting mapserver into
an infinite loop.
-Steve W
On 7/12/2015 7:14 PM, Stephen Woodbridge wrote:
Hi all,
I have run into a very strange problem doing a
mapserver query. I'm upgrading a system that was
running fine. The details are below. I also tried
to put the template inside a CLASS ... END block
that didn't help.
Can anyone else reproduce something like this? Got
any ideas on how to work around it?
Thanks, -Steve W
Old system: Ubuntu 12.04 LTS
MapServer version 6.2.1 OUTPUT=GIF OUTPUT=PNG
OUTPUT=JPEG SUPPORTS=PROJ SUPPORTS=GD SUPPORTS=AGG
SUPPORTS=FREETYPE SUPPORTS=CAIRO
SUPPORTS=SVG_SYMBOLS SUPPORTS=ICONV
SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER
SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER
SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI
SUPPORTS=THREADS SUPPORTS=GEOS INPUT=JPEG
INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE
New system: Ubuntu 14.04 LTS
MapServer version 6.4.1 OUTPUT=GIF OUTPUT=PNG
OUTPUT=JPEG OUTPUT=KML SUPPORTS=PROJ SUPPORTS=GD
SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=CAIRO
SUPPORTS=SVG_SYMBOLS SUPPORTS=RSVG SUPPORTS=ICONV
SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER
SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER
SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI
SUPPORTS=THREADS SUPPORTS=GEOS INPUT=JPEG
INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE
The query layer looks like this:
LAYER NAME "model_query" STATUS on TYPE raster
TILEINDEX "NOAA_Coastal_Relief_Model/index.shp"
TILEITEM "LOCATION" TEMPLATE
"/maps/wms/pixel.value.html" END
cat "/maps/wms/pixel.value.html" <!-- MapServer
Template --> [value_0]
dbf_dump -info
wms/data/Bathy/NOAA_Coastal_Relief_Model/index.dbf
Filename:
wms/data/Bathy/NOAA_Coastal_Relief_Model/index.dbf
Version: 0x03 (ver. 3) Num of records: 61
Header length: 65 Record length: 256 Last
change: 1995/7/26 Num fields: 1 Field info: Num
Name Type Len Decimal 1. LOCATION
C 255 0
dbf_dump
wms/data/Bathy/NOAA_Coastal_Relief_Model/index.dbf
| head NOAA_Coastal_Relief_Model/BSM-1035.tif
NOAA_Coastal_Relief_Model/BSM-1156.tif
NOAA_Coastal_Relief_Model/BSM-1193.tif
NOAA_Coastal_Relief_Model/BSM-1239.tif
NOAA_Coastal_Relief_Model/BSM-1261.tif
NOAA_Coastal_Relief_Model/BSM-1326.tif
NOAA_Coastal_Relief_Model/BSM-1343.tif
NOAA_Coastal_Relief_Model/BSM-1536.tif
NOAA_Coastal_Relief_Model/BSM-1557.tif
NOAA_Coastal_Relief_Model/BSM-1801.tif
ogrinfo -so
wms/data/Bathy/NOAA_Coastal_Relief_Model/index.shp
index INFO: Open of
`wms/data/Bathy/NOAA_Coastal_Relief_Model/index.shp'
using driver `ESRI Shapefile' successful.
Layer name: index Geometry: Polygon Feature Count:
61 Extent: (-130.000417, 16.999583) - (-65.999599,
49.000401) Layer SRS WKT: (unknown) location:
String (255.0)
When I issue a query like:
/usr/lib/cgi-bin/mapserv
QUERY_STRING='map=/maps/wms/bathymetry.map&mode=query&layers=model_query&mapxy=-100.98125+41.58125'
The old server responded with:
Content-Type: text/html
<HTML> <HEAD><TITLE>MapServer
Message</TITLE></HEAD> <!-- MapServer version 6.2.1
OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG SUPPORTS=PROJ
SUPPORTS=GD SUPPORTS=AGG SUPPORTS=FREETYPE
SUPPORTS=CAIRO SUPPORTS=SVG_SYMBOLS SUPPORTS=ICONV
SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER
SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER
SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI
SUPPORTS=THREADS SUPPORTS=GEOS INPUT=JPEG
INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE
--> <BODY BGCOLOR="#FFFFFF"> msQueryByPoint():
Search returned no results. No matching record(s)
found. </BODY></HTML>You have new mail in
/var/mail/root
I get an infinite number of lines like the
following on the new server:
# /usr/lib/cgi-bin/mapserv
QUERY_STRING='map=/maps/wms/bathymetry.map&mode=query&layers=model_query&mapxy=-100.98125+41.58125'
| head
Content-Type: text/html
-99999 -99999 -99999 -99999 -99999 -99999 -99999
-99999 ^C
# gdalinfo
wms/data/Bathy/NOAA_Coastal_Relief_Model/BSM-1156.tif
Driver: GTiff/GeoTIFF Files:
wms/data/Bathy/NOAA_Coastal_Relief_Model/BSM-1156.tif
Size is 4801, 4801 Coordinate System is: GEOGCS["WGS
84", DATUM["WGS_1984", SPHEROID["WGS
84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]], AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]] Origin =
(-102.000833330000006,41.000817329999997) Pixel Size
= (0.000833330000000,-0.000833330000000) Metadata:
AREA_OR_POINT=Point Image Structure Metadata:
INTERLEAVE=BAND Corner Coordinates: Upper Left
(-102.0008333, 41.0008173) (102d 0' 3.00"W, 41d 0'
2.94"N) Lower Left (-102.0008333, 37.0000000)
(102d 0' 3.00"W, 37d 0' 0.00"N) Upper Right (
-98.0000160, 41.0008173) ( 98d 0' 0.06"W, 41d 0'
2.94"N) Lower Right ( -98.0000160, 37.0000000) ( 98d
0' 0.06"W, 37d 0' 0.00"N) Center (-100.0004247,
39.0004087) (100d 0' 1.53"W, 39d 0' 1.47"N) Band 1
Block=256x256 Type=Float32, ColorInterp=Gray NoData
Value=-99999 Overviews: 2401x2401, 1201x1201,
601x601, 301x301, 151x151, 76x76, 38x38
You have to ^C to kill it, even when using head.
This seems like a serious bug as it runs for about
10 mins taking 100% CPU. If you call it via
file_get_contents() in php is hangs the request
and freezea my OL app until it is killed or kills
php reading all the data.
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________ mapserver-users
mailing list [email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________ mapserver-users
mailing list [email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
