This may be a clue.
I gdalwarp'd your typic_xerofluvents.tif to a proper -f COG format and
did stats on both. The block sizes are wildly different. Try making a
proper COG out of your .tif and see if you get the same results.
Block=1766x4, as opposed to Block=512x512
Here's what I'm looking at:
gdalinfo -stats typic_xerofluvents.tif
Band 1 Block=1766x4 Type=Byte, ColorInterp=Gray
Minimum=1.000, Maximum=95.000, Mean=10.684, StdDev=12.230
NoData Value=0
Overviews: 883x1262, 442x631, 221x316, 111x158
Metadata:
STATISTICS_MINIMUM=1
STATISTICS_MAXIMUM=95
STATISTICS_MEAN=10.684186169509
STATISTICS_STDDEV=12.229580166735
STATISTICS_VALID_PERCENT=0.3393
####
gdalwarp -f COG typic_xerofluvents.tif tst.tif
gdalinfo -stats tst.tif
Band 1 Block=512x512 Type=Byte, ColorInterp=Gray
Minimum=1.000, Maximum=95.000, Mean=10.684, StdDev=12.230
NoData Value=0
Overviews: 883x1262, 441x631, 220x315, 110x157
Metadata:
STATISTICS_MINIMUM=1
STATISTICS_MAXIMUM=95
STATISTICS_MEAN=10.684186169509
STATISTICS_STDDEV=12.229580166735
STATISTICS_VALID_PERCENT=0.3393
On 1/10/25 11:30, Dylan Beaudette wrote:
Hi Rich,
Now that is a good question! Perhaps an implied use of those overviews
has changed across versions, or a previously-default mapfile parameter
has changed. We didn't see anything in the changelog, but to be honest,
a lot has changed between versions and maybe I missed something.\
Here is a direct link to the file, in case that is helpful:
https://soilmap4-1.lawr.ucdavis.edu/taxa-grid-tms/taxsubgrp/
typic_xerofluvents.tif <https://soilmap4-1.lawr.ucdavis.edu/taxa-grid-
tms/taxsubgrp/typic_xerofluvents.tif>
Here is a snippet from the mapfile, the same on both machines. Note that
we are using a combination of pattern matching and expressions to select
files and apply a color styling. Perhaps there was an implied change to
"processing" or "use of overviews" directives in the more recent
mapserver version?
MAP
OUTPUTFORMAT
NAME "png"
IMAGEMODE RGBA
DRIVER "AGG/PNG"
MIMETYPE "image/png"
EXTENSION "png"
FORMATOPTION "GAMMA=0.75"
END
STATUS ON
# set the output projection of map images to 'spherical
mercator' used by leaflet maps
PROJECTION
"init=epsg:3857"
END
##### blue 9 from dylan
LAYER
NAME blue9dylan
STATUS ON
DATA "%type%/%name%.tif"
TYPE RASTER
# set the input projection of the source data
PROJECTION
"init=epsg:3857"
END
VALIDATION
"type" "^taxorder|taxsuborder|taxgrtgroup|
taxsubgrp|fm\/gg|fm\/sg$"
"name" "^[a-zA-z_\-]+$"
"minvalue" "^[0-9]{1,3}$"
END
CLASS # NoData Values
EXPRESSION ([pixel] = 0)
STYLE
COLOR "#00000000"
END
END
CLASS
EXPRESSION ([pixel] >= %minvalue% AND [pixel] <= 2)
STYLE
COLOR "#E0F7E1"
END
END
[...]
Thanks,
Dylan
On Fri, Jan 10, 2025 at 11:10 AM Richard Greenwood
<richard.greenw...@gmail.com <mailto:richard.greenw...@gmail.com>> wrote:
Hi Dylan,
Welcome back. Why do you think the issue is related to the use of
overviews and not something else?
Rich
On Fri, Jan 10, 2025 at 10:50 AM Dylan Beaudette via MapServer-users
<mapserver-users@lists.osgeo.org <mailto:mapserver-
us...@lists.osgeo.org>> wrote:
Thanks Scott,
I tried that: md5sums on the GeoTiffs and diff on the .map files
confirms that they are identical.
The only difference between these two systems is the version of
Mapserver.
MapServer version 7.2.2 OUTPUT=PNG OUTPUT=JPEG OUTPUT=KML
SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=CAIRO
SUPPORTS=SVG_SYMBOLS SUPPORTS=RSVG SUPPORTS=ICONV
SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT
SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER
SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS
SUPPORTS=GEOS SUPPORTS=PBF INPUT=JPEG INPUT=POSTGIS INPUT=OGR
INPUT=GDAL INPUT=SHAPEFILE
MapServer version 7.6.4 OUTPUT=PNG OUTPUT=JPEG OUTPUT=KML
SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=CAIRO
SUPPORTS=SVG_SYMBOLS SUPPORTS=RSVG SUPPORTS=ICONV
SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT
SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER
SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS
SUPPORTS=GEOS SUPPORTS=POINT_Z_M SUPPORTS=PBF INPUT=JPEG
INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE
The newer version is provided via apt:
cgi-mapserver 7.6.4-2build2
amd64 CGI executable for MapServer
Any other leads?
Thanks!
Dylan
On Thu, Jan 9, 2025 at 2:45 PM Scott via MapServer-users
<mapserver-users@lists.osgeo.org <mailto:mapserver-
us...@lists.osgeo.org>> wrote:
I would do a sanity check first, just to make sure the 2
raster files
are truly the same. Try something like:
gdalinfo -stats -hist file1.tif
gdalinfo -stats -hist file2.tif
Then, I would continue with the .map files, to make sure
they are the
same. Something like:
diff serv1.map serv2.map
See what falls out.
On 1/9/25 13:10, Dylan Beaudette via MapServer-users wrote:
>
> Hi Everyone,
>
> I've got identical GeoTiffs and .map files on two servers
with varying
> versions of Mapserver that is being used as a TMS.
Mapserver 7.2.2 is (I
> think) correctly using the overviews while Mapserver
7.6.4 is not.
>
> Here is an example of the same file, tile, and styling
applied across
> both versions of Mapserver.
>
> # Mapserver 7.2.2:
> https://soilmap2-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/
data1/website/ <https://soilmap2-1.lawr.ucdavis.edu/cgi-bin/
mapserv?map=/data1/website/>
> ste/
>
taxa.map&mode=tile&tilemode=gmap&tile=5+12+5&layer=blue9dylan&type=taxsubgrp&name=typic_xerofluvents&minvalue=5
<https://soilmap2-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/data1/website/ste/taxa.map&mode=tile&tilemode=gmap&tile=5+12+5&layer=blue9dylan&type=taxsubgrp&name=typic_xerofluvents&minvalue=5
<https://soilmap2-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/data1/website/ste/taxa.map&mode=tile&tilemode=gmap&tile=5+12+5&layer=blue9dylan&type=taxsubgrp&name=typic_xerofluvents&minvalue=5>>
>
> Mapserver 7.6.4:
> https://soilmap4-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/
data1/website/ <https://soilmap4-1.lawr.ucdavis.edu/cgi-bin/
mapserv?map=/data1/website/>
> ste/
>
taxa.map&mode=tile&tilemode=gmap&tile=5+12+5&layer=blue9dylan&type=taxsubgrp&name=typic_xerofluvents&minvalue=5
<https://soilmap4-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/data1/website/ste/taxa.map&mode=tile&tilemode=gmap&tile=5+12+5&layer=blue9dylan&type=taxsubgrp&name=typic_xerofluvents&minvalue=5
<https://soilmap4-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/data1/website/ste/taxa.map&mode=tile&tilemode=gmap&tile=5+12+5&layer=blue9dylan&type=taxsubgrp&name=typic_xerofluvents&minvalue=5>>
>
> It appears to me that the newer version is either
ignoring or
> interpreting the overviews differently. Each file has an
internal tiling
> and overview scheme like this:
>
>
----------------------------------------------------------------------------------
> Band 1 Block=1766x4 Type=Byte, ColorInterp=Gray
> NoData Value=0
> Overviews: 883x1262, 442x631, 221x316, 111x158
>
----------------------------------------------------------------------------------
>
> Does anyone have a clue about the differences? Is either
version in error?
>
> Thanks, and good to be back on the mailing list after a
nearly 15 year
> hiatus!
> Dylan
>
>
>
>
> _______________________________________________
> MapServer-users mailing list
> MapServer-users@lists.osgeo.org <mailto:MapServer-
us...@lists.osgeo.org>
> https://lists.osgeo.org/mailman/listinfo/mapserver-users
<https://lists.osgeo.org/mailman/listinfo/mapserver-users>
_______________________________________________
MapServer-users mailing list
MapServer-users@lists.osgeo.org <mailto:MapServer-
us...@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/mapserver-users
<https://lists.osgeo.org/mailman/listinfo/mapserver-users>
_______________________________________________
MapServer-users mailing list
MapServer-users@lists.osgeo.org <mailto:MapServer-
us...@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/mapserver-users
<https://lists.osgeo.org/mailman/listinfo/mapserver-users>
--
Richard W. Greenwood
www.greenwoodmap.com <http://www.greenwoodmap.com>
_______________________________________________
MapServer-users mailing list
MapServer-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users
