Hi Seth,
I tried with both of these in the Mapfile, and also setting in the
command window (SET CPL_CREATE_ZIP64=YES) but still getting a corrupt zip:
CONFIG "CPL_CREATE_ZIP64" "NO"
CONFIG "CPL_CREATE_ZIP64" "YES"
Is there any ogr command to test the low level zip funcion in GDAL?
No
I thought "ogr2ogr foo.shp.zip something" command was a higher level
one which meant it might not be using the same approach.
I believe it should be representative enough of how MapServer uses GDAL
zip API.
I tried both the Windows unzip, 7-zip (x64) and opening on Linux with
Ark and unzip all with the same errors.
The strangest thing is that saving via IIS through the browser is
producing a valid zip using the same OUTPUTFORMAT..
I'm going to set up the msautotest and roads example in IIS to test if
that works.
Could it be related to a change in the Windows cmd shell?
Maybe? But as I mentioned before, I'd thought this would rather be some
msIO_needBinaryStdout() call missing. Perhaps IIS already sets stdout to
binary mode ?
For the sake of testing, what if you add a call to
msIO_needBinaryStdout() in mapserv.c, let's say at
https://github.com/MapServer/MapServer/blob/main/mapserv.c#L243 . I'm
not saying this is necessarily the right fix
Even
Seth
--
web:http://geographika.co.uk
twitter: @geographika
On Sun, Nov 21, 2021, at 4:54 PM, Even Rouault wrote:
Le 21/11/2021 à 16:43, Seth G a écrit :
Steve - in my case WMS are all fine, it is only related to zipped
outputs using OUTPUTFORMAT blocks.
I was trying to find a working commit for the bisect, but went back
to 7-2 without finding one.
I built MapServer main, and the latest 7-6, 7-4, and 7-2 branches
and ran the msautotest:
cd /D D:\GitHub\mapserver\msautotest\wxs
mapserv -nh
"QUERY_STRING=map=wfs_ogr.map&service=WFS&REQUEST=GetFeature&TYPENAME=road&version=2.0.0&outputformat=shapezip"
> output.zip
All zips were invalid. However the same command does work with the
https://www.gisinternals.com/query.html?content=filelist&file=release-1911-x64-gdal-2-4-4-mapserver-7-4-3.zip
<https://www.gisinternals.com/query.html?content=filelist&file=release-1911-x64-gdal-2-4-4-mapserver-7-4-3.zip>
build. It looks like something changed between GDAL 2.4 and 3.3
Likely function is CPLCreateZip (as this is used by MapServer as
mentioned by Even in the other thread), found in:
https://github.com/OSGeo/gdal/commits/v3.3.3/gdal/port/cpl_minizip_zip.cpp
<https://github.com/OSGeo/gdal/commits/v3.3.3/gdal/port/cpl_minizip_zip.cpp>
There are only a couple of commits between versions that look like
they might be relevant?
https://github.com/OSGeo/gdal/commit/0e159e1ba8ee9e7b4a56bd0b53d16de0acf89485#diff-b156d979e3531903e620bc0616c120584c8e28945247f50b29e81c5a7a49df0b
<https://github.com/OSGeo/gdal/commit/0e159e1ba8ee9e7b4a56bd0b53d16de0acf89485#diff-b156d979e3531903e620bc0616c120584c8e28945247f50b29e81c5a7a49df0b>
https://github.com/OSGeo/gdal/commit/edcdc5b0a57b4fab159b14180141ee42a0408830#diff-b156d979e3531903e620bc0616c120584c8e28945247f50b29e81c5a7a49df0b
<https://github.com/OSGeo/gdal/commit/edcdc5b0a57b4fab159b14180141ee42a0408830#diff-b156d979e3531903e620bc0616c120584c8e28945247f50b29e81c5a7a49df0b>
Is there a test / output zip file in the GDAL tests that could be
used to verify?
Not necessarily generated by recent GDAL versions, and if they were,
most certainly from Linux builds
What is weird is that you reported that .shp.zip files generated by
recent gisinternals version with "ogr2ogr foo.shp.zip something" were
correct.
Hum actually I see that .shp.zip are generated with the
CPL_CREATE_ZIP64 environment variable set to NO (the default is YES)
Can you try that ?
It might be that your .zip reader aren't Zip64 ready, or that GDAL
write them in a slightly invalid way
I have a working and invalid zip from msautotest if that helps.
yes, could be useful (but I suspect this is a difference between
regular ZIP vs Zip64 one)
Seth
--
web:http://geographika.co.uk <http://geographika.co.uk>
twitter: @geographika
On Sun, Nov 21, 2021, at 3:51 PM, Steve Lime wrote:
This seems related...
https://github.com/MapServer/MapServer/issues/6412
<https://github.com/MapServer/MapServer/issues/6412>
WMS fails but shp2img works, as does WFS. Could it be WMS specific?
Would be helpful to try mode=map and see if that works.
On Sun, Nov 21, 2021 at 8:45 AM Seth G <[email protected]
<mailto:[email protected]>> wrote:
Thanks Jérome. and Even. Sorry I should have checked for the
headers in that zip first.
I'll attempt the bisecting now!
--
web:http://geographika.co.uk <http://geographika.co.uk>
twitter: @geographika
On Sun, Nov 21, 2021, at 1:12 PM, Even Rouault wrote:
> Seth,
>
> The .zip extension of this test file is a bit misleading, as
the HTTP
> headers are not stripped (testing HTTP headers is something
good to test
> in some tests). I've issued
> https://github.com/MapServer/MapServer/pull/6440
<https://github.com/MapServer/MapServer/pull/6440> to rename
the expected
> file though to avoid the confusion
>
> You can obtain a valid .zip file without headers with
>
> ../../build/mapserv -conf ../etc/mapserv.conf -nh
>
QUERY_STRING="map=wfs_ogr.map&SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&TYPENAME=road&featureid=road.1521&OUTPUTFORMAT=SHAPEZIP"
> > out.zip
>
> Even
>
> Le 21/11/2021 à 12:32, Seth G a écrit :
>> Hi all,
>>
>> Following on from the post at
https://lists.osgeo.org/pipermail/mapserver-users/2021-November/082429.html
<https://lists.osgeo.org/pipermail/mapserver-users/2021-November/082429.html>
I've been trying to find where the cause of the corrupt zips
may be.
>>
>> There is a msautotest to create zip files and check against
the expected output, however the expected output also seems to
be an invalid zip file - I tried on both Windows and Linux.
>>
>> Even stranger this is both in the main branch (simply
download the file with the following URL to test), and also
going back in the history to 2016.
>>
>>
https://github.com/MapServer/MapServer/blob/main/msautotest/wxs/expected/wfsogr10_shapezip.zip
<https://github.com/MapServer/MapServer/blob/main/msautotest/wxs/expected/wfsogr10_shapezip.zip>
>>
>> I had a working process using shapezips in a 7.6 release of
MapServer so its hard to understand what could have changed,
MapServer, GDAL, or zip formats themselves?
>> I'll attempt a Git bisect but if the issue is outside
MapServer it won't find an issue.
>>
>> If anyone is able to open the zip file successfully please
let me know (tried with Ark and unzip on Linux and Windows
inbuilt unzip and 7-zip on Windows). Error seems to be "87
extra bytes at beginning or within zip file".
>>
>> Seth
>>
>> --
>> web:http://geographika.co.uk <http://geographika.co.uk>
>> twitter: @geographika
>> _______________________________________________
>> MapServer-dev mailing list
>> [email protected]
<mailto:[email protected]>
>> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
<https://lists.osgeo.org/mailman/listinfo/mapserver-dev>
>
> --
> http://www.spatialys.com <http://www.spatialys.com>
> My software is free, but my time generally not.
_______________________________________________
MapServer-dev mailing list
[email protected]
<mailto:[email protected]>
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
<https://lists.osgeo.org/mailman/listinfo/mapserver-dev>
--
http://www.spatialys.com <http://www.spatialys.com>
My software is free, but my time generally not.
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
MapServer-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
