Dear GDAL, We are seeing some strange behavior when reading and writing files with the GZip VSI, and I want to ask in the dev forum first before assuming it's a bug and filing an issue. I can't find anything in the docs or the forum that seems relevant.
When writing (say) a .geojson.gz using the original GeoJSON driver, it seems to create, and then delete, a sibling .properties file during the write. Based on the comment at... https://github.com/OSGeo/gdal/blob/master/gdal/port/cpl_vsil_gzip.cpp#L68 ...this seems fair enough. It also seems to try to create the .properties file when READING a GZip file, which is also fair enough, if it can. In the case of a read-only file-system, or other read-only VSI source, however, I assume a failure to write this file is supposed to be benign, and indeed I wouldn't expect any user-facing message or error at all. We are able to read (say) a .geojson.gz from an S3 bucket with no issues. However, when we try to read a file from within a gzipped TAR file on an S3 bucket, there is a problem. GDAL throws the following error: DoSinglePartPUT of /vsis3/omnisci-import-test/example.geojson.tar.gz.properties failed (1) ...and in our CI environment with an ASAN test build, ASAN reports a bunch of leaks, which appear to be to do with Curl. I don't have these messages to hand, but I can get them from my colleague if necessary. >From the code (we're still running 2.4.2, but 3.1.2 appears to be the same) it seem that DoSinglePartPUT is only called from VSIS3WriteHandle::Close(), so I'm wondering if it's trying to flush out the write of the .properties file while closing the data source, and perhaps not knowing to squash any error. Here is the path to the file in what should be an open public bucket (in region us-west-1) if you want to try it yourself: s3://omnisci-import-test/example.geojson.tar.gz with subpath /example/example.geojson Thanks in advance! -- <http://www.omnisci.com/> Simon Eves Senior Graphics Engineer, Rendering Group 100 Montgomery St (5th Floor), San Francisco, CA 94104, USA Email: [email protected] | Cell: +1 (415) 902-1996
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
