Hi Javier, A POST url to COG content only makes sense if the intent is to support full download of the file. POST is normally for adding/updating data. I took a quick look and yes no way to directly access the COGs other than use the search UI and get a list.
We see this many times, the good news is that they are formatted as COGs, but the value prop of COGs, being able to use them directly via range-gets is not there. We are hoping to make this easier for our public sector customers this year by simplifying STAC deployments on top serverless. I did notice that download UI is very slow to load but the WMTS service used by the Viewer tool is performant. https://www.ign.es/wmts/pnoa-ma?SERVICE=WMTS&REQUEST=GetTile&VERSION=1.0.0&LAYER=OI.OrthoimageCoverage&STYLE=default&FORMAT=image/jpeg&TILEMATRIXSET=EPSG:4326&TILEMATRIX=EPSG:4326:18&TILEROW=77554&TILECOL=257526 Depending on what you are looking to do, it might actually be faster to use GDAL or GDAL tools via QGIS to point a the relevant data using the WMTS service and output COG. Of course you would lose any meta data in the source COGS. WMS I did not try. Mark Date: Tue, 2 Jan 2024 13:14:09 +0100 From: Javier Jimenez Shaw <j...@jimenezshaw.com <mailto:j...@jimenezshaw.com>> To: gdal dev <gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>> Subject: [gdal-dev] COG via HTTP POST Message-ID: <CADRrdKvusz8UsaHyVMGLXN+zR8+7Q=sg-qQVVGDb9hMNAZSD=a...@mail.gmail.com <mailto:a...@mail.gmail.com>> Content-Type: text/plain; charset="utf-8" Hi Does it make any sense to provide COG via HTTP POST method? If I understand correctly, the "magic" of downloading only what is needed from the server is done with HTTP Range requests. But I am not sure if that works in POST method (it does work in GET method, when enabled ;). In case it could work with POST, I do not know how to tell GDAL or QGIS to use a POST method. Why am I asking this? the CNIG (Natial Center of Geographic Information in Spain) is doing it. https://centrodedescargas.cnig.es/CentroDescargas/index.jsp <https://centrodedescargas.cnig.es/CentroDescargas/index.jsp> The link is just the starting point. Unfortunately they do not provide link to the final page where you can download the files (yes, ugly). So for now I have to download the full file, and do whatever I want later. Thanks Javier .___ ._ ..._ .. . ._. .___ .. __ . _. . __.. ... .... ._ .__ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/cedd5d7b/attachment-0001.htm> <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/cedd5d7b/attachment-0001.htm>> ------------------------------ Message: 3 Date: Tue, 2 Jan 2024 13:22:00 +0100 From: Javier Jimenez Shaw <j...@jimenezshaw.com <mailto:j...@jimenezshaw.com>> To: Even Rouault <even.roua...@spatialys.com <mailto:even.roua...@spatialys.com>> Cc: gdal dev <gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>> Subject: Re: [gdal-dev] GDAL 3.8.3 RC1 is available, and motion to approve it Message-ID: <cadrrdktf+dgx0jib75dld3swqiovi96969meazluasnubyg...@mail.gmail.com <mailto:cadrrdktf+dgx0jib75dld3swqiovi96969meazluasnubyg...@mail.gmail.com>> Content-Type: text/plain; charset="utf-8" +1 Javier On Tue, 2 Jan 2024, 12:56 Even Rouault via gdal-dev, < gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>> wrote: > Hi, > > I have prepared an advanced-of-time GDAL/OGR 3.8.3 release candidate, > mostly > to fix a 3.8.0 regression that has been reported lately > (https://github.com/OSGeo/gdal/issues/8998 > <https://github.com/OSGeo/gdal/issues/8998>), that causes content of > boolean > fields in GeoPackage or FlatGeobuf datasets to be incorrectly read when > using > the ArrowArray interface, for example when used as source datasets in > ogr2ogr > workflows. > > Pick up an archive among the following ones (by ascending size): > > https://download.osgeo.org/gdal/3.8.3/gdal-3.8.3rc1.tar.xz > <https://download.osgeo.org/gdal/3.8.3/gdal-3.8.3rc1.tar.xz> > https://download.osgeo.org/gdal/3.8.3/gdal-3.8.3rc1.tar.gz > <https://download.osgeo.org/gdal/3.8.3/gdal-3.8.3rc1.tar.gz> > https://download.osgeo.org/gdal/3.8.3/gdal383rc1.zip > <https://download.osgeo.org/gdal/3.8.3/gdal383rc1.zip> > > A snapshot of the gdalautotest suite is also available: > > https://download.osgeo.org/gdal/3.8.3/gdalautotest-3.8.3rc1.tar.gz > <https://download.osgeo.org/gdal/3.8.3/gdalautotest-3.8.3rc1.tar.gz> > https://download.osgeo.org/gdal/3.8.3/gdalautotest-3.8.3rc1.zip > <https://download.osgeo.org/gdal/3.8.3/gdalautotest-3.8.3rc1.zip> > > The NEWS file is here: > > https://github.com/OSGeo/gdal/blob/v3.8.3RC1/NEWS.md > <https://github.com/OSGeo/gdal/blob/v3.8.3RC1/NEWS.md> > > Motion: adopt 3.8.3 RC1 as final 3.8.3 release > > Starting with my +1, > > Best regards, > > Even > > -- > http://www.spatialys.com <http://www.spatialys.com> > My software is free, but my time generally not. > > _______________________________________________ > gdal-dev mailing list > gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org> > https://lists.osgeo.org/mailman/listinfo/gdal-dev > <https://lists.osgeo.org/mailman/listinfo/gdal-dev> > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/4928e12e/attachment-0001.htm> <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/4928e12e/attachment-0001.htm>> ------------------------------ Message: 4 Date: Tue, 2 Jan 2024 13:39:45 +0100 From: Even Rouault <even.roua...@spatialys.com <mailto:even.roua...@spatialys.com>> To: Javier Jimenez Shaw <j...@jimenezshaw.com <mailto:j...@jimenezshaw.com>>, gdal dev <gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>> Subject: Re: [gdal-dev] COG via HTTP POST Message-ID: <d6946b88-f82c-4cee-a3c5-56cd0e479...@spatialys.com <mailto:d6946b88-f82c-4cee-a3c5-56cd0e479...@spatialys.com>> Content-Type: text/plain; charset="utf-8"; Format="flowed" Javier, According to https://httpwg.org/specs/rfc7233.html#header.range <https://httpwg.org/specs/rfc7233.html#header.range> , HTTP Range requests only apply to GET requests.? So COGs delivered through POST can indeed only be usable after full download. BTW, someone should ask CNIG to include GeoTIFF SRS tags. At least, for file pnt_sentinel2_2020_invierno_mosaico_islas_canarias_b843_hu28_8bits.tif which has only ModelTiepointTag and ModelPixelScaleTag geotiff keys giving the geospatial extent, but no CRS keys. Even Le 02/01/2024 ? 13:14, Javier Jimenez Shaw via gdal-dev a ?crit?: > Hi > > Does it make any sense to provide COG via HTTP POST method? > If I understand correctly, the "magic" of downloading only what is > needed from the server is done with HTTP Range requests. But I am not > sure if that works in POST method (it does work in GET method, when > enabled ;). > In case it could work with POST, I do not know how to tell GDAL or > QGIS to use a POST method. > > Why am I asking this? the CNIG (Natial Center of Geographic > Information in Spain) is doing it. > https://centrodedescargas.cnig.es/CentroDescargas/index.jsp > <https://centrodedescargas.cnig.es/CentroDescargas/index.jsp> The link > is just the starting point. Unfortunately they do not provide link to > the final page where you can download the files (yes, ugly). > > So for now I have to download the full file, and do whatever I want later. > > Thanks > Javier > .___ ._ ..._ .. . ._.? .___ .. __ . _. . __..? ... .... ._ .__ > > _______________________________________________ > gdal-dev mailing list > gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org> > https://lists.osgeo.org/mailman/listinfo/gdal-dev > <https://lists.osgeo.org/mailman/listinfo/gdal-dev> -- http://www.spatialys.com <http://www.spatialys.com> My software is free, but my time generally not. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/94ae3cae/attachment-0001.htm> <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/94ae3cae/attachment-0001.htm>> ------------------------------ Message: 5 Date: Tue, 2 Jan 2024 13:43:59 +0100 From: Javier Jimenez Shaw <j...@jimenezshaw.com <mailto:j...@jimenezshaw.com>> To: Even Rouault <even.roua...@spatialys.com <mailto:even.roua...@spatialys.com>> Cc: gdal dev <gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>> Subject: Re: [gdal-dev] COG via HTTP POST Message-ID: <CADRrdKtWswA+eokeQwtQWDuVmhjAD641SGOZ8WDk=fbqkot...@mail.gmail.com <mailto:fbqkot...@mail.gmail.com>> Content-Type: text/plain; charset="utf-8" Thanks Even. I will tell them about the missing SRS On Tue, 2 Jan 2024, 13:39 Even Rouault, <even.roua...@spatialys.com <mailto:even.roua...@spatialys.com>> wrote: > Javier, > > According to https://httpwg.org/specs/rfc7233.html#header.range > <https://httpwg.org/specs/rfc7233.html#header.range> , HTTP > Range requests only apply to GET requests. So COGs delivered through POST > can indeed only be usable after full download. > > BTW, someone should ask CNIG to include GeoTIFF SRS tags. At least, for > file pnt_sentinel2_2020_invierno_mosaico_islas_canarias_b843_hu28_8bits.tif > which has only ModelTiepointTag and ModelPixelScaleTag geotiff keys giving > the geospatial extent, but no CRS keys. > > Even > Le 02/01/2024 ? 13:14, Javier Jimenez Shaw via gdal-dev a ?crit : > > Hi > > Does it make any sense to provide COG via HTTP POST method? > If I understand correctly, the "magic" of downloading only what is needed > from the server is done with HTTP Range requests. But I am not sure if that > works in POST method (it does work in GET method, when enabled ;). > In case it could work with POST, I do not know how to tell GDAL or QGIS to > use a POST method. > > Why am I asking this? the CNIG (Natial Center of Geographic Information in > Spain) is doing it. > https://centrodedescargas.cnig.es/CentroDescargas/index.jsp > <https://centrodedescargas.cnig.es/CentroDescargas/index.jsp> The link is > just the starting point. Unfortunately they do not provide link to the > final page where you can download the files (yes, ugly). > > So for now I have to download the full file, and do whatever I want later. > > Thanks > Javier > .___ ._ ..._ .. . ._. .___ .. __ . _. . __.. ... .... ._ .__ > > _______________________________________________ > gdal-dev mailing listgdal-...@lists.osgeo.orgh > <mailto:listgdal-...@lists.osgeo.orgh>ttps://lists.osgeo.org/mailman/listinfo/gdal-dev > > -- http://www.spatialys.com <http://www.spatialys.com> > My software is free, but my time generally not. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/8627e23b/attachment-0001.htm> <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/8627e23b/attachment-0001.htm>> ------------------------------ Message: 6 Date: Tue, 2 Jan 2024 15:09:23 +0100 From: leo.fuhrmann.l...@icloud.com <mailto:leo.fuhrmann.l...@icloud.com> To: gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org> Subject: [gdal-dev] Error when importing GeoPackage into PostgreSQL database ("Worker thread task has not expected m_iStartShapeId value") Message-ID: <a0357b4d-4dfb-4070-86f4-eb7975c41...@icloud.com <mailto:a0357b4d-4dfb-4070-86f4-eb7975c41...@icloud.com>> Content-Type: text/plain; charset="utf-8" Hi, I?m using ogr2ogr 3.8.2 to import a (layer of a) GeoPackage into a PostGIS database: ``` ogr2ogr --debug ON -f PostgreSQL "PG:dbname=XX host=XX port=XX user=XX password=XX? <path/to/gpkg.gpkg> <layer> ``` However, I?m seeing an error in the logs that I cannot sort out (see mail subject and/or logs below). Despite the error, the data gets written to the database, yet I?m worried that something is messed up along the way. Using the official docker image to execute the command (from a MacBook with M1 Pro) works without an error. Then again, using a cloud service (IBM Code Engine in this case) with the official image also throws the error. I tried with different image versions but found no difference. I tried to track down the error and found https://github.com/OSGeo/gdal/blob/1df855eb676e17278c337e9b14774d1bb298df90/ogr/ogrsf_frmts/gpkg/ogrgeopackagetablelayer.cpp#L8352 <https://github.com/OSGeo/gdal/blob/1df855eb676e17278c337e9b14774d1bb298df90/ogr/ogrsf_frmts/gpkg/ogrgeopackagetablelayer.cpp#L8352> The comment suggests that I might have messed with GetNextFeature(), but I?m sure I haven?t. Can someone help me to make sense of this error? I?m sorry to say that I can?t share the GeoPackage at the moment. Thanks, Leo Here are the logs: GPKG: GeoPackage v1.2.0 GDAL: GDALOpen(<path/to/gpkg.gpkg>, this=0x14c053e00) succeeds as GPKG. GDAL: GDALDriver::Create(PostgreSQL,PG:dbname=XX host=XX port=XX user=XX password=XX PG: Client encoding: 'UTF8' PG: PostGIS schema: 'public' PG: PostgreSQL version string : 'PostgreSQL 16.1 (Debian 16.1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit' PG: PostGIS version string : '3.4 USE_GEOS=1 USE_PROJ=1 USE_STATS=1' PG: Could not retrieve table oid for <layer> GDALVectorTranslate: Using FID=fid and -preserve_fid PG: Could not retrieve table oid for <layer> OGR2OGR: Using WriteArrowBatch() GPKG: GeoPackage v1.2.0 GPKG: GeoPackage v1.2.0 GPKG: GeoPackage v1.2.0 ERROR 1: Worker thread task has not expected m_iStartShapeId value PG: PQputCopyEnd() GDAL: GDALClose(<path/to/gpkg.gpkg>, this=0x14c053e00) GDAL: GDALClose(PG:dbname=XX host=XX port=XX user=XX password=XX this=0x14b721fb0) GDAL: In GDALDestroy - unloading GDAL shared library. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/3eca1d4d/attachment.htm> <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240102/3eca1d4d/attachment.htm>> ------------------------------ Subject: Digest Footer _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org> https://lists.osgeo.org/mailman/listinfo/gdal-dev <https://lists.osgeo.org/mailman/listinfo/gdal-dev> ------------------------------ End of gdal-dev Digest, Vol 236, Issue 1 **************************************** _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
