Hi,
I have a lot of COGs hosted online in HTTP RANGE compatible storage. I want to
create a catalog/index of them so that GDAL and tools that use GDAL (like QGIS)
can extract the relevant data from the COGs for specific areas. For example,
I’d like to use `gdal raster clip` to clip this big ‘virtual raster’ to a
specific bbox or polygon. Ideally I’d also like to be able to view the big
raster on-the-fly in QGIS, but I’m aware that may be more difficult. My tiles
are ’true tiles’ in that they tessellate properly and each location is in a
single file.
I’ve been looking at the GDAL Tile Index format, but have been getting a bit
confused, and some things haven’t been working quite how I expect. A few
questions below:
1. Is there an easy way to generate a GDAL Tile Index file from a file
containing a list of remote URLs? I couldn’t find the right way to specify a
file containing URLs on the command line, so I ended up writing a short Python
script that loads the list of URLs from a file and then passes them to
gdal.TileIndex. Is that the best way to do this? One disadvantage of this
method is that I can’t seem to get a progress bar to display.
2. My current way of doing that requires GDAL loading the bbox of each file by
requesting it over the network, which is fairly slow, and I have thousands of
files to do this for. Is there a way to get the GTI creation to use local files
on my hard disk for getting the extent of each file, but save the location as a
/vsicurl/ URL to the remote file? I’ve found --base-url in `gdal raster index`
though I’m slightly confused by the description of it ("Base URL for
STAC-GeoParquet the href property of the id property in a STAC-GeoParquet
index.” - does that do what I want and put a base URL in front of each file it
finds?). That option doesn’t seem to be present in `gdal driver gti create`
though, and seems like it may be STAC-GeoParquet specific.
3. Once I’ve created the file, I can do `gdal raster clip` operations on the
command-line quite quickly, but if I load the GTI raster into QGIS then QGIS
just hangs. I hoped it would just request the relevant COGs for the area I was
looking at. I’ve tried loading it in by dragging and dropping the .gti.gpkg
file and then selecting the raster in the layers popup, and also adding it via
Add Layer->Raster and putting ‘GTI:<path>’. I have a web map working which
loads these COG files on the fly depending on where you’re looking (using a
minimal STAC catalog in a JSON file) and was hoping to have the same thing
working in QGIS, without the effort of setting up something like titiler or
titiler-pgstac, as I’m currently managing to get away without running any
servers (as opposed to static data stores) for this data.
If there’s an alternative method I should be looking at for doing either part
of this (the clipping or the viewing in QGIS) then let me know - I’m aware I
may be pushing the boundaries of what can be sensibly done with the GTI format.
Best regards,
Robin
Dr Robin Wilson
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
