Re: [gdal-dev] Discrepancy in utilizing overviews with gdalwarp

[Rahkonen Jukka via gdal-dev]

Hi,

Those two commands behave in quite different ways. With “-oo OVERVIEW_LEVEL=1” 
the output size is the size of the second overview. In my test case the 
original has 12000 x 12000 pixels and the command creates a 3000 x 3000 sized 
output

GDAL: GDALOverviewDataset(vrttest.vrt, this=01CB3D728890) creation.
GDAL: Using GTiff driver
GDAL: Computing area of interest: 26.8805, 63.2011, 27, 63.255
Creating output file that is 3000P x 3000L.
…
GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=0,0,3000x3000 Dst=0,0,3000x3000
...10...20...30...40...50...60...70...80...90...100 - done.

With -ovr 1 the output has the size of the full resolution image, 12000 x 12000 
pixels, even the data is read from the 3000 x 3000 overview. The pixels will be 
upsampled.

GDAL: Using GTiff driver
GDAL: Computing area of interest: 26.8805, 63.2011, 27, 63.255
Creating output file that is 12000P x 12000L.
WARP: Selecting overview level 1 for vrttest.vrt
…
GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=0,0,1500x750 Dst=0,0,6000x3000
...10.GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=1500,0,1500x750 Dst=6000,0,6000x3000
..20..GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=0,750,1500x750 Dst=0,3000,6000x3000
.30...GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=1500,750,1500x750 Dst=6000,3000,6000x3000
40...50GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=0,1500,1500x750 Dst=0,6000,6000x3000
...60.GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=1500,1500,1500x750 Dst=6000,6000,6000x3000
..70..GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=0,2250,1500x750 Dst=0,9000,6000x3000
.80...GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() 
Src=1500,2250,1500x750 Dst=6000,9000,6000x3000
90...100 - done.
GDAL: Flushing dirty blocks: 
0...10...20...30...40...50...60...70...80...90...100 - done.

The behavior is the same when using a TIFF with internal overviews as input 
instead of VRT. Obviously the -ovr option has been designed to force the use of 
some better-quality overview as source data when warping yields a smaller, 
downsampled image. The forced upsampling does not make much sense to me. I 
wonder if gdalwarp should even allow that?

The gdal_translate utility has also -ovr option 
https://gdal.org/programs/gdal_translate.html#cmdoption-gdal_translate-ovr. It 
behaves in a different way  when it comes to the size of the output, but with 
gdal_translate it makes sense.
“When -ovr is specified to an integer value, and none of -outsize and -tr is 
specified, the size of the overview will be used as the output size.”

Maybe the thing to do is to improve the documentation of the gdalwarp option 
https://gdal.org/programs/gdalwarp.html#cmdoption-gdalwarp-ovr and mention that 
this option does not have on effect on the size of the output.

-Jukka Rahkonen-




Lähettäjä: gdal-dev  Puolesta Denis Rykov via 
gdal-dev
Lähetetty: maanantai 10. kesäkuuta 2024 2.27
Vastaanottaja: gdal dev 
Aihe: [gdal-dev] Discrepancy in utilizing overviews with gdalwarp

I'd anticipate that both commands would yield the same outcome:

$ gdalwarp -oo OVERVIEW_LEVEL=1 -dstalpha input.vrt ovr1.tif
$ gdalwarp -ovr 1 -dstalpha input.vrt ovr1.tif

However, the reality is different. The first command behaves as intended, 
utilizing overviews from *.vrt.ovr, while the second command operates 
noticeably slower, seemingly disregarding the overviews.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdaladdo slowness for VRTs

[Rahkonen Jukka via gdal-dev]

Hi,

A better reference file can be created by materializing the VRT. The one 
created by gdal_create has the same raster values on all the pixels but the 
materialized one contains the same original image data. Maybe in your test case 
with uncompressed outputs the difference is not so big, but please test it 
anyway.

gdal_translate -of GTiff -co tiled=yes -co compress=LZW input.vrt output.tif
gdaladdo -ro output.tif

What is the difference in speed now? Ten times slower performance with vrt 
feels quite a lot.

-Jukka Rahkonen-


Lähettäjä: gdal-dev  Puolesta Denis Rykov via 
gdal-dev
Lähetetty: keskiviikko 5. kesäkuuta 2024 4.36
Vastaanottaja: gdal dev 
Aihe: [gdal-dev] gdaladdo slowness for VRTs

Hi,

I spotted a slow calculation of overviews and I'm wondering what could be the 
reason.

$ gdal_create in.tif -if 20240602_230818_SN26_RR_VISUAL_MS.vrt
$ time gdaladdo -ro in.tif
gdaladdo -ro in.tif  4,25s user 6,23s system 34% cpu 30,623 total

But on the file of the same size and num of bands but VRT it takes much more 
time:

$ time gdaladdo -ro 20240602_230818_SN26_RR_VISUAL_MS.vrt
gdaladdo -ro   55,36s user 5,06s system 44% cpu 2:14,79 total

Here is the output of gdalinfo of the VRT:

$ gdalinfo 20240602_230818_SN26_RR_VISUAL_MS.vrt
Driver: VRT/Virtual Raster
Files: 20240602_230818_SN26_RR_VISUAL_MS.vrt
   20240602_230818_SN26_RR_VISUAL_MS_340_5020.tif
   20240602_230818_SN26_RR_VISUAL_MS_340_5040.tif
   20240602_230818_SN26_RR_VISUAL_MS_360_5020.tif
   20240602_230818_SN26_RR_VISUAL_MS_360_5040.tif
   20240602_230818_SN26_RR_VISUAL_MS_360_5060.tif
Size is 20779, 46754
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 59S",
BASEGEOGCRS["WGS 84",
DATUM["World Geodetic System 1984",
ELLIPSOID["WGS 84",6378137,298.257223563,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4326]],
CONVERSION["UTM zone 59S",
METHOD["Transverse Mercator",
ID["EPSG",9807]],
PARAMETER["Latitude of natural origin",0,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8801]],
PARAMETER["Longitude of natural origin",171,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8802]],
PARAMETER["Scale factor at natural origin",0.9996,
SCALEUNIT["unity",1],
ID["EPSG",8805]],
PARAMETER["False easting",50,
LENGTHUNIT["metre",1],
ID["EPSG",8806]],
PARAMETER["False northing",1000,
LENGTHUNIT["metre",1],
ID["EPSG",8807]]],
CS[Cartesian,2],
AXIS["easting",east,
ORDER[1],
LENGTHUNIT["metre",1]],
AXIS["northing",north,
ORDER[2],
LENGTHUNIT["metre",1]],
ID["EPSG",32759]]
Data axis to CRS axis mapping: 1,2
Origin = (353999.7988358,5068000.62999888241)
Pixel Size = (0.770,-0.770)
Corner Coordinates:
Upper Left  (  353999.800, 5068000.630) (169d 9'45.50"E, 44d31'35.55"S)
Lower Left  (  353999.800, 5032000.050) (169d 9' 8.52"E, 44d51' 1.68"S)
Upper Right (  36.630, 5068000.630) (169d21'50.06"E, 44d31'46.58"S)
Lower Right (  36.630, 5032000.050) (169d21'17.12"E, 44d51'12.83"S)
Center  (  361999.715, 505.340) (169d15'30.36"E, 44d41'24.33"S)
Band 1 Block=128x128 Type=Byte, ColorInterp=Red
  NoData Value=0
Band 2 Block=128x128 Type=Byte, ColorInterp=Green
  NoData Value=0
Band 3 Block=128x128 Type=Byte, ColorInterp=Blue
  NoData Value=0
Band 4 Block=128x128 Type=Byte, ColorInterp=Undefined
  NoData Value=0

And gdalinfo output for one of the underlying rasters:

gdalinfo 20240602_230818_SN26_RR_VISUAL_MS_360_5040.tif
Driver: GTiff/GeoTIFF
Files: 20240602_230818_SN26_RR_VISUAL_MS_360_5040.tif
Size is 10390, 25974
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 59S",
BASEGEOGCRS["WGS 84",
DATUM["World Geodetic System 1984",
ELLIPSOID["WGS 84",6378137,298.257223563,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4326]],
CONVERSION["UTM zone 59S",
METHOD["Transverse Mercator",
ID["EPSG",9807]],
PARAMETER["Latitude of natural origin",0,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8801]],
PARAMETER["Longitude of natural origin",171,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8802]],
PARAMETER["Scale factor at natural origin",0.9996,
SCALEUNIT["unity",1],
ID["EPSG",8805]],
PARAMETER["False easting",50,
LENGTHUNIT["metre",1],
ID["EPSG",8806]],
PARAMETER["False northing",1000,
LENGTHUNIT["metre",1],
ID["EPSG",8807]]],
CS[Cartesian,2],
AXIS["(E)",east,
ORDER[1],

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Rahkonen Jukka via gdal-dev]

via gdal-dev a écrit :
Because this is not clear in documentation, especially in WMTS driver docs, my 
question is strictly about this issue to programmers who have already explored 
this topic.

Regards
Michal

W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 napisał(a):
Hi,

No, I am not sure because I do not program myself. Reading this document 
https://gdal.org/user/multithreading.html makes me feel that it is possible to 
do some things in parallel, but the programmer must know how to do it right.

-Jukka Rahkonen-

Lähettäjä: Javier Jimenez Shaw 
mailto:j...@jimenezshaw.com>>
Lähetetty: keskiviikko 29. toukokuuta 2024 10.41
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Kopio: Michał Kowalczuk 
mailto:michkowalc...@gmail.com>>; 
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation



On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Hi,

When you have a RasterBand  from the WMTS data source, it is abstracted and you 
can read the raster data just like from any other data source and raster band 
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data “There are a 
few ways to read raster data, but the most common is via the 
GDALRasterBand::RasterIO() method. This method will automatically take care of 
data type conversion, up/down sampling and windowing.” GDAL knows which tiles 
to read.

I do not know if the WMTS driver can do parallel tile downloads. If not, it is 
possible to run many RasterIO() at the same time, each reading data from a 
different window like in this rasterio document 
https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.

Jukka, are you sure you can run several RasterIO in parallel (on the same 
dataset)? in GDAL GeoTIFF you cannot: the cache may be corrupted. In that case 
I open several datasets over the same file. I do not know about WMTS.


-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Michal Kowalczuk via gdal-dev
Lähetetty: keskiviikko 29. toukokuuta 2024 9.08
Vastaanottaja: gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: [gdal-dev] GDAL WMTS get tiles no documentation

Hi GDAL fellows
This is my first post on this mailing list, so I'm asking for understanding.

As all we know the purpose of using WMTS over WMS, I'd like to implement 
parallel downloading tiles from service using C API.
In my opinion GDAL documentation (https://gdal.org/drivers/raster/wmts.html) 
says nothing on this topic.
I can get capabilities from WMTS, I can open the selected subdataset but how to 
get tiles for given extent? I could not find any information how to do it, even 
in the GDAL tests on github.

I am kindly asking for tips. It also can be in python. How using pure GDAL API 
fetch tiles to dynamically complete the displayed map. This is my goal.

Thank you!
Michal
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev


___

gdal-dev mailing list

gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>

https://lists.osgeo.org/mailman/listinfo/gdal-dev

--

http://www.spatialys.com<http://www.spatialys.com/>

My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Rahkonen Jukka via gdal-dev]

(resending, need to cut old discussion due to too many bytes in the body)

Hi,

gdalinfo 
"WMTS:http://maps.wien.gv.at/wmts/1.0.0/WMTSCapabilities.xml,layer=fmzk3dviewer;
Driver: WMTS/OGC Web Map Tile Service
Files: none associated
Size is 305719, 257379

The unit of size is pixels. You used -srcwin 1820157 6144777 512 512, with 
offsets far outsize the layer. This returns me something greenish.

gdal_translate 
"WMTS:http://maps.wien.gv.at/wmts/1.0.0/WMTSCapabilities.xml,layer=fmzk3dviewer;
 -srcwin 10 10 512 512 tile.jpg

-Jukka Rahkonen-


Lähettäjä: Michał Kowalczuk 
mailto:michkowalc...@gmail.com>>
Lähetetty: perjantai 31. toukokuuta 2024 16.58
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Kopio: Even Rouault 
mailto:even.roua...@spatialys.com>>; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation

As you proposed I tested gdal_translate utility but for the following sample 
command I get an empty image (tested with jpg and png)

gdal_translate 
"WMTS:http://maps.wien.gv.at/wmts/1.0.0/WMTSCapabilities.xml,layer=fmzk3dviewer;
 -srcwin 1820157 6144777 512 512 tile.jpg
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Rahkonen Jukka via gdal-dev]

Hi,

Before starting to program you can familiar yourself with the topic by using 
gdal_translate gdal_translate — GDAL 
documentation<https://gdal.org/programs/gdal_translate.html>.
Play with the -srcwin and -projwin / -projwin_srs options and gather some trust 
that is is possible to select and download any part of the large WMTS layer. 
Add “-- debug on” and you will see that GDAL downloads tiles separately just 
like QGIS. Actually, with WMTS it is the only way to download data. Maybe you 
have read the standard already https://portal.ogc.org/files/?artifact_id=35326. 
It is not necessary because GDAL knows what to do, but a review every now and 
then does not harm. Especially the TileMatrix and TileMatrixSet things are not 
self-evident, but they provide the answer to the question about how a WMTS 
client knows which row, col, and zoom to download.

Do not touch the DataWindow. It defines the extent of the whole WMTS layer and 
it is used for example if gdal_translate is run without spatial selection 
(-srcwin or -projwin). Touch in your future code things which are equivalent to 
-srcwin or -projwin of gdal_translate.

-Jukka Rahkonen-

Lähettäjä: Michał Kowalczuk 
Lähetetty: perjantai 31. toukokuuta 2024 15.45
Vastaanottaja: Even Rouault 
Kopio: Rahkonen Jukka ; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation

Hi, thank you for your replies!
As I'm still a newbie especially with WMTS. I need to take a step back to the 
basics. Let's forget about multithreading for now.

  *   I'm in the stage that I can open a dataset using WMTS service path from 
data provider
  *   I can list subdatasets and open the selected layer as a new dataset using 
path from SUBDATASET_id_NAME.
  *   This dataset reports large height and width values, which is obvious.
  *   The first question is how to read any fragment from this dataset using 
GDALDatasetRasterIO.
Is there a similar way to SpatialFIlter from vector layers?
I see in the docs there is a  tag for a local service descriptions 
which defines extents of the data. Does it mean that for each map extent change 
I need to create a new local XML file and open a new dataset to read demanded 
bbox?
Is this the only way for doing it?
  *   From other site I noticed that QGIS downloads tiles separately? But how 
it knows which row/col and zoom should be used in GET request? Does it use GDAL 
for WMTS reading at all?
[cid:image001.png@01DAB373.3E20F850]
I will be grateful for any tip and trucks for a newbie ;-)

Regards,
Michal

śr., 29 maj 2024 o 12:41 Even Rouault 
mailto:even.roua...@spatialys.com>> napisał(a):

Hi,

The WMS cache (which is used underneath by the WMTS driver) writes each tiles 
in a separate file. So if you use multi-threaded, as long as you read in 
parallel different GDALDataset* object on the WMTS filename, and read disjoint 
areas, that should be fine. There might be a slight risk of things going bad if 
you happened to read the same tile from different GDALDataset* object at the 
same time, as it could potentially read a partially written cached tile, or 
concurrently write the tile in the cache and corrupt it. The logic is in 
frmts/wms/gdalwmscache.cpp

Even
Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
Because this is not clear in documentation, especially in WMTS driver docs, my 
question is strictly about this issue to programmers who have already explored 
this topic.

Regards
Michal

W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 napisał(a):
Hi,

No, I am not sure because I do not program myself. Reading this document 
https://gdal.org/user/multithreading.html makes me feel that it is possible to 
do some things in parallel, but the programmer must know how to do it right.

-Jukka Rahkonen-

Lähettäjä: Javier Jimenez Shaw 
mailto:j...@jimenezshaw.com>>
Lähetetty: keskiviikko 29. toukokuuta 2024 10.41
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Kopio: Michał Kowalczuk 
mailto:michkowalc...@gmail.com>>; 
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation



On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Hi,

When you have a RasterBand  from the WMTS data source, it is abstracted and you 
can read the raster data just like from any other data source and raster band 
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data “There are a 
few ways to read raster data, but the most common is via the 
GDALRasterBand::RasterIO() method. This method will automatically take care of 
data type conversion, up/down sampling and windowing.” GDAL knows which tiles 
to read.

I do not know if the WMTS driver can do parallel tile downloads. If not, it is 
possible to run many RasterIO() at the same time, each reading data from a 
differ

Re: [gdal-dev] OGR SQL CAST geometry example

[Rahkonen Jukka via gdal-dev]

Hi,

As far I know the the Spatialite functions 
https://www.gaia-gis.it/gaia-sins/spatialite-sql-4.2.0.html by design work row 
by row and one input row returns one output row. Therefore DissolvePoints 
creates one row with a multipoint geometry, not several rows like ST_Dump does. 
You can discuss about that in the spatialite-users forum 
https://groups.google.com/g/spatialite-users.
It is possible to extract the points from linestrings with SpatiaLite SQL by 
using NumPoints and then PointN in a loop, but such SQL is not possible to 
execute with ogrinfo/ogr2ogr. It is possible to do amazing things with ogr2ogr 
and SQL but it does make it the right tool for demanding ETL processes. 
Typically people turn to use Python at that point.

If you have a real need to extract points from a linestring by using only GDAL 
command line utilities, it should be possible to do in two steps. Create an 
interim file by using the Spatialite function DissolvePoints, and then run 
ogr2ogr with the -explodecollections option.

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Dan Jacobson 
via gdal-dev
Lähetetty: perjantai 31. toukokuuta 2024 8.36
Vastaanottaja: Daniel Baston 
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] OGR SQL CAST geometry example

Alas, even for MULTIPOINT, as there are no facilities for extracting from 
aggregates, not even ST_Dump, in the end all one can do is $ ogrinfo 
0.lines.kml -al | perl -nwle \
   'if(/LINESTRING.*\((.*)\)/){print for split /,/, $1}'
-100.79005679 36.38022 1000
-100.79008013 36.38531 1000
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Rahkonen Jukka via gdal-dev]

Hi,

No, I am not sure because I do not program myself. Reading this document 
https://gdal.org/user/multithreading.html makes me feel that it is possible to 
do some things in parallel, but the programmer must know how to do it right.

-Jukka Rahkonen-

Lähettäjä: Javier Jimenez Shaw 
Lähetetty: keskiviikko 29. toukokuuta 2024 10.41
Vastaanottaja: Rahkonen Jukka 
Kopio: Michał Kowalczuk ; gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation



On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Hi,

When you have a RasterBand  from the WMTS data source, it is abstracted and you 
can read the raster data just like from any other data source and raster band 
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data "There are a 
few ways to read raster data, but the most common is via the 
GDALRasterBand::RasterIO() method. This method will automatically take care of 
data type conversion, up/down sampling and windowing." GDAL knows which tiles 
to read.

I do not know if the WMTS driver can do parallel tile downloads. If not, it is 
possible to run many RasterIO() at the same time, each reading data from a 
different window like in this rasterio document 
https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.

Jukka, are you sure you can run several RasterIO in parallel (on the same 
dataset)? in GDAL GeoTIFF you cannot: the cache may be corrupted. In that case 
I open several datasets over the same file. I do not know about WMTS.


-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Michal Kowalczuk via gdal-dev
Lähetetty: keskiviikko 29. toukokuuta 2024 9.08
Vastaanottaja: gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: [gdal-dev] GDAL WMTS get tiles no documentation

Hi GDAL fellows
This is my first post on this mailing list, so I'm asking for understanding.

As all we know the purpose of using WMTS over WMS, I'd like to implement 
parallel downloading tiles from service using C API.
In my opinion GDAL documentation (https://gdal.org/drivers/raster/wmts.html) 
says nothing on this topic.
I can get capabilities from WMTS, I can open the selected subdataset but how to 
get tiles for given extent? I could not find any information how to do it, even 
in the GDAL tests on github.

I am kindly asking for tips. It also can be in python. How using pure GDAL API 
fetch tiles to dynamically complete the displayed map. This is my goal.

Thank you!
Michal
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Rahkonen Jukka via gdal-dev]

Hi,

When you have a RasterBand  from the WMTS data source, it is abstracted and you 
can read the raster data just like from any other data source and raster band 
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data "There are a 
few ways to read raster data, but the most common is via the 
GDALRasterBand::RasterIO() method. This method will automatically take care of 
data type conversion, up/down sampling and windowing." GDAL knows which tiles 
to read.

I do not know if the WMTS driver can do parallel tile downloads. If not, it is 
possible to run many RasterIO() at the same time, each reading data from a 
different window like in this rasterio document 
https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Michal 
Kowalczuk via gdal-dev
Lähetetty: keskiviikko 29. toukokuuta 2024 9.08
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL WMTS get tiles no documentation

Hi GDAL fellows
This is my first post on this mailing list, so I'm asking for understanding.

As all we know the purpose of using WMTS over WMS, I'd like to implement 
parallel downloading tiles from service using C API.
In my opinion GDAL documentation (https://gdal.org/drivers/raster/wmts.html) 
says nothing on this topic.
I can get capabilities from WMTS, I can open the selected subdataset but how to 
get tiles for given extent? I could not find any information how to do it, even 
in the GDAL tests on github.

I am kindly asking for tips. It also can be in python. How using pure GDAL API 
fetch tiles to dynamically complete the displayed map. This is my goal.

Thank you!
Michal
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Gdal_grid produces a lot of empty cells

[Rahkonen Jukka via gdal-dev]

Hi,

I forgot to say that I think it is not a good idea to use gdal_grid with 
EPSG:3857 at all. The scale of Web Mercator is correct only at the equator. In 
your data from Poland, if the real ellipsoidal distance between two points on 
the ground is 10 meters, the cartesian distance in EPSG:3857 claims that the 
distance is 15 meters. I think it would be better to compute the grid in a CRS 
that has approximately correct scale, and re-project the raster into Web 
Mercator if needed. That said, perhaps there is a bug in the gdal_grid utility.

-Jukka Rahkonen-


Lähettäjä: Paul Meems 
Lähetetty: keskiviikko 22. toukokuuta 2024 18.24
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka 
Aihe: Re: [gdal-dev] Gdal_grid produces a lot of empty cells

Sorry Jukka,

Forgot all about that ;)

I've attached the csv from the field in Poland with the original lon-lat values 
and the X and Y in EPSG:3857

Op wo 22 mei 2024 om 16:57 schreef Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>:
Hi,

At least I cannot resolve your issue on paper. Please share some test data.

-Jukka-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Paul Meems via gdal-dev
Lähetetty: keskiviikko 22. toukokuuta 2024 17.31
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Gdal_grid produces a lot of empty cells

Hello List,

I have a CSV-file with sensor data and GPS coordinates in WGS84.
I convert these GPS coordinates to a projected projection. Mostly a UTM zone, 
depending on the location.
These X and Y values are added to the CSV-file and saved with a different name.
Then I create a VRT-file to get the proper Z-column:


01 field data-EPSG32634.csv
wkbPoint




This VRT file is sent to Gdal_grid to create a tiff-file using these arguments:
-a invdist:power=2:
   smoothing=20:
   min_points=16:
   max_points=56:
   radius1=40:
   radius2=40:
   nodata=1.7014100071e+38
-txe 273360 274940
-tye 5559440 5561020
-tr 12.5 12.5
-a_srs EPSG:32634
"01 field data-EPSG32634-TC.vrt" "tmpmrmq4d.tiff"

This tiff-file looks like this: https://pasteboard.co/tKP1ZJ1cAY2r.png

This process works fine and we've been using this for several years now.

Recently we added a process step to export one of the sensor data to a PostGIS 
database and with the help of Geoserver and OpenLayers we've created a very 
simple webpage.
In this step, things are getting weird. The WebMap is in EPSG:3857 
(Pseudo/Web-Mercator).
I start with the original CSV-file with the sensor data and add the X and Y 
columns again, now in EPSG:3857. Then I create a similar vrt-file:


01 field data-EPSG3857.csv
wkbPoint




And call gdal_grid with similar arguments:
-a invdist:power=2:
   smoothing=20:
   min_points=16:
   max_points=56:
   radius1=40:
   radius2=40:
   nodata=1.7014100071e+38
-txe 1984580 1987000
-tye 6471320 6473740
-tr 12.5 12.5
-a_srs EPSG:3857
"01 field data-EPSG3857-TC.vrt" "tmpun25fo.tiff"

But now the result is: https://pasteboard.co/bcMp7rP5eyff.png

A huge amount of empty cells. I don't understand this.
Both EPSG:32634 (WGS 84 / UTM zone 34N) and EPSG:3857 are in meters so I 
thought the same settings should work.
But in fact only with very different settings I can get a proper grid:
 -a invdist:power=2:
   smoothing=60:
   min_points=4:
   max_points=56:
   radius1=80:
   radius2=80:
   nodata=1.7014100071e+38
-txe 1984580 1987000
-tye 6471320 6473740
-tr 20 20
-a_srs EPSG:3857
"01 field data-EPSG3857-TC.vrt" "tmpsvlhgy.tiff"

I got these values through trial and error.
https://pasteboard.co/7pypGsynS09K.png

But now we loose a lot of variation because the search radius and grid cell 
size are much higher.

To make it even more weird. Other fields nearby don't have this problem and I 
also have this problem with other projections like:

Belgium (EPSG:3857): https://pasteboard.co/ZXLTsEuvpZ3s.png
Belgium (EPSG:31370): https://pasteboard.co/yNpF6yujF3tV.png

The Netherlands (EPSG:3857): https://pasteboard.co/8PpDJrDi1Wxq.png
The Netherlands (EPSG:28992): https://pasteboard.co/38SlnHVCog4m.png

Norway (EPSG:3857): https://pasteboard.co/q69jQHRK40c6.png
Norway (EPSG: 25833): https://pasteboard.co/gs9HNAmnaxSJ.png

We do have enough data points: https://pasteboard.co/4CmBs7kgQGVO.png
https://pasteboard.co/J8ZbMHKeErZu.png

Thanks if you made it all the way here ;)

My questions are:

  *   Am I using Gdal_Grid in the proper way or do I need to change one or more 
arguments?
  *   What might be an explanation for the empty cells when using EPSG:3857?

Thanks,

Paul Meems











___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Gdal_grid produces a lot of empty cells

[Rahkonen Jukka via gdal-dev]

Hi,

I do not know what happens, but it seems that the algorithm finds rather few 
points within the search ellipse. The radius of the circle is 40 m and by 
looking at the data there are typically at least 30 points within that 
distance. However, for filling the whole raster I had to use "min_points=4".
I could repeat the behavior by opening the 3857 data into QGIS and running the 
invdist from there. And when I reprojected the point data into EPSG:32634 then 
QGIS created a good gridded raster. The difference in scale between EPSG:3857 
and EPSG:32634 is not so big in Poland that it could be the reason IMHO.
For removing possible errors due to CSV format and VRT I materialized the 
datasets into OpenJUMP JML format. It could have been any other GIS format.

Unfortunately I cannot help you more. If I were a programmer I think I would 
try to catch how many points the algorithm finds within the search radius in 
EPSG:3857, compare how many points it finds from EPSG:32634 data, and finally 
consider if the results make sense.

-Jukka Rahkonen-



Lähettäjä: Paul Meems 
Aihe: Re: [gdal-dev] Gdal_grid produces a lot of empty cells

Sorry Jukka,

Forgot all about that ;)

I've attached the csv from the field in Poland with the original lon-lat values 
and the X and Y in EPSG:3857

Op wo 22 mei 2024 om 16:57 schreef Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>:
Hi,

At least I cannot resolve your issue on paper. Please share some test data.

-Jukka-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Paul Meems via gdal-dev
Lähetetty: keskiviikko 22. toukokuuta 2024 17.31
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Gdal_grid produces a lot of empty cells

Hello List,

I have a CSV-file with sensor data and GPS coordinates in WGS84.
I convert these GPS coordinates to a projected projection. Mostly a UTM zone, 
depending on the location.
These X and Y values are added to the CSV-file and saved with a different name.
Then I create a VRT-file to get the proper Z-column:


01 field data-EPSG32634.csv
wkbPoint




This VRT file is sent to Gdal_grid to create a tiff-file using these arguments:
-a invdist:power=2:
   smoothing=20:
   min_points=16:
   max_points=56:
   radius1=40:
   radius2=40:
   nodata=1.7014100071e+38
-txe 273360 274940
-tye 5559440 5561020
-tr 12.5 12.5
-a_srs EPSG:32634
"01 field data-EPSG32634-TC.vrt" "tmpmrmq4d.tiff"

This tiff-file looks like this: https://pasteboard.co/tKP1ZJ1cAY2r.png

This process works fine and we've been using this for several years now.

Recently we added a process step to export one of the sensor data to a PostGIS 
database and with the help of Geoserver and OpenLayers we've created a very 
simple webpage.
In this step, things are getting weird. The WebMap is in EPSG:3857 
(Pseudo/Web-Mercator).
I start with the original CSV-file with the sensor data and add the X and Y 
columns again, now in EPSG:3857. Then I create a similar vrt-file:


01 field data-EPSG3857.csv
wkbPoint




And call gdal_grid with similar arguments:
-a invdist:power=2:
   smoothing=20:
   min_points=16:
   max_points=56:
   radius1=40:
   radius2=40:
   nodata=1.7014100071e+38
-txe 1984580 1987000
-tye 6471320 6473740
-tr 12.5 12.5
-a_srs EPSG:3857
"01 field data-EPSG3857-TC.vrt" "tmpun25fo.tiff"

But now the result is: https://pasteboard.co/bcMp7rP5eyff.png

A huge amount of empty cells. I don't understand this.
Both EPSG:32634 (WGS 84 / UTM zone 34N) and EPSG:3857 are in meters so I 
thought the same settings should work.
But in fact only with very different settings I can get a proper grid:
 -a invdist:power=2:
   smoothing=60:
   min_points=4:
   max_points=56:
   radius1=80:
   radius2=80:
   nodata=1.7014100071e+38
-txe 1984580 1987000
-tye 6471320 6473740
-tr 20 20
-a_srs EPSG:3857
"01 field data-EPSG3857-TC.vrt" "tmpsvlhgy.tiff"

I got these values through trial and error.
https://pasteboard.co/7pypGsynS09K.png

But now we loose a lot of variation because the search radius and grid cell 
size are much higher.

To make it even more weird. Other fields nearby don't have this problem and I 
also have this problem with other projections like:

Belgium (EPSG:3857): https://pasteboard.co/ZXLTsEuvpZ3s.png
Belgium (EPSG:31370): https://pasteboard.co/yNpF6yujF3tV.png

The Netherlands (EPSG:3857): https://pasteboard.co/8PpDJrDi1Wxq.png
The Netherlands (EPSG:28992): https://pasteboard.co/38SlnHVCog4m.png

Norway (EPSG:3857): https://pasteboard.co/q69jQHRK40c6.png
Norway (EPSG: 25833): https://pasteboard.co/gs9HNAmnaxSJ.png

We do have enough data points: https://pasteboard.co/4CmBs7kgQGVO.png
https://pasteboard.co/J8ZbMHKeErZu.png

Thanks if you made it all the way here ;)

My questions are:

  *   Am I using Gdal_Grid in the proper way or do I need to change one or more 
arguments?
  *   What might be an explanation for the empty 

Re: [gdal-dev] Gdal_grid produces a lot of empty cells

[Rahkonen Jukka via gdal-dev]

Hi,

At least I cannot resolve your issue on paper. Please share some test data.

-Jukka-

Lähettäjä: gdal-dev  Puolesta Paul Meems via 
gdal-dev
Lähetetty: keskiviikko 22. toukokuuta 2024 17.31
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Gdal_grid produces a lot of empty cells

Hello List,

I have a CSV-file with sensor data and GPS coordinates in WGS84.
I convert these GPS coordinates to a projected projection. Mostly a UTM zone, 
depending on the location.
These X and Y values are added to the CSV-file and saved with a different name.
Then I create a VRT-file to get the proper Z-column:


01 field data-EPSG32634.csv
wkbPoint




This VRT file is sent to Gdal_grid to create a tiff-file using these arguments:
-a invdist:power=2:
   smoothing=20:
   min_points=16:
   max_points=56:
   radius1=40:
   radius2=40:
   nodata=1.7014100071e+38
-txe 273360 274940
-tye 5559440 5561020
-tr 12.5 12.5
-a_srs EPSG:32634
"01 field data-EPSG32634-TC.vrt" "tmpmrmq4d.tiff"

This tiff-file looks like this: https://pasteboard.co/tKP1ZJ1cAY2r.png

This process works fine and we've been using this for several years now.

Recently we added a process step to export one of the sensor data to a PostGIS 
database and with the help of Geoserver and OpenLayers we've created a very 
simple webpage.
In this step, things are getting weird. The WebMap is in EPSG:3857 
(Pseudo/Web-Mercator).
I start with the original CSV-file with the sensor data and add the X and Y 
columns again, now in EPSG:3857. Then I create a similar vrt-file:


01 field data-EPSG3857.csv
wkbPoint




And call gdal_grid with similar arguments:
-a invdist:power=2:
   smoothing=20:
   min_points=16:
   max_points=56:
   radius1=40:
   radius2=40:
   nodata=1.7014100071e+38
-txe 1984580 1987000
-tye 6471320 6473740
-tr 12.5 12.5
-a_srs EPSG:3857
"01 field data-EPSG3857-TC.vrt" "tmpun25fo.tiff"

But now the result is: https://pasteboard.co/bcMp7rP5eyff.png

A huge amount of empty cells. I don't understand this.
Both EPSG:32634 (WGS 84 / UTM zone 34N) and EPSG:3857 are in meters so I 
thought the same settings should work.
But in fact only with very different settings I can get a proper grid:
 -a invdist:power=2:
   smoothing=60:
   min_points=4:
   max_points=56:
   radius1=80:
   radius2=80:
   nodata=1.7014100071e+38
-txe 1984580 1987000
-tye 6471320 6473740
-tr 20 20
-a_srs EPSG:3857
"01 field data-EPSG3857-TC.vrt" "tmpsvlhgy.tiff"

I got these values through trial and error.
https://pasteboard.co/7pypGsynS09K.png

But now we loose a lot of variation because the search radius and grid cell 
size are much higher.

To make it even more weird. Other fields nearby don't have this problem and I 
also have this problem with other projections like:

Belgium (EPSG:3857): https://pasteboard.co/ZXLTsEuvpZ3s.png
Belgium (EPSG:31370): https://pasteboard.co/yNpF6yujF3tV.png

The Netherlands (EPSG:3857): https://pasteboard.co/8PpDJrDi1Wxq.png
The Netherlands (EPSG:28992): https://pasteboard.co/38SlnHVCog4m.png

Norway (EPSG:3857): https://pasteboard.co/q69jQHRK40c6.png
Norway (EPSG: 25833): https://pasteboard.co/gs9HNAmnaxSJ.png

We do have enough data points: https://pasteboard.co/4CmBs7kgQGVO.png
https://pasteboard.co/J8ZbMHKeErZu.png

Thanks if you made it all the way here ;)

My questions are:

  *   Am I using Gdal_Grid in the proper way or do I need to change one or more 
arguments?
  *   What might be an explanation for the empty cells when using EPSG:3857?

Thanks,

Paul Meems











___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to exclude a layer from a translation using ogr2ogr

[Rahkonen Jukka via gdal-dev]

Hi,

See this recent thread 
https://lists.osgeo.org/pipermail/gdal-dev/2024-May/058993.html. I think that 
authoring a vector VRT file is also the answer to your question.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Abel Pau via 
gdal-dev
Lähetetty: torstai 16. toukokuuta 2024 8.56
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] How to exclude a layer from a translation using ogr2ogr

Hi,
I want to convert a Geopackage into MiraMonVector format.
I use this command line:
ogr2ogr "D:\mapes\2024\044 XP Topografia 
territorial\topografia-territorial-v1r0-3d-2022_menys_90_modelatge_terreny" 
"D:\mapes\2024\044 XP Topografia 
territorial\topografia-territorial-v1r0-3d-2022.gpkg" -f MiraMonVector

This gpkg has several layers and I'd like to exclude the one called 
_90_modelatge_terreny_l

I've tried several methods but they don't work.
Anyone know how to do that?
Thanks!

Abel Pau Garcia
GIS developer
[https://www.creaf.cat/sites/default/files/creaf-signature.png]
a@creaf.uab.cat 
Let's chat on Teams! 

Tel. +34 934814277
[https://www.creaf.cat/sites/default/files/so-cat-signature.png]
[https://www.creaf.cat/sites/default/files/primary_x.png][https://www.creaf.cat/sites/default/files/primary_linkedin.png][https://www.creaf.cat/sites/default/files/primary_youtube.png][https://www.creaf.cat/sites/default/files/primary_instagram.png]
www.creaf.cat  | http://blog.creaf.cat 

[https://www.creaf.cat/sites/default/files/cap_0.png]
CREAF. Campus UAB. Edifici C. 08193 Bellaterra (Barcelona)
Abans d'imprimir aquest missatge electrònic penseu en el medi ambient.
[https://www.creaf.cat/sites/default/files/corner_right_0.png]


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] FileGDB that does not open

[Rahkonen Jukka via gdal-dev]

Hi,

FileGDB data from 
https://vgin.maps.arcgis.com/home/item.html?id=29627d7c051a47dc8ce71b4484531ab3 
"Virginia_Parcel_Dataset_2024Q1.gdb" does not open with GDAL 3.9.0dev.  The 
error is:

Warning 1: VA_Parcels layer has a 
Virginia_Parcel_Dataset_2024Q1.gdb\a0009.gdbtable.cdf file whose format is 
unhandled

I do not need the data. I am only testing, but maybe someone wants to study 
what is special in the a0009.gdbtable.cdf file.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: approve GDAL 3.9.02RC2 as final release

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: keskiviikko 8. toukokuuta 2024 14.13
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Motion: approve GDAL 3.9.02RC2 as final release

Hi,

Motion:

Adopt GDAL 3.9.0RC2 as final 3.9.0 release

Starting with my +1

Even

-- 
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Total Viewshed and GDAL

[Rahkonen Jukka via gdal-dev]

Hi,

Can this tool utilize digital surface model and deal with obstacles like 
buildings and trees?

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Luis Felipe 
Romero via gdal-dev
Lähetetty: maanantai 29. huhtikuuta 2024 12.16
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] Total Viewshed and GDAL

Hi,

After several months working on other things, I have returned to an application 
of the total viewshed.

What I have done is use Tamas Szekeres' C++ code as a template and, at least on 
Windows, I have managed to compile and run. With just a GPU it is quite fast: 
on a 6000x6000 raster it does the calculations in less than a minute, although 
it can take two or three hours without a GPU on a PC with 8 cores. In multiple 
viewshed mode, the time is reduced proportionally.

On this page I have uploaded more info and some images of a national parks 
project I am working on. https://github.com/luisfromero/ppnn

It's still a little premature to share it, because I need to do many more 
tests, compile in Linux, etc., but my main doubt right now is that I need to 
shape the part of the tool necessary for calculating solar radiation:

In the so-called "mode 3" the visible horizon from each point is calculated, 
which together with other parameters such as tilt and head allow the 
calculation of solar radiation in the presence of shadows.

My question is the file format for the horizon data. For now, it's just an 
array with dimx·dimy·360 byte (using a one degree precision), and I'm 
considering different formats. For example, save it simply as an output file 
(compressed, preferably), but it could be compacted into a geopackage (as a 
metadata file or in a database, I don't know). They could even be saved in a 
multiband raster along with the elevation, tilt and head data from the  DEM. In 
short, everything necessary to then easily calculate the radiation in a fixed 
period of time.

Thanks for any suggestions!

Luis F Romero
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Reading interpolated values on DSM

[Rahkonen Jukka via gdal-dev]

Hi,

How about making a VRT file with doubled pixel size by applying your favourite 
resampling method and then making a query from the VRT file with 
gdallocationinfo?

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Javier Jimenez 
Shaw via gdal-dev
Lähetetty: keskiviikko 24. huhtikuuta 2024 11.33
Vastaanottaja: gdal dev 
Aihe: [gdal-dev] Reading interpolated values on DSM

Hi

I would like to read in QGIS or GDAL an interpolated value in a DSM (well, 
actually it is a geoid model, but it is the same behaviour). See that I do not 
want the pixel value, but the linear interpolation among the neighbour pixels, 
assuming that the pixel value is in the center of the pixel.
For instance, this file
https://www.isgeoid.polimi.it/Geoid/Asia/Japan/japan2000_g.html

Is there any way to get it (without implementing the interpolation myself)?
If I understood correctly gdallocationinfo is not interpolating, just giving 
the pixel value.

Thanks

.___ ._ ..._ .. . ._.  .___ .. __ . _. . __..  ...  ._ .__
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.9.0beta1 available for testing

[Rahkonen Jukka via gdal-dev]

Hi,

The mask thing may happen due to https://github.com/OSGeo/gdal/pull/9604.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Sean Gillies 
via gdal-dev
Lähetetty: maanantai 22. huhtikuuta 2024 19.15
Vastaanottaja: Even Rouault 
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL 3.9.0beta1 available for testing

Hi Even,

Rasterio's test suite has 4 errors with GDAL 3.9.0beta1.

Metadata output of gdalinfo has changed. As soon as there are docker images 
I'll look more closely at this.

Writing a mask to a Rasterio RGB.byte.tif dataset no longer creates a 
RGB.byte.tif.msk file: 
https://github.com/rasterio/rasterio/actions/runs/8786444142/job/24109342188#step:8:2231.

The text of an error message which was "tests/data/corrupt.tif, band 1: 
IReadBlock failed at X offset 1, Y offset 1: TIFFReadEncodedTile() failed." in 
3.8 has changed to "corrupt.tif, band 1: IReadBlock failed at X offset 1, Y 
offset 1: TIFFReadEncodedTile() failed." This is not a big deal, we can adjust 
Rasterio's test.

Behavior of SRC_METHOD/DST_METHOD=NO_GEOTRANSFORM for warp options seems to 
have changed: 
https://github.com/rasterio/rasterio/actions/runs/8786444142/job/24109342188#step:8:2287.
 I'm not entirely sure what the intent is in Rasterio's test, will dig into 
that. I just wanted to say that I see a change in GDAL.


On Mon, Apr 22, 2024 at 6:12 AM Even Rouault via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Hi,

I've prepared a beta1 of GDAL 3.9.0 to get feedback from early testers.

The NEWS file is here:

   https://github.com/OSGeo/gdal/blob/v3.9.0beta1/NEWS.md

For packagers,
https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html may
make it more attractive to build some drivers that typically have heavy
dependencies as plugins, installable in separate packages, due to the
load time penalty having being improved. It is let to the appreciation
of packagers to decide which drivers are worth building as plugins
installable in a separate package.
You may also pass CMake options
([GDAL/OGR]_DRIVER__PLUGIN_INSTALLATION_MESSAGE=xxx) so
user get a hint of which package they need to install when GDAL detects
that a file may be opened by a plugin which is not available on the file
system but known to have been built as a plugin. Cf
https://gdal.org/development/building_from_source.html#deferred-loaded-plugins
for more details.

For end-users, the following utilities have been updated to use a new
argument parsing framework: gdaladdo, gdalinfo, gdal_translate,
gdalwarp, gdal_grid, gdal_viewshed, gdalbuildvrt, nearblack, ogrinfo,
ogr2ogr, sozip.
This helps detecting errors such as specifying twice an argument that
should be specified once (where generally the last instance was the one
used). While we have tried to retain backwards compatibility for nominal
use cases, obviously if you have scripts that accidentally specify an
argument several times whereas it should be specified at most once, they
will have to be corrected.

Docker images with 3.9.0beta1 are currently cooking.

master is now marked as 3.10.0dev, and the release/3.9 branch has been
created. All bugfixes for 3.9 should be backported into it.

Source snapshots at:

- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.xz
- https://download.osgeo.org/gdal/3.9.0/gdal380beta1.zip

Autotest snapshots:

- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.zip

Even


--
Sean Gillies
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Issue with 3D GeoJSON to 3D DXF Conversion

[Rahkonen Jukka via gdal-dev]

Hi,

By reading the documentation 
https://gdal.org/drivers/vector/dxf.html#dxf-writer what you see is expected. 
--config DXF_WRITE_HATCH FALSE is writing polylines and polylines have vertices 
with different Z coordinates. --config DXF_WRITE_HATCH TRUE writes polygons. 
The polygons have same elevation value at each vertex, and that is also 
documented:
“You may need to do this [to use DXF_WRITE_HATCH FALSE] if your geometries do 
not have a constant elevation, as the DXF HATCH entity cannot represent such 
geometries.”

-Jukka Rahkonen-



Lähettäjä: gdal-dev  Puolesta Shahrukh Mirza 
Nawandish via gdal-dev
Lähetetty: maanantai 22. huhtikuuta 2024 13.26
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Issue with 3D GeoJSON to 3D DXF Conversion

Dear GDAL Team,

I hope this email finds you well. I'm encountering an issue converting 3D 
GeoJSON polygons to DXF format using ogr2ogr. During conversion, the elevation 
data becomes uniform for all points within the polygon. Additionally, when 
using "DXF_WRITE_HATCH," the geometry becomes a polyline instead of a polygon.

Attached are the GeoJSON data and conversion code for your reference. Could you 
please assist in resolving this discrepancy?
ogr2ogr -f "DXF" output.dxf input.geojson -t_srs EPSG:32737 --config 
DXF_WRITE_HATCH FALSE


Thank you for your attention to this matter.

--
Saygılarımla / Kind Regards

Shahrukh Mirza NAWANDISH
Havacılık Bilgi Sistemleri / Aeronautical Information Systems
Yazılım Geliştirme Uzm. Yrd. / Software Development Assist. Specialist
Bilgisayar Programcısı / Computer Programmer MCA

T:+90 312 266 66 98
M:+90 538 519 72 67
 E:mirza.nawand...@haritaevi.com
▶: Ankara-Istanbul-Bucharest
www.haritaevi.com
www.obstacleanalyze.com
www.aixm.haritaevi.com
www.aerodatamarket.com
[https://avatars.mds.yandex.net/get-mail-signature/2434044/d24bf65965f2be07edc47ff3c25b0348/orig][https://avatars.mds.yandex.net/get-mail-signature/200369/c75763261eac7fb671922d147573080b/orig][https://avatars.mds.yandex.net/get-mail-signature/2434044/1cd6e615d44979bf9b0d6b0ea9c47182/orig][https://avatars.mds.yandex.net/get-mail-signature/232821/ec6941b191f27730d3a5f92fd846beb4/orig][https://avatars.mds.yandex.net/get-mail-signature/232821/0545b2da2dc9f49f532774615ec45e51/orig][https://www.haritaevi.com/wp-content/uploads/2022/02/disclaimer_1.png]
 
[https://avatars.mds.yandex.net/get-mail-signature/2434044/58c47a3a908e28fdb209656734002f36/orig]
  
[https://avatars.mds.yandex.net/get-mail-signature/2423644/1ccfc2184635533811e43aeeae4bf6c1/orig]
  
[https://avatars.mds.yandex.net/get-mail-signature/2423644/405b95fb41fb8db77dcee35ff542f333/orig]
  
[https://avatars.mds.yandex.net/get-mail-signature/1539903/c89e02562564f32028015473ea475742/orig]
 

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Potential geopackage rtree problem on MacOS ARM64

[Rahkonen Jukka via gdal-dev]

Hi,

It seems that at least two MacOS ARM64 users have faced a problem with the new, 
faster rtree creation method so maybe it is worth having a look. Obviously a 
big dataset and right hardware is needed for testing 
https://gis.stackexchange.com/questions/479958/how-to-fix-failed-to-prepare-sql-error-when-creating-gpkg-file-from-osm-extrac/479964#479964

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] LAUNDER option for geopackages

[Rahkonen Jukka via gdal-dev]

Hi,

The GeoPackage standard recommends some laundering to be done by the clients:

Implementers should be aware of the fact that SQLite table names are not case
sensitive and that table names in sqlite_master and gpkg_extensions may not have
the same case. When searching for table name references, it is recommended to
transform table names to lower case with the lower() function. See the Abstract
Test Suite for an example.

If GeoNode does not have any other reason for its requirement than what you 
have pointed out, then maybe it is too strict.

Having launder in GPKG driver feels still reasonable. SQLite driver has it 
already https://gdal.org/drivers/vector/sqlite.html.

"LAUNDER=[YES/NO]: Defaults to YES. Controls whether layer and field names will 
be laundered for easier use in SQLite. Laundered names will be converted to 
lower case and some special characters(' - #) will be changed to underscores."

It is either not documented or not implemented, but laundering identifiers to 
work without "quotas" would also require removing numbers from the beginning of 
the name. And at least % should be added to the list of special characters. But 
total laundering requires quite a lot 
https://www.sqlite.org/draft/tokenreq.html.

Laundering characters like äöå for GeoNode feels too much. Who would say into 
what characters to launder?

-Jukka Rahkonen-



Lähettäjä: gdal-dev  Puolesta Robert Hewlett 
via gdal-dev
Lähetetty: keskiviikko 3. huhtikuuta 2024 17.51
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] LAUNDER option for geopackages

Hi,

Is there a LAUNDER for geopackages?

I ask this because at the following site:

https://www.geopackage.org/guidance/getting-started.html

Gives this specific guidance:

Note: For maximum interoperability, start your database identifiers (table 
names, column names, etc.) with a lowercase character and only use lowercase 
characters, numbers 0-9, and underscores (_).

Which then turned into a constraint in GeoNode (I could not upload a certain 
geopackage from the township of langley) where I eventually discovered in the 
celery logs this error:

ERROR/ForkPoolWorker-958] {'validation_code': 'RQ1', 'validation_description': 
'Layer names must start with a letter, and valid char
acters are lowercase a-z, numbers or underscores.', 'level': 'error', 
'locations': ['Error layer: Parcel_Attributes']}

I would love LAUNDER to be a more global option.

Upper for: Oracle and Shapefiles.
Lower for: Postgresql, Mysql, geopackages

Just wondering,

Rob


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Gdaladdo and external mask band

[Rahkonen Jukka via gdal-dev]

Hi,

Great, I just found 21000 .msk files without overviews from my computer. My 
scanned map files are so small that I have not noticed that they are 
unnecessary slow to view. Only trying someone else's data with 7 by 8 
pixels draw my attention.

-Jukka-

Lähettäjä: Even Rouault 
Lähetetty: keskiviikko 3. huhtikuuta 2024 17.18
Vastaanottaja: Rahkonen Jukka ; 
'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) 
Aihe: Re: [gdal-dev] Gdaladdo and external mask band


Hi Jukka,

I've realized this behaviour was actually documented in 
https://gdal.org/drivers/raster/gtiff.html#overviews-and-nodata-masks : 
"Internal overviews, external nodata mask: when running BuildOverviews() in 
update mode on the .tif file, only the overviews of the main bands will be 
generated. Overviews of the external .msk file must be explicitly generated by 
running BuildOverviews() on the .msk."

... but just after fixing it in https://github.com/OSGeo/gdal/pull/9603 to do 
the expected thing: that is build overviews of the main bands and of the .msk

I think it is also high time to reconsider switching the default value of 
GDAL_TIFF_INTERNAL_MASK to YES : https://github.com/OSGeo/gdal/pull/9604

Even
Le 03/04/2024 à 10:56, Rahkonen Jukka via gdal-dev a écrit :
Hi,

I was making some tests with filegdb raster from Region 5 - State 
(usda.gov)<https://www.fs.usda.gov/detail/r5/communityforests/?cid=fseprd1009698>,
 file 
https://usfs-public.box.com/shared/static/q96eev9gnsa9hw5bwpmwpmxerbdat0vi.zip
I converted the data into GeoTIFF, added overviews, and noticed, that the 
GeoTIFF is very slow to browse with QGIS at small scales. I noticed that it is 
because gdal_translate creates an external .msk mask band, and gdaladdo does 
not create overviews for the mask.
Is is somehow possible to create overviews for the .msk mask either 
automatically or manually? Or would it create a bad mask that would not match 
with the corresponding overviews of the data bands?

-Jukka Rahkonen-



___

gdal-dev mailing list

gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>

https://lists.osgeo.org/mailman/listinfo/gdal-dev

--

http://www.spatialys.com<http://www.spatialys.com/>

My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Gdaladdo and external mask band

[Rahkonen Jukka via gdal-dev]

Hi,

I was making some tests with filegdb raster from Region 5 - State 
(usda.gov),
 file 
https://usfs-public.box.com/shared/static/q96eev9gnsa9hw5bwpmwpmxerbdat0vi.zip
I converted the data into GeoTIFF, added overviews, and noticed, that the 
GeoTIFF is very slow to browse with QGIS at small scales. I noticed that it is 
because gdal_translate creates an external .msk mask band, and gdaladdo does 
not create overviews for the mask.
Is is somehow possible to create overviews for the .msk mask either 
automatically or manually? Or would it create a bad mask that would not match 
with the corresponding overviews of the data bands?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.5RC1 available, and motion to adopt it

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: tiistai 2. huhtikuuta 2024 13.17
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL 3.8.5RC1 available, and motion to adopt it

Hi,

I have prepared a GDAL/OGR 3.8.5 release candidate. This is likely to be the 
last bug fix release in the 3.8.x series, with 3.9.0 coming beginning of May.
...

Motion: adopt 3.8.5RC1 as final 3.8.5

Starting with my +1

Best regards,

Even
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ERROR 1: Couldn't fetch requested layer when using a filter in the ogr2ogr- GetFeature request

[Rahkonen Jukka via gdal-dev]

Hi,

Works for me with an URL encoded request:

ogrinfo 
"https://isk.geobasis-bb.de/ows/feldblockkataster_wfs?SERVICE=WFS=2.0.0=GetFeature=refbb:Feldblock=EPSG:25833=%3Cfes:Filter%20xmlns:fes=%22http://www.opengis.net/fes/2.0%22%20xmlns:xsi=%22http://www.w3.org/2001/XMLSchema-instance%22%20xsi:schemaLocation=%22http://www.opengis.net/fes/2.0http://schemas.opengis.net/filter/2.0/filterAll.xsd%22%3E%20%3Cfes:And%3E%20%3Cfes:PropertyIsEqualTo%3E%20%3Cfes:ValueReference%3EFLIK%3C/fes:ValueReference%3E%20%3Cfes:Literal%3EDEBBLI1873413738%3C/fes:Literal%3E%20%3C/fes:PropertyIsEqualTo%3E%20%3Cfes:PropertyIsEqualTo%3E%20%3Cfes:ValueReference%3EFACHL_GUELTIG_AB%3C/fes:ValueReference%3E%20%3Cfes:Literal%3E2022-01-01%3C/fes:Literal%3E%20%3C/fes:PropertyIsEqualTo%3E%20%3C/fes:And%3E%20%3C/fes:Filter%3E;
 -al

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Michael Otto 
via gdal-dev
Lähetetty: keskiviikko 27. maaliskuuta 2024 15.29
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] ERROR 1: Couldn't fetch requested layer when using a filter in 
the ogr2ogr- GetFeature request

Hello,

I'm trying to query a free WFS with filter values via ogr2ogr and write the 
results to a GeoJson file.

If I use the following URL incl. filter:

https://isk.geobasis-bb.de/ows/feldblockkataster_wfs?SERVICE=WFS=2.0.0=GetFeature=refbb:Feldblock=EPSG:25833=http://www.opengis.net/fes/2.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://www.opengis.net/fes/2.0http://schemas.opengis.net/filter/2.0/filterAll.xsd;>
   
FLIK 
DEBBLI1873413738  
 
FACHL_GUELTIG_AB 
2022-01-01   


get the error message:

ERROR 1: Couldn't fetch requested layer 
'http://schemas.opengis.net/filter/2.0/filterAll. xsd>  
 FLIK 
DEBBLI1873413738   FACHL_GUELTIG_AB 
2022-01-01   
'!

and no results.

If I submit the GetFeature request directly in the browser, I get the expected 
result (namely exactly 2 features).
Without the filter on the URL, my GetFeature requests also work as expected 
with ogr2ogr.

What am I doing wrong?

Michael
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Estonian WMS failure

[Rahkonen Jukka via gdal-dev]

Hi,

Works for me from the opposite side of the Gulf of Finland. I get a list of 215 
subdatasets with
gdalinfo "WMS:https://kaart.maaamet.ee/wms/alus-geo?; --debug on

No problem with VERSION=1.3.0 either. I tested with GDAL 3.8.4, released 
2024/02/08 (OSGeo4W).

-Jukka Rahkonen-


Lähettäjä: gdal-dev  Puolesta Elena Ruiz via 
gdal-dev
Lähetetty: perjantai 22. maaliskuuta 2024 13.37
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Estonian WMS failure


Hi, I am using GDAL v3.06 to read map web services.
A customer from Estonia tells us that the official and public servers in their 
country are not working,
and as you can see below, they return the following error:

>gdalinfo "WMS:https://kaart.maaamet.ee/wms/alus-geo?;
ERROR 1: Error returned by server : unknown (56)
gdalinfo failed - unable to open 'WMS:https://kaart.maaamet.ee/wms/alus-geo?'.

>gdalinfo "WMS:https://kaart.maaamet.ee/wms/alus-geo?version=1.3.0;
ERROR 1: Error returned by server : unknown (56)
gdalinfo failed - unable to open 
'WMS:https://kaart.maaamet.ee/wms/alus-geo?version=1.3.0'.

However, there are some of those Estonian servers that do work like the 
following:
teenus.maaamet.ee/ows/maardlad?service=WMS=1.3.0=GetCapabilities

But I don't see the difference, both are version 1.3.0. Could you tell me what 
is causing this error?
Thank you and regards.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdal_rasterize processed GTiff filesize question

[Rahkonen Jukka via gdal-dev]

Hi,

I suppose that something similar than with gdalwarp 
https://trac.osgeo.org/gdal/wiki/UserDocs/GdalWarp#GeoTIFFoutput-coCOMPRESSisbroken
 happens but with gdal_rasterize I think there are no tricks that could help.

-Jukka Rahkonen-

Lähettäjä: Andrey VI 
Lähetetty: tiistai 19. maaliskuuta 2024 18.09
Vastaanottaja: Rahkonen Jukka 
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] gdal_rasterize processed GTiff filesize question

Yes, the target files already exists. Yes, I’ve tried to investigate this issue 
on some small and medium files. For example:
original hillshade.tif size is 11469 bytes, after gdal_rasterize its size is 
26465 bytes, and after gdal_translate - 15426 bytes,
original - 7477 K, gdal_rasterize - 8705 K, gdal_translate - 5964 K,
original - 39110 K, gdal_rasterize - 39112 K, gdal_translate - 446 K.
So, it depends on the content of the raster.
I would like to avoid intermediate processing if possible. This will require 
additional time for processing and space for the data. Moreover, in some cases 
a lot of space needs to be allocated for intermediate files.

Andrey
Tuesday, March 19, 2024 4:25 PM +03:00 from Rahkonen Jukka 
>:


Hi,



So the target files dem.tif and hillshade.tif exist already? Maybe it is not so 
simple to update already compressed data in place. Have you tried what happens 
if you re-write the results into new files

gdal_translate -co tiled=yes -co compress=LZW hillshade.tif final_hillshade.tif.



-Jukka Rahkonen-



Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Andrey VI via gdal-dev
Lähetetty: tiistai 19. maaliskuuta 2024 14.43
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] gdal_rasterize processed GTiff filesize question



Hello.



I’m using gdal_rasterize to remove some data from DEM (Int16) and hillshade 
(Byte) rasters with shape-file, like this:

gdal_rasterize -burn -32768 shape.shp dem.tif and gdal_rasterize -burn 0 
shape.shp hillshade.tif

Both DEM and hillshade GTiffs created with -co COMPRESS=LZW -co TILED=YES 
options. After processing with gdal_rasterize the size of GTiffs becomes 1.5 
times larger. Why it happens? I assumed that it should be the other way around, 
because a significant portion of the data is replaced with the same values and 
should be compressed well.



Andrey
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdal_rasterize processed GTiff filesize question

[Rahkonen Jukka via gdal-dev]

Hi,

So the target files dem.tif and hillshade.tif exist already? Maybe it is not so 
simple to update already compressed data in place. Have you tried what happens 
if you re-write the results into new files

gdal_translate -co tiled=yes -co compress=LZW hillshade.tif final_hillshade.tif.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Andrey VI via 
gdal-dev
Lähetetty: tiistai 19. maaliskuuta 2024 14.43
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] gdal_rasterize processed GTiff filesize question

Hello.

I’m using gdal_rasterize to remove some data from DEM (Int16) and hillshade 
(Byte) rasters with shape-file, like this:
gdal_rasterize -burn -32768 shape.shp dem.tif and gdal_rasterize -burn 0 
shape.shp hillshade.tif
Both DEM and hillshade GTiffs created with -co COMPRESS=LZW -co TILED=YES 
options. After processing with gdal_rasterize the size of GTiffs becomes 1.5 
times larger. Why it happens? I assumed that it should be the other way around, 
because a significant portion of the data is replaced with the same values and 
should be compressed well.

Andrey
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] WFS: Why to query the schema of all feature types if asking for just one?

[Rahkonen Jukka via gdal-dev]

Hi,

I noticed that this query:
ogrinfo 
WFS:https://maps.helcom.fi/arcgis/services/MADS/Human_Activities/MapServer/WFSServer
 Human_Activities:Dredging_sites_points_36_2 --debug on

builds first a DescribeCoverage request that has a list all the 156 
featuretypes:
Fetch(https://maps.helcom.fi/arcgis/services/MADS/Human_Activities/MapServer/WFSServer?SERVICE=WFS=2.0.0=DescribeFeatureType=Human_Activities:Dredging_sites_points_36_2,Human_Activities:Dredging_sites_areas_36_2,
 ...

I believe that the query string gets too long and the WFS server sends http 404 
error. After that GDAL makes DescribeFeatureType for just 
"Human_Activities:Dredging_sites_points_36_2" and that query is successful. But 
I wonder why to make the first query at all.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Segmentation fault (core dumped) occurs when running ogr2ogr command on container

[Rahkonen Jukka via gdal-dev]

Hi,

I had a try with your dataset on Windows and I had no segmentation fault. 
Perhaps not surprise because I do not use the same container than you. I had 
some encoding problem but I believe that it is not related. But perhaps you 
have some use for knowing that the unzipping part is not necessary, shapefile 
can be read directly from the zip with /vsizip/.

ogr2ogr -f GeoJSON -oo ENCODING=MS932 d1_raw.geojson /vsizip/test.zip d1  
--config CPL_DEBUG ON
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll 
using GDALRegister_ECW_JP2ECW.
GDAL: GDALOpen(/vsizip/test.zip, this=023F19871250) succeeds as ESRI 
Shapefile.
GDAL: GDALDriver::Create(GeoJSON,d1_raw.geojson,0,0,0,Unknown,)
Shape: DBF Codepage = SJIS for /vsizip/test.zip/d1.shp
Shape: Treating as encoding 'MS932'.
Shape: Cannot recode from 'MS932'. Disabling recoding
GDALVectorTranslate: 5315 features written in layer 'd1'
Shape: 5315 features read on layer 'd1'.
GDAL: GDALClose(/vsizip/test.zip, this=023F19871250)
GDAL: GDALClose(d1_raw.geojson, this=023F19876CF0)
GDAL: In GDALDestroy - unloading GDAL shared library.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta ?? ?? via 
gdal-dev
Lähetetty: tiistai 12. maaliskuuta 2024 10.56
Vastaanottaja: Abel Pau ; gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] Segmentation fault (core dumped) occurs when running 
ogr2ogr command on container

Hi,

I tried executing with --config CPL_DEBUG ON, but it did not provide any useful 
information.

I'm sending you the command that's causing the issue.
We're using curl command to obtain the Shape file from the internet.

---

  1.  Obtaining the Shape file:
$curl -f --output-dir ./ -O 
https://data.bodik.jp/dataset/d4b9a284-58c0-4559-9e48-b7c83a4275f0/resource/28aa8f13-7cf0-4dc5-9dbf-a77f58c3f2c8/download/shape.zip


  1.  Unzipping the ZIP file:
$unzip -o -j -d ./ ./shape.zip

③ Converting from Shape format to GeoJSON format:
$ogr2ogr -f GeoJSON -oo ENCODING=MS932 ./d1_raw.geojson ./d1.shp --config 
CPL_DEBUG ON
---
These commands are executed within the convert.sh called from the Dockerfile.

Best regards,

From: Abel Pau mailto:a@creaf.uab.cat>>
Sent: Wednesday, March 6, 2024 5:53 PM
To: 寛座　智大 
mailto:tomohiro.kanza@baycurrent.co.jp>>;
 gdal-dev@lists.osgeo.org
Subject: RE: Segmentation fault (core dumped) occurs when running ogr2ogr 
command on container

Hi,

I reduced the amount of memory requested. I could do that because I am 
developing the driver that failed.
After my mail I discovered another error, so, perhaps, this was the real cause.

4Mb it’s so small, so I is not the cause...
Perhaps you can add --config  CPL_DEBUG ON at your command line to see if this 
give you more information about the place where it crashes.

I’m sorry not being able to help you more.
Perhaps you can send me the shape and the command line to see if it crashes in 
my computer and I can find out the problem.

See you

De: 寛座　智大 
mailto:tomohiro.kanza@baycurrent.co.jp>>
Enviado el: dimecres, 6 de març de 2024 7:20
Para: Abel Pau mailto:a@creaf.uab.cat>>; 
gdal-dev@lists.osgeo.org
Asunto: RE: Segmentation fault (core dumped) occurs when running ogr2ogr 
command on container

Abel Pau,

Hi. Thank you for your response.

>I reduced this capacity 10 times and it worked.
Does this mean you reduced the size of the Shape file?
Or does it mean that you reduced the amount of memory requested?
( If it's the latter, I'd like to learn how to do it.)

Shape files are typically around 4MB, so I don't consider them to be very large 
files.
Additionally, I tried it on an AKS with a free memory size of over 20GB, but 
encountered the same issue.
It seems unlikely that there is a lack of specifications in the execution 
environment.

Could the issue possibly be due to a bug in the command?

From: Abel Pau mailto:a@creaf.uab.cat>>
Sent: Monday, March 4, 2024 4:57 PM
To: 寛座　智大 
mailto:tomohiro.kanza@baycurrent.co.jp>>
Subject: RE: Segmentation fault (core dumped) occurs when running ogr2ogr 
command on container

Hi,
I did similar tests with a driver I am implementing and I obtained exact the 
same behavior as yours.
After a few speculations I deduced that my driver was asking for too much 
memory for the docker capacity.
I reduced this capacity 10 times and it worked.
So, perhaps, Shape or Geojson asks for too much memory for the docker capacity.
I don’t know if it’s possible to make it bigger.

Abel.

De: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> En 
nombre de ?? ?? via gdal-dev
Enviado el: dilluns, 4 de març de 2024 3:30
Para: gdal-dev@lists.osgeo.org
Asunto: [gdal-dev] Segmentation fault (core dumped) occurs when running ogr2ogr 
command on container


I am implementing batch processing to convert Shape files, publicly available 
as open data,

into Geojson format within a container environment.



I installed 

Re: [gdal-dev] Utilizing GDAL for adding overviews layer in python language

[Rahkonen Jukka via gdal-dev]

Hi,

External overviews that gdaladdo creates when the -ro option is used are TIFF 
files with a file name extension .ovr. If you can create similar TIFFs with the 
other library and name them as .ovr then GDAL should use them automatically.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Pradeep Mahato 
via gdal-dev
Lähetetty: tiistai 12. maaliskuuta 2024 9.00
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Utilizing GDAL for adding overviews layer in python language

Dear Team,
We are working on a project with Geo-spatial images using python language 
for our development purpose. To visualize the images properly pyramid layer is 
necessary. GDAL buildoverview functionality of python, we are using to generate 
the overview but it is utilizing only the CPU. We are able to generate the 
pyramid layer using other python library utilizing the GPU but not able to add 
this pyramid layer into the geospatial images of ".tiff" file format as 
overview layer.

Please help us in this regards if any GDAL function is there to add the pyramid 
as overview layer.

Thanks and Regards,
Pradeep Kumar Mahato
Phone: 080-2504 9199/ +91 8762020114



[https://email.gov.in/videos/images/75.jpg]

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] (no subject) force PAM

[Rahkonen Jukka via gdal-dev]

Hi,

I only managed to force the creation of the .aux.gml file with a workaround.

gdal_translate p4433h.tif test.tif -co profile=baseline

In this case it seems that not even --config GDAL_PAM_ENABLED YES is needed but 
test.tif.aux.xml gets created. Maybe the same PAM file works also for the 
non-baseline versions of that tiff file.

With GeoTIFF, COG, or VRT as output format  --config GDAL_PAM_ENABLED YES does 
not trigger the creation of the PAM file.
I think that storing the same metadata into two distinct places (GeoTIFF tags 
and .aux.xml) is generally speaking a bad idea because sooner or later the two 
versions do not match any more. But I guess that it could be convenient to read 
the plain text aux.xml file from the cloud before touching the main TIFF.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Shaima Alsharif 
via gdal-dev
Lähetetty: perjantai 8. maaliskuuta 2024 11.58
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] (no subject)

Dear Developers,


I am reaching out to seek advice or potential solutions regarding the creation 
of .aux.xml files for GeoTIFF Or GOG using GDAL. I have attempted using the 
following commands, like so:



os.environ['GDAL_PAM_ENABLED'] = 'YES'

os.environ['GDAL_GEOREF_SOURCES'] = 'WORLDFILE,PAM,INTERNAL'

os.environ['ESRI_XML_PAM'] = 'TRUE'



gdalwarp -overwrite -t_srs EPSG:4326 -r cubic JED240207122504.MAXBERJ  
output.tiff



gdaldem color-relief -alpha output.tiff titan.txt output.colored.tiff



gdal_translate -of COG -co TILING_SCHEME=GoogleMapsCompatible -co COMPRESS=LZW  
output.colored.tiff  output.cog.tiff



gdal_translate -of GTiff -co TILED=YES -co COMPRESS=DEFLATE input.tif 
output.tif --config GDAL_PAM_ENABLED YES



gdalinfo --build
PAM_ENABLED=YES
OGR_ENABLED=YES
GEOS_ENABLED=YES
GEOS_VERSION=3.11.1-CAPI-1.17.1
PROJ_BUILD_VERSION=9.1.1
PROJ_RUNTIME_VERSION=9.1.1
COMPILER=GCC 12.2.0



 gdalinfo --version

GDAL 3.8.4, released 2024/02/08

also tried  gdal-3.6.2



OS is Ubuntu Aarch64





However, this does not result in the creation of an .aux.xml file. Could you 
provide any insights into why the .aux.xml file may not be generated in this 
instance?



Any assistance or suggestions you could offer would be greatly appreciated.



Best regards,

Shaima A.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GeoJSON: json as string

[Rahkonen Jukka via gdal-dev]

Thanks,

So maybe the following SQL with the OGR SQL dialect is then the easiest method 
for getting the desired result with GDAL:

ogrinfo jsonjson.json -sql "select * EXCEPT (key), cast(key as character) from 
file" jsonjson.json
...
key (String) = { "start": "2000-01-01", "end": "2023-11-30" }

However, that trick is not exposed to QGIS but maybe there is some other 
workaround for QGIS.

-Jukka-



Lähettäjä: Even Rouault 
Lähetetty: torstai 22. helmikuuta 2024 21.32
Vastaanottaja: Rahkonen Jukka ; 
'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) 
Aihe: Re: [gdal-dev] GeoJSON: json as string

Jukka,

I am looking at a question in gis.stackexchange 
https://gis.stackexchange.com/questions/476489/geojson-item-considered-a-json-instead-of-a-string.
 I thought that open option ARRAY_AS_STRING would have some effect on opening 
the test data, but it does not. Ogrinfo returns String(JSON) with both YES and 
NO
key (String(JSON)) = { "start": "2000-01-01", "end": "2023-11-30" }
Have I understood something wrong?

{ "start": "2000-01-01", "end": "2023-11-30" } is not a JSON array, but a JSON 
object/dict, hence the ARRAY_AS_STRING has no effect on that property.

It would only affect something like [ "2000-01-01", "2023-11-30" ]

Even

--

http://www.spatialys.com

My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GeoJSON: json as string

[Rahkonen Jukka via gdal-dev]

Hi,

I am looking at a question in gis.stackexchange 
https://gis.stackexchange.com/questions/476489/geojson-item-considered-a-json-instead-of-a-string.
 I thought that open option ARRAY_AS_STRING would have some effect on opening 
the test data, but it does not. Ogrinfo returns String(JSON) with both YES and 
NO
key (String(JSON)) = { "start": "2000-01-01", "end": "2023-11-30" }
Have I understood something wrong?
I was testing with GDAL 3.9.0dev (OSGeo4W) on Windows.

-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Question about PG_LIST_ALL_TABLES in the documentation

[Rahkonen Jukka via gdal-dev]

Hi,

I don't undestand this part of the PosgGIS driver documentation 
https://gdal.org/drivers/vector/pg.html#faqs
"Permission issues on geometry_columns and/or spatial_ref_sys tables can be 
generally confirmed if you can see the tables by setting the configuration 
option PG_LIST_ALL_TABLES to YES. (e.g. ogrinfo - -config PG_LIST_ALL_TABLES 
YES PG:x)"

For the first, was the meaning to say the opposite, that there are permission 
issues if you cannot see the tables? And for the second, with current GDAL and 
PostGIS versions user does not see geometry_columns and spatial_ref_sys on the 
ogrinfo layer list even with this config option. It seems that they are 
filtered away from the result. I do see that the config option has an effect 
and for example raster_columns and raster_overviews are added to the layer list.

An easy way to check the permissions is to query those views/tables directly.
ogrinfo PG:x spatial_ref_sys
ogrinfo PG:x geometry_columns

If ogrinfo lists something else than an error, then permissions are OK. Is this 
method good enough so I could edit the documentation?

-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: Adopt GDAL 3.8.4RC1 as 3.8.4 release

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: perjantai 9. helmikuuta 2024 11.13
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Motion: Adopt GDAL 3.8.4RC1 as 3.8.4 release

Hi,

Motion:

Adopt GDAL 3.8.4RC1 as 3.8.4 release

Starting with my +1

Even

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL Maintainers Meeting Minutes

[Rahkonen Jukka via gdal-dev]

Hi,

The spatialreference.org site looks very good! Thanks for the refactoring, and 
for the links to the definitions at epsg.org.
About the translations, by experience I can tell that translating technical 
text is harder than you may think because the translator should also understand 
what the documented thing is doing. Translating word-by-word may lead into 
badly misleading result. I am not at all against translations, for example QGIS 
is certainly more popular now in Finland because the user interface is 
localized. However, I think it took about three years from a group of 
volunteers before the translation was more useful than funny.

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Howard Butler 
via gdal-dev
Lähetetty: maanantai 29. tammikuuta 2024 20.43
Vastaanottaja: gdal dev 
Aihe: [gdal-dev] GDAL Maintainers Meeting Minutes

Howard Butler, Even Rouault, Dan Baston, Sean Gilles, Javier Jiminez Shaw, and 
Alessandro Pasotti held the monthly GDAL Maintainers Meeting on 11/23/2023. The 
following items were discussed and reported upon:

Fundraising update


Annual renewals are now going out to sponsors, so if you are a coordinator of 
GDAL funding in your organization, please be on the lookout for them. 

Maintenance activities update
--

Dan

No activity this month

Alessandro

* Add GetExtent3D "fast" implementation for GeoJSON 
* GDAL Raster Tile Index driver review. Find out more at 
https://gdal.org/drivers/raster/gti.html
Even

* 3.8.3 Bugfix release. See https://github.com/OSGeo/gdal/blob/v3.8.3/NEWS.md 
for release notes
* Added CodeQL analyzer via GitHub Actions. It identified nearly 300 issues, 
though almost all of them were false positives or unconcerning.
* Submitted GDAL to the OpenSSF for a security scorecard. It initially ranked 
5.8/10, and Even completed actions to improve its score such as providing PR 
submissions to CodeQL. It currently ranks 9.0/10.0, with the item most 
contributing to its slightly reduced score being the need for more code and PR 
reviews (we need more PR reviews from GDAL collaborators!) See  
https://securityscorecards.dev/viewer/?uri=github.com/OSGeo/gdalfor details on 
the score. 

Javier

* Javier refactored https://spatialreference.org/. It now loads very fast, it 
is statically generated, and it is open source software that users can deploy 
in their own environments. Thanks again Javier!

Sean

* Sean added GitHub Actions that listen to the GDAL release tag and fire and 
runs rasterio's tests when a GDAL release is issued. 
* Sean added exception chaining in rasterio and this prompted a discussion 
about lack of enthusiasm for changing any of GDAL's error handling behaviors.

Translation Discussion

Alessandro began a discussion about the need to add language translation of 
open options for drivers, how the project could go about managing them. 
Discussion evolved to covering translation of contextual error messages and the 
challenge of maintaining and providing that. No action was to be taken at this 
time, but if this is a topic that's interesting to you, please chime in on the 
mailing list or in a ticket.

The next GDAL Maintainers Meeting is 02/22/2024 at 9:00 EDT. Any PSC members 
are welcome to join by reaching out to me for an invite.

Howard
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdal_translate shifted values

[Rahkonen Jukka via gdal-dev]

Hi,

I used the gdal_compare script https://gdal.org/programs/gdalcompare.html

gdalcompare  st4_pr.2017092016.01h st4_pr.2017092016.01h.tif

and it reports three differences:

  *   Files differ at the binary level (normal, GRIB and tif are not the same)
  *   Difference in SRS (but they are alike, * IsSame() reports them as 
equivalent.)
  *   Difference in Dataset metadata key count ( ['AREA_OR_POINT'] added)

So if I read the report right the pixels in the files are identical. All I can 
conclude is that gdal_compare and your R script are doing something in 
different ways.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Furuya, 
Takahiro via gdal-dev
Lähetetty: torstai 1. helmikuuta 2024 1.53
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] gdal_translate shifted values

Hi gdal-dev, I am Takahiro, a student who is using gdal_translate.
I have an issue with gdal_translate and would like to ask a question.
I used gdal_translate to transform a NCEP GRIB file (downloadable from 
https://data.eol.ucar.edu/cgi-bin/codiac/fgr_form/id=21.093, or attached to 
this email) to GeoTiff, and compare those 2 data formats. The command I used is
>gdal_translate -of GTiff st4_pr.2017092016.01h st4_pr.2017092016.01h.tif

(The GRIB file I used was st4_pr.2017092016.01h, which can be downloaded from 
the above link by specifying "Begin Date/Time" to "20170920"/"15:00:00"  and 
"End Date/Time" to "20170920"/"15:59:59".)

Then I read the 2 data files in R and checked if the original value at a 
certain grid cell in the GRIB file remained at the same grid cell in the 
GeoTiff file, but the value was shifted by 6 rows (the value I checked was 9.75 
[mm/hour] that was stored in index [100,86] in the original GRIB file, but it 
was shifted to [106,86] in the GeoTiff file).

You can see the Python script used for reading the original GRIB file here: 
https://bpa.st/ZNOSI
The R script used for checking the tif-translated data: https://bpa.st/WZLSY

The version of gdal_translate I used is 3.0.2.
Is this a known issue of gdal_translate? Or is this issue not supposed to 
happen (is this the mistake of my side?)?

Best regards,
Takahiro



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Is there any RHEL repo for relatively new GDAL versions?

[Rahkonen Jukka via gdal-dev]

Hi,

The newest GDAL in the official RHEL repo is 3.4.3, released 2022-04-22. There 
used to be GDAL 3.7 available from the PostgreSQL repo 
https://yum.postgresql.org/repopackages/ and we have been using that, but the 
version was removed recently. Some other user was unhappy as well and created a 
bug
Bug #7943: Providers missing from pgdg-common for GDAL 3.7 
(RHEL/CentOS/AlmaLinux 8) - PostgreSQL YUM Repository - Redmine - PostgreSQL 
community

That user got this feedback:
"Why did you install GDAL 3.7? The packages are in the repo, but they were 
never required by any released GIS suite. I even removed them from git a couple 
of weeks ago:
https://git.postgresql.org/gitweb/?p=pgrpms.git;a=commitdiff;h=059f17f664ab093af35389dc7ec7b62c5a9f3fe2
Commit comment: gdal35 and gdal37: Remove from the repo. Not used in any 
packages.

By our agency's IT policy compiling GDAL by ourselves is not possible. Can 
anyone suggest other acceptable means for getting at least GDAL 3.7 for RHEL 8?

-Jukka Rahkonen-



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] vector NODATA for Z values?

[Rahkonen Jukka via gdal-dev]

Hi,

I do understand the use case and we use such nodata-Z in some of our datasets. 
For example, in our densified contour lines where the computed densification 
points get Z-value -. We prefer to use the nodata value because we cannot 
guarantee that the interpolated Z would match with the ground truth.
However, none of the 3D programs that I know can handle such data correctly, 
despite the one that we have written ourselves.

I somehow think that I do not like the nodata values in Z at all. It is a 
misuse of XYZ coordinates which should define the location in 3D space. I think 
that nobody is planning to support nodata in X and Y. How is it possible to 
compute the volume of some 3D solid if Z-values are unknown? Z that may be 
unknown is rather a measure than a Z-coordinate.  Or maybe even an attribute 
instead of a coordinate. On the other hand, I recognize that nodata-Z can be 
useful because anything better and standardized exists yet. OGC to the rescue?

-Jukka Rahkonen-

Lähettäjä: Abel Pau 
Lähetetty: perjantai 26. tammikuuta 2024 11.59
Vastaanottaja: Rahkonen Jukka ; 
gdal-dev@lists.osgeo.org
Aihe: RE: vector NODATA for Z values?

Hi again,
I understand...

Perhaps an option could be using something like -lco Z_NODATA_VALUE_DST="X" to 
translate NODATA  from origin to this value X (NaN, 0, or whatever it may be). 
The driver can offer this number instead of the own NODATA and the user can 
know that the X value in destination will mean NODATA.

On the other hand, a user could also specify -lco Z_NODATA_VALUE_SRC="Y" so 
that a driver can identify a value considered NODATA in the source and 
translate it to the own NODATA value (or the one specified by the first 
parameter (if it exists)).

Is this too complicated? Are there any repercussions that I'm not seeing?
Thanks.




De: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Enviado el: dijous, 25 de gener de 2024 18:27
Para: Abel Pau mailto:a@creaf.uab.cat>>; 
gdal-dev@lists.osgeo.org
Asunto: Re: vector NODATA for Z values?

Hi,

I think that it depends on the file format. Rather often it is set to zero but 
I have seen NaN in many places. However, shapefiles explicitly denies NaN

"...Positive infinity, negative infinity, and Not-a-Number (NaN) values are not 
allowed in shapefiles. Nevertheless, shapefiles support the concept of "no 
data" values, but they are currently used only for measures. Any floating point 
number smaller than -1038 is considered by a shapefile reader to represent a 
"no data" value.

With the SQLite dialect it is possible to select the value but NaN is not 
accepted.
Can be tested with
ogrinfo -dialect SQLite -sql "select CastToXYZ(ST_GeomFromText('POINT (1 
2)'),3)" point.json

-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Abel Pau via gdal-dev
Lähetetty: torstai 25. tammikuuta 2024 19.07
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] vector NODATA for Z values?

Hi,
there is any value in GDAL for VECTORS that indicates that a concrete value of 
a Z is not known (z nodata value)?
I couldn't find it anywhere.

In MiraMon format we use one concrete number documented in our format pdf 
(-1.0E+300) an in the driver it's planned to translate it to the same number. I 
could translate it to the one I am asking. And the same for detecting nodata Z 
and translate them to -1.0E+300 when reading another format.

Thanks!


Abel Pau Garcia
GIS developer
[https://www.creaf.cat/sites/default/files/creaf-signature.png]
a@creaf.uab.cat
Let's chat on 
Teams!
Tel. +34 934814277
[https://www.creaf.cat/sites/default/files/so-en-signature.png]
[https://www.creaf.cat/sites/default/files/twitter-icon-signature.png][https://www.creaf.cat/sites/default/files/linkedin-icon-signature.png][https://www.creaf.cat/sites/default/files/youtube-icon-signature.png][https://www.creaf.cat/sites/default/files/instagram-icon-signature.png]
www.creaf.cat | 
http://blog.creaf.cat
[https://www.creaf.cat/sites/default/files/uab_logo_signatura.png]
CREAF. Campus UAB. Edifici C. 08193 Bellaterra (Barcelona)

Before printing this electronic message, think about the environment.



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] vector NODATA for Z values?

[Rahkonen Jukka via gdal-dev]

Hi,

I think that it depends on the file format. Rather often it is set to zero but 
I have seen NaN in many places. However, shapefiles explicitly denies NaN

"...Positive infinity, negative infinity, and Not-a-Number (NaN) values are not 
allowed in shapefiles. Nevertheless, shapefiles support the concept of "no 
data" values, but they are currently used only for measures. Any floating point 
number smaller than -1038 is considered by a shapefile reader to represent a 
"no data" value.

With the SQLite dialect it is possible to select the value but NaN is not 
accepted.
Can be tested with
ogrinfo -dialect SQLite -sql "select CastToXYZ(ST_GeomFromText('POINT (1 
2)'),3)" point.json

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Abel Pau via 
gdal-dev
Lähetetty: torstai 25. tammikuuta 2024 19.07
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] vector NODATA for Z values?

Hi,
there is any value in GDAL for VECTORS that indicates that a concrete value of 
a Z is not known (z nodata value)?
I couldn't find it anywhere.

In MiraMon format we use one concrete number documented in our format pdf 
(-1.0E+300) an in the driver it's planned to translate it to the same number. I 
could translate it to the one I am asking. And the same for detecting nodata Z 
and translate them to -1.0E+300 when reading another format.

Thanks!


Abel Pau Garcia
GIS developer
[https://www.creaf.cat/sites/default/files/creaf-signature.png]
a@creaf.uab.cat
Let's chat on 
Teams!
Tel. +34 934814277
[https://www.creaf.cat/sites/default/files/so-en-signature.png]
[https://www.creaf.cat/sites/default/files/twitter-icon-signature.png][https://www.creaf.cat/sites/default/files/linkedin-icon-signature.png][https://www.creaf.cat/sites/default/files/youtube-icon-signature.png][https://www.creaf.cat/sites/default/files/instagram-icon-signature.png]
www.creaf.cat | 
http://blog.creaf.cat
[https://www.creaf.cat/sites/default/files/uab_logo_signatura.png]
CREAF. Campus UAB. Edifici C. 08193 Bellaterra (Barcelona)

Before printing this electronic message, think about the environment.



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo in GDAL 3.8.3

[Rahkonen Jukka via gdal-dev]

Hi,

Certainly, consistent behavior is expected but I did not notice that your VRT 
was created with GDAL.
I edited your vrt a bit:

and with gdalinfo I seem to get still the same
GCP[  0]: Id=0, Info=
  (0,0) -> (0,0,0)

And I tried also this:
gdal_translate -gcp 0 0 fin fin test.tif inftest.tif

gdalinfo inftest.tif
GCP[  0]: Id=1, Info=
  (0,0) -> (0,0,0)

Now perhaps GDAL could inform that “FIN” and “fin” are not valid double values, 
but on the other hand, I think that I have also some influence on the wrong 
behavior. But the issue with INF/inf seems to be fixed now by Even.

-Jukka Rahkonen-

Lähettäjä: Pierluigi Guasqui 
Lähetetty: torstai 25. tammikuuta 2024 15.04
Vastaanottaja: Rahkonen Jukka 
Aihe: Re: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo 
in GDAL 3.8.3

Jukka,

it is not documented but I expect GDAL to be consistent with itself. If I set 
GCP (0, 0, inf, inf, -inf) with GDAL I would expect to read the same values 
with GDAL.

Pierluigi

On Thu, Jan 25, 2024 at 1:24 PM Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
Hi,

I guessed so. Is that your own invention or is it documented somewhere that 
undefined GCPs should be marked that way by using INF? I ask because if it is 
something that should be reliable then I may have use for that feature in the 
future.

Problem may be limited to vrt. This test seems to pass
gdal_translate -gcp 0 0 inf inf test.tif inftest.tif

gdalinfo inftest.tif
…
GCP[  0]: Id=1, Info=
  (0,0) -> (inf,inf,0)

-Jukka Rahkonen-

Lähettäjä: Pierluigi Guasqui mailto:guas...@actgate.com>>
Lähetetty: torstai 25. tammikuuta 2024 14.16
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Aihe: Re: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo 
in GDAL 3.8.3

Hello Jukka,

in general using infinite values is to mark undefined or invalid GCPs.

Pierluigi

On Thu, Jan 25, 2024 at 12:44 PM Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
Hi,

Out of curiosity, what is the purpose of using inf or -inf as values? Is it to 
use them as placeholders telling that you do not know yet what the 
georeferenced coordinates of the ground control points are?

-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Pierluigi Guasqui via gdal-dev
Lähetetty: torstai 25. tammikuuta 2024 13.34
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo in 
GDAL 3.8.3

Dear GDAL team,

I am using GDAL version 3.8.3 and I am trying to extract GCPs values with 
gdalinfo tool from a VRT file (attached).

As you can see by looking at the VRT file with a text editor GCP XYZ values are 
infinite values. When I read GCPs from the attached VRT file with gdalinfo 
(latest version GDAL v. 3.8.3) I am getting all zeros:

GCP[  0]: Id=0, Info=
  (0,0) -> (0,0,0)

Instead, if I use gdalinfo (GDAL v. 3.6.3) I am getting the correct values:

GCP[  0]: Id=0, Info=
  (0,0) -> (inf,inf,-inf)

Is this a bug with GDAL 3.8.3?

Thank you for your help!

--
Pierluigi Guasqui
Software Engineer

Applied Coherent Technology Corporation (ACT)
www.actgate.com


--
Pierluigi Guasqui
Software Engineer

Applied Coherent Technology Corporation (ACT)
www.actgate.com


--
Pierluigi Guasqui
Software Engineer

Applied Coherent Technology Corporation (ACT)
www.actgate.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo in GDAL 3.8.3

[Rahkonen Jukka via gdal-dev]

Hi,

I guessed so. Is that your own invention or is it documented somewhere that 
undefined GCPs should be marked that way by using INF? I ask because if it is 
something that should be reliable then I may have use for that feature in the 
future.

Problem may be limited to vrt. This test seems to pass

gdal_translate -gcp 0 0 inf inf test.tif inftest.tif

gdalinfo inftest.tif
…
GCP[  0]: Id=1, Info=
  (0,0) -> (inf,inf,0)

-Jukka Rahkonen-

Lähettäjä: Pierluigi Guasqui 
Lähetetty: torstai 25. tammikuuta 2024 14.16
Vastaanottaja: Rahkonen Jukka 
Aihe: Re: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo 
in GDAL 3.8.3

Hello Jukka,

in general using infinite values is to mark undefined or invalid GCPs.

Pierluigi

On Thu, Jan 25, 2024 at 12:44 PM Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
Hi,

Out of curiosity, what is the purpose of using inf or -inf as values? Is it to 
use them as placeholders telling that you do not know yet what the 
georeferenced coordinates of the ground control points are?

-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Pierluigi Guasqui via gdal-dev
Lähetetty: torstai 25. tammikuuta 2024 13.34
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo in 
GDAL 3.8.3

Dear GDAL team,

I am using GDAL version 3.8.3 and I am trying to extract GCPs values with 
gdalinfo tool from a VRT file (attached).

As you can see by looking at the VRT file with a text editor GCP XYZ values are 
infinite values. When I read GCPs from the attached VRT file with gdalinfo 
(latest version GDAL v. 3.8.3) I am getting all zeros:

GCP[  0]: Id=0, Info=
  (0,0) -> (0,0,0)

Instead, if I use gdalinfo (GDAL v. 3.6.3) I am getting the correct values:

GCP[  0]: Id=0, Info=
  (0,0) -> (inf,inf,-inf)

Is this a bug with GDAL 3.8.3?

Thank you for your help!

--
Pierluigi Guasqui
Software Engineer

Applied Coherent Technology Corporation (ACT)
www.actgate.com


--
Pierluigi Guasqui
Software Engineer

Applied Coherent Technology Corporation (ACT)
www.actgate.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo in GDAL 3.8.3

[Rahkonen Jukka via gdal-dev]

Hi,

Out of curiosity, what is the purpose of using inf or -inf as values? Is it to 
use them as placeholders telling that you do not know yet what the 
georeferenced coordinates of the ground control points are?

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Pierluigi 
Guasqui via gdal-dev
Lähetetty: torstai 25. tammikuuta 2024 13.34
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Bad values when reading GCPs from a VRT file with gdalinfo in 
GDAL 3.8.3

Dear GDAL team,

I am using GDAL version 3.8.3 and I am trying to extract GCPs values with 
gdalinfo tool from a VRT file (attached).

As you can see by looking at the VRT file with a text editor GCP XYZ values are 
infinite values. When I read GCPs from the attached VRT file with gdalinfo 
(latest version GDAL v. 3.8.3) I am getting all zeros:

GCP[  0]: Id=0, Info=
  (0,0) -> (0,0,0)

Instead, if I use gdalinfo (GDAL v. 3.6.3) I am getting the correct values:

GCP[  0]: Id=0, Info=
  (0,0) -> (inf,inf,-inf)

Is this a bug with GDAL 3.8.3?

Thank you for your help!

--
Pierluigi Guasqui
Software Engineer

Applied Coherent Technology Corporation (ACT)
www.actgate.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] units=km not working

[Rahkonen Jukka via gdal-dev]

Hi,

Because the units mean the units of the projected coordinates. An example:
https://epsg.io/map#srs=3067-1149=295545.168414=6836388.322477=4=streets
Here x and y are in meters: "+proj=utm +zone=35 +ellps=GRS80 
+towgs84=0,0,0,0,0,0,0 +units=m +no_defs +type=crs".

By changing the the units into kilometers the x value would change into 
295.545168414 any y into 6836.388322477 but that does not change the size of 
the Earth.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Wilco K via 
gdal-dev
Lähetetty: tiistai 23. tammikuuta 2024 21.57
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] units=km not working

Hi,

why is units=km not working?

gdaltransform -s_srs EPSG:4326 -t_srs "+proj=stere +lat_0=90 +lon_0=0 
+lat_ts=60 +a=6378.14 +b=6356.75 +x_0=0 +y_0=0 +units=km"

The following error is returned:
ERROR 1: PROJ: proj_create_operations: Source and target ellipsoid do not 
belong to the same celestial body

Converting +a and +b to meters and it does works:
gdaltransform -s_srs EPSG:4326 -t_srs "+proj=stere +lat_0=90 +lon_0=0 
+lat_ts=60 +a=6378140 +b=6356750 +x_0=0 +y_0=0"

Any ideas?

Wilco



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Layerstack multiple images, of different extents and variable number of bands

[Rahkonen Jukka via gdal-dev]

Hi,

I recommend to study gdalbuildvrt https://gdal.org/programs/gdalbuildvrt.html 
and consider if it might be good for you with the -separate option.

-Jukka Rahkonen-

Lähettäjä: lef...@gmail.com 
Lähetetty: tiistai 23. tammikuuta 2024 21.51
Vastaanottaja: Rahkonen Jukka 
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] Layerstack multiple images, of different extents and 
variable number of bands

You've interpreted that correctly.

On Tue, Jan 23, 2024 at 12:13 PM Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
Hi,

Please clarify what is the desired result.  A multiband image that covers the 
union of extents of all the input images, with a number of bands that is the 
sum of bands in all the inputs or what?

-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta lefsky--- via gdal-dev
Lähetetty: tiistai 23. tammikuuta 2024 19.01
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Layerstack multiple images, of different extents and variable 
number of bands

I've been incorporating gdal into my processing pipelines and have a question 
about a feature.

I need to be able to layer stack multiple images with different extents and 
varying numbers of bands, usually exceeding 1. I haven't been able to find a 
single command to do this in gdal. Am I missing something? If there is no such 
feature, has anyone developed a script out there that can robustly do all of 
those things (i.e. with checking for violation of assumptions about files and 
error handling)?

Michael

--
Michael Lefsky (He/His)
Home Location: HVHF+GH
Cell: 970-980-9036
http://www.researcherid.com/rid/A-7224-2009

“for being prematurely, and worse, intuitively right — there’s a heavy price. 
But for being wrong — no, not so long as you’re wrong in a pack." Gary Brecher 
/ Portis

*I acknowledge that I live and work on stolen land. This is the land of the 
Cheyenne, Arapaho, Ute, and Ocheithi Sakowin people. To learn more about these 
nations, please visit;
http://www.utemountainutetribe.com/
http://www.cheyennenation.com/
https://cheyenneandarapaho-nsn.gov/
https://native-land.ca/



--
Michael Lefsky (He/His)
Home Location: HVHF+GH
Cell: 970-980-9036
http://www.researcherid.com/rid/A-7224-2009

“for being prematurely, and worse, intuitively right — there’s a heavy price. 
But for being wrong — no, not so long as you’re wrong in a pack." Gary Brecher 
/ Portis

*I acknowledge that I live and work on stolen land. This is the land of the 
Cheyenne, Arapaho, Ute, and Ocheithi Sakowin people. To learn more about these 
nations, please visit;
http://www.utemountainutetribe.com/
http://www.cheyennenation.com/
https://cheyenneandarapaho-nsn.gov/
https://native-land.ca/

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Layerstack multiple images, of different extents and variable number of bands

[Rahkonen Jukka via gdal-dev]

Hi,

Please clarify what is the desired result.  A multiband image that covers the 
union of extents of all the input images, with a number of bands that is the 
sum of bands in all the inputs or what?

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta lefsky--- via 
gdal-dev
Lähetetty: tiistai 23. tammikuuta 2024 19.01
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Layerstack multiple images, of different extents and variable 
number of bands

I've been incorporating gdal into my processing pipelines and have a question 
about a feature.

I need to be able to layer stack multiple images with different extents and 
varying numbers of bands, usually exceeding 1. I haven't been able to find a 
single command to do this in gdal. Am I missing something? If there is no such 
feature, has anyone developed a script out there that can robustly do all of 
those things (i.e. with checking for violation of assumptions about files and 
error handling)?

Michael

--
Michael Lefsky (He/His)
Home Location: HVHF+GH
Cell: 970-980-9036
http://www.researcherid.com/rid/A-7224-2009

"for being prematurely, and worse, intuitively right - there's a heavy price. 
But for being wrong - no, not so long as you're wrong in a pack." Gary Brecher 
/ Portis

*I acknowledge that I live and work on stolen land. This is the land of the 
Cheyenne, Arapaho, Ute, and Ocheithi Sakowin people. To learn more about these 
nations, please visit;
http://www.utemountainutetribe.com/
http://www.cheyennenation.com/
https://cheyenneandarapaho-nsn.gov/
https://native-land.ca/

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Non able to manage properly vitual file, warping and black edges

[Rahkonen Jukka via gdal-dev]

Hi,

Please try to imagine that you are a stranger that might volunteer to help by 
running similar commands with same kind of imagery. Imagine a list of 
information and commands that you would need to know before you can do that. 
Then please share that information with us.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta andy via 
gdal-dev
Lähetetty: tiistai 23. tammikuuta 2024 15.45
Vastaanottaja: Even Rouault 
Kopio: gdal dev 
Aihe: Re: [gdal-dev] Non able to manage properly vitual file, warping and black 
edges

Even,
the verbose process lasts 24 minutes, the COG process lasts 36 minutes.
There is a 50% difference.

Best regards


--
___

Andrea Borruso
website: https://medium.com/tantotanto
38° 7' 48" N, 13° 21' 9" E, EPSG:4326
___

"cercare e saper riconoscere chi e cosa,
 in mezzo all'inferno, non è inferno,
e farlo durare, e dargli spazio"

Italo Calvino
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Non able to manage properly vitual file, warping and black edges

[Rahkonen Jukka via gdal-dev]

Hi,

Could you add a small image to show how the result looks like? I fear my 
imagination may be inaccurate.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta andy via 
gdal-dev
Lähetetty: maanantai 22. tammikuuta 2024 18.54
Vastaanottaja: gdal dev 
Aihe: [gdal-dev] Non able to manage properly vitual file, warping and black 
edges

Hi,
I have four tif input files, these have 3 bands RGB and are JPEG compressed.
No null data set.

My goal is to mosaic these and then warp the mosaic.

I start it with:

gdalbuildvrt mosaic.vrt file1.tif  file2.tif  file3.tif  file4.tif

Then I warp the mosaic:

gdalwarp -dstalpha -of vrt -overwrite -s_srs EPSG:32632 -t_srs EPSG:7791 
mosaic.vrt warped.vrt

Then I run

gdal_translate -b 1 -b 2 -b 3 -b 4 -co BIGTIFF=YES -co COMPRESS=DEFLATE -co 
ALPHA=YES -co TILED=YES -co NUM_THREADS=ALL_CPUS  warped.vrt  output.tif

The final tiff file has black edges.
What's wrong with my procedure? How to avoid this?

Thank you

--
___

Andrea Borruso
website: https://medium.com/tantotanto
38° 7' 48" N, 13° 21' 9" E, EPSG:4326
___

"cercare e saper riconoscere chi e cosa,
 in mezzo all'inferno, non è inferno,
e farlo durare, e dargli spazio"

Italo Calvino
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdal2tiles custom tms configuration not found

[Rahkonen Jukka via gdal-dev]

Hi,

Support has been added in https://github.com/OSGeo/gdal/pull/2757/files and PR 
seems to include some tests.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Andreas Neumann 
via gdal-dev
Lähetetty: perjantai 19. tammikuuta 2024 13.03
Vastaanottaja: Andreas Neumann 
Kopio: gdal dev 
Aihe: Re: [gdal-dev] gdal2tiles custom tms configuration not found


Hi,

Just found 
https://github.com/OSGeo/gdal/blob/master/swig/python/gdal-utils/osgeo_utils/gdal2tiles.py#L272
 where it seems to look for all files following the naming scheme "tms_*.json". 
Therefore I don't know why my custom tms configuration files is not found - any 
ideas?

My custom tms json file also has an identifier set to '2056_28'.

Here is the error message I get:
gdal2tiles.py: error: option -p: invalid choice: '2056_28' (choose from 
'mercator', 'geodetic', 'raster', 'LINZAntarticaMapTilegrid', 'APSTILE')

Andreas



On 2024-01-19 11:36, Andreas Neumann via gdal-dev wrote:

Hi,

I would like to use gdal2tiles with a custom tms profile file. I prepared the 
json file attached in this mail and put this json file in the /usr/share/gdal 
folder, there there are already other such files, e.g. from New Zealand or 
Antartica.

However, when I try to use this custon tms config, gdal2tiles tells me it can't 
find this profile. Here is my command I used:

gdal2tiles.py -p 2056_28 -z 16 -w openlayers -t "Swiss Hillshading Grayscale" 
-r cubic -s EPSG:2056 -a 0.0 -e -n 
lv_16_switzerland_hillshading_small-scale_swissalti3d-srtm_100m_jpeg.tif tiles

Do I need to register the custom TMS config json file somewhere in addition to 
copying it to the /usr/share/gdal directory?

Thank you for your help!

Andreas

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GPKG raster creation : error with too small resolution

[Rahkonen Jukka via gdal-dev]

Hi,

I am not sure how standard the level 25 is for the InspireCRS84Quad, the 
INSPIRE technical guidance stops at 17. But I believe that you know your own 
use case and have considered also that EPSG:3857 at 1 mm resolution is good for 
you.

The number of tiles at level 25 is 2^50= 1.1259E+15. TileCol and TileRow would 
get rather large numbers but I suppose that it is not an issue.

-Jukka Rahkonen-

Lähettäjä: Philippe Ghesquiere 
Lähetetty: torstai 18. tammikuuta 2024 16.06
Vastaanottaja: Rahkonen Jukka 
Kopio: gdal-dev 
Aihe: Re: [gdal-dev] GPKG raster creation : error with too small resolution

Hi Jukka,

I have no choice but to use the standard InspireCRS84Quad tiling grid.

Thanks for the "-a_ullr fix"

Philippe


On Thu, Jan 18, 2024 at 2:42 PM Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
Hi,

Can you imagine using a custom tiling schema instead of the world-wide 
InspireCRS84Quad? These commands work

gdal_create -of GTiff -co COMPRESS=DEFLATE -outsize 2000 2000 -burn 255 -burn 0 
-burn 0 -bands 3 -ot Byte -a_srs epsg:32631 -a_ullr 100 102 102 
100 1mm.tif

gdal_translate -of GPKG  1mm.tif 1mm.gpkg

Notice the fixed ullr for making the image north-up.

-Jukka Rahkonen-



gdal_translate -of GPKG  1mm.tif 1mm.gpkg
Input file size is 2000, 2000
0ERROR 6: Only north-up non rotated geotransform supported
..ERROR 1: Raster table 1mm not correctly initialized due to missing call to 
SetGeoTransform()

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Philippe Ghesquiere via gdal-dev
Lähetetty: torstai 18. tammikuuta 2024 15.22
Vastaanottaja: gdal-dev 
mailto:gdal-dev@lists.osgeo.org>>
Aihe: [gdal-dev] GPKG raster creation : error with too small resolution


Dear all,

I just had an error while converting an image to GPKG. This image has a very 
small resolution and gdal_translate pops up an error message.

Test image :
gdal_create -of GTiff -co COMPRESS=DEFLATE -outsize 2000 2000 -burn 255 -burn 0 
-burn 0 -bands 3 -ot Byte -a_srs epsg:32631 -a_ullr 100 100 102 
102 1mm.tif

Conversion to GPKG :
gdal_translate -of GPKG -co TILING_SCHEME=InspireCRS84Quad -co 
ZOOM_LEVEL_STRATEGY=UPPER 1mm.tif 1mm.gpkg
Input file size is 2000, 2000
ERROR 1: Could not find an appropriate zoom level

Analysis
The error is due to  
ogrgeopackagedatasource.cpp
 (line 3169), and it does make sense : the zoom level is too high (Z>25).

However, wouldn't it be possible to modify gdal_translate behavior ?
For example, we could :

  *   let GDAL compute the appropriate zoom level : -co 
ZOOM_LEVEL_STRATEGY=AUTO/LOWER/UPPER,
  *   optionally specify a maximum zoom level : -co MAX_ZOOM_LEVEL=18
What do you think ?

Philippe

The information in this e-mail is confidential. The contents may not be 
disclosed or used by anyone other than the addressee. Access to this e-mail by 
anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus immediately and 
delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or completeness of 
this e-mail as it has been sent over public networks. If you have any concerns 
over the content of this message or its Accuracy or Integrity, please contact 
Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated virus 
scanning software but you should take whatever measures you deem to be 
appropriate to ensure that this message and any attachments are virus free.
The information in this e-mail is confidential. The contents may not be 
disclosed or used by anyone other than the addressee. Access to this e-mail by 
anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus immediately and 
delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or completeness of 
this e-mail as it has been sent over public networks. If you have any concerns 
over the content of this message or its Accuracy or Integrity, please contact 
Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated virus 
scanning software but you should take whatever measures you deem to be 
appropriate to ensure that this message and any attachments are virus free.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GPKG raster creation : error with too small resolution

[Rahkonen Jukka via gdal-dev]

Hi,

Can you imagine using a custom tiling schema instead of the world-wide 
InspireCRS84Quad? These commands work

gdal_create -of GTiff -co COMPRESS=DEFLATE -outsize 2000 2000 -burn 255 -burn 0 
-burn 0 -bands 3 -ot Byte -a_srs epsg:32631 -a_ullr 100 102 102 
100 1mm.tif

gdal_translate -of GPKG  1mm.tif 1mm.gpkg

Notice the fixed ullr for making the image north-up.

-Jukka Rahkonen-



gdal_translate -of GPKG  1mm.tif 1mm.gpkg
Input file size is 2000, 2000
0ERROR 6: Only north-up non rotated geotransform supported
..ERROR 1: Raster table 1mm not correctly initialized due to missing call to 
SetGeoTransform()

Lähettäjä: gdal-dev  Puolesta Philippe 
Ghesquiere via gdal-dev
Lähetetty: torstai 18. tammikuuta 2024 15.22
Vastaanottaja: gdal-dev 
Aihe: [gdal-dev] GPKG raster creation : error with too small resolution


Dear all,

I just had an error while converting an image to GPKG. This image has a very 
small resolution and gdal_translate pops up an error message.

Test image :
gdal_create -of GTiff -co COMPRESS=DEFLATE -outsize 2000 2000 -burn 255 -burn 0 
-burn 0 -bands 3 -ot Byte -a_srs epsg:32631 -a_ullr 100 100 102 
102 1mm.tif

Conversion to GPKG :
gdal_translate -of GPKG -co TILING_SCHEME=InspireCRS84Quad -co 
ZOOM_LEVEL_STRATEGY=UPPER 1mm.tif 1mm.gpkg
Input file size is 2000, 2000
ERROR 1: Could not find an appropriate zoom level

Analysis
The error is due to  
ogrgeopackagedatasource.cpp
 (line 3169), and it does make sense : the zoom level is too high (Z>25).

However, wouldn't it be possible to modify gdal_translate behavior ?
For example, we could :

  *   let GDAL compute the appropriate zoom level : -co 
ZOOM_LEVEL_STRATEGY=AUTO/LOWER/UPPER,
  *   optionally specify a maximum zoom level : -co MAX_ZOOM_LEVEL=18
What do you think ?

Philippe

The information in this e-mail is confidential. The contents may not be 
disclosed or used by anyone other than the addressee. Access to this e-mail by 
anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus immediately and 
delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or completeness of 
this e-mail as it has been sent over public networks. If you have any concerns 
over the content of this message or its Accuracy or Integrity, please contact 
Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated virus 
scanning software but you should take whatever measures you deem to be 
appropriate to ensure that this message and any attachments are virus free.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Nearblack and the datatype of the output

[Rahkonen Jukka via gdal-dev]

Hi,

Do I understand right that the raster drivers like the GTiff driver do not have 
a creation option for the datatype? Most utilities like gdal_translate, 
gdalwarp, and gdal_create have the -ot option, but there are couple of 
utilities that don't even it might make sense. For example nearblack can only 
create float32 output from float32 input and user cannot do anything for that.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.3 RC3 is available, and motion to approve it (Re: GDAL 3.8.3 RC1 is available, and motion to approve it)

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Javier Jimenez 
Shaw via gdal-dev
Lähetetty: torstai 4. tammikuuta 2024 21.11
Vastaanottaja: Even Rouault 
Kopio: gdal dev 
Aihe: Re: [gdal-dev] GDAL 3.8.3 RC3 is available, and motion to approve it (Re: 
GDAL 3.8.3 RC1 is available, and motion to approve it)

+1 Javier

On Thu, 4 Jan 2024, 19:32 Even Rouault via gdal-dev, 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Hi,

A third (and hopefully final) release candidate for 3.8.3 with the
following additional fixes on top of RC2:
- re-enable multi-threaded ArrowArray interface in GPKG driver now that
the issue has been reproduced and fixed
(https://github.com/OSGeo/gdal/pull/9026)
- Internal libjson: resync random_seed.c with upstream, and use
getrandom() implementation when available (#9024)
- Fix build of optional utility gdal2ogr

Pick up an archive among the following ones (by ascending size):

   https://download.osgeo.org/gdal/3.8.3/gdal-3.8.3rc3.tar.xz
   https://download.osgeo.org/gdal/3.8.3/gdal-3.8.3rc3.tar.gz
   https://download.osgeo.org/gdal/3.8.3/gdal383rc3.zip

A snapshot of the gdalautotest suite is also available:

https://download.osgeo.org/gdal/3.8.3/gdalautotest-3.8.3rc3.tar.gz
   https://download.osgeo.org/gdal/3.8.3/gdalautotest-3.8.3rc3.zip

The NEWS file is here:

https://github.com/OSGeo/gdal/blob/v3.8.3RC3/NEWS.md



Retracting previous motion to approve RC2, and replacing it with:

Motion: adopt 3.8.3 RC3 as final 3.8.3 release

Starting with my +1,

~

Even

--

http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.3 RC2 is available, and motion to approve it (Re: GDAL 3.8.3 RC1 is available, and motion to approve it)

[Rahkonen Jukka via gdal-dev]

+1 

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: keskiviikko 3. tammikuuta 2024 22.46
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL 3.8.3 RC2 is available, and motion to approve it (Re: 
GDAL 3.8.3 RC1 is available, and motion to approve it)

Hi,

I've issued a 2nd release candidate for 3.8.3 with the following additional 
fixes:

...
Retracting previous motion to approve RC1, and replacing it with:

Motion: adopt 3.8.3 RC2 as final 3.8.3 release

Starting with my +1,

~

Even

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Any difference between VRT SimpleSource and no-op ComplexSource?

[Rahkonen Jukka via gdal-dev]

Hi,

About -separate and multi-band inputs, see 
https://gdal.org/programs/gdalbuildvrt.html#cmdoption-gdalbuildvrt-separate

“Starting with GDAL 3.8, all bands of each input file are added as separate VRT 
bands, unless -b is specified to select a subset of them. Before GDAL 3.8, only 
the first band of each input file was placed into a new VRT band, and -b was 
ignored.”

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Lauren?iu 
Nicola via gdal-dev
Lähetetty: torstai 21. joulukuuta 2023 16.09
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] Any difference between VRT SimpleSource and no-op 
ComplexSource?

On Thu, Dec 21, 2023, at 15:51, Even Rouault wrote:

It only uses a ComplexSource when it needs to, that is to say when there is 
nodata or a mask band.
I didn't realize this. I think I'm hitting a corner case with -separate, which 
is practically the only way I've ever used it:

$ gdalbuildvrt test.vrt T35TMK_20230520T090559_B02_10m.jp2 && cat test.vrt
0...10...20...30...40...50...60...70...80...90...100 - done.

  PROJCS["WGS 84 / UTM zone 
35N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 
84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",27],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",50],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","32635"]]
3.9996e+05,  1.e+01,  
0.e+00,  5.4000e+06,  0.e+00, 
-1.e+01
  
Gray

  T35TMK_20230520T090559_B02_10m.jp2
  1
  
  
  

  
  2 4 8 16


$ gdalbuildvrt -separate test.vrt T35TMK_20230520T090559_B02_10m.jp2 && cat 
test.vrt
0...10...20...30...40...50...60...70...80...90...100 - done.

  PROJCS["WGS 84 / UTM zone 
35N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 
84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",27],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",50],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","32635"]]
3.9996e+05,  1.e+01,  
0.e+00,  5.4000e+06,  0.e+00, 
-1.e+01
  

  T35TMK_20230520T090559_B02_10m.jp2
  1
  
  
  

  


Otherwise it will use SimpleSource. There is a performance hit in using 
ComplexSource when it is not needed. In 3.8, there has been an optimization to 
improve ComplexSource performance in common nodata-only scenarios: cf 
https://github.com/OSGeo/gdal/commit/cd67491d3909f5f05160fa209609b38d973c42de
Oh, that's great!

PS: are there good reasons for not allowing -separate for multi-band inputs (by 
including all bands) or propagating the input block size if it's the same for 
all of them?

Laurentiu
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] MVT tileset with limited output extent

[Rahkonen Jukka via gdal-dev]

Hi,

Have you tried -sql? Like -sql “select * from table where 
st_intersects(geom,reference_geom)”.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Jan Wolski via 
gdal-dev
Lähetetty: keskiviikko 20. joulukuuta 2023 17.07
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] MVT tileset with limited output extent

Thanks for the reply.

I can see how spatially filtering the input (-spat) is useful, but I don't want 
to clip the geometries (other than clipping by tile edges that MVT driver does 
by default). What I'm after is spatial filtering of the output tiles based on 
each tile extent.

So to further clarify, I could do the translation using -spat  and then 
iterate over all of the generated MVT files and delete the ones whose tile 
extent does not intersect with . But that would waste resources in a big 
way.

Thanks,
Jan Wolski

On Tue, Dec 5, 2023 at 4:05 PM Even Rouault 
mailto:even.roua...@spatialys.com>> wrote:

You can use the -spat + (-clipsrc or -clipdst) switches of ogr2ogr
Le 05/12/2023 à 14:57, Jan Wolski via gdal-dev a écrit :
Hi,

I'm creating MVT tilesets for the ETRS-TM35FIN tiling scheme using GDAL/OGR. 
I'm prototyping with ogr2ogr but later I will be using Python bindings to 
create the production workflow.

Is there a way to limit the geographic extent of where the MVT driver creates 
tiles? By default it seems to create all tiles that intersect any input 
feature. I would like to create only tiles that intersect a given extent. This 
is to enable partial update of a MVT tileset.

This is the ogr2ogr command I have now:

ogr2ogr -f MVT stand MV_R5232B.gpkg stand -dsco "MAXZOOM=11" -dsco "MINZOOM=8" 
-dsco "TILE_EXTENSION=mvt" -dsco 
"TILING_SCHEME=EPSG:3067,-548576,8388608,2097152" -dsco "TYPE=overlay" -dsco 
"COMPRESS=NO"


Thanks,
Jan Wolski


___

gdal-dev mailing list

gdal-dev@lists.osgeo.org

https://lists.osgeo.org/mailman/listinfo/gdal-dev

--

http://www.spatialys.com

My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.2 RC1 is available, and motion to approve it

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: lauantai 16. joulukuuta 2023 16.01
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL 3.8.2 RC1 is available, and motion to approve it

Hi,

I have prepared an advanced-of-time GDAL/OGR 3.8.2 release candidate, mostly to 
fix a 3.8.0 regression that has been reported lately 
(https://github.com/OSGeo/gdal/issues/8967), which caused an excessive use of 
RAM when translating a VRT with many VRTComplexSource (typically used when 
nodata or mask are involved), making such use cases unworkable.

Pick up an archive among the following ones (by ascending size):

   https://download.osgeo.org/gdal/3.8.2/gdal-3.8.2rc1.tar.xz
   https://download.osgeo.org/gdal/3.8.2/gdal-3.8.2rc1.tar.gz
   https://download.osgeo.org/gdal/3.8.2/gdal382rc1.zip

A snapshot of the gdalautotest suite is also available:

https://download.osgeo.org/gdal/3.8.2/gdalautotest-3.8.2rc1.tar.gz
   https://download.osgeo.org/gdal/3.8.2/gdalautotest-3.8.2rc1.zip

The NEWS file is here:

   https://github.com/OSGeo/gdal/blob/v3.8.2RC1/NEWS.md

Motion: adopt 3.8.2 RC1 as final 3.8.2 release

Starting with my +1,

Best regards,

Even

--
http://www.spatialys.com/
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [BULK] Re: [EXTERNAL] Re: GTiff bit shuffle compression feature request

[Rahkonen Jukka via gdal-dev]

Hi,

Could Zarr be used as Sozipped https://gdal.org/programs/sozip.html?

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Meyer, Jesse R. 
(GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev
Lähetetty: perjantai 8. joulukuuta 2023 21.44
Vastaanottaja: Even Rouault ; gdallists 

Aihe: Re: [gdal-dev] [BULK] Re: [EXTERNAL] Re: GTiff bit shuffle compression 
feature request

Unfortunately Zarr has a design choice that won't work for us: blocks are 
individual files on a file system.  Our datasets are massive and this will 
explode our inode allocations.  While we could archive the folder into a zip 
archive, it adds a step for anyone to work with the data.  Curiously, this 
sparse friendly representation seems totally baked into the format and there's 
no way to opt out.  I'm not quite ready to share compression ratio findings, 
but initial results are consistent with my expectations.  Bitshuffle is 
effective for our data and works very well with another Zarr feature 
"DELTA_DTYPE" which _can_ be a form of lossless compression if the max delta is 
known.

I understand not wanting to make third party tiff compliance worse than it 
already is from the GDAL project perspective.  That would be minimized if the 
functionality were added to libtiff proper, so any project that depends on 
libtiff could benefit from the enhancement.

From: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> on 
behalf of "Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] 
via gdal-dev" mailto:gdal-dev@lists.osgeo.org>>
Reply-To: "Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC]" 
mailto:jesse.r.me...@nasa.gov>>
Date: Friday, December 8, 2023 at 12:40 PM
To: Even Rouault 
mailto:even.roua...@spatialys.com>>, gdallists 
mailto:gdal-dev@lists.osgeo.org>>
Subject: [BULK] Re: [gdal-dev] [EXTERNAL] Re: GTiff bit shuffle compression 
feature request

Thanks for the suggestion Even, we'll see how effective Zarr is for our 
datasets.

Jesse

From: Even Rouault 
mailto:even.roua...@spatialys.com>>
Date: Friday, December 8, 2023 at 12:20 PM
To: "Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC]" 
mailto:jesse.r.me...@nasa.gov>>, gdallists 
mailto:gdal-dev@lists.osgeo.org>>
Subject: [EXTERNAL] Re: [gdal-dev] GTiff bit shuffle compression feature request

CAUTION: This email originated from outside of NASA.  Please take care when 
clicking links or opening attachments.  Use the "Report Message" button to 
report suspicious messages to the NASA SOC.


Jesse,

This would break interoperability with other TIFF readers... Even adding a new 
TIFF tag to advertize that bit shuffling is applied would probably not be a 
sufficient guard, as existing readers wouldn't read it, and would just display 
garbage, which is worth that not being able to open the file at all. The only 
way I can think off of doing that in a safe way would be to use new values for 
the Compression tag, which isn't pretty either.

You should probably try Zarr which has such capability with the Blosc codec. Cf 
https://gdal.org/drivers/raster/zarr.html : BLOSC_SHUFFLE

I'm curious however to know which typical compression gain you get with that.

Even


Le 08/12/2023 à 18:06, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND 
APPLICATIONS INC] via gdal-dev a écrit :
Hi,

When using horizonal differencing to reduce the numerical range of band data, 
the upper bytes in the produced stream are typically 0 which leverages LZ's 
byte based compression model.  But the least significant bytes can still have 
many significant bits as 0. Unless the whole byte is replicated, LZ compressors 
can't do much to leverage the pattern however.  For data with temporal and or 
spatial coherence, 'shuffling' is another effective strategy to losslessly 
reform the data stream to be favorable to LZ style compressors.  And plays 
nicely off gains already provided by the PREDICTOR functionality.

The notion is to arrange the bit stream where the Nth "shuffled" byte contains 
the Nth bit from each byte in the sequence.  The sequence length is usually 
determined by the data type bit length.

For example (for brevity, assume bytes are 4 bits long)

Byte 1,  Byte 2, Byte 3, Byte 4
0001, 0011, 0111, 0001

They all share the top 0 bit and the bottom 1 bit,

"Shuffled"
, 0010, 0110, 

The algorithm is pretty simple to implement, and can be SIMD accelerated for 
high performance.

While we specifically are users of the GTIFF format, such a strategy could be 
employed generically for most raster and even vector formats.

Best,
Jesse


___

gdal-dev mailing list

gdal-dev@lists.osgeo.org

https://lists.osgeo.org/mailman/listinfo/gdal-dev

--

http://www.spatialys.com

My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org

Re: [gdal-dev] Motion: adopt RFC 98: Build requirements for GDAL 3.9

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Kurt Schwehr 
via gdal-dev
Lähetetty: perjantai 1. joulukuuta 2023 18.28
Vastaanottaja: Even Rouault 
Kopio: gdal dev 
Aihe: Re: [gdal-dev] Motion: adopt RFC 98: Build requirements for GDAL 3.9

+1 KurtS

On Fri, Dec 1, 2023, 7:34 AM Even Rouault via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Hi,

Motion: adopt RFC 98: Build requirements for GDAL 3.9

https://github.com/OSGeo/gdal/pull/8802

Starting with my +1,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to determine which creation options are supported by a driver?

[Rahkonen Jukka via gdal-dev]

Hi,

Run ”gdalinfo –format GTiff”.  For me it prints:

NONE
LZW
PACKBITS
JPEG
CCITTRLE
CCITTFAX3
CCITTFAX4
DEFLATE
ZSTD
  

So my GDAL build does not support LERC.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Meyer, Jesse R. 
(GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev
Lähetetty: torstai 30. marraskuuta 2023 22.19
Vastaanottaja: Even Rouault via gdal-dev 
Aihe: [gdal-dev] How to determine which creation options are supported by a 
driver?

Hi,

I have gdal 3.8.1 installed via Brew which includes liblerc and zstd as part of 
their formulae, so I assumed I could use the creation options 
“COMPRESS=LERC_ZSTD” and “MAX_Z_ERROR=x” as documented on the GTIFF page when 
creating a new gtif image.

But when I do GDAL states:

“Warning 6: 'LERC_ZSTD' is an unexpected value for COMPRESS creation option of 
type string-select.
Warning 6: driver GTiff does not support creation option MAX_Z_ERROR”

So I am not sure if this is an issue with the driver itself or with the brew 
formula.  Please advise,

Thanks,
Jesse
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.1RC3 is available & motion to approve it

[Rahkonen Jukka via gdal-dev]

+1

-Jukka-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: tiistai 28. marraskuuta 2023 17.33
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL 3.8.1RC3 is available & motion to approve it

Hi,

well, another annoying 3.8.0 regression related to ogr2ogr GPKG -> Shapefile 
when field names are longer than 10 characters (ah
Shapefile!!!) popped up today, so here's a RC3 (crossing fingers it's the good 
one):
...
A snapshot of the gdalautotest suite is also available:
...
The changes since RC2 are:

* CMake: make GDAL_USE_LIBKML and GDAL_USE_OPENJPEG honor GDAL_USE_EXTERNAL_LIBS
* Detect failure in installation of the Python bindings
* Fix installation issue with Python 3.12 on Debian
* GDALOverviewDataset::IRasterIO(): use parent dataset when possible for more
   efficiency
* gdal_footprint: fix wrong taking into account of alpha band (#8834)
* gdal_footprint: fix taking into account of individual bands that have nodata
* COG: for JPEG compression, convert single band+alpha as single band JPEG + 
1-bit mask band
* GTIFF SRS reader: include VertCRS name from EPSG in CompoundCRS name if 
there's no citation geokey
* ogr2ogr: fix GPKG -> Shapefile when field names are truncated (#8849,
3.8.0 regression)
* CSV writer: do not quote integer fields by default (only if 
STRING_QUOTING=ALWAYS is specified)
* GPX: make detection of extensions element more robust (#8827)
* Shapefile: recognize '  0' as a null date, fix writing an invalid 
"/00/00" date
* Python bindings: add a combineBands option to gdal.Footprint()


Withdrawing previous motion and:

==> Motion: adopt 3.8.1 RC3 as final 3.8.1 release

Starting with my +1,

Even

--
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.1 RC2 is available (was Re: GDAL 3.8.1 RC1 is available, and motion to approve it)

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: perjantai 24. marraskuuta 2023 13.06
Vastaanottaja: Sebastiaan Couwenberg ; 
gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL 3.8.1 RC2 is available (was Re: GDAL 3.8.1 RC1 is 
available, and motion to approve it)

Hi,

OK, I've reverted that change. I'll guess someone will have to figure out how 
to have both .py scripts and .exe/.sh launchers at the same time...

Pick up an archive among the following ones (by ascending size):
...

Withdrawing previous motion and:

Motion: adopt 3.8.1 RC2 as final 3.8.1 release

Starting with my +1,

Even

Le 24/11/2023 à 06:34, Sebastiaan Couwenberg via gdal-dev a écrit :
> On 11/23/23 22:56, Even Rouault via gdal-dev wrote:
>>  From a packaging point of view, the following change might have
>> impacts on build recipes:
>
> The Python utilities are no longer installed with the .py extension,
> this broke their use in QGIS when we removed this in the Debian
> package some time ago. Don't know if that's still the case. Either
> way, this doesn't seem like an appropriate change for a .1 patch release.
>
> usr/bin/__init__ is now also installed which not correct.
>
> Kind Regards,
>
> Bas
>
--
http://www.spatialys.com/
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: adopt RFC 97: OGRFeatureDefn, OGRFieldDefn and OGRGeomFieldDefn "sealing"

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: keskiviikko 22. marraskuuta 2023 14.06
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Motion: adopt RFC 97: OGRFeatureDefn, OGRFieldDefn and 
OGRGeomFieldDefn "sealing"

Hi,

Motion: adopt RFC 97: OGRFeatureDefn, OGRFieldDefn and OGRGeomFieldDefn 
"sealing"

...

Starting with my +1,

Even

--
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: adopt RFC 96: Deferred C++ plugin loading

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: keskiviikko 15. marraskuuta 2023 11.52
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Motion: adopt RFC 96: Deferred C++ plugin loading

Hi,

Motion: adopt RFC 96: Deferred C++ plugin loading
...
Starting with my +1,

Even

-- 

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: adopt GDAL 3.8.0RC2 as 3.8.0 release

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: torstai 9. marraskuuta 2023 12.12
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Motion: adopt GDAL 3.8.0RC2 as 3.8.0 release

Retracting the motion to adopt 3.8.0RC1 and updating it to:

Motion:

Adopt GDAL 3.8.0RC2 as 3.8.0 release

Starting with my +1

Even

Le 08/11/2023 à 09:39, Even Rouault via gdal-dev a écrit :
> Hi,
>
> Motion:
>
> Adopt GDAL 3.8.0RC1 as 3.8.0 release
>
> Starting with my +1
>
> Even
>
-- 

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: keskiviikko 8. marraskuuta 2023 10.39
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release

Hi,

Motion:

Adopt GDAL 3.8.0RC1 as 3.8.0 release

Starting with my +1

Even

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] tps - gdalwarp vs ogr2ogr

[Rahkonen Jukka via gdal-dev]

Hi,

The slowness feels bad. I fear I must also have a try with your dataset.

-Jukka-

Lähettäjä: Stijn Tallir 
Lähetetty: tiistai 7. marraskuuta 2023 9.30
Vastaanottaja: Rahkonen Jukka 
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] tps - gdalwarp vs ogr2ogr

Hi Jukka,

The transformation lasted 3h20min vs 9min (before the fix)!

The results for raster and vector tps transformation are very similar now and 
nothing changed for the vector result. Do I have to conclude the previous 
rasters tps transformation method was faulty then?

Stijn

Op ma 6 nov 2023 om 16:32 schreef Stijn Tallir 
mailto:stijn%2bgdal-...@strict.be>>:
Hi,

Does that mean the old raster tps transformation was "wrong" or the old vector 
transformation?

I'm doing a test and trying to transform my raster image with the latest dev 
version in osgeo4W but it takes forever to process now. Don't know in how many 
days I will see the result :(

The vector transformation with the latest dev version was the same (time and 
result).

Stijn

Op ma 6 nov 2023 om 14:48 schreef Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>:
Hi,

See the issue https://github.com/OSGeo/gdal/issues/8572. Maybe your problem is 
also resolved by https://github.com/OSGeo/gdal/pull/8573. The fix is included 
in the GDAL 3.8 RC1 version that was released 3 hour ago. Do you have an option 
to make a test?

-Jukka Rahkonen-

Lähettäjä: Stijn Tallir mailto:st...@strict.be>>
Lähetetty: maanantai 6. marraskuuta 2023 14.47
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] tps - gdalwarp vs ogr2ogr

Hi Jukka,

I finally found the time to produce a test set.

You can download it here: 
https://drive.google.com/file/d/1Y08Q-tIm5dxyKFKNdVqilvAO3H7FFFbx/view?usp=sharing

I started from an unreferenced raster file (raster2tps.tif) with gcp's 
(gcp4tps.gcp) and transformed it with tps (tpsraster.tif).
Then polygonized the unreferenced raster file (vector2tps.shp) and transformed 
the result with  the same gcp's (gcp4tps.gcp) and with tps (tpsvector.shp).

The vector2tps.shp polygons are "flipped" because of the different Y-origin for 
rasters and vectors but this way both datasets can use the exact same gcp's.

When you lay the tpsvector-result on top of the tpsraster-result (in QGis for 
instance) you'll see the differences in how both are transformed.

Kind regards,

Stijn



Op wo 16 aug 2023 om 13:16 schreef Stijn Tallir 
mailto:stijn%2bgdal-...@strict.be>>:
Yes, I checked them visually for both raster and vector.

I compared the results also visually. The rasters are transformed in a way that 
the end ponts of the gcp's align exactly with the result so that is why I 
referred to it as "right". The vector data result is in the neighbourhood of 
the end points (sometimes a rather significant distance).

The result is different from order 1-3 transformations so I presume the tps 
option isn't ignored.

Stijn

Op wo 16 aug 2023 om 11:52 schreef Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>:
Hi,

Did you check the ground control points? What is your reference when you say 
that one result is right, and another wrong? Have you used some other software 
for comparison? Or do you only know that the results are different?

-Jukka-

Lähettäjä: Stijn Tallir mailto:st...@strict.be>>
Lähetetty: keskiviikko 16. elokuuta 2023 12.27
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] tps - gdalwarp vs ogr2ogr

Hi Jukka,

I thought of the density as an option for the "error" as you suggested and I 
made a point-file with a point for every pixel in my original image and used 
this as a source for the ogr2ogr transformation. So you could say the desnity 
for both sources raster and vector) are then alike.

The results were still the same (and wrong) ...

Stijn


Op wo 16 aug 2023 om 10:22 schreef Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>:
Hi,

Without test data it is very hard to say much. I believe that the promise of 
tps is that the ground control points stay where they are set. The intermediate 
points follow the least tension surfaces and I do not know how exactly those 
spline algorithms are defined. Raster data is full of points to warp but 
probably in the vector data the transformation is done vertex by vertex. I 
would first check if the GCPs are in the same place in both outputs. Then I 
would make a test by densifying the vector data before georeferencing to have 
much more vertices and see if it has any effect on the result.

-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Stijn Tallir
Lähetetty: keskiviikko 16. elokuuta 2023 10.29
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] tps - gdalwarp vs ogr2ogr

Hi,

According to the 

Re: [gdal-dev] tps - gdalwarp vs ogr2ogr

[Rahkonen Jukka via gdal-dev]

Hi,

See the issue https://github.com/OSGeo/gdal/issues/8572. Maybe your problem is 
also resolved by https://github.com/OSGeo/gdal/pull/8573. The fix is included 
in the GDAL 3.8 RC1 version that was released 3 hour ago. Do you have an option 
to make a test?

-Jukka Rahkonen-

Lähettäjä: Stijn Tallir 
Lähetetty: maanantai 6. marraskuuta 2023 14.47
Vastaanottaja: Rahkonen Jukka 
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] tps - gdalwarp vs ogr2ogr

Hi Jukka,

I finally found the time to produce a test set.

You can download it here: 
https://drive.google.com/file/d/1Y08Q-tIm5dxyKFKNdVqilvAO3H7FFFbx/view?usp=sharing

I started from an unreferenced raster file (raster2tps.tif) with gcp's 
(gcp4tps.gcp) and transformed it with tps (tpsraster.tif).
Then polygonized the unreferenced raster file (vector2tps.shp) and transformed 
the result with  the same gcp's (gcp4tps.gcp) and with tps (tpsvector.shp).

The vector2tps.shp polygons are "flipped" because of the different Y-origin for 
rasters and vectors but this way both datasets can use the exact same gcp's.

When you lay the tpsvector-result on top of the tpsraster-result (in QGis for 
instance) you'll see the differences in how both are transformed.

Kind regards,

Stijn



Op wo 16 aug 2023 om 13:16 schreef Stijn Tallir 
mailto:stijn%2bgdal-...@strict.be>>:
Yes, I checked them visually for both raster and vector.

I compared the results also visually. The rasters are transformed in a way that 
the end ponts of the gcp's align exactly with the result so that is why I 
referred to it as "right". The vector data result is in the neighbourhood of 
the end points (sometimes a rather significant distance).

The result is different from order 1-3 transformations so I presume the tps 
option isn't ignored.

Stijn

Op wo 16 aug 2023 om 11:52 schreef Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>:
Hi,

Did you check the ground control points? What is your reference when you say 
that one result is right, and another wrong? Have you used some other software 
for comparison? Or do you only know that the results are different?

-Jukka-

Lähettäjä: Stijn Tallir mailto:st...@strict.be>>
Lähetetty: keskiviikko 16. elokuuta 2023 12.27
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] tps - gdalwarp vs ogr2ogr

Hi Jukka,

I thought of the density as an option for the "error" as you suggested and I 
made a point-file with a point for every pixel in my original image and used 
this as a source for the ogr2ogr transformation. So you could say the desnity 
for both sources raster and vector) are then alike.

The results were still the same (and wrong) ...

Stijn


Op wo 16 aug 2023 om 10:22 schreef Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>:
Hi,

Without test data it is very hard to say much. I believe that the promise of 
tps is that the ground control points stay where they are set. The intermediate 
points follow the least tension surfaces and I do not know how exactly those 
spline algorithms are defined. Raster data is full of points to warp but 
probably in the vector data the transformation is done vertex by vertex. I 
would first check if the GCPs are in the same place in both outputs. Then I 
would make a test by densifying the vector data before georeferencing to have 
much more vertices and see if it has any effect on the result.

-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Stijn Tallir
Lähetetty: keskiviikko 16. elokuuta 2023 10.29
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] tps - gdalwarp vs ogr2ogr

Hi,

According to the documentation gdal and ogr use the same algorithm for the 
tps-transformation but I don't seem to get the same results using the same set 
of gcp's for images and vectors.

I have images that are unreferenced and vector data digitised on these images 
(in pixel coordinates).

The images are then georeferenced with +100 gcp's and warped with gdalwarp 
using the "tps" option.

When I use the same gcp's (with adjusted y-origin to lower left corner) to 
georeference the vector data with ogr2ogr and the "tps" option I get different 
results. The vector-result is similar to the image-result but never exactly the 
same and differences can be substantial.

Any thoughts?

Stijn


--
Stijn Tallir - StrICT BV

Wijnveld 8
9112 Sinaai-Waas

GSM: 0486 750220

E-mail: i...@strict.be
Web: www.strict.be

BTW: BE 0567.559.668

[http://www.strict.be/strict_web2.jpg]


--
Stijn Tallir - StrICT BV

Wijnveld 8
9112 Sinaai-Waas

GSM: 0486 750220

E-mail: i...@strict.be
Web: www.strict.be

BTW: BE 0567.559.668

[http://www.strict.be/strict_web2.jpg]
___
gdal-dev mailing list

Re: [gdal-dev] oblique cuts on a raster using python GDAL

[Rahkonen Jukka via gdal-dev]

Hi,

Please add gdalinfo of the source image. Even better if you can share the image.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Naima Dambrine 
via gdal-dev
Lähetetty: sunnuntai 5. marraskuuta 2023 17.35
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] oblique cuts on a raster using python GDAL

Hi ,

I have problems with oblique cuts on a raster using python GDAL (3.6.2)

- with this line i obtain black borders around :
gdal.warp('raster-dst' , raster-src', cutLineDSName='geometry-extraction.shp', 
cropToCutline=True)

- with this one, the crop is not clean on closer inspection: residual black 
pixels around image and white pixels appear in the image.
gdal. warp( 'raster-dst' , raster-src', 
cutLineDSName='geometry-extraction.shp', cropToCutline=True, copyMetaData=True, 
dstNodata=0)

I tried, without success, to refine with outputBounds=[minX, maxX, minY, maxY], 
under QGIS directly ….
I've run out of ideas :/
A (naive) question comes to mind: Is it possible to make oblique cuts with 
gdal.warp() & co?
Naïma
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Show isBigTIFF in image metadata

[Rahkonen Jukka via gdal-dev]

Hi,

The exact use case is for the GDAL community help desk. I was wondering how 
this user 
https://gis.stackexchange.com/questions/469512/cannot-write-geotiff-larger-than-4gb-with-gdalwarp-despite-using-bigtiff-createo
 could check if their TIFF is created as BigTIFF or not.

-Jukka Rahkonen-

Lähettäjä: Kurt Schwehr 
Lähetetty: perjantai 3. marraskuuta 2023 3.08
Vastaanottaja: Even Rouault 
Kopio: Rahkonen Jukka ; 
'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) 
Aihe: Re: [gdal-dev] Show isBigTIFF in image metadata

Jukka,

What's the exact use case for needing to know if a tiff is traditional or 
BigTIFF? Is there a key tool that doesn't understand BigTIFF? The only one I 
know of is Autodesk Civil3D.

Thanks,
-Kurt

On Thu, Nov 2, 2023 at 2:49 PM Even Rouault via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Jukka,

Does it feel reasonable? I know that overviews may be standard TIFFs while the 
main image is BigTIFF but maybe the information from the header would be 
enough. Or have I missed some existing tool? I thought that tiffinfo at least 
would report TIFF/BigTIFF but it doesn’t.

tiffdump does:

$ tiffdump byte.tif
byte.tif:
Magic: 0x4949  Version: 0x2a 

vs

$ tiffdump bigtiff.tif
bigtiff.tif:
Magic: 0x4949  Version: 0x2b 
Even

--

http://www.spatialys.com

My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Show isBigTIFF in image metadata

[Rahkonen Jukka via gdal-dev]

Hi,

I tried to find some easy way for checking with GDAL if an image is a standard 
TIFF or BigTIFF but I could not find any. It seems that "more big.tiff" on 
Windows finds "II+" from the beginning of the file if the image is BigTIFF and 
"II*" otherwise. Or then I can open the image with a hex editor for seeing if 
the third byte is 2A or 2B. However, I would like to see the information with 
gdalinfo for example as

Image Structure Metadata:
  INTERLEAVE=BAND
  VERSION=BigTIFF

Does it feel reasonable? I know that overviews may be standard TIFFs while the 
main image is BigTIFF but maybe the information from the header would be 
enough. Or have I missed some existing tool? I thought that tiffinfo at least 
would report TIFF/BigTIFF but it doesn't.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: Adopt GDAL 3.7.3RC1 as 3.7.3 release

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: keskiviikko 1. marraskuuta 2023 13.14
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Motion: Adopt GDAL 3.7.3RC1 as 3.7.3 release

Hi,

Motion:

Adopt GDAL 3.7.3RC1 as 3.7.3 release

Starting with my +1

Even

--
http://www.spatialys.com/
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.0beta1 available for testing

[Rahkonen Jukka via gdal-dev]

Hi,

I made a simple test with ogr2ogr and geopackage to geopackage on Windows. With 
GDAL 3.8.0dev-3e4dc710a2 (no arrow, old R-Tree) the timing was 36 minutes, with 
GDAL 3.8.0dev-6bbd2c080a the same conversion took 21 minutes. The gpkg file is 
75 GB, with 120 layers.

Could it be that the new R-Tree implementation does not print anything into 
debug? With older GDAL I see
GPKG: Creating insert/delete feature_count triggers
GPKG: 194505 rows inserted into rtree_virtausnuoli_geom
but with the brand new just
GPKG: Creating insert/delete feature_count triggers
GPKG: Creating insert/delete feature_count triggers

-Jukka Rahkonen-




-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Even Rouault 
via gdal-dev
Lähetetty: tiistai 31. lokakuuta 2023 16.33
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL 3.8.0beta1 available for testing

Hi,

I've prepared a beta1 of GDAL 3.8.0 to get feedback from earlier testers.

Sorry no updated NEWS.md file yet, but I'd in particular be interesting by 
testing of ogr2ogr workflows, since they have underwent significant changes in 
the underlying implementation:

- when the source layer is a layer implementing the ArrowStream API (that is  
GeoPackage, FlatGeoBuf, Arrow or Parquet), and when no ogr2ogr options than 
-of, -where, -spat, -lco, -dsco, -gt, -append/-overwrite/-update are used (and 
-sql as well for GeoPackage).
When enabling CPL_DEBUG, you'll see a "OGR2OGR: Using WriteArrowBatch()"
trace when that new code path is taken. If specifying other options, the 
feature-iteration-based traditional implementation is used

- and/or when the output layer is GeoPackage (new layer), due to the revamped 
much faster spatial index creation.  This enhanced spatial index creation is 
not ogr2ogr specific and is actually available more generally for CreateLayer() 
+ CreateFeature() or CreateLayer() +
WriteArrowBatch() scenarios.

Point of attention would be when in situations with large files and/or with low 
RAM.

The ghcr.io/osgeo/gdal:ubuntu-small-latest,
ghcr.io/osgeo/gdal:ubuntu-full-latest,
ghcr.io/osgeo/gdal:alpine-normal-latest Docker images have been refreshed with 
3.8.0beta1 (ghcr.io/osgeo/gdal:alpine-small-latest still building at time of 
writing).

(Note: the GDAL master conda builds mentioned at
https://gdal.org/download.html#gdal-master-conda-builds have been broken for a 
couple weeks and are thus not usable to test beta1 currently. I'm
investigating)

Source snapshots at:

- https://download.osgeo.org/gdal/3.8.0/gdal-3.8.0beta1.tar.gz

- https://download.osgeo.org/gdal/3.8.0/gdal-3.8.0beta1.tar.xz

- https://download.osgeo.org/gdal/3.8.0/gdal380beta1.zip

Autotest snapshots:

- https://download.osgeo.org/gdal/3.8.0/gdalautotest-3.8.0beta1.tar.gz

- https://download.osgeo.org/gdal/3.8.0/gdalautotest-3.8.0beta1.zip

Even

--
http://www.spatialys.com/
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gif gfw and prj

[Rahkonen Jukka via gdal-dev]

Hi,

I made a quick test with some random .gif file from my computer and your world 
file and those two do find each other without problems. The format of 
a_datum.prj is not familiar to me. Maybe it is Global Mapper specific. Maybe 
you can just delete it and use the EPSG:32632 code instead in gdal_translate as 
-a_srs EPSG:32632.

gdalinfo gif.gif
Driver: BIGGIF/Graphics Interchange Format (.gif)
Files: gif.gif
   gif.gfw
Size is 12000, 12000
Origin = (243987.500,5120012.500)
Pixel Size = (25.000,-25.000)
Image Structure Metadata:
  INTERLACED=NO
Corner Coordinates:
Upper Left  (  243987.500, 5120012.500)
Lower Left  (  243987.500, 4820012.500)
Upper Right (  543987.500, 5120012.500)
Lower Right (  543987.500, 4820012.500)
Center  (  393987.500, 4970012.500)

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta 
mauge.philippe--- via gdal-dev
Lähetetty: tiistai 31. lokakuuta 2023 11.19
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] gif gfw and prj

Hi,
I have a gif file named img_512_244000.gif 

with a world file named img_512_244000.gfw that contains  :

25.00
0.00
0.00
-25.00
244000.00
512.00

and a prj file named a_datum.prj that contains :

Projection UTM
Datum  WGS84
Zunits NO
Units  METERS
Zone   32
Xshift 0.00
Yshift 0.00
Parameters

The 3 files are in the same folder.

When I open the gif file with GDAL using GDALOpen, GDAL doesn't read the gfw 
and the prj files and there is no projection.
Is there a way to say to GDAL to read the gfw and the prj files ?

I have successfully loaded the file with Global Mapper that read the gfw and 
the prj files.

The gif file can be uploaded at :
[link removed]

If you have any clue, please let me know.

Have a good day

Philippe
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Driver for ISOXML format

[Rahkonen Jukka via gdal-dev]

Hi,

Because it is XML I would first try what the GMLAS driver 
https://gdal.org/drivers/vector/gmlas.htmlcan can get out of it.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Matthias Loeks 
via gdal-dev
Lähetetty: keskiviikko 25. lokakuuta 2023 12.41
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Driver for ISOXML format


Hello all,

apologies if this has been asked before - I couldn't find anything, but might 
have missed past messages.

Does anyone know if someone ever considered implementing a GDAL driver for the 
ISOXML format used in Digital Agriculture / Precision Farming?

It would be very useful to support it in GDAL, so both GIS software and Farm 
Management Information Systems (FMIS) could much more easily integrate it, 
offering read/write support.

To my knowledge, there is no standardized support for this as of today. 
However, even though ISOXML has been around for a while, it is an established 
industry standard, and widely used throghout the growing Smart Farming 
community.

I am a software engineer and would be willing to work on a driver, even though 
I'm not familiar with GDAL core code at all so far. Anyone who would be 
interested in joining? My employer might offer options for funding.

Many thanks for your input!

Best regards,
Matthias

--

Further reading

. https://www.iso.org/standard/61581.html
. https://www.isobus.net/isobus/
. https://www.dev4agriculture.de/en/isoxml-library/
. 
https://gis.stackexchange.com/questions/354092/converting-geojson-to-isoxml-isobus-iso-11783-10
. https://github.com/aparshin/isoxml-visualization
. https://adaptframework.org/ (working on standardizing agricultural data flows)


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] WMS layer issue at higher zoom level

[Rahkonen Jukka via gdal-dev]

Hi,

Do you have another WMS server as a source for this layer? The error message 
that you have this time suggests so: “GDALWMS: Unable to download block 497, 
740.” If that is the case, do you admin also the remote WMS?

-Jukka Rahkonen-



Lähettäjä: gdal-dev  Puolesta Peter Schmitt 
via gdal-dev
Lähetetty: perjantai 20. lokakuuta 2023 20.33
Vastaanottaja: Raghav Singh 
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] WMS layer issue at higher zoom level

If you know the querystring that generated the failure you could run even more 
verbose logging on the command line. Something like:

env CPL_DEBUG=ON CPL_TIMESTAMP=ON MS_DEBUGLEVEL=5 MS_ERRORFILE=/dev/stderr 
mapserv 
QUERY_STRING='SERVICE=WMS=1.3.0=GetMap=image/png=256=256&'

and maybe you could copy the image locally and try something like

gdal_translate -tr 256 256 -projwin ulx uly lrx lry in.tif out.tif

to glean more information.  It's possible the input image (or an overview 
within the image) is corrupted in some way.  If it turns out an overview is 
corrupted, you could try regenerating it with gdaladdo?


On Fri, Oct 20, 2023 at 10:53 AM Raghav Singh via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
I ran mapserver locally ran the specific wms request locally so getting this 
error - Assertion failed: (MS_RENDERER_RAWDATA(image->format) || 
(MS_RENDERER_PLUGIN(image->format) && rb)), function msDrawRasterLayerGDAL, 
file mapdrawgdal.c, line 126.
Abort trap: 6
Any idea regarding this?

and in logs it's coming like this -

Fri Oct 20 21:56:56 2023].137990 msDrawMap(): rendering using outputformat 
named mypng (AGG/PNG).

[Fri Oct 20 21:56:56 2023].138020 msDrawMap(): WMS/WFS set-up and query, 0.000s

[Fri Oct 20 21:56:56 2023].198332 
msDrawRasterLayerLow(PICT-INHEND19-psQwiZ1L4t): entering.

[Fri Oct 20 21:56:58 2023].803667 
msDrawRasterLayerLow(PICT-INHEND19-psQwiZ1L4t): entering.

[Fri Oct 20 22:10:13 2023].219765 drawGDAL() error coming from here: Unable to 
access file. GDALDatasetRasterIO() failed: IReadBlock failed at X offset 497, Y 
offset 740: GDALWMS: Unable to download block 497, 740.






From: Raghav Singh
Sent: 04 October 2023 11:09
To: gdal-dev@lists.osgeo.org 
mailto:gdal-dev@lists.osgeo.org>>
Subject: WMS layer issue at higher zoom level

Hi,

I am trying to get a layer from wms and at higher zoom levels I am getting this 
error - msDrawMap(): Image handling error. Failed to draw layer named 
'PICT-INHEND19-psQwiZ1L4t'. drawGDAL(): Unable to access file. 
GDALDatasetRasterIO() failed: /tmp/mapfilePassthru-v2-temp-HxmyxI, band 1: 
IReadBlock failed at X offset 1064, Y offset 1551.

So, for this I checked from the imageservice side the mapfile is being created 
correctly and it is also present there and the mapfile contains a gserve 
request for getting the tiles - that is also working fine. So getting this 
error from mapserver only and tried to put debug level 5 and other things also 
in the mapfile but couldn't get any other error, only this is coming. So need 
help in this if anybody knows what exactly is this and i can i debug mapserver 
locally if i want to then i can figure out the exact point where the error is 
coming from.

Thanks,
Raghav Singh
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
Pete
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDALPolygonize fails with "failed to execute insert : disk I/O error"

[Rahkonen Jukka via gdal-dev]

Hi,

SQLite is known to have problems with network disks (NFS) but that is not your 
case. But maybe WSL can have same kind of problems, at least there has been 
some https://github.com/microsoft/WSL/issues/2395.

I would try to run the program on true Linux or true Windows and see if the 
issue happens to go away.

-Jukka Rahkonen-

Lähettäjä: Shayan Ghofrany 
Lähetetty: tiistai 17. lokakuuta 2023 19.57
Vastaanottaja: Rahkonen Jukka 
Aihe: Re: [gdal-dev] GDALPolygonize fails with "failed to execute insert : disk 
I/O error"

Thanks for the response,

I've had this issue also with Shapefiles, but haven't tested the other formats 
you mentioned. I am not sure what you mean by what kind of file system I'm 
using, my program is being run on Windows Subsystem for Linux. I'm not familiar 
with journal mode or pragma synchronous, so it's most likely the default option.

Shayan G.

On Tue, Oct 17, 2023 at 12:45 PM Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
Hi,

I believe that the i/o error is something that comes from SQLite. Does error 
happen if you save into other formats like shapefile flatgeobuf, or parquet? In 
what kind of file system the new geopackage is? What journal mode do you use? 
Is “pragma synchronous” on or off?

-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Shayan Ghofrany via gdal-dev
Lähetetty: tiistai 17. lokakuuta 2023 18.52
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDALPolygonize fails with "failed to execute insert : disk I/O 
error"

I'm attempting to polygonize a GeoTIFF using the built-in GDALPolygonize, but 
sometimes it fails to write to file. I'm not attempting any sort of 
multi-threaded functionality so everything is running synchronously. Here is 
the code I'm using to polygonize: 
https://gist.github.com/Irantwomiles/c5f38b3e81fc3014be4e859048ad523f
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDALPolygonize fails with "failed to execute insert : disk I/O error"

[Rahkonen Jukka via gdal-dev]

Hi,

I believe that the i/o error is something that comes from SQLite. Does error 
happen if you save into other formats like shapefile flatgeobuf, or parquet? In 
what kind of file system the new geopackage is? What journal mode do you use? 
Is "pragma synchronous" on or off?

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Shayan Ghofrany 
via gdal-dev
Lähetetty: tiistai 17. lokakuuta 2023 18.52
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDALPolygonize fails with "failed to execute insert : disk I/O 
error"

I'm attempting to polygonize a GeoTIFF using the built-in GDALPolygonize, but 
sometimes it fails to write to file. I'm not attempting any sort of 
multi-threaded functionality so everything is running synchronously. Here is 
the code I'm using to polygonize: 
https://gist.github.com/Irantwomiles/c5f38b3e81fc3014be4e859048ad523f
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] including higher-level library in GPX creator string

[Rahkonen Jukka via gdal-dev]

Hi,

It seems to be hardcoded in 
https://github.com/OSGeo/gdal/blob/2cfeb1a9b03716c8348d6f1f4c611f2589d53e76/ogr/ogrsf_frmts/gpx/ogrgpxdatasource.cpp#L744

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Greg Troxel via 
gdal-dev
Lähetetty: perjantai 13. lokakuuta 2023 15.52
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] including higher-level library in GPX creator string

I am a member of the GPX standards list, and we are discussing a new version 
and also dealing with broken data.

I wrote a gpx (from geojson), with just ogr2ogr, and the creator looks like 
this:

  creator="GDAL 3.5.3"

that's arguably totally fine with ogr2ogr.

But it seems that if some random big program links with gdal and uses it to 
produce a file, the creator is just GDAL, and not "qgis 3.28.5 via GDAL 3.5.3" 
or something like that.  I groveled over all my gpx files and found none that 
indicated anything like this.

It would be helpful to the GPX community if files were attributed to the larger 
software as well, so that they can be chased down or at least understood to be 
separately broken.

I don't see anything at

that could be used to do this.

Does this make sense to people here?

Is there any facility that would do this, that people just aren't using?

Any other thoughts?

Greg
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: Annual Contracts for Maintainers

[Rahkonen Jukka via gdal-dev]

+1

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Howard Butler 
via gdal-dev
Lähetetty: keskiviikko 11. lokakuuta 2023 19.14
Vastaanottaja: gdal dev 
Aihe: [gdal-dev] Motion: Annual Contracts for Maintainers

PSC,

I'm a little late but I would like to make the following motions in regards to 
GDAL maintainers:

* Authorize Alessandro Pasotti for up to 30 hrs/month with a rate increase of 
10 euros/hr until 30 NOV 2024.
* Authorize Even Rouault for up to 90 hrs/month with a rate increase of 10 $/hr 
until 30 JUL 2024

Howard
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] help needed with pygdaltools library, documentation sparse

[Rahkonen Jukka via gdal-dev]

Hi,

The actual error seems to be that you try to write some geometry that is not a 
linestring into a GeoPackage table that has created with a constraint that 
makes it accept only linestrings. You do that by using "-nlt linestring".
You have two options. The fast one is to use "-nlt geometry" that makes 
GeoPackage table to accept all kind of geometries. The other option is to 
filter the source data by geometry type and select only linestrings. Here is an 
example about how to do that with the OGR SQL dialect 
https://gdal.org/user/ogr_sql_dialect.html#ogr-geometry.

Don't try to use "-nlt geometry" with shapefiles because that format does not 
support mixed geometrytypes.

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Catania, Luke A 
ERDC-RDE-GRL-VA CIV via gdal-dev
Lähetetty: keskiviikko 11. lokakuuta 2023 8.59
Vastaanottaja: Scott ; gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] help needed with pygdaltools library, documentation sparse

I have not even gotten to see that anything is being ignored because I have not 
yet added any options.  My first attempt was with adding skipfailures, but I 
can't add any of the options I have listed because I have no idea how to.  I 
thought it would be gdal_tools.config_options() but I get pylint tells me it is 
not callable

-Original Message-
From: gdal-dev  On Behalf Of Scott via 
gdal-dev
Sent: Wednesday, October 11, 2023 1:45 AM
To: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] help needed with pygdaltools library, documentation 
sparse

-nln is being ignored because the data layer you're -updating -appending to 
exists. You can't rename a layer if it has a name. Remove -nln option

Next you are trying to write a collection as a linestring. Use 
-explodecollections (if exists in your install) and/or use -nlt PROMOTE_TO_MULTI

Try not to use -skipfailures

On 10/10/23 22:26, Catania, Luke A ERDC-RDE-GRL-VA CIV via gdal-dev wrote:
> I am running simple code to convert dxf to gpkg and I get the error
> below.  -am an trying to figure out how to add a bunch of options ( -f
> GPKG -append -update --config DXF_ENCODING CP1252 --config
> DXF_INCLUDE_RAW_CODE_VALUES TRUE --config
> DXF_TRANSLATE_ESCAPE_SEQUENCES FALSE --config
> DXF_MERGE_BLOCK_GEOMETRIES FALSE --config DXF_INLINE_BLOCKS TRUE -nlt
> LINESTRING -nln 32111_F)
>
> But can't even figure out how to add the recommended skipfailures flag.
> I tried adding
>
> gdal_tools.config_options("-skipfailures")
>
> but I get pylint tells me it is not callable.
>
> Anyone use pygdaltools?  There is some documentation, but nothing
> showing you how to set these command line parameters.
>
> import gdaltools
>
> gdal_tools = gdaltools.ogr2ogr()
>
> gdal_tools.BASEPATH = r"C:\Program Files\QGIS 3.16.16\bin"
>
> gdal_tools.set_encoding("UTF-8")
>
> gdal_tools.set_input(self.dxf_out_path)
>
> gdal_tools.set_output(self.gpkg_out)
>
> gdal_tools.execute()
>
> And I get the ERROR:root:b"Warning 1: Layer creation options ignored
> since an existing layer is\r\n being appended to.\r\nWarning 6:
> Normalized/laundered field name: 'EntityHandle' to
> 'EntityHa_2'\r\nERROR
> 1: Attempt to write non-linestring (GEOMETRYCOLLECTION) geometry to
> ARC type shapefile.\r\nERROR 1: Unable to write feature 0 from layer
> entities.\r\nERROR 1: Terminating translation prematurely after
> failed\r\ntranslation of layer entities (use -skipfailures to skip
> errors)\r\n"
>
> Traceback (most recent call last):
>
>File
> "c:\Users\RDTECLAC\AppData\Roaming\QGIS\QGIS3\profiles\default\python\
> plugins\site_selection\tools\master_planning\load_mp_designs.py", line
> 227, in 
>
>  main()
>
>File
> "c:\Users\RDTECLAC\AppData\Roaming\QGIS\QGIS3\profiles\default\python\
> plugins\site_selection\tools\master_planning\load_mp_designs.py", line
> 196, in main
>
>  mp_designs.gdal_convert()
>
>File
> "c:\Users\RDTECLAC\AppData\Roaming\QGIS\QGIS3\profiles\default\python\
> plugins\site_selection\tools\master_planning\load_mp_designs.py", line
> 185, in gdal_convert
>
>  gdal_tools.execute()
>
>File
> "C:\Users\RDTECLAC\AppData\Roaming\Python\Python39\site-packages\gdalt
> ools\ogr2ogrcmd.py", line 330, in execute
>
>  return self._do_execute(args)
>
>File
> "C:\Users\RDTECLAC\AppData\Roaming\Python\Python39\site-packages\gdalt
> ools\basetypes.py", line 108, in _do_execute
>
>  raise GdalToolsError(rc, err)
>
> gdaltools.basetypes.GdalToolsError: (1, b"Warning 1: Layer creation
> options ignored since an existing layer is\r\n being appended
> to.\r\nWarning 6: Normalized/laundered field name: 'EntityHandle' to
> 'EntityHa_2'\r\nERROR 1: Attempt to write non-linestring
> (GEOMETRYCOLLECTION) geometry to ARC type shapefile.\r\nERROR 1:
> Unable to write feature 0 from layer entities.\r\nERROR 1: Terminating
> translation prematurely after failed\r\ntranslation of layer entities
> (use -skipfailures to skip errors)\r\n")error:
>
>
> 

[gdal-dev] OAPIF: how does the driver utilize schema?

[Rahkonen Jukka via gdal-dev]

Hi,

The documentation in https://gdal.org/drivers/vector/oapif.html#layer-schema is 
wrong when it claims this:
OGR needs a fixed schema per layer, but OGC API - Features Core doesn't impose 
fixed schema. So the driver will retrieve the first page of features (10 
features) and establish a schema from this.

I can see from the debug info and from the source code that if GDAL finds a 
relation of type "describedBy" for a collection then it reads and interprets 
the schema from the link. However, I am not sure if  GDAL in this case trusts 
the schema and skips scanning of the first ten features for harvesting the 
attributes.

BTW, the name of the relation type has been changed into "describedby" due to 
https://github.com/opengeospatial/ogcapi-features/issues/338. It is better to 
support both describedBy and describedby.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] The relationship between overview levels and zoom levels?

[Rahkonen Jukka via gdal-dev]

Hi,

There are documentation about overviews in 
https://gdal.org/drivers/raster/cog.html, 
https://gdal.org/drivers/raster/gtiff.html, and 
https://gdal.org/programs/gdal_translate.html. Probably in other places, too, 
but reading those carefully is a good start. They deal also a bit with zoom 
levels.

What happens when a client opens an image with only one overview [16] is 
totally up to the client. It knows that the 1/16 downsampled version is 
available and it may decide to use the closest available overview: first full 
resolution till 1/8 and then jump to use 1/16. This is the AUTO approach of 
gdal_translate. Some other client would prefer better quality and use the full 
resolution up till 1/16 to avoid upsampling data that has downsampled earlier.

The COG driver has options for creating the overviews to match with the zoom 
levels of the selected tiling schema. Then there will be no need to resample 
the overview data again when the client makes zoom level based requests.

-Jukka Rahkonen-



Lähettäjä: gdal-dev  Puolesta James Sammone 
via gdal-dev
Lähetetty: sunnuntai 8. lokakuuta 2023 1.02
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] The relationship between overview levels and zoom levels?

I'm not sure if this is the best channel to ask this question as it might be 
beyond the scope, but I've asked it in a few others and have had no responses 
aside from others also being curious.

I am trying to understand the relationship between overviews and zoom levels so 
I know how to make more efficient Cloud Optimized GeoTIFFs. Using gdaladdo or 
gdal.BuildOverviews(), I can create overviews at [2,4,8,16] or at just [16]. 
From my understanding, this means the size is being divided by those values to 
provide downsample arrays of the original source. In the first example 
[2,4,8,16], I've created 4 separate overview arrays into the GeoTIFF that are 
2x, 4x, 8x, and 16x downsampled. And in the second example using only [16], 
I've built one overview array into the GeoTIFF that is 16x downsampled.

How can I understand how these overviews are applied when it comes to zoom 
levels? Does the 16x downsample appear sooner in the second example when 
zooming out than for the first example due to being first in order? Or do the 
16x downsamples appear at the same zoom level for both cases but the second 
example has additional 2x, 4x, and 8x downsamples that also appear before 
getting there?

Thanks for any insight into this anyone can provide. Despite using overviews 
all the time, I've struggled with this for a while and had largely consigned to 
not understanding it.

Best,

James
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Default page size of the OAPIF driver

[Rahkonen Jukka via gdal-dev]

Hi,

The OAPIF driver is using page size of 10 features by default 
https://gdal.org/drivers/vector/oapif.html#open-options.

"PAGE_SIZE=: Defaults to 10. Number of features to retrieve per 
request. Minimum is 1, maximum 1."

I know that 10 is the default page size ("limit") also in the standard but I 
think that this is too small page size, both for the users and especially for 
the service providers. Most users run GDAL utilities with the defaults. The 
buildings collection in our OGC API Features service has about 5 million 
features, which means that GDAL users are fetching the data by sending half a 
million request. That is much slower for the users than making 500 request, 
1 features each. And think about our log files.

The better default page size would be the biggest that the OAPIF server 
supports. By the standard the maximum is 1 but service providers may use 
bigger or smaller maximum page sizes. By the standard the server can always 
return less features than client is asking with "limit=" and the client must be 
prepared to follow the next links.

My suggestion for the new page size default is 1. If that feels too big for 
some reason, then would 1000 feel better?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Questions regarding temporary database file (ogr2ogr with MVT driver)

[Rahkonen Jukka via gdal-dev]

Hi,

I tested with this environmental variable and command

set OGR_MVT_REMOVE_TEMP_FILE=NO
ogr2ogr -f mvt -dsco temporary_db=c:\data\mtk\temp\temp.db out.mvt 
mtkmaasto.gpkg rakennus --debug on

Once ogr2ogr stopped I had the temp.db file left in the directory. It seems to 
be a simple SQLite database. Without setting the environmental variable the 
temporary database was immediately deleted. During the process the file 
explorer only shows that the file exists but with file size of 0 bytes. I 
believe that this is normal behavior with SQLite.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Leo Fuhrmann 
via gdal-dev
Lähetetty: perjantai 6. lokakuuta 2023 16.08
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Questions regarding temporary database file (ogr2ogr with MVT 
driver)

Hi,

I'm creating vector tiles from a PostGIS database using ogr2ogr with the MVT 
driver. The region is quite large and I'm creating tiles for levels 0-15, so 
the process takes quite some time (hours). The tiles are written directly to a 
S3 storage using the /vsis3/ virtual file system (see: 
https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files).

Looking at the logs, I can see that, firstly, a "temporary database" file 
(*.temp.db) is created. Some things are notable:

- this step seems to be CPU intensive compared to the creation of the actual 
pbf files
- the temporary database file is not being written to the output file directory 
(S3 storage in this case) as stated in the documentation
- the dataset creation option `TEMPORARY_DB` (see: 
https://gdal.org/drivers/vector/mvt.html#dataset-creation-options) seems to be 
ignored (I tried writing the file to a local directory but no such file shows 
up)
- my guess is that the file is only kept in memory (which would be /vsimem/ if 
I'm correct)

Is it possible to skip the step of first creating the temp db and directly 
create tiles from PostGIS or any other data source? Where/how exactly is the 
database file stored? Any ideas on optimizing the process?

I'm looking forward to some discussion on this topic. I will happily try to 
contribute where I can.

Regards,

Leo
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] layer.GetSpatialRef() fails on linux for shapefiles

[Rahkonen Jukka via gdal-dev]

Hi,

Then they should add that environment if they do not know that they do not 
belong to "most users" https://proj.org/en/9.3/usage/environmentvars.html

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Jonathan Moules 
via gdal-dev
Lähetetty: torstai 28. syyskuuta 2023 14.10
Vastaanottaja: gdal-dev@lists.osgeo.org; Even Rouault 

Aihe: Re: [gdal-dev] layer.GetSpatialRef() fails on linux for shapefiles

Hi Even,

The colleague doesn't have either a PROJ_LIB or a PROJ_DATA environment 
variable.

I asked another colleague to try it; they're on Ubuntu 20.04, and it worked for 
them. I believe using the same setup instructions.

Cheers,

Jonathan

On 2023-09-24 22:37, Jonathan Moules via gdal-dev wrote:
> Thanks Even. I don't have access to the machine either as the
> colleague is moving to another project. I'll have to see if it fails
> for another *nix user.
>
> On 2023-09-24 22:35, Even Rouault wrote:
>>
>> Le 24/09/2023 à 23:22, Jonathan Moules via gdal-dev a écrit :
>>>
>>> > Also check if the environment isn't messed up regarding PROJ and
>>> the PROJ_LIB/PROJ_DATA environment variable
>>>
>>> Thanks Even; sorry, what does this line mean? I'm guessing you're
>>> referring to:
>>> https://proj.org/en/9.3/usage/environmentvars.html - what would a "messed 
>>> up" one look like?
>>>
>> Hard to say without access to the machine. Perhaps just try to
>> recreate a new Conda env from scratch
>>
>>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://list/
> s.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev=05%7C01%7Cjukka.rahko
> nen%40maanmittauslaitos.fi%7C1f6517888cb34fa40aed08dbc01389dd%7Cc4f8a6
> 3255804a1c92371d5a571b71fa%7C0%7C0%7C638314962370381350%7CUnknown%7CTW
> FpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6
> Mn0%3D%7C3000%7C%7C%7C=3fVru6K6Ndpkv35FnAbMOlT%2BM96USO7wywqx550
> uRUs%3D=0

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Using ogr2ogr with limited memory

[Rahkonen Jukka via gdal-dev]

Hi,

Do you mean that your conversions fail because ogr2ogr does not get enough 
memory, or do you want to tell ogr2ogr to use max xxx MB of memory even there 
is much more available?

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Scott via 
gdal-dev
Lähetetty: keskiviikko 27. syyskuuta 2023 6.16
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] Using ogr2ogr with limited memory

Any tips for using ogr2ogr to use only a specified amount of RAM? I'm not 
having any luck.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev