Re: [gdal-dev] cmake error: IMPORTED_LOCATION not set for imported target "GEOTIFF::GEOTIFF"

[Even Rouault via gdal-dev]

Hi,

I don't reproduce this on my Linux setup. Can you give more information on:

- your operating system

- CMake version

- cmake command line you used to build libgeotiff

- cmake command line you use to build GDAL

Also, if you've reused an existing GDAL build directory that was used 
with a prior libgeotiff version, perhaps try rebuilding in a fresh build 
directory. libgeotiff CMake


Even

Le 04/06/2024 à 16:29, Hernán De Angelis via gdal-dev a écrit :

Hello!

While building GDAL 3.9.0 with libtiff 1.7.3 I am getting this error 
from ccmake when generating:


CMake Error in CMakeLists.txt:
   IMPORTED_LOCATION not set for imported target "GEOTIFF::GEOTIFF"
   configuration "Release".

I am used to build GDAL from source but never seen this before. Any 
hint on what does this mean and how to get around?


Thanks in advance!

Hernán


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Thread-safe raster access

[Even Rouault via gdal-dev]

Andrew,

what would be the purpose of thread-safe access: just making it 
thread-safe without any particular requirement on how efficient this 
would be (1), or hope for true concurrent access with ideally close to 
linear scalability with the number of threads (2) ?


If (1), then we could add a GDALMutexedDataset class, similarly to 
https://github.com/OSGeo/gdal/blob/master/ogr/ogrsf_frmts/generic/ogrmutexeddatasource.h 
which exists on the vector side (just used by the FileGDB driver due to 
the fact that the underlying SDK is not even re-entrant), which uses the 
decorator pattern around all public API entry points to call the 
underlying dataset under a mutex.  One could imagine to have a 
GDAL_OF_THREADSAFE open flag that GDALOpen() would use to return such 
instance. Shouldn't be too hard to implement, but probably not that 
useful IMHO. I can anticipate most users would have higher expectations 
than a mutex-based implementation.


If (2), it seems to me that it would require a huge effort, and the 
programming language we use (C++) offers hardly any safety belt to make 
sure we don't make mistakes, the main one being forgetting to lock 
things that should be locked, or dead locks situation. If we go into 
doing that, I'm not even sure how we can reliably identify all parts of 
the code that must be modified


Neither GDAL raster core nor any driver are designed to be thread-safe. 
For core, at least gcore/gdalarraybandblockcache.cpp and 
gcore/gdalhashsetbandblockcache.cpp which interact with the block cache 
should be made thread-safe, and "just" adding a lock would defeat the 
aim to achieve linear scalability. The change in GDALDataset::RasterIO() 
I did in 
https://github.com/OSGeo/gdal/commit/7f3a0e582eb189744bc7cb8e4a751135edaecaf5 
isn't thread-safe either (would be easy to make thread-safe though)


Once GDAL raster code is ready, the main challenge is making drivers 
themselves thread-safe. Raster drivers may directly read from a 
VSILFILE* handle, which isn't thread safe when using the standard Seek() 
+ Read() pair. A few VSIVirtualFileSystem have a PRead() implementation, 
which is thread-safe, but not all). Or they rely on using some instance 
of a "reader" returned by a third-party library (libtiff, libjpeg, 
libpng, sqlite3, etc.) (which in most cases also uses a VSILFILE*), none 
of which are thread-safe (except sqlite3 that can be made thread-safe by 
passing a flag at sqlite3_open() time, that will basically applies 
strategy (1) by protecting all calls with a mutex). Perhaps using 
thread-specific instances of VSILFILE* and third-party "reader" objects 
could be a way of solving this. But realistically doing a pass in all 
GDAL drivers would be a multi-month-man to multi-year-man type of 
effort. A realistic plan should be designed to allow combining (1) and 
(2): (2) for a few select drivers, and (1) as a fallback for most 
drivers that wouldn't be updated.


Even

Le 03/06/2024 à 15:44, Andrew Bell via gdal-dev a écrit :

Hi,

I am aware that there isn't thread-safe raster access with the current 
GDAL interface for various reasons. Given the state of processors, I 
was wondering if it would be valuable to take a look at providing the 
ability to do Raster I/O (at least reads) in a thread-safe way. This 
could be done through a new set of API calls or perhaps by 
modifications to what currently exists -- I don't know what makes 
sense at this point. I would be happy to spend some time looking at 
this if there is interest, but I would also like to learn from 
existing experience as to what kinds of things that I'm surely not 
considering would have to be dealt with.


Thanks,

--
Andrew Bell
andrew.bell...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalwarp across the antimeridian with a source extent >180E

[Even Rouault via gdal-dev]

Michael,

https://github.com/OSGeo/gdal/pull/10108 will fix it.

You can also workaround the issue by overriding the extent of the source 
dataset to be exactly -179.995,89.995,180.005,-89.995 with


 
"vrt://NETCDF:2024010209-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc:analysed_sst?a_srs=EPSG:4326_ullr=-179.995,89.995,180.005,-89.995"

The issue was twofolds:

- the dataset uses lon/lat arrays with single precision floating-point 
numbers. Hence -179.99 gets actually read a a read as 
-179.99000549316406 as a double-precision number, and thus the maxx - 
minxx difference was slightly over 360 degrees. I've added a heuristics 
to try to round -179.99-single-precision as -179.99-double-precision


- which defeated a heuristics in the warper to identify the center 
longitude of a dataset registered in a geographic CRS. This center 
longitude is just (min_lon + max_lon) / 2 if the dataset doesn't cover 
more than 360 degrees of longitude. This value, when computed, is passed 
as a hint OGRCoordinateTransformation so that it can post-correct 
longitudes to apply a +/- 360 degree offset, to be in the range of the 
source dataset. I've relaxed the sanity check to allow slighly more than 
360 degrees


Even


Le 01/06/2024 à 08:19, Michael Sumner via gdal-dev a écrit :
This data source has an odd georeferencing, it's a 36000x17999 raster 
in the extent -179.995 -89.995 180.005 89.995.


vrt://NETCDF:/vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2024010109-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc:analysed_sst?a_srs=EPSG:4326

(Why is it 17999: well there's one missing row - you'd expect 18000 at 
0.01 resolution and that manifests as half a cell short at the top and 
bottom).


But also the x registration is half a cell to the east from what you 
would expect.


It's correctly registered in that frame though, if we shift it west by 
the half cell I've verified visually that it "looks wrong" compared to 
coastlines etc.


The point of this report is, gdalwarp misses the western window of 
data for a request across the antimeridian (there's no data in the 
output east of 180).


gdalwarp $dsn -te 173.00 -19.85 185.00 -15.00 out1.tif -co COMPRESS=LZW

If we "fix" the extent to be more longitudinally pleasing, we get the 
data east of the antimeridian, the map is properly filled either side 
of the antimeridian.



vrt://NETCDF:/vsicurl/https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/2024010109-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc:analysed_sst?a_srs=EPSG:4326_ullr=-180,89.995,180,-89.995 



gdalwarp $dsnfix -te 173.00 -19.85 185.00 -15.00 out2.tif -co COMPRESS=LZW


Which is good, but as mentioned above the workaround means we are now 
half a cell out. It works fine for projected windows over the 
antimeridian in either situation.


The compressed file sizes reflect the missing data in the second one, 
these are 625K and 1.1Mb.


I'd like the warper to be able to correctly fill the data from the 
western window in the original case, we have determined that we can't 
feasibly "fix" the extent.


Or maybe something else I'm missing? I don't *think* SAMPLE_GRID or 
SOURCE_EXTRA helps here. I know that I could craft a meridian-crossing 
combination in VRT but I'd rather like this workflow to work as-is.  
(Chasing down the weird grid referencing is another project, but it's 
a really massive dataset so I'm pretty unconfident of that occurring).


Cheers, Mike

--
Michael Sumner
Research Software Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] OGR SQL CAST geometry example

[Even Rouault via gdal-dev]

According to http://www3.sqlite.org/series.html, generate_series() is 
only built in the sqlite3 command line utility, not in the library 
itself. That said, with some work, the source at 
https://www.sqlite.org/src/file/ext/misc/series.c could potentially be 
embedded into libgdal


Le 31/05/2024 à 02:49, Dan Jacobson a écrit :

$ ogrinfo 0.lines.kml -q -dialect SQLITE -sql \
'SELECT ST_PointN(GEOMETRY, generate_series(1, ST_NPoints(GEOMETRY))) FROM "0"'
ERROR 1: In ExecuteSQL(): sqlite3_prepare_v2(
   no such function: generate_series


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] OGR SQL CAST geometry example

[Even Rouault via gdal-dev]

Ah, I misunderstood and thought you wanted to extract points from 
linestrings.


Given

$ cat test.csv
id,my_wkt
1,"POINT Z (1 2 3)"

you can do:

$ ogrinfo test.csv -sql "select id, cast(my_wkt as geometry(pointz, 
4326)) from test"

INFO: Open of `test.csv'
  using driver `CSV' successful.

Layer name: test
Geometry: 3D Point
Feature Count: 1
Extent: (1.00, 2.00) - (1.00, 2.00)
Layer SRS WKT:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
    MEMBER["World Geodetic System 1984 (Transit)"],
    MEMBER["World Geodetic System 1984 (G730)"],
    MEMBER["World Geodetic System 1984 (G873)"],
    MEMBER["World Geodetic System 1984 (G1150)"],
    MEMBER["World Geodetic System 1984 (G1674)"],
    MEMBER["World Geodetic System 1984 (G1762)"],
    MEMBER["World Geodetic System 1984 (G2139)"],
    MEMBER["World Geodetic System 1984 (G2296)"],
    ELLIPSOID["WGS 84",6378137,298.257223563,
    LENGTHUNIT["metre",1]],
    ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
    ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
    AXIS["geodetic latitude (Lat)",north,
    ORDER[1],
    ANGLEUNIT["degree",0.0174532925199433]],
    AXIS["geodetic longitude (Lon)",east,
    ORDER[2],
    ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
    SCOPE["Horizontal component of 3D system."],
    AREA["World."],
    BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Geometry Column = my_wkt
id: String (0.0)
OGRFeature(test):1
  id (String) = 1
  POINT Z (1 2 3)

Note that probably counterintuitively, if instead of pointz, you try to 
specify point (or even linestring), that will only affect the *declared* 
geometry column type, but will have no consequence on the actual feature 
geometry, and no consistency check is done to ensure they match  
(contrary to PostGIS that is very strict, and would not even allow 
casting that POINT Z (1 2 3) as a geometry(point), but only a 
geometry(pointz))



Le 31/05/2024 à 01:29, Dan Jacobson a écrit :

"ER" == Even Rouault  writes:

ER> you can't do operations on geometries with OGR SQL...

OK, I'll try it. Thanks.
The page should still have an example of actual use of
   Casting ... POINT[Z], LINESTRING[Z]

Without examples people will try things like POINTZ, "POINTS[Z]"


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] OGR SQL CAST geometry example

[Even Rouault via gdal-dev]

Dan,

you can't do operations on geometries with OGR SQL, but selecting it. 
You'd typically need to use the ST_PointN() from Spatialite with the 
SQLite dialect


Even

Le 31/05/2024 à 00:32, Dan Jacobson via gdal-dev a écrit :

Hi. I will add an example to
https://gdal.org/user/ogr_sql_dialect.html#changing-the-type-of-the-fields
showing users how to e.g., CAST LINESTRING Z to POINTS Z, e.g.,
to extract the points from:
$ ogrinfo 0.lines.kml -q -sql 'SELECT "_ogr_geometry_" FROM "0"'
Layer name: 0
OGRFeature(0):1
   LINESTRING Z (-100.79005679 36.38022 1000,-100...
Alas, there are no examples I can find of how to write it.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] RFC 96 adjustment: Re: Call for review and discussion on RFC96: Deferred in-tree C++ plugin loading

[Even Rouault via gdal-dev]

Hi,

I just wanted to point that a shortcoming of RFC 96 
(https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html) 
was discovered. https://github.com/OSGeo/gdal/pull/10068 will address 
it. Please refer to it for the details.


This PR is aimed at being backported for 3.9.1. Pedantically this will 
be an ABI change (between in-tree drivers built as plugins and core), 
although I suspect that it shouldn't be noticed for most 
use scenarios. Not clear if that's worth a SONAME bump for 
3.9.1, or if that would be more an annoyance.


Even

Le 02/11/2023 à 12:59, Even Rouault via gdal-dev a écrit :

Hi,

I'm seeking for feedback and review on a new RFC (RFC 96: Deferred 
in-tree C++ plugin loading),

detailed in https://github.com/OSGeo/gdal/pull/8648, whose summary is:

This RFC adds a mechanism to defer the loading of in-tree C++ plugin 
drivers to
the point where their executable code is actually needed, and converts 
a number
of relevant drivers to use that mechanism. The aim is to allow for 
more modular

GDAL builds, while improving the performance of plugin loading.

(This is material only for GDAL 3.9 of course)

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Even Rouault via gdal-dev]

Hi,

The WMS cache (which is used underneath by the WMTS driver) writes each 
tiles in a separate file. So if you use multi-threaded, as long as you 
read in parallel different GDALDataset* object on the WMTS filename, and 
read disjoint areas, that should be fine. There might be a slight risk 
of things going bad if you happened to read the same tile from different 
GDALDataset* object at the same time, as it could potentially read a 
partially written cached tile, or concurrently write the tile in the 
cache and corrupt it. The logic is in frmts/wms/gdalwmscache.cpp


Even

Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
Because this is not clear in documentation, especially in WMTS driver 
docs, my question is strictly about this issue to programmers who have 
already explored this topic.


Regards
Michal

W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka 
 napisał(a):


Hi,

No, I am not sure because I do not program myself. Reading this
document https://gdal.org/user/multithreading.html makes me feel
that it is possible to do some things in parallel, but the
programmer must know how to do it right.

-Jukka Rahkonen-

*Lähettäjä:*Javier Jimenez Shaw 
*Lähetetty:* keskiviikko 29. toukokuuta 2024 10.41
*Vastaanottaja:* Rahkonen Jukka 
*Kopio:* Michał Kowalczuk ;
gdal-dev@lists.osgeo.org
*Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation

On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev
mailto:gdal-dev@lists.osgeo.org>> wrote:

Hi,

When you have a RasterBand  from the WMTS data source, it is
abstracted and you can read the raster data just like from any
other data source and raster band
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data

“There
are a few ways to read raster data, but the most common is via
the GDALRasterBand::RasterIO() method. This method will
automatically take care of data type conversion, up/down
sampling and windowing.” GDAL knows which tiles to read.

I do not know if the WMTS driver can do parallel tile
downloads. If not, it is possible to run many RasterIO() at
the same time, each reading data from a different window like
in this rasterio document
https://rasterio.readthedocs.io/en/latest/topics/concurrency.html
.

Jukka, are you sure you can run several RasterIO in parallel (on
the same dataset)? in GDAL GeoTIFF you cannot: the cache may be
corrupted. In that case I open several datasets over the same
file. I do not know about WMTS.

-Jukka Rahkonen-

*Lähettäjä:*gdal-dev mailto:gdal-dev-boun...@lists.osgeo.org>> *Puolesta *Michal
Kowalczuk via gdal-dev
*Lähetetty:* keskiviikko 29. toukokuuta 2024 9.08
*Vastaanottaja:* gdal-dev@lists.osgeo.org

*Aihe:* [gdal-dev] GDAL WMTS get tiles no documentation

Hi GDAL fellows

This is my first post on this mailing list, so I'm asking for
understanding.

As all we know the purpose of using WMTS over WMS, I'd like to
implement parallel downloading tiles from service using C API.

In my opinion GDAL documentation
(https://gdal.org/drivers/raster/wmts.html
) says nothing on
this topic.

I can get capabilities from WMTS, I can open the selected
subdataset but how to get tiles for given extent? I could not
find any information how to do it, even in the GDAL tests on
github.

I am kindly asking for tips. It also can be in python. How
using pure GDAL API fetch tiles to dynamically complete the
displayed map. This is my goal.

Thank you!

Michal

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org 
https://lists.osgeo.org/mailman/listinfo/gdal-dev



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] libgeotiff 1.7.3 is released

[Even Rouault via gdal-dev]

Hi,

Follow-up release of 1.7.2 to address a regression with CMake builds 
related to a typo in CMAKE_INSTALL_BINDIR install target that affects 
location of geotiff.dll (#117)


Source packages can be found at:

https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.3.tar.gz
https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.3.zip

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] libgeotiff 1.7.2 is released

[Even Rouault via gdal-dev]

Hi,

Source packages can be found at:

https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.2.tar.gz
https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.2.zip

News is:

* GTIFGetDatumInfoEx(): handle dynamic datums
* CMake: adopt GNUInstallDirs
* CMake: export TIFF as a public dependency
* Add XTIFFClientOpenExt() with re-entrant error callbacks
* listgeo: add a -no_corners option to avoid printing corner
  coordinates which might be PROJ version sensitive (refs #81)
* Fix test failures with PROJ 9.1.1 and 9.3
* Remove trailing spaces from many files.
* Localize variables and add const when possible in a number of files
* geo_normalize.c: sprintf --> snprintf
* Add missing includes
* Fix -Werror=calloc-transposed-args with gcc 14
* FindPROJ.cmake: proj_experimental;h -> proj_experimental.h
* CMake: add a BUILD_SHARED_LIBS option, and make it default to ON
* autoconf/CMake: add generation of libgeotiff.pc
* Remove INSTALL and install-sh autoconf-generated files

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] building just the python bindings (cmake)

[Even Rouault via gdal-dev]

Le 22/05/2024 à 16:59, Greg Troxel a écrit :

I got 3.5 to work, so I'm focusing on the 3.9 upgrade.   It seems that
3.9, vs 3.5, has withdrawn the swig generated files from the distfile.

   With 3.9, you could also do a more manual approach (not sure the
   python_generated_files target is available in 3.5)
   cmake ..
   make GDAL  # build the library
   cd swig/python
   make python_generated_files  # generate SWIG generated files
   python setup.py build

That looks like it works.  Except that I omitted the 'make GDAL'  -- in
my case, gdal is already installed.   I don't see why one would need to
build, as long as the headers and libs are available in the system.
Does that seem reasonable, or am I missing something?
The  GDAL Python (or CSharp or Java) bindings, being in-tree, have a 
special status compared to other bindings in dedicated code repository, 
since they get built as one of the many components built by the build 
process, and do not rely on installed libs/headers in the default 
process. You've figured out it is still possible to build a wheel (or 
use the source pakage on pypi https://pypi.org/project/GDAL/ which are 
generated following the procedure documented in HOWTO-RELEASE) that will 
manage to find the installed libs/headers


It would be great if README in swig/python explained this.  The
"installation" instructions are more pointers to packaging systems, and
not so much guidance for people trying to build from source.  I realize
that almost all people who use gdal use it from packaging and do not
have the slightest idea how to build perhaps anything, but on the other
hand I would expect that a README in the sources is only read by people
who are trying to build it.

Feel free to contribute documentation :-)


Also in README, it talks about numpy being a depdendency but not
formally required.   I think two clarificaions would be useful:

   What is lost specifically, vs "many examples and utilities"?  If you
   use it with qgis, without numpy, are things missing there too?  I'm
   willing to believe it's a big deal, but it's hard to understand.

   I am guessing that the things that need numpy work fine if you install
   this built in an environment without numpy and then later also install
   numpy, but it is unclear if numpy is a run-time-only dependency or if
   it is needed at build time.


It is a built-time for the osgeo.gdal_array component that is for 
example used when using the osgeo.gdal.Dataset.ReadAsArray() method.


For generic purpose packages (unless you're very very tight on the 
binary footprint), I'd strongly suggest building the Python bindings 
with the numpy dependency, and make it a run-time one as well.



--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] building just the python bindings (cmake)

[Even Rouault via gdal-dev]

Greg,

If the build requirements for the Python bindings are met (python + swig 
available), then a "make && make install" cycle will build and install 
libgdal and the python bindings, like it did in autoconf era. The CMake 
build target "python_binding" has libgdal as a dependency.


The idea for packaging is similar to autoconf era: that is  you just do 
a build and separate the artifacts into separate packages. That's what 
Alpine (https://git.alpinelinux.org/aports/tree/community/gdal/APKBUILD) 
or Debian 
(https://salsa.debian.org/debian-gis-team/gdal/-/blob/master/debian/rules?ref_type=heads)


With 3.9, you could also do a more manual approach (not sure the 
python_generated_files target is available in 3.5)

cmake ..
make GDAL  # build the library
cd swig/python
make python_generated_files  # generate SWIG generated files
python setup.py build

Even

Le 21/05/2024 à 19:26, Greg Troxel via gdal-dev a écrit :

I had not gotten to converting the pkgsrc package to cmake, because it
seemed like it would take a long time.  So gdal is at 3.5.3, the last
one that works with autoconf.  Yes, I know that 3.5 was about two years
ago.  It turns out that it is taking hours to do the conversion, and the
good news is that the main "gdal-lib" package is converted.

I also know 3.5 is not current, but I am trying to convert it to cmake
(which presumably works fine, given that 3.6 dropped autoconf), before
updating to newer versions.

I first  tried running cmake in swig/python (in a build dir), but the
generated makefile doesn't really build anything.

I then tried cmake at top level, and tried to run gmake in swig/python
and it seems to be building large amounts of things that are not the
python binding.  Perhaps all of gdal lib and bin.  The install target
indeed installs libgdal and bin/gdalfoo, regular non-python things, vs
the expected python tools.


The README talks about setuptools.  Should I conclude that cmake is a
red herring here, despite the presence of CMakeLists.txt?   And that the
build should just be setuptools in that dir, with no configure stage?





___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [Geotiff] libgeotiff 1.7.2 release candidate

[Even Rouault via gdal-dev]

oops. Here are the corrected URLs:

https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.2rc1.tar.gz
https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.2rc1.zip

Le 21/05/2024 à 15:44, Greg Troxel via Geotiff a écrit :

Even Rouault via Geotiff  writes:


do you mean

https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.2rc1.tar.gz

assuming yes builds and passes tests on NetBSD 10 amd64.
___
Geotiff mailing list
geot...@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/geotiff


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] libgeotiff 1.7.2 release candidate

[Even Rouault via gdal-dev]

Hi,

I've prepared a libgeotiff 1.7.2 release candidate.

https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.1rc2.tar.gz
https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-1.7.1rc2.zip

I'll promote it to final later this week if nothing serious is reported 
on it before.



News is:

libgeotiff 1.7.2


* GTIFGetDatumInfoEx(): handle dynamic datums
* CMake: adopt GNUInstallDirs
* CMake: export TIFF as a public dependency
* Add XTIFFClientOpenExt() with re-entrant error callbacks
* listgeo: add a -no_corners option to avoid printing corner
  coordinates which might be PROJ version sensitive (refs #81)
* Fix test failures with PROJ 9.1.1 and 9.3
* Remove trailing spaces from many files.
* Localize variables and add const when possible in a number of files
* geo_normalize.c: sprintf --> snprintf
* Add missing includes
* Fix -Werror=calloc-transposed-args with gcc 14
* FindPROJ.cmake: proj_experimental;h -> proj_experimental.h
* CMake: add a BUILD_SHARED_LIBS option, and make it default to ON
* autoconf/CMake: add generation of libgeotiff.pc
* Remove INSTALL and install-sh autoconf-generated files

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Conda failure in GA

[Even Rouault via gdal-dev]

As suggested in the comment of #939, attempting 
https://github.com/rouault/gdal/commit/1000cfaa950a18bd14b0c72109439cad58c789d5



That fixed it. Merged into master

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Conda failure in GA

[Even Rouault via gdal-dev]

It looks like the numpy 2.0 update was pushed to the gdal-feedstock 
https://github.com/conda-forge/gdal-feedstock/pull/939 (looking like 
 GDAL must also build and test clean against Numpy 2.0 at this point too).


The error is the Conda builder complaining that it can't find Numpy 
2.0 because our recipe doesn't use the Numpy 2.0 release candidate 
channel. I think something is busted by this change to the recipe but 
I have not had time to investigate it.


As suggested in the comment of #939, attempting 
https://github.com/rouault/gdal/commit/1000cfaa950a18bd14b0c72109439cad58c789d5


My understanding is that the addition of this channel should only be a 
build requirement, and not impose to add this channel when using the 
generated package given than building against numpy 2.0 also yields 
binaries that are compatible with the oldest supported numpy 1.x


Even

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Data Type Conversion

[Even Rouault via gdal-dev]

Andrew,

Data type conversion is done by GDALCopyWords[64]: 
https://gdal.org/api/raster_c_api.html#_CPPv415GDALCopyWords64PKv12GDALDataTypeiPv12GDALDataTypei10GPtrDiff_t


The doc was a bit outdated, regarding conversion from floating point 
data types to integer data types. I've just adjusted it in 
https://github.com/OSGeo/gdal/commit/139e891b9b4a7b928c66e3daef83361fa530e6ac


Website should be refreshed soon

Even

Le 20/05/2024 à 17:01, Andrew Bell via gdal-dev a écrit :

Hi,

The documentation for GDALRasterBand::RasterIO says that it will 
convert data from the source data type to the destination data type. I 
searched the source code a bit I'm not sure I located the bit that 
does this conversion. I'm trying to understand what happens in this 
conversion process. Is this a numeric conversion as would be done with 
a C/C++ cast or is this conversion simply add or truncate bytes in 
order to fit in a destination data type. Or perhaps something else?




Thanks,

--
Andrew Bell
andrew.bell...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GPKG to PG single transaction, different schemas

[Even Rouault via gdal-dev]

Le 14/05/2024 à 17:30, Matteo Ghetta via gdal-dev a écrit :

Hi all,

I've a geopackage with many layers (in the following example let's 
pretend just 2) that I want to import in a single transaction 
(important!) into a PG database.


GPKG and PG table names are the same, but it can happen that the 
tables are in different schemas. I couldn't find the correct flags 
combination to specify the destination schema.table_name.


With the following command features of first_table are appended to the
second_table table:

ogr2ogr --debug ON -f PostgreSQL "PG:host=localhost user=matteo 
password=my_pws dbname=test" -append /home/matteo/export_db.gpkg 
first_schema.first_layer second_schema.second_layer


same with the -nln option:

ogr2ogr --debug ON -f PostgreSQL "PG:host=localhost user=matteo 
password=my_pws dbname=test" -append /home/matteo/export_db.gpkg 
first_layer -nln first_schema.first_layer second_layer -nln 
second_schema.second_layer


There's no formal concept of schema in GeoPackage. You can't use -nln 
multiple times. It works only when importing a single layer


Yes, you'll likely have to use ogr2vrt 
(https://github.com/OSGeo/gdal/blob/master/swig/python/gdal-utils/osgeo_utils/samples/ogr2vrt.py) 
and rename manually the layers


And add "-ds_transaction -gt unlimited" to make sure everything is 
imported in the same transaction



--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Discrepancy in reported Spatial Reference b/n gdalinfo and arcpy

[Even Rouault via gdal-dev]

Le 14/05/2024 à 18:08, Liyuneh Gebre a écrit :

hi,
thanks for your kind commitment.  I tried the docker version and 
unfortunately it shows the same result.
That's expected. The fix has been done post 3.9.0. This will be 
available in 3.9.1



On Sun, May 12, 2024 at 4:02 PM Even Rouault 
 wrote:


Hi,

fix in https://github.com/OSGeo/gdal/pull/9909

Le 12/05/2024 à 12:06, Javier Jimenez Shaw via gdal-dev a écrit :


I do not know if there is any documentation from esri about that
specific format for LAMBERT_AZIMUTHAL. It would be great.
And also I am surprised that in 13 years nobody complained.


Digging a bit in the web, I found some. See code comments in the
pull request:

https://github.com/OSGeo/gdal/pull/9909/files#diff-ed43401b4bc3e43302f36a0502b4106a7fa0866cd5e3c6ac4ccb3823d10f3085R436

Even

-- 
http://www.spatialys.com

My software is free, but my time generally not.



--
/*Liyuneh Gebre*//, Associate Researcher///
/Ethiopian Institute of Agricultural Research,EIAR
/
/Cell Phone:+251 911858155
/
/linkedIn: https://www.linkedin.com/in/liyuneh-gebre-b842b440/
/
//skype:liyenew_1//
/Addis Ababa, Ethioipa/


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.9.0 is released

[Even Rouault via gdal-dev]

Hi,

The GDAL 3.9.0 Docker images are available:

* ghcr.io/osgeo/gdal:alpine-small-3.9.0
* ghcr.io/osgeo/gdal:alpine-normal-3.9.0
* ghcr.io/osgeo/gdal:ubuntu-small-3.9.0
* ghcr.io/osgeo/gdal:ubuntu-full-3.9.0

Even

Le 10/05/2024 à 16:10, Even Rouault via gdal-dev a écrit :

Hi,

On behalf of the GDAL/OGR development team and community, I am pleased to
announce the release of GDAL/OGR 3.9.0. GDAL/OGR is a C++ geospatial data
access library for raster and vector file formats, databases and web 
services.

It includes bindings for several languages, and a variety of command line
tools.

http://gdal.org/

The 3.9.0 release is a new feature release with the following highlights:

* [RFC 
96](https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html): 
Deferred C++ plugin loading
* [RFC 
97](https://gdal.org/development/rfc/rfc97_feature_and_fielddefn_sealing.html): 
OGRFeatureDefn, OGRFieldDefn and OGRGeomFieldDefn "sealing"
* [RFC 
98](https://gdal.org/development/rfc/rfc98_build_requirements_gdal_3_9.html): 
Build requirements for GDAL 3.9
* [RFC 
99](https://gdal.org/development/rfc/rfc99_geometry_coordinate_precision.html): 
Geometry coordinate precision
* Add [S104](https://gdal.org/drivers/raster/s104.html) (Water Level 
Information for Surface Navigation Product) and
  [S111](https://gdal.org/drivers/raster/s111.html) (Surface Currents 
Product) raster read-only drivers (required libhdf5)
* Add raster [GTI](https://gdal.org/drivers/raster/gti.html) (GDAL 
Raster Tile Index) driver to support catalogs with huge number of 
sources.
* Add vector 
[MiraMonVector](https://gdal.org/drivers/vector/miramon.html) 
read/creation driver (#9688)

* Deprecated ARG driver has been removed (#7920)
* Bump of shared lib major version

More complete information on the new features and fixes in the 3.9.0 
release can be found at:


https://github.com/OSGeo/gdal/blob/v3.9.0/NEWS.md

The release can be downloaded from:
  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.gz - source 
as .tar.gz

  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.gz.md5 - md5sum
  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.gz.sig - 
signature
  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.xz - source 
as .tar.xz

  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.xz.md5 - md5sum
  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.xz.sig - 
signature

  * https://download.osgeo.org/gdal/3.9.0/gdal390.zip - source as a zip
  * https://download.osgeo.org/gdal/3.9.0/gdal390.zip.md5 - md5sum
  * https://download.osgeo.org/gdal/3.9.0/gdal390.zip.sig - signature
  * https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0.tar.gz - 
test suite
  * https://download.osgeo.org/gdal/3.9.0/gdal390doc.zip - 
documentation / website


Note to packagers: obviously GCC 14 has just been released. It breaks 
mingw64

builds. Fix in https://github.com/OSGeo/gdal/pull/9894

Best regards,

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Discrepancy in reported Spatial Reference b/n gdalinfo and arcpy

[Even Rouault via gdal-dev]

Hi,

fix in https://github.com/OSGeo/gdal/pull/9909

Le 12/05/2024 à 12:06, Javier Jimenez Shaw via gdal-dev a écrit :


I do not know if there is any documentation from esri about that 
specific format for LAMBERT_AZIMUTHAL. It would be great.

And also I am surprised that in 13 years nobody complained.


Digging a bit in the web, I found some. See code comments in the pull 
request: 
https://github.com/OSGeo/gdal/pull/9909/files#diff-ed43401b4bc3e43302f36a0502b4106a7fa0866cd5e3c6ac4ccb3823d10f3085R436


Even

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.9.0 is released

[Even Rouault via gdal-dev]

Hi,

On behalf of the GDAL/OGR development team and community, I am pleased to
announce the release of GDAL/OGR 3.9.0. GDAL/OGR is a C++ geospatial data
access library for raster and vector file formats, databases and web 
services.

It includes bindings for several languages, and a variety of command line
tools.

http://gdal.org/

The 3.9.0 release is a new feature release with the following highlights:

* [RFC 
96](https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html): 
Deferred C++ plugin loading
* [RFC 
97](https://gdal.org/development/rfc/rfc97_feature_and_fielddefn_sealing.html): 
OGRFeatureDefn, OGRFieldDefn and OGRGeomFieldDefn "sealing"
* [RFC 
98](https://gdal.org/development/rfc/rfc98_build_requirements_gdal_3_9.html): 
Build requirements for GDAL 3.9
* [RFC 
99](https://gdal.org/development/rfc/rfc99_geometry_coordinate_precision.html): 
Geometry coordinate precision
* Add [S104](https://gdal.org/drivers/raster/s104.html) (Water Level 
Information for Surface Navigation Product) and
  [S111](https://gdal.org/drivers/raster/s111.html) (Surface Currents 
Product) raster read-only drivers (required libhdf5)
* Add raster [GTI](https://gdal.org/drivers/raster/gti.html) (GDAL 
Raster Tile Index) driver to support catalogs with huge number of sources.
* Add vector 
[MiraMonVector](https://gdal.org/drivers/vector/miramon.html) 
read/creation driver (#9688)

* Deprecated ARG driver has been removed (#7920)
* Bump of shared lib major version

More complete information on the new features and fixes in the 3.9.0 
release can be found at:


https://github.com/OSGeo/gdal/blob/v3.9.0/NEWS.md

The release can be downloaded from:
  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.gz - source as 
.tar.gz

  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.gz.md5 - md5sum
  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.gz.sig - signature
  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.xz - source as 
.tar.xz

  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.xz.md5 - md5sum
  * https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0.tar.xz.sig - signature
  * https://download.osgeo.org/gdal/3.9.0/gdal390.zip - source as a zip
  * https://download.osgeo.org/gdal/3.9.0/gdal390.zip.md5 - md5sum
  * https://download.osgeo.org/gdal/3.9.0/gdal390.zip.sig - signature
  * https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0.tar.gz - 
test suite
  * https://download.osgeo.org/gdal/3.9.0/gdal390doc.zip - 
documentation / website


Note to packagers: obviously GCC 14 has just been released. It breaks 
mingw64

builds. Fix in https://github.com/OSGeo/gdal/pull/9894

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: approve GDAL 3.9.02RC2 as final release

[Even Rouault via gdal-dev]

Motion passed with +1 from PSC members EvenR, JukkaR and JavierJS

Le 08/05/2024 à 13:13, Even Rouault via gdal-dev a écrit :

Hi,

Motion:

Adopt GDAL 3.9.0RC2 as final 3.9.0 release

Starting with my +1

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Doxygen Arguments

[Even Rouault via gdal-dev]

Andrew,

this is now fixed per 
https://github.com/OSGeo/gdal/commit/393eee77b40ae16751e4d2c0ad3e53386b22d025 
. Doxygen apparently takes the function/method at declaration time, and 
not definition time, and we had a number of unnamed parameters in 
declarations. (I also struggled with weird things when trying to fix 
that locally due not using a too recent Doxygen version. The HTML output 
was fine, but once integrated with Sphinx/Breathe, the parameters were 
still missing. Updating to Doxygen 1.9.8 as now used by our CI fixed things)


Even

Le 09/05/2024 à 15:20, Andrew Bell via gdal-dev a écrit :

Hi,

I'm looking at the documentation for GDALRasterBand::RasterIO and it's 
surprising that the names of the arguments aren't included in the 
function prototype despite them being in the doxygen function 
definition. The arguments are described well below the function 
prototype, making it hard to match things up and you have to assume 
that the order matches.


Does some doxygen expert know how the argument names could somehow be 
included in the function prototype?


Web page: 
https://gdal.org/api/gdalrasterband_cpp.html#classGDALRasterBand_1a75d4af97b3436a4e79d9759eedf89af4


Source: https://fossies.org/linux/gdal/gcore/gdalrasterband.cpp (line 110)

--
Andrew Bell
andrew.bell...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] SQL / gdal_get_pixel_value tips requested

[Even Rouault via gdal-dev]

Jesse,

the answer lies in exploring available Spatialite SQL functions: 
https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.1.0.html


ST_Centroid() returns a Point geometry. You may get its coordinates with 
ST_X() and ST_Y(). I suppose you'd need to use a sub-select construct to 
get things working, like


SELECT gdal_get_pixel_value(path, band_num, 'georef', ST_X(centroid), 
ST_Y(centroid)) FROM (SELECT ST_Centroid(geometry) AS centroid FROM ds) x


untested obviously!

Le 08/05/2024 à 19:09, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND 
APPLICATIONS INC] via gdal-dev a écrit :


Hi,

I’m wanting to execute a SQL query on a vector database like:

vector_mem_ds.ExecuteSQL(/f/"select ST_Centroid(GEOMETRY) as cnt, 
gdal_get_pixel_value('{raster_fp}', {ndvi_band_num}, 'georef', 
cnt.STX, cnt.STY) from ds",/dialect/="SQLITE")


In particular, I want to use the geometry’s centroid for 
gdal_get_pixel_value, but the latter function takes two parameters, 
and ST_Centroid provides 1 return value (a compound type Point), so I 
am hoping for some way to unpack this into the form necessary expected 
from get_pixel_value. Apparently some GIS query engines do expose a 
STX and STY field but that’s if you trust random google results.  But 
as is this fails.


Is this possible largely as-is?

Thanks,

Jesse


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Motion: approve GDAL 3.9.02RC2 as final release

[Even Rouault via gdal-dev]

Hi,

Motion:

Adopt GDAL 3.9.0RC2 as final 3.9.0 release

Starting with my +1

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.9.0 rc2 is available (Re: GDAL 3.9.0 rc1 is available)

[Even Rouault via gdal-dev]

Hi,

I've prepared a RC2 to fix a regression in ogr2ogr workflows from 
Shapefile (or other datasources with layer metadata items) to PostGIS.


  https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0rc2.tar.xz
  https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0rc2.tar.gz
  https://download.osgeo.org/gdal/3.9.0/gdal390rc2.zip


https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0rc2.tar.gz
  https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0rc2.zip

Even

Le 06/05/2024 à 15:43, Even Rouault via gdal-dev a écrit :

Hi,

I have prepared a GDAL/OGR 3.9.0 release candidate.

NEWS at:

  https://github.com/OSGeo/gdal/blob/v3.9.0RC1/NEWS.md

Pick up an archive among the following ones (by ascending size):

  https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0rc1.tar.xz
  https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0rc1.tar.gz
  https://download.osgeo.org/gdal/3.9.0/gdal390rc1.zip

A snapshot of the gdalautotest suite is also available :

https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0rc1.tar.gz
  https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0rc1.zip

A snapshot of the documentation is at:

  https://download.osgeo.org/gdal/3.9.0/gdal390doc.zip

I'll call for a vote promoting it later this week if no serious 
problems are

reported before.

There are no major changes, impacting packaging or integration, since 
beta2, but

functional bugfixes and enhancements:
* /vsigs/: fix RmdirRecursive() on an empty directory
* /vsigs/ UnlinkBatch(): make sure that path config options are 
checked for the

  deleted files and not '/vsigs/batch/
* /vsiaz/ UnlinkBatch(): make sure that path config options are 
checked for the

  deleted files
* Various fixes in MiramonVector driver
* Warper: add a NODATA_VALUES_PCT_THRESHOLD option
* gdal2tiles: add --nodata-values-pct-threshold option
* support HDF5EOS metadata where the GROUP=GridStructure has an empty sub
  GROUP=Dimension
* HDF5: implement GetOffset() and GetScale() for netCDF metadata items 
add_offset

  and scale_factor
* TileDB: multidim: prevent infinite recursion when opening an array 
in a group

  where there are 2 or more 2D+ arrays
* TileDB: multidim: OpenMDArray(): fix opening an array, member of a 
group,

  which has no explicit name
* Fix scripts/gdal-bash-completion.sh

Best regards,

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Mass operations to raster block?

[Even Rouault via gdal-dev]

Hi Andrew,

no, there's no such support in GDAL itself. That could potentially be 
done using VRT and pixel functions, but I'm not sure that would qualify 
as "optimal", and that would require a bit of setup. You're probably 
better instantiating your for(y: ...) for (x: ...) loop manually with 
the operation you want to apply


xtensor (https://xtensor.readthedocs.io/en/latest/operator.html) (in a 
nutshell, this is similar to numpy but in C++) could potentially be used 
in combination of GDAL, but I don't think we want to drag this 
dependency in GDAL itself


Even

Le 06/05/2024 à 20:43, Andrew Bell via gdal-dev a écrit :

Hi,

Is there any support in the library for applying some arithmetic 
operations to all entries in a block? For example, can I optimally 
subtract or add some scalar to every entry?


Thanks,

--
Andrew Bell
andrew.bell...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.9.0 rc1 is available

[Even Rouault via gdal-dev]

Hi,

I have prepared a GDAL/OGR 3.9.0 release candidate.

NEWS at:

  https://github.com/OSGeo/gdal/blob/v3.9.0RC1/NEWS.md

Pick up an archive among the following ones (by ascending size):

  https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0rc1.tar.xz
  https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0rc1.tar.gz
  https://download.osgeo.org/gdal/3.9.0/gdal390rc1.zip

A snapshot of the gdalautotest suite is also available :

https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0rc1.tar.gz
  https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0rc1.zip

A snapshot of the documentation is at:

  https://download.osgeo.org/gdal/3.9.0/gdal390doc.zip

I'll call for a vote promoting it later this week if no serious problems are
reported before.

There are no major changes, impacting packaging or integration, since 
beta2, but

functional bugfixes and enhancements:
* /vsigs/: fix RmdirRecursive() on an empty directory
* /vsigs/ UnlinkBatch(): make sure that path config options are checked 
for the

  deleted files and not '/vsigs/batch/
* /vsiaz/ UnlinkBatch(): make sure that path config options are checked 
for the

  deleted files
* Various fixes in MiramonVector driver
* Warper: add a NODATA_VALUES_PCT_THRESHOLD option
* gdal2tiles: add --nodata-values-pct-threshold option
* support HDF5EOS metadata where the GROUP=GridStructure has an empty sub
  GROUP=Dimension
* HDF5: implement GetOffset() and GetScale() for netCDF metadata items 
add_offset

  and scale_factor
* TileDB: multidim: prevent infinite recursion when opening an array in 
a group

  where there are 2 or more 2D+ arrays
* TileDB: multidim: OpenMDArray(): fix opening an array, member of a group,
  which has no explicit name
* Fix scripts/gdal-bash-completion.sh

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Efficient raster bounding box transformation?

[Even Rouault via gdal-dev]

==> GDALCreateApproxTransformer(): 
https://gdal.org/api/gdal_alg.html#_CPPv427GDALCreateApproxTransformer19GDALTransformerFuncPvd


Otherwise there's OGRCoordinateTransformation::TransformBounds(), but 
this works from a rectangle in the source CRS


Most simple would be to use GDALAutoCreateWarpedVRT() 
(https://gdal.org/api/gdalwarp_cpp.html#_CPPv423GDALAutoCreateWarpedVRT12GDALDatasetHPKcPKc15GDALResampleAlgdPK15GDALWarpOptions) 
to EPSG:4326 and just take the extent of the warped VRT.


Le 03/05/2024 à 23:25, Simon Eves via gdal-dev a écrit :

I like it.

On Fri, May 3, 2024 at 2:24 PM Javier Jimenez Shaw 
 wrote:


Now I think I understand what you mean.
One effective way is to convert the four sides of the image. To
avoid the conversion of every pixel, you can start a partition
process. Then compare the difference between the transformed
central point and just the mid point between the extremes. If that
different is bigger than an certain threshold, keep subdividing
each side. With that you have a good approximation of the borders
of the image transformed, and then you can compute your bounding box.
The idea is not mine. Even Rouault mentioned it last year in
FOSS4G. IIRC, it is used by gdalwarp to not reproject every single
point; once the subdivision is enough, then it does linear
interpolation.

On Fri, 3 May 2024 at 22:18, Simon Eves  wrote:

Yes, but that's just the corners.

Consider, say, a raster that contains a satellite sweep which
is not axis-aligned but forms a "windshield wiper" shape in
lon/lat space. The bounding box of ALL the pixels is not just
the bounding box of the corners.

On Fri, May 3, 2024 at 1:09 PM Javier Jimenez Shaw
 wrote:

Maybe I don't understand your question, but in gdalinfo
you have the four corners as lat-lon

On Fri, 3 May 2024, 21:46 Simon Eves via gdal-dev,
 wrote:

So we are trying to optimize our raster import
process, and particularly the steps to derive the
final WGS84/4326 bounding box for a raster file or
tile thereof.

Obviously in the general case, the transform is from
integer pixel coordinate through the Affine
Transformation matrix and then whatever
OGRCoordinateTransformation is required to get to
WGS84/4326, and perhaps a GCP-based mesh
transformation too.

Currently we are deriving the bounding box by passing
all pixels of the four edges of the file/tile through
the full transform, except in the simple Affine-only
case where we just transform the four corners.

Is there any shortcut API provided by GDAL or PROJ to
allow the bounding box to be computed (or at least
safely over-estimated) in the general case? I'm
assuming that even a non-GCP  OGRCT could still be
non-linear such that just transforming the corners is
insufficient.

Thanks in advance,

Simon

-- 
Simon Eves

Senior Rendering Engineer
+1 (415) 902-1996
simon.e...@heavy.ai

<http://www.heavy.ai>

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.9.0beta2 available for testing

[Even Rouault via gdal-dev]

Hi,

The Docker images are now ready. Testing of the Ubuntu-based ones would 
be appreciated due to having switched to 24.04 since beta1:


- ghcr.io/osgeo/gdal:ubuntu-full-3.9.0beta2
- ghcr.io/osgeo/gdal:ubuntu-small-3.9.0beta2
- ghcr.io/osgeo/gdal:alpine-normal-3.9.0beta2
- ghcr.io/osgeo/gdal:alpine-small-3.9.0beta2

Even

Le 29/04/2024 à 14:26, Even Rouault via gdal-dev a écrit :

Hi,

I've prepared a beta2 of GDAL 3.9.0 as there have been a few 
non-trivial changes since beta1


The major ones are:

* pyproject.toml: use numpy>=2.0.0rc1 for python >=3.9 (#9751)
* Docker: update ubuntu-small and ubuntu-full to 24.04
* TileDB: migrate away from deprecated APIs, and bump minimum version 
to 2.15


Other bugfixes:

* CMake: add missing dependency to generate_gdal_version_h target when 
building {plugin}_core targets

* CMake: fix generated find_dependency
* Miramon: various fixes
* GDALSuggestedWarpOutput2(): make changes of PR #9336 honour 
SRC_METHOD=NO_GEOTRANSFORM. Fixes failure on rasterio 
tests/test_warpedvrt.py::test_transformer_options__width_height
* Update p-ranav/argparse to latest master; ogrinfo and ogr2ogr: 
update to use store_into(GIntBig&)
* GDALOpen(): avoid double '.' in error messages related to plugin 
drivers

* Add OSRIsDerivedProjected() / OGRSpatialReference::IsDerivedProjected()
* OGR_CT: use PROJJSON internally rather than in WKT:2019 (#9732)
* GDALArgumentParser: fix dealing with positional arguments that are 
negative numeric values
* GDALVectorInfo(): fix 3.9.0beta1 regression regarding passing layer 
names

* Internal libtiff: updated to latest upstream
* gdalinfo and ogrinfo -json: add newline character at end of JSON 
output (qgis#57266)
* OGRFeature::SetField(): add warnings when detecting some lossy 
conversions (refs #9792)
* Arrow/Parquet: fix writing empty point Z with 
GEOMETRY_ENCODING=GEOARROW_INTERLEAVED, and test spatial filtering of 
that encoding (qgis#57228)
* OGRLayer::WriteArrowBatch(): add tolerance for field type mismatches 
if int32/int64/real; Also add an option IF_FIELD_NOT_PRESERVED=ERROR 
to error out when lossy conversion occurs. (#9792)
* Parquet: fix ResetReading() implementation, when using the 
ParquetDataset API and when there's a single batch
* Parquet: fix opening single file Parquet datasets with the 
ParquetDataset API when using PARQUET:filename.parquet

* OGRCloneArrowArray(): add missing support for 'tss:' Arrow type

Docker images with 3.9.0beta2 are currently cooking.

Source snapshots at:

- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta2.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta2.tar.xz
- https://download.osgeo.org/gdal/3.9.0/gdal390beta2.zip

Autotest snapshots:

- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta2.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta2.zip

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Test problem After Merge

[Even Rouault via gdal-dev]

Andrew,

interestingly adding the debug traces I suggested to you to the GDAL CI 
MacOS 14 build witch no longer worked turns out to reveal the exact same 
issue as yours! (funny that I didn't even consider doing that myself ! 
interesting case where helping others actually is helping oneself :-))  
_gdal.cpython-312-darwin.so was also linking against libgdal.so.34 (GDAL 
3.8.5) instead of libgdal.so.35 (libgdal 3.10dev). The reason is that 
this CI configuration uses conda-forge to install dependencies, and it 
accidentally installed libgdal itself, instead of just its dependencies 
(it add a "conda install --only-deps libgdal libgdal-arrow-parquet" 
line, but I missed that libgdal *is* a dependency of 
libgdal-arrow-parquet... )


Looking at the compilation log, I see:

arm64-apple-darwin20.0.0-clang++ -bundle -undefined dynamic_lookup 
-Wl,-rpath,/Users/runner/miniconda3/envs/test/lib 
-L/Users/runner/miniconda3/envs/test/lib 
-Wl,-rpath,/Users/runner/miniconda3/envs/test/lib 
-L/Users/runner/miniconda3/envs/test/lib 
-Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs 
-Wl,-rpath,/Users/runner/miniconda3/envs/test/lib 
-L/Users/runner/miniconda3/envs/test/lib -ftree-vectorize -fPIC 
-fstack-protector-strong -O2 -pipe -isystem 
/Users/runner/miniconda3/envs/test/include -D_FORTIFY_SOURCE=2 -isystem 
/Users/runner/miniconda3/envs/test/include 
build/temp.macosx-11.0-arm64-cpython-312/extensions/gdal_wrap.o 
-L/Users/runner/work/gdal/gdal/build -lgdal -o 
build/lib.macosx-11.0-arm64-cpython-312/osgeo/_gdal.cpython-312-darwin.so


So something in the Python extension building logic (conda 
customizations?) adds this -L/Users/runner/miniconda3/envs/test/lib 
where the "libgdal" from conda-forge is found, which apparently takes 
precedence over the "-L/Users/runner/work/gdal/gdal/build -lgdal" 
towards the end of the line, which is the bit that links against the 
libgdal of the build tree.


For my CI case removing libgdal from the installed conda-forge packages 
fixed the issue, but it would be nice if we could avoid/detect the issue 
in the first place. If so, that would likely be some magic to add into 
swig/python/setup.py.in. It might be tricky to do so though. Like 
detecting -Larg in the linking options and looking for a 
libgdal.so/dylib in them... ? Yuck. Or perhaps do some postprocessing 
check that the resulting extension .so links against the expected libgdal?


Even

Le 30/04/2024 à 17:41, Andrew Bell a écrit :
On Tue, Apr 30, 2024 at 9:18 AM Even Rouault 
 wrote:


Even,

Also try in a Python interpreter "from osgeo import gdal" to see
if the exception is more verbose. If your GDAL lib links against
libraries that are not in the default library search path, GDAL
command line utilities might still be able to find them, but
loading libgdal through the Python bindings might fail (at least
that happens to me on Linux, cf
https://github.com/OSGeo/gdal/pull/9783)

Also try otool -L on libgdal.dylib and on
_gdal.cpython-312-darwin.so <http://gdal.cpython-312-darwin.so>

This was indeed the issue. I didn't think there was a version of GDAL 
in my environment, but it must have gotten in by error at some point. 
When the SWIG library _gdal...so was created, a dependency was created 
for the system-located library (version 34) rather than the one in the 
build (version 35). I'd have to do more research to figure out how to 
rectify this, if desired.


Thanks for your help. The python failure message wasn't too helpful on 
its own.


--
Andrew Bell
andrew.bell...@gmail.com


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Test problem After Merge

[Even Rouault via gdal-dev]

Andrew,

first, check that you can use the command line utilities like gdalinfo, 
so you may need to adjust your PATH, DYLD_LIBRARY_PATH and GDAL_DATA. 
The Bash script scripts/setdevenv.sh might help setting up stuff (not 
totally sure if it works on Mac though).


Also try in a Python interpreter "from osgeo import gdal" to see if the 
exception is more verbose. If your GDAL lib links against libraries that 
are not in the default library search path, GDAL command line utilities 
might still be able to find them, but loading libgdal through the Python 
bindings might fail (at least that happens to me on Linux, cf 
https://github.com/OSGeo/gdal/pull/9783)


Also try otool -L on libgdal.dylib and on _gdal.cpython-312-darwin.so

Actually your issue looks superficially like the one we have in our CI 
since a few days : https://github.com/OSGeo/gdal/issues/9723 / 
https://github.com/actions/setup-python/issues/849 . Python bindings on 
macos-14 arm64 github were working fine, and suddenly broke at runtime 
with an exception similar to you whereas nobody significant changed 
between working and non-working builds. It looked correlated to some 
update on the github action components, but perhaps that's something 
else. Do you know if there was an operating system update on your system 
recently?


Even

Le 30/04/2024 à 14:51, Andrew Bell via gdal-dev a écrit :

Hi all,

I removed all files from the build and started over, building using 
the same procedure that I used with the source from a couple of weeks 
ago without problem and now I get the following error:


(gdal) [master] $ pytest -v autotest/utilities/test_gdal_viewshed.py
ImportError while loading conftest 
'/Users/abell/gdal.2/build/autotest/conftest.py'.

autotest/conftest.py:9: in 
    from osgeo import gdal, ogr, osr
swig/python/osgeo/__init__.py:35: in 
    _gdal = swig_import_helper()
swig/python/osgeo/__init__.py:32: in swig_import_helper
    return importlib.import_module('_gdal')
E   ModuleNotFoundError: No module named '_gdal'


PYTHONPATH is set as follows:

(gdal) [master] $ env | grep PYTHONPATH
PYTHONPATH=/Users/abell/gdal.2/build/swig/python:


The osgeo directory referenced from PYTHONPATH contains the following:

(gdal) [master] $ ls -l /Users/abell/gdal.2/build/swig/python/osgeo
total 6136
-rw-r--r--  1 abell  staff    4758 Apr 30 08:00 __init__.py
drwxr-xr-x  3 abell  staff      96 Apr 30 08:01 __pycache__
-rwxr-xr-x  1 abell  staff  970248 Apr 30 08:00 
_gdal.cpython-312-darwin.so <http://gdal.cpython-312-darwin.so>
-rwxr-xr-x  1 abell  staff  136376 Apr 30 08:00 
_gdal_array.cpython-312-darwin.so 
<http://gdal_array.cpython-312-darwin.so>
-rwxr-xr-x  1 abell  staff   54352 Apr 30 08:00 
_gdalconst.cpython-312-darwin.so <http://gdalconst.cpython-312-darwin.so>
-rwxr-xr-x  1 abell  staff  104072 Apr 30 08:00 
_gnm.cpython-312-darwin.so <http://gnm.cpython-312-darwin.so>
-rwxr-xr-x  1 abell  staff  717640 Apr 30 08:00 
_ogr.cpython-312-darwin.so <http://ogr.cpython-312-darwin.so>
-rwxr-xr-x  1 abell  staff  416120 Apr 30 08:00 
_osr.cpython-312-darwin.so <http://osr.cpython-312-darwin.so>

-rw-r--r--  1 abell  staff  336675 Apr 30 08:00 gdal.py
-rw-r--r--  1 abell  staff   33746 Apr 30 08:00 gdal_array.py
-rw-r--r--  1 abell  staff   12517 Apr 30 08:00 gdalconst.py
-rw-r--r--  1 abell  staff     214 Apr 30 08:00 gdalnumeric.py
-rw-r--r--  1 abell  staff   14413 Apr 30 08:00 gnm.py
-rw-r--r--  1 abell  staff  221167 Apr 30 08:00 ogr.py
-rw-r--r--  1 abell  staff   84266 Apr 30 08:00 osr.py

I would have thought the file _gdal.cpython-312-darwin.so 
<http://gdal.cpython-312-darwin.so> would have been found and loaded, 
but my expertise in this is limited. I tried tracing with 
PYTHONVERBOSE set, but the only references to _gdal I saw were in the 
same error message quoted above.


Thanks for any help,

On Mon, Apr 29, 2024 at 5:37 PM Even Rouault 
 wrote:


Andrew,

Perhaps you're running updated GDAL Python bindings against a
libgdal that hasn't been rebuilt recently? OSRIsDerivedProjected()
has been added recently both in libgdal and the bindings


--
Andrew Bell
andrew.bell...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] another problem HDF5

[Even Rouault via gdal-dev]

Hi,

oh well, so this i actually a HDF5 file using netCDF conventions, but 
likely created with the HDF5 API itself. Per se, this isn't the issue, 
as you've figured it, we can convince the netCDF driver to open it. The 
lack of CRS comes from the fact that the land_mask_map variable has this 
attribute:


    land_mask_map:grid_mapping = "crs: grid_x crs: grid_y" ;

Which tries to point to the "crs" variable where the CRS is defined. In 
99% of the products, the value of grid_mapping would be just "crs", and 
life would be good.


Here they use the "second format" for grid_mapping, described at 
https://cfconventions.org/Data/cf-conventions/cf-conventions-1.11/cf-conventions.html#grid-mappings-and-projections


"In the second format, it is a blank-separated list of words 
":  [ …​] 
[: …​]", which identifies one 
or more grid mapping variables, and with each grid mapping associates 
one or more coordinatesVariables, i.e. coordinate variables or auxiliary 
coordinate variables."


That documentation isn't 100% crystal clear but the later example 5.10 
"British National Grid" gives an example:


temp:grid_mapping = "crsOSGB: x y crsWGS84: lat lon" ;

So you can point to several crs variables, but for each you need to 
define *all* the relevant axis. The way it is done in the product you 
point doesn't match that philosophy (if they wanted to use the second 
format that should rather set "crs: grid_x grid_y", but using the second 
format here is just asking for troubles), and GDAL doesn't manage to 
parse it.


You could tune netCDFDataset::SetProjectionFromVar() to recognize this 
weird value for grid_mapping in netCDFDataset::SetProjectionFromVar(), 
around line 3287, but I don't feel myself personally motivated to do 
that, if that product is just an outlier, so I leave it to you as an 
exercise if you wish :-). I guess just testing against the hardcoded 
"crs: grid_x crs: grid_y" value would be good enough to handle that outlier.


(Someone, super bored, should author a book about with horror stories 
with netCDF and HDF georeferencing. There's a lot of material. Although 
likely not to be a best seller)


Even

Le 30/04/2024 à 09:45, Michael Sumner via gdal-dev a écrit :

This time, with:

https://n5eil01u.ecs.nsidc.org/ATLAS/ATL20.004/2018.10.14/ATL20-01_20181001010615_00370101_004_01.h5

NETCDF gets the geotransform (from x_grid,y_grid which report in the 
metadata) but no crs:


  gdalinfo NETCDF:ATL20-02_20181001010615_00370101_004_01.h5 -sd 1 -nomd
Driver: netCDF/Network Common Data Format
Files: ATL20-02_20181001010615_00370101_004_01.h5
Size is 316, 332
Origin = (-395.000,435.000)
Pixel Size = (25000.000,-25000.000)
Corner Coordinates:
Upper Left  (-395.000, 435.000)
Lower Left  (-395.000,-395.000)
Upper Right ( 395.000, 435.000)
Lower Right ( 395.000,-395.000)
Center      (       0.000,  20.000)
Band 1 Block=500x500 Type=Float64, ColorInterp=Undefined
  Unit Type: degrees_north


HDF5 doesn't get geotransform or crs:

gdalinfo ATL20-02_20181001010615_00370101_004_01.h5 -sd 1 -nomd
Driver: HDF5Image/HDF5 Dataset
Files: ATL20-02_20181001010615_00370101_004_01.h5
Size is 316, 332
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  332.0)
Upper Right (  316.0,    0.0)
Lower Right (  316.0,  332.0)
Center      (  158.0,  166.0)
Band 1 Block=500x500 Type=Float32, ColorInterp=Undefined
  NoData Value=3.4028235e+38

This is unproblematically NSIDC 25km south, there's no mix of poles in 
this one. I honestly don't know what to expect from these files 
generally though I get my hopes up sometimes haha.


I'll have to keep notes and explore more deeply with native HDF5 tools.

Cheers, Mike




--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Re-post, user-defined

[Even Rouault via gdal-dev]

At first sight, I'd say that the ArcPy output is incorrect. The SRS 
definition of its output doesn't match what you want since it has


PARAMETER["Latitude of natural origin",20,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8801]],
PARAMETER["Longitude of natural origin",6378137,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8802]],
PARAMETER["False easting",5,
LENGTHUNIT["metre",1],
ID["EPSG",8806]],

The "Longitude of natural origin" is completely wrong, and should be 5 
given the PROJ.4 string you mentionned


And the the "False easting" should be 0.

The output of GDAL looks plausible.


Le 30/04/2024 à 00:00, Liyuneh Gebre a écrit :

Hi Even,
thank you for your response,
gdal version is: GDAL 3.6.4, released 2023/04/17
I got the defined target projection included in the link. yes, it's 
geocentric and for further exploration the custom project 
transformation file .gtf extension is also included.

https://u.pcloud.link/publink/show?code=kZj5wh0ZtFWXtI9UGjQiD6KXxn5A9hiWDpEX

On Mon, Apr 29, 2024 at 4:23 PM Even Rouault 
 wrote:


Hi,

it is difficult to help you with just the elements you've provided.

What could be helpful is:

- the output of gdalinfo on the input file

- the output of gdalinfo on the output file generate by ArcPy

- how you define the target projection in ArcPy

- the output of gdalinfo on the output file generate by GDAL

- links to all the above files

- the GDAL version you use


Totally blind guess: try to add "+towgs84=0,0,0 " in your -t_srs
PROJ string, to ask PROJ to consider that your target CRS datum
has the same center and axis orientation as WGS84 (and thus go to
geographic <--> geocentric conversion when changing datums).
Otherwise without datum shifts, PROJ will consider that
longitude,latitude coordinates of the geographic CRS underlying
the source and target CRS is the same.

Even

Le 29/04/2024 à 14:57, Liyuneh Gebre via gdal-dev a écrit :

Dear all,
I am requesting again if you kind have some responds or
direction, thank you
I have difficulties in accurately reprojecting a raster file to a
user-defined projection.
I used the following commands in gdal while the output is not
similar to arcpy results.  kindly offer your guidance to achieve
the same result.  The arcpy uses custom transformation and then
reprojection.
gdalwarp -t_srs '+proj=laea +a=6378137.0 +f=0.0 +pm=0 +x_0=0.0
+y_0=0.0 +lon_0=20.0 +lat_0=5.0 +units=m +axis=enu +no_defs' 
-dstalpha -wo SOURCE_EXTRA=1000 -r nearest rain_in.bil
rain_projected.bil


/*Liyuneh Gebre*//, Associate Researcher///
/Ethiopian Institute of Agricultural Research,EIAR
/
/Cell Phone:+251 911858155
/
/linkedIn: https://www.linkedin.com/in/liyuneh-gebre-b842b440/
/
//skype:liyenew_1//
/Addis Ababa, Ethioipa/

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


-- 
http://www.spatialys.com

My software is free, but my time generally not.



--
/*Liyuneh Gebre*//, Associate Researcher///
/Ethiopian Institute of Agricultural Research,EIAR
/
/Cell Phone:+251 911858155
/
/linkedIn: https://www.linkedin.com/in/liyuneh-gebre-b842b440/
/
//skype:liyenew_1//
/Addis Ababa, Ethioipa/


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Test problem After Merge

[Even Rouault via gdal-dev]

Andrew,

Perhaps you're running updated GDAL Python bindings against a libgdal 
that hasn't been rebuilt recently? OSRIsDerivedProjected() has been 
added recently both in libgdal and the bindings


Even

Le 29/04/2024 à 23:27, Andrew Bell via gdal-dev a écrit :

Hi,

I just merged master into my branch and I'm now getting an error when 
trying to run a test. If someone might know what happened, I'd 
appreciate a tip.


(gdal) [master] $ pytest -v autotest/utilities/test_gdal_viewshed.py
ImportError while loading conftest 
'/Users/abell/gdal.2/build/autotest/conftest.py'.

autotest/conftest.py:9: in 
    from osgeo import gdal, ogr, osr
swig/python/osgeo/gdal.py:3942: in 
    from . import ogr
swig/python/osgeo/ogr.py:608: in 
    from . import osr
swig/python/osgeo/osr.py:13: in 
    from . import _osr
E   ImportError: 
dlopen(/Users/abell/gdal.2/build/swig/python/osgeo/_osr.cpython-312-darwin.so 
, 0x0002): symbol not found in flat 
namespace '_OSRIsDerivedProjected'


--
Andrew Bell
andrew.bell...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] HDF5 and geolocation arrays

[Even Rouault via gdal-dev]

Michael,

actually support for those products was already there at 95% since they 
use the HDF5EOS GRID formalism which we support since 3.7. But that 
particular type of products has an oddity. They have an empty 
GROUP=Dimension within the GROUP=GridStructure in the HDF5EOS metadata, 
which our parser didn't like as we use the information in this group to 
map dimension names to dimension index and size. Fixed/worked around in 
https://github.com/OSGeo/gdal/pull/9807 . Hard to tell if those products 
are in fault given that the HDF5EOS specification at 
https://www.earthdata.nasa.gov/s3fs-public/imported/ESDS-RFC-008-v1.1.pdf 
, appendix A, is extremely minimum regarding the HDF5EOS metadata itself 
(for once, I complain for a spec to be too minimal !), and the example 
they give for grids doesn't make sense to me (the declared dimension 
names and sizes have nothing to do with the ones use in the grid)


I now get:

$ gdalinfo 
HDF5:"AMSR_U2_L3_SeaIce12km_B04_20120702.he5"://HDFEOS/GRIDS/SpPolarGrid12km/Data_Fields/SI_12km_SH_18H_ASC

Driver: HDF5Image/HDF5 Dataset
Files: AMSR_U2_L3_SeaIce12km_B04_20120702.he5
Size is 632, 664
Coordinate System is:
PROJCRS["unnamed",
    BASEGEOGCRS["Unknown datum based upon the custom spheroid",
    DATUM["Not specified (based on custom spheroid)",
    ELLIPSOID["Custom spheroid",6378273,0,
    LENGTHUNIT["metre",1,
    ID["EPSG",9001,
    PRIMEM["Greenwich",0,
    ANGLEUNIT["degree",0.0174532925199433,
    ID["EPSG",9122,
    CONVERSION["Polar Stereographic (variant B)",
    METHOD["Polar Stereographic (variant B)",
    ID["EPSG",9829]],
    PARAMETER["Latitude of standard parallel",-70,
    ANGLEUNIT["degree",0.0174532925199433],
    ID["EPSG",8832]],
    PARAMETER["Longitude of origin",0,
    ANGLEUNIT["degree",0.0174532925199433],
    ID["EPSG",8833]],
    PARAMETER["False easting",0,
    LENGTHUNIT["metre",1],
    ID["EPSG",8806]],
    PARAMETER["False northing",0,
    LENGTHUNIT["metre",1],
    ID["EPSG",8807]]],
    CS[Cartesian,2],
    AXIS["(E)",north,
    MERIDIAN[90,
    ANGLEUNIT["degree",0.0174532925199433,
    ID["EPSG",9122]]],
    ORDER[1],
    LENGTHUNIT["Meter",1]],
    AXIS["(N)",north,
    MERIDIAN[0,
    ANGLEUNIT["degree",0.0174532925199433,
    ID["EPSG",9122]]],
    ORDER[2],
    LENGTHUNIT["Meter",1]]]
Data axis to CRS axis mapping: 1,2
Origin = (-395.000,435.000)
Pixel Size = (12500.000,-12500.000)
Metadata:
  Conventions=CF-1.6
  HDFEOS_INFORMATION_HDFEOSVersion=HDFEOS_5.1.15
  history=This version of the Sea Ice processing code contains updates 
provided by the science team on September 16, 2019. For details on these 
updates, see the release notes provided in the DAP.

  institution=NASA's AMSR Science Investigator-led Processing System (SIPS)
  references=Please cite these data as: Markus, T., J. C. Comiso, and 
W. N. Meier. 2018. AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness 
Temperatures, Sea Ice Concentration, Motion & Snow Depth Polar Grids, 
Version 1. [Indicate subset used]. Boulder, Colorado USA. NASA National 
Snow and Ice Data Center Distributed Active Archive Center. doi: 
https://doi.org/10.5067/RA1MIJOYPK3P.

  source=satellite observation
  title=AMSR-E/AMSR2 Unified L3 Daily 12.5 km Brightness Temperatures, 
Sea Ice Concentration, Motion & Snow Depth Polar Grids

Corner Coordinates:
Upper Left  (-395.000, 435.000) ( 42d14'27.21"W, 39d11'27.54"S)
Lower Left  (-395.000,-395.000) (135d 0' 0.00"W, 41d23'59.41"S)
Upper Right ( 395.000, 435.000) ( 42d14'27.21"E, 39d11'27.54"S)
Lower Right ( 395.000,-395.000) (135d 0' 0.00"E, 41d23'59.41"S)
Center  (   0.000,  20.000) (  0d 0' 0.01"E, 88d 8'51.76"S)
Band 1 Block=632x1 Type=Int32, ColorInterp=Undefined
  NoData Value=0
  Metadata:
    add_offset=0
    coordinates=lon lat
    long_name=18.7 GHz horizontal daily average ascending Tbs
    packing_convention=netCDF
    packing_convention_description=unpacked = scale_factor x packed + 
add_offset

    scale_factor=0.1
    standard_name=brightness_temperature
    units=degree_kelvin
    _FillValue=0

Even

Le 29/04/2024 à 21:10, Michael Sumner via gdal-dev a écrit :
This HDF5 (requires earthdata credentials your "Authorization: Bearer 
" in GDAL_HTTP_HEADERS, or equiv) presents without geolocation 
arrays.


gdalinfo 
"/vsicurl/https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5; 
-sd 26

Driver: HDF5Image/HDF5 Dataset
Files: 
/vsicurl/https://n5eil01u.ecs.nsidc.org/AMSA/AU_SI12.001/2012.07.02/AMSR_U2_L3_SeaIce12km_B04_20120702.he5

Size is 608, 896
Metadata:
  Conventions=CF-1.6
  

Re: [gdal-dev] Inquire mapped column names for conversion to shapefile

[Even Rouault via gdal-dev]

Hi,

Something like:


from osgeo import ogr
original_field_names = [ "toolongforshapefile1", "toolongforshapefile2"]
map_to_shp = {}
tmpfilename = "/vsimem/temp.shp"
ds = ogr.GetDriverByName("ESRI Shapefile").CreateDataSource(tmpfilename)
lyr = ds.CreateLayer("temp")
lyr_defn = lyr.GetLayerDefn()
for name in original_field_names:
    lyr.CreateField(ogr.FieldDefn(name))
    shp_field_name = lyr_defn.GetFieldDefn(lyr_defn.GetFieldCount() - 
1).GetNameRef()

    map_to_shp[name] = shp_field_name
ds = None
ogr.GetDriverByName("ESRI Shapefile").DeleteDataSource(tmpfilename)
print(map_to_shp)

Even

Le 29/04/2024 à 16:23, Jan Heckman via gdal-dev a écrit :

Hi everyone,
I want to make a map of original (let's say postgresql/postgis) column 
names to those used in a shapefile after conversion; assuming that at 
least some column names in postgis will be over 10 characters in 
length, and shortening may produce conflicts. Basically I know how 
this is done, appending _, but I hope that there is preferably a 
python method available in Qgis python to get the list of column names 
used in the shapefile.

Any help or pointers are welcome!
Thanks,
Jan

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Re-post, user-defined

[Even Rouault via gdal-dev]

Hi,

it is difficult to help you with just the elements you've provided.

What could be helpful is:

- the output of gdalinfo on the input file

- the output of gdalinfo on the output file generate by ArcPy

- how you define the target projection in ArcPy

- the output of gdalinfo on the output file generate by GDAL

- links to all the above files

- the GDAL version you use


Totally blind guess: try to add "+towgs84=0,0,0 " in your -t_srs PROJ 
string, to ask PROJ to consider that your target CRS datum has the same 
center and axis orientation as WGS84 (and thus go to geographic <--> 
geocentric conversion when changing datums). Otherwise without datum 
shifts, PROJ will consider that longitude,latitude coordinates of the 
geographic CRS underlying the source and target CRS is the same.


Even

Le 29/04/2024 à 14:57, Liyuneh Gebre via gdal-dev a écrit :

Dear all,
I am requesting again if you kind have some responds or direction, 
thank you
I have difficulties in accurately reprojecting a raster file to a 
user-defined projection.
I used the following commands in gdal while the output is not similar 
to arcpy results.  kindly offer your guidance to achieve the same 
result.  The arcpy uses custom transformation and then reprojection.
gdalwarp -t_srs '+proj=laea +a=6378137.0 +f=0.0 +pm=0 +x_0=0.0 
+y_0=0.0 +lon_0=20.0 +lat_0=5.0 +units=m +axis=enu +no_defs'  
-dstalpha -wo SOURCE_EXTRA=1000 -r nearest rain_in.bil rain_projected.bil



/*Liyuneh Gebre*//, Associate Researcher///
/Ethiopian Institute of Agricultural Research,EIAR
/
/Cell Phone:+251 911858155
/
/linkedIn: https://www.linkedin.com/in/liyuneh-gebre-b842b440/
/
//skype:liyenew_1//
/Addis Ababa, Ethioipa/

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.9.0beta2 available for testing

[Even Rouault via gdal-dev]

Hi,

I've prepared a beta2 of GDAL 3.9.0 as there have been a few non-trivial 
changes since beta1


The major ones are:

* pyproject.toml: use numpy>=2.0.0rc1 for python >=3.9 (#9751)
* Docker: update ubuntu-small and ubuntu-full to 24.04
* TileDB: migrate away from deprecated APIs, and bump minimum version to 
2.15


Other bugfixes:

* CMake: add missing dependency to generate_gdal_version_h target when 
building {plugin}_core targets

* CMake: fix generated find_dependency
* Miramon: various fixes
* GDALSuggestedWarpOutput2(): make changes of PR #9336 honour 
SRC_METHOD=NO_GEOTRANSFORM. Fixes failure on rasterio 
tests/test_warpedvrt.py::test_transformer_options__width_height
* Update p-ranav/argparse to latest master; ogrinfo and ogr2ogr: update 
to use store_into(GIntBig&)

* GDALOpen(): avoid double '.' in error messages related to plugin drivers
* Add OSRIsDerivedProjected() / OGRSpatialReference::IsDerivedProjected()
* OGR_CT: use PROJJSON internally rather than in WKT:2019 (#9732)
* GDALArgumentParser: fix dealing with positional arguments that are 
negative numeric values

* GDALVectorInfo(): fix 3.9.0beta1 regression regarding passing layer names
* Internal libtiff: updated to latest upstream
* gdalinfo and ogrinfo -json: add newline character at end of JSON 
output (qgis#57266)
* OGRFeature::SetField(): add warnings when detecting some lossy 
conversions (refs #9792)
* Arrow/Parquet: fix writing empty point Z with 
GEOMETRY_ENCODING=GEOARROW_INTERLEAVED, and test spatial filtering of 
that encoding (qgis#57228)
* OGRLayer::WriteArrowBatch(): add tolerance for field type mismatches 
if int32/int64/real; Also add an option IF_FIELD_NOT_PRESERVED=ERROR to 
error out when lossy conversion occurs. (#9792)
* Parquet: fix ResetReading() implementation, when using the 
ParquetDataset API and when there's a single batch
* Parquet: fix opening single file Parquet datasets with the 
ParquetDataset API when using PARQUET:filename.parquet

* OGRCloneArrowArray(): add missing support for 'tss:' Arrow type

Docker images with 3.9.0beta2 are currently cooking.

Source snapshots at:

- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta2.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta2.tar.xz
- https://download.osgeo.org/gdal/3.9.0/gdal390beta2.zip

Autotest snapshots:

- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta2.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta2.zip

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] lost python api doc?

[Even Rouault via gdal-dev]

Michael,

The Python doc got reorganized one month ago 
(https://github.com/OSGeo/gdal/pull/9575), so perhaps Google hasn't 
updated yet the new references


If you look for gdal.warp on gdal.org directly : 
https://gdal.org/search.html?q=gdal.warp_keywords=yes=default


that leads to https://gdal.org/api/python/utilities.html#osgeo.gdal.Warp

Even

Le 29/04/2024 à 00:52, Michael Sumner via gdal-dev a écrit :
I'm confused about how to browse the python api docs, I used to just 
web search "osgeo.gdal Warp", and then scan down the first result page 
to find "Warp(" and I had the documentation I needed.


Where is that now? (Why can't I find it, sorry - grepping the sources 
for "Warp(" only results in RFC 59 references, so I think this might 
be a real problem).


If I'm just navigating the API docs incorrectly I'd appreciate a 
walkthrough to find the signature and param descripts for "Warp()". 
Thanks!


Cheers, Mike


--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] don't search side car files on s3 with gdalinfo

[Even Rouault via gdal-dev]

yes, GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR is the trick to both disable 
directory listing and preventing individual file probing.


Javier, do you have GDAL_DISABLE_READDIR_ON_OPEN=YES also set? Otherwise 
if GDAL_DISABLE_READDIR_ON_OPEN is not set, GDAL will normally issue a 
S3 directory listing call, which should prevent later individual file 
probing (if we have the list of files in the directory, we just need to 
check the list, and don't need to do individual open()/GET attempts). 
But that director listing is sometimes still undesirable if doing it in 
a folder with a large number of files hence 
GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR. Or maybe you don't have 
directory listing permission in that folder/bucket?


Related to that, Howard filed https://github.com/OSGeo/gdal/issues/9443 
recently. but it is not that simple to find a different default behavior 
that doesn't result in functional regressions.


I was wondering if a solution, restricted to COG, couldn't be to put a 
hint in the so-called "header ghost area" 
(https://gdal.org/drivers/raster/cog.html#header-ghost-area), like 
SIDECAR_FILES=NO, that could be used by the GTiff driver. Not sure if it 
is the right way to address the issue however (and would work only for 
new files of course, and probably only on explicit user choice at file 
creation, because there could be scenarios where a user would want to 
provide side car files)


Le 27/04/2024 à 13:12, Michael Smith via gdal-dev a écrit :


Javier,

You can control with the GDAL_DISABLE_READDIR_ON_OPEN configuration 
parameter (https://gdal.org/user/configoptions.html). Typically, I set 
GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR which will disable this.


Mike

--

Michael Smith

Remote Sensing/GIS Center

US Army Corps of Engineers

*From: *gdal-dev  on behalf of 
Javier Jimenez Shaw via gdal-dev 

*Reply-To: *Javier Jimenez Shaw 
*Date: *Saturday, April 27, 2024 at 7:01 AM
*To: *gdal dev 
*Subject: *[gdal-dev] don't search side car files on s3 with gdalinfo

Hi

I am accessing a big file in S3. All works fine (when I learned how to 
set the authentication environment variables), but gdalinfo is 
checking for a lot of files that do not exist. I know there is only a 
GeoTIFF - COG (no side car files)


However gdalinfo is trying to find several side car files, that are 
not there. S3 returns a 403. It takes its time to search for all those 
files.


Is there a way to tell to gdalinfo that it should not check for those 
files?


Bonus: QGIS is also taking time to open it (just using 'qgis' instead 
of 'gdalinfo' in the command line). I think it is for the same reason. 
It there a hack there as well?


Thanks.

$ AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... 
AWS_SESSION_TOKEN=... gdalinfo /vsis3/blah/folder/ortho.tiff

Driver: GTiff/GeoTIFF
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.tiff.aux.xml: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.aux: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.AUX: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.tiff.aux: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.tiff.AUX: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.tiff.ovr: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.tiff.OVR: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.tiff.msk: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.tiff.MSK: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.XML: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.xml: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.IMD: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.imd: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.RPB: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.rpb: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.PVL: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho.pvl: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho_rpc.txt: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho_RPC.TXT: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho_metadata.txt: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho_METADATA.TXT: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho_MTL.txt: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/ortho_MTL.TXT: 403
Warning 1: HTTP response code on 
https://blah.s3.amazonaws.com/folder/METADATA.DIM: 403
Warning 1: HTTP response code on 

Re: [gdal-dev] Unable to open KMZ datasource using libkml

[Even Rouault via gdal-dev]

Hi,

you can try something kike ogrinfo /vsizip/your.kmz/your.kml

Even

Le 25/04/2024 à 14:32, Johannes Paul via gdal-dev a écrit :

Hello,
using GDAL 3.8.3 compiled with libkml, I'm unable to read a specific 
KMZ file, using ogrinfo (error "unable to open datasource ...")
However if I extract the .kml index file for the kmz archive, GDAL can 
provide ogrinfo on this .kml index file.
I noticed from the LIBKML driver documentation that GDAL will default 
to look for a /doc.kml/ index file which is not the case here, so I 
tried to rename my index file and I also tried to use 
/--config LIBKML_USE_DOC.KML=NO/ without success.
Any idea to get GDAL to read the kmz file without unarchiving the kml 
index ?

Thanks
Johannes

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Bumping TileDB minimum from 2.7 to 2.15 for GDAL 3.9?

[Even Rouault via gdal-dev]

Le 25/04/2024 à 00:39, Andrew C Aitchison via gdal-dev a écrit :


On Wed, 24 Apr 2024, Even Rouault via gdal-dev wrote:

A future TileDB version will remove various deprecated API that the 
GDAL TileDB driver currently uses. 
https://github.com/OSGeo/gdal/pull/9725 migrates away from those 
deprecated APIs, but that causes the minimum requirement from TileDB 
to go from 2.7 to 2.15. It would probably be wise to backport this 
cleanup in the 3.9 branch, so that it doesn't cause later packaging 
issues, typically with conda-forge builds as soon as they will 
package the future TileDB version removing the deprecated APIs, which 
might occur during the GDAL 3.9.x life cycle. Does anyone see an 
issue in doing this bump? The few distributions I'm aware of shipping 
TileDB meet the >= 2.15 requirement: Conda-forge already ships TileDB 
2.22, Alpine Linux is a 2.17.4.


https://docs.tiledb.com/main/how-to/installation/building-from-source
suggests that TileDB requires a C++20 compiler.

Is that an issue ?


I don't think so. For now, the public C++ API of TileDB even of the 
latest versions is C++17 compatible.  And to build TileDB itself, 
digging into their CMakeLists.txt, I see that up to 2.16, it used to be 
C++17 compatible, so it should be possible to have a GDAL build against 
TileDB with only a C++17 compiler by sticking to TileDB 2.15 or 2.16. 
The requirement for C++20 to build TileDB has started with 2.17.0.


On a somewhat related note, I should not that the upcoming version of 
Poppler (2024.05) requires a C++20 compiler, including to include its 
headers from GDAL (as we use a somewhat semi-public/semi-private C++ 
API, not much care is done on it to be usable). So some of our 
dependencies might require C++20 in their latest versions. And before we 
switched GDAL to require C++17 we already had to force C++17 for some 
drivers like TileDB, Poppler, PDFIUM.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Bumping TileDB minimum from 2.7 to 2.15 for GDAL 3.9?

[Even Rouault via gdal-dev]

Hi,

A future TileDB version will remove various deprecated API that the GDAL 
TileDB driver currently uses. https://github.com/OSGeo/gdal/pull/9725 
migrates away from those deprecated APIs, but that causes the minimum 
requirement from TileDB to go from 2.7 to 2.15. It would probably be 
wise to backport this cleanup in the 3.9 branch, so that it doesn't 
cause later packaging issues, typically with conda-forge builds as soon 
as they will package the future TileDB version removing the deprecated 
APIs, which might occur during the GDAL 3.9.x life cycle. Does anyone 
see an issue in doing this bump? The few distributions I'm aware of 
shipping TileDB meet the >= 2.15 requirement: Conda-forge already ships 
TileDB 2.22, Alpine Linux is a 2.17.4.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Reading interpolated values on DSM

[Even Rouault via gdal-dev]

Le 24/04/2024 à 15:00, Michael Sumner a écrit :
Or a grouping function that returned the cell index for neighbours and 
weighting that are involved in whatever calculation summary is wanted.
That doesn't seem super user friendly, as users would still be left to 
do the raster value extraction and applying the weights, taking into 
account nodata, etc. Not trivial. What is the advantage of this compared 
to returning the interpolated value? The only one I see is to 
potentially save a bit of computation if you need to interpolate values 
at the same location in multiple bands, but the performance gain would 
probably be marginal (or if not, then a variant of the function dealing 
with multiple bands could be offered)


Maybe the warper could return this as a starting point rather than 
doing the "task at hand". ?
The warper code has indeed a "FilterFuncType 
GWKGetFilterFunc(GDALResampleAlg eResampleAlg)" method that returns a 
function that returns interpolation weights and int 
GWKGetFilterRadius(GDALResampleAlg eResampleAlg). The code in 
GDALRPCGetDEMHeight() has an interesting logic where it caches a window 
of interest around the first queried pixel so that subsequent queries in 
the neighbouroud can be honoured without going to RasterIO(). This 
substantially improves performance in the RPC case, in particular during 
reverse transformation where you use an iterative method and thus may 
need a lot of DEM extraction to compute a single point.




On Wed, Apr 24, 2024 at 8:51 PM Even Rouault via gdal-dev 
 wrote:


Hi,

I guess this discussion, and past similar ones, calls for an
enhancement. A new API function, like CPLErr
GDALRasterInterpolateAtPoint(GDALRasterBandH, double dfPixel,
double dfLocation, GDALRIOResampleAlg eInterpolation, double
*pdfValue), that could be used by a new mode of gdallocationinfo.
The GDALRPCGetDEMHeight() function in alg/gdal_rpc.cpp is a
plausible candidate implementation for bilinear and bicubic (we
could potentially restrict to that at the moment).

Even

Le 24/04/2024 à 10:33, Javier Jimenez Shaw via gdal-dev a écrit :

Hi

I would like to read in QGIS or GDAL an interpolated value in a
DSM (well, actually it is a geoid model, but it is the same
behaviour). See that I do not want the pixel value, but the
linear interpolation among the neighbour pixels, assuming that
the pixel value is in the center of the pixel.
For instance, this file
https://www.isgeoid.polimi.it/Geoid/Asia/Japan/japan2000_g.html

Is there any way to get it (without implementing the
interpolation myself)?
If I understood correctly gdallocationinfo is not interpolating,
just giving the pixel value.

Thanks

.___ ._ ..._ .. . ._.  .___ .. __ . _. . __..  ...  ._ .__

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


-- 
http://www.spatialys.com

My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev



--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Reading interpolated values on DSM

[Even Rouault via gdal-dev]

Hi,

I guess this discussion, and past similar ones, calls for an 
enhancement. A new API function, like CPLErr 
GDALRasterInterpolateAtPoint(GDALRasterBandH, double dfPixel, double 
dfLocation, GDALRIOResampleAlg eInterpolation, double *pdfValue), that 
could be used by a new mode of gdallocationinfo. The 
GDALRPCGetDEMHeight() function in alg/gdal_rpc.cpp is a plausible 
candidate implementation for bilinear and bicubic (we could potentially 
restrict to that at the moment).


Even

Le 24/04/2024 à 10:33, Javier Jimenez Shaw via gdal-dev a écrit :

Hi

I would like to read in QGIS or GDAL an interpolated value in a DSM 
(well, actually it is a geoid model, but it is the same behaviour). 
See that I do not want the pixel value, but the linear interpolation 
among the neighbour pixels, assuming that the pixel value is in the 
center of the pixel.

For instance, this file
https://www.isgeoid.polimi.it/Geoid/Asia/Japan/japan2000_g.html

Is there any way to get it (without implementing the interpolation 
myself)?
If I understood correctly gdallocationinfo is not interpolating, just 
giving the pixel value.


Thanks

.___ ._ ..._ .. . ._.  .___ .. __ . _. . __..  ...  ._ .__

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.9.0beta1 available for testing

[Even Rouault via gdal-dev]

The Docker images are now available:

ghcr.io/osgeo/gdal:alpine-small-3.9.0beta1
ghcr.io/osgeo/gdal:ubuntu-small-3.9.0beta1
ghcr.io/osgeo/gdal:alpine-normal-3.9.0beta1
ghcr.io/osgeo/gdal:ubuntu-full-3.9.0beta1

Le 22/04/2024 à 14:12, Even Rouault via gdal-dev a écrit :

Hi,

I've prepared a beta1 of GDAL 3.9.0 to get feedback from early testers.

The NEWS file is here:

  https://github.com/OSGeo/gdal/blob/v3.9.0beta1/NEWS.md

For packagers, 
https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html 
may make it more attractive to build some drivers that typically have 
heavy dependencies as plugins, installable in separate packages, due 
to the load time penalty having being improved. It is let to the 
appreciation of packagers to decide which drivers are worth building 
as plugins installable in a separate package.
You may also pass CMake options 
([GDAL/OGR]_DRIVER__PLUGIN_INSTALLATION_MESSAGE=xxx) so 
user get a hint of which package they need to install when GDAL 
detects that a file may be opened by a plugin which is not available 
on the file system but known to have been built as a plugin. Cf 
https://gdal.org/development/building_from_source.html#deferred-loaded-plugins 
for more details.


For end-users, the following utilities have been updated to use a new 
argument parsing framework: gdaladdo, gdalinfo, gdal_translate, 
gdalwarp, gdal_grid, gdal_viewshed, gdalbuildvrt, nearblack, ogrinfo, 
ogr2ogr, sozip.
This helps detecting errors such as specifying twice an argument that 
should be specified once (where generally the last instance was the 
one used). While we have tried to retain backwards compatibility for 
nominal use cases, obviously if you have scripts that accidentally 
specify an argument several times whereas it should be specified at 
most once, they will have to be corrected.


Docker images with 3.9.0beta1 are currently cooking.

master is now marked as 3.10.0dev, and the release/3.9 branch has been 
created. All bugfixes for 3.9 should be backported into it.


Source snapshots at:

- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.xz
- https://download.osgeo.org/gdal/3.9.0/gdal380beta1.zip

Autotest snapshots:

- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.zip

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.9.0beta1 available for testing

[Even Rouault via gdal-dev]

Holger,

thanks for the report. This should be fixed per 
https://github.com/OSGeo/gdal/commit/fba559b5bd8d33aac215681df4f6a613517a6c43 
which I've backported to the release/3.9 branch


A workaround is to explicitly run "cmake --target 
generate_gdal_version_h" before building all other targets


Even

Le 22/04/2024 à 23:28, Holger Jaekel via gdal-dev a écrit :

Hi Even,

Am 22.04.24 um 14:12 schrieb Even Rouault via gdal-dev:

I've prepared a beta1 of GDAL 3.9.0 to get feedback from early testers.


I tried to build on Alpine Linux Edge and got the following error:


/builds/hjaekel/aports/community/gdal/src/gdal-3.9.0beta1/ogr/ogr_core.h:38:10: 


fatal error: gdal_version.h: No such file or directory
   38 | #include "gdal_version.h"
  |  ^~~~
compilation terminated.


gdal_version.h seems not to be generated to build/gcore.

You can find the full build logs here:
https://gitlab.alpinelinux.org/hjaekel/aports/-/pipelines/228841

Holger

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.9.0beta1 available for testing

[Even Rouault via gdal-dev]

It should have read https://download.osgeo.org/gdal/3.9.0/gdal390beta1.zip

Le 22/04/2024 à 21:29, Abel Pau via gdal-dev a écrit :

Hi,
I only want to advice that 
https://download.osgeo.org/gdal/3.9.0/gdal380beta1.zip is not found.
Thanks.

-Mensaje original-
De: gdal-dev  En nombre de Even Rouault via 
gdal-dev
Enviado el: dilluns, 22 d’abril de 2024 14:13
Para: gdal-dev@lists.osgeo.org
Asunto: [gdal-dev] GDAL 3.9.0beta1 available for testing

Hi,

I've prepared a beta1 of GDAL 3.9.0 to get feedback from early testers.

The NEWS file is here:

    https://github.com/OSGeo/gdal/blob/v3.9.0beta1/NEWS.md

For packagers,
https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html may make it 
more attractive to build some drivers that typically have heavy dependencies as 
plugins, installable in separate packages, due to the load time penalty having 
being improved. It is let to the appreciation of packagers to decide which 
drivers are worth building as plugins installable in a separate package.
You may also pass CMake options
([GDAL/OGR]_DRIVER__PLUGIN_INSTALLATION_MESSAGE=xxx) so user get a 
hint of which package they need to install when GDAL detects that a file may be 
opened by a plugin which is not available on the file system but known to have been 
built as a plugin. Cf 
https://gdal.org/development/building_from_source.html#deferred-loaded-plugins
for more details.

For end-users, the following utilities have been updated to use a new argument 
parsing framework: gdaladdo, gdalinfo, gdal_translate, gdalwarp, gdal_grid, 
gdal_viewshed, gdalbuildvrt, nearblack, ogrinfo, ogr2ogr, sozip.
This helps detecting errors such as specifying twice an argument that should be 
specified once (where generally the last instance was the one used). While we 
have tried to retain backwards compatibility for nominal use cases, obviously 
if you have scripts that accidentally specify an argument several times whereas 
it should be specified at most once, they will have to be corrected.

Docker images with 3.9.0beta1 are currently cooking.

master is now marked as 3.10.0dev, and the release/3.9 branch has been created. 
All bugfixes for 3.9 should be backported into it.

Source snapshots at:

- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.xz
- https://download.osgeo.org/gdal/3.9.0/gdal380beta1.zip

Autotest snapshots:

- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.zip

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.9.0beta1 available for testing

[Even Rouault via gdal-dev]

Sean,

Rasterio's test suite has 4 errors with GDAL 3.9.0beta1.

Metadata output of gdalinfo has changed.


ok, I've run locally rasterio tests and I see gdalinfo now outputs:

Metadata:
  a=1
  b=2
  AREA_OR_POINT=Area

whereas the test expects

Metadata:
  a=1
  AREA_OR_POINT=Area
  b=2

Order of metadata items shouldn't be relied upon. I believe we have 
changed something internally (likely changing from char** to 
CPLStringList that does sorting) that might indeed have affected it. You 
could also potentially use "gdalinfo -json" output to get something 
easier to parse. But don't you have a rasterio API to get GDAL metadata 
that would avoid using gdalinfo at all?



As soon as there are docker images I'll look more closely at this.

ghcr.io/osgeo/gdal:ubuntu-full-3.9.0beta1 is already available


Writing a mask to a Rasterio RGB.byte.tif dataset no longer creates a 
RGB.byte.tif.msk file: 
https://github.com/rasterio/rasterio/actions/runs/8786444142/job/24109342188#step:8:2231.


Yes, this is intended. Due to this change:

GTiff driver:
 * change default value of GDAL_TIFF_INTERNAL_MASK config option to YES

You might run the test under GDAL_TIFF_INTERNAL_MASK=NO if you want to 
test external mask.




The text of an error message which was "tests/data/corrupt.tif, band 
1: IReadBlock failed at X offset 1, Y offset 1: TIFFReadEncodedTile() 
failed." in 3.8 has changed to "corrupt.tif, band 1: IReadBlock failed 
at X offset 1, Y offset 1: TIFFReadEncodedTile() failed." This is not 
a big deal, we can adjust Rasterio's test.
Yes, related to 
https://github.com/OSGeo/gdal/commit/cb5161d907e8b4fa5cb68aaf891c1a258d6475b0


Behavior of SRC_METHOD/DST_METHOD=NO_GEOTRANSFORM for warp options 
seems to have changed: 
https://github.com/rasterio/rasterio/actions/runs/8786444142/job/24109342188#step:8:2287. 
I'm not entirely sure what the intent is in Rasterio's test, will dig 
into that. I just wanted to say that I see a change in GDAL.


ok, this is related to https://github.com/OSGeo/gdal/pull/9336 , which 
indeed wasn't honoring the SRC_METHOD=NO_GEOTRANSFORM and was using 
directly the geotransform from the source dataset.


Fix queued in https://github.com/OSGeo/gdal/pull/9724

(that said, tests reprojecting from 4326 to 3857 while ignoring the 
geotransform are a bit weird)


Even


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.9.0beta1 available for testing

[Even Rouault via gdal-dev]

Robert,

If you want to use ready-made binaries, then using Conda gdal-master 
builds might be the easiest way for all major platforms: 
https://gdal.org/download.html#gdal-master-conda-builds . They track 
gdal master, but that's more or less the same as 3.9.0beta1 right now.


Otherwise if you want to build from source, you can take inspiration 
from the build recipe we use for continuous integration:


https://github.com/OSGeo/gdal/blob/master/.github/workflows/ubuntu_22.04/Dockerfile.ci
https://github.com/OSGeo/gdal/blob/master/.github/workflows/ubuntu_22.04/build.sh

(you don't necessarily need all the dependencies)

And as I mentioned, Docker images are building. Should be ready by tomorrow.

Even

Le 22/04/2024 à 15:18, Robert Hewlett via gdal-dev a écrit :
Any advice on how to become a tester such as testing on prefered OS 
and setup?


I have access to a proxmox server but I usually stick to LINUX OSes.

Or is at easy as following the notes here:
https://gdal.org/development/dev_environment.html

Ubuntu 22.x is the goto right now.

Any advice would be appreciated.

Rob


On Mon, Apr 22, 2024 at 5:12 AM Even Rouault via gdal-dev 
 wrote:


Hi,

I've prepared a beta1 of GDAL 3.9.0 to get feedback from early
testers.

The NEWS file is here:

https://github.com/OSGeo/gdal/blob/v3.9.0beta1/NEWS.md

For packagers,
https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html
may
make it more attractive to build some drivers that typically have
heavy
dependencies as plugins, installable in separate packages, due to the
load time penalty having being improved. It is let to the
appreciation
of packagers to decide which drivers are worth building as plugins
installable in a separate package.
You may also pass CMake options
([GDAL/OGR]_DRIVER__PLUGIN_INSTALLATION_MESSAGE=xxx) so
user get a hint of which package they need to install when GDAL
detects
that a file may be opened by a plugin which is not available on
the file
system but known to have been built as a plugin. Cf

https://gdal.org/development/building_from_source.html#deferred-loaded-plugins

for more details.

For end-users, the following utilities have been updated to use a new
argument parsing framework: gdaladdo, gdalinfo, gdal_translate,
gdalwarp, gdal_grid, gdal_viewshed, gdalbuildvrt, nearblack, ogrinfo,
ogr2ogr, sozip.
This helps detecting errors such as specifying twice an argument that
should be specified once (where generally the last instance was
the one
used). While we have tried to retain backwards compatibility for
nominal
use cases, obviously if you have scripts that accidentally specify an
argument several times whereas it should be specified at most
once, they
will have to be corrected.

Docker images with 3.9.0beta1 are currently cooking.

master is now marked as 3.10.0dev, and the release/3.9 branch has
been
created. All bugfixes for 3.9 should be backported into it.

Source snapshots at:

- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.xz
- https://download.osgeo.org/gdal/3.9.0/gdal380beta1.zip

Autotest snapshots:

- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.zip

Even

-- 
http://www.spatialys.com

My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.9.0beta1 available for testing

[Even Rouault via gdal-dev]

Hi,

I've prepared a beta1 of GDAL 3.9.0 to get feedback from early testers.

The NEWS file is here:

  https://github.com/OSGeo/gdal/blob/v3.9.0beta1/NEWS.md

For packagers, 
https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html may 
make it more attractive to build some drivers that typically have heavy 
dependencies as plugins, installable in separate packages, due to the 
load time penalty having being improved. It is let to the appreciation 
of packagers to decide which drivers are worth building as plugins 
installable in a separate package.
You may also pass CMake options 
([GDAL/OGR]_DRIVER__PLUGIN_INSTALLATION_MESSAGE=xxx) so 
user get a hint of which package they need to install when GDAL detects 
that a file may be opened by a plugin which is not available on the file 
system but known to have been built as a plugin. Cf 
https://gdal.org/development/building_from_source.html#deferred-loaded-plugins 
for more details.


For end-users, the following utilities have been updated to use a new 
argument parsing framework: gdaladdo, gdalinfo, gdal_translate, 
gdalwarp, gdal_grid, gdal_viewshed, gdalbuildvrt, nearblack, ogrinfo, 
ogr2ogr, sozip.
This helps detecting errors such as specifying twice an argument that 
should be specified once (where generally the last instance was the one 
used). While we have tried to retain backwards compatibility for nominal 
use cases, obviously if you have scripts that accidentally specify an 
argument several times whereas it should be specified at most once, they 
will have to be corrected.


Docker images with 3.9.0beta1 are currently cooking.

master is now marked as 3.10.0dev, and the release/3.9 branch has been 
created. All bugfixes for 3.9 should be backported into it.


Source snapshots at:

- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdal-3.9.0beta1.tar.xz
- https://download.osgeo.org/gdal/3.9.0/gdal380beta1.zip

Autotest snapshots:

- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.tar.gz
- https://download.osgeo.org/gdal/3.9.0/gdalautotest-3.9.0beta1.zip

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Block cache and VRT sources

[Even Rouault via gdal-dev]

Le 19/04/2024 à 17:28, Sean Gillies a écrit :

Even,

Does the shared attribute of a VRT sourceFilename element not affect 
caching at all?


If shared is set to 0, then one GDALDataset per VRTSource will be 
opened. This has little benefit.


The scope of sharing was initially greater since it extented to 
potentially several GDALDataset instance of VRT, but this wasn't safe in 
multithreaded scenarios where you would read one VRT in a thread, 
another one in another thread, but both would point to let's same the 
same TIFF GDALDataset. Hence the scope of sharing was reduced to a 
single GDALDataset VRT instance to be safe by default.


Is the cache avoided so that potentially stale data isn't propagated, 
or for other reasons?
Are you reacting to "if it is triggered. VRTSource::IRasterIO() calls 
IRasterIO() on the source band, which doesn't necessarily always trigger 
block-based reading" ?  I just meant that calling IRasterIO() on a 
GDALDataset/GDALRasterBand doesn't necessarily cause the block cache to 
trigger. This is dependent of the driver and the parameters of the 
request. And this isn't specific to use it from a VRT.


On Fri, Apr 19, 2024 at 9:09 AM Even Rouault 
 wrote:


Sean,

Within a given GDALDataset opened on a VRT, if the VRT references
several times the same source, only one GDALDataset will be opened
for it, so you may benefit from the block cache mechanism (if it
is triggered. VRTSource::IRasterIO() calls IRasterIO() on the
source band, which doesn't necessarily always trigger block-based
reading).  But if you open another VRT (or the same one), it will
not share the same GDALDataset for sources that may be common with
the first one, so no re-use of existing block cache. For network
sources, the I/O cache at the /vsicurl/ level works however on
filenames, not VSIFILE* instances, so you will save network reads

Even

Le 19/04/2024 à 16:48, Sean Gillies via gdal-dev a écrit :

Happy Friday, folks!

Are the source rasters of a VRT added to the block cache such
that different VRTs using the same source can avoid reads from
disk or the network? Or is it intended that the VSI cache covers
this need instead?

Thanks,




--
Sean Gillies


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Block cache and VRT sources

[Even Rouault via gdal-dev]

Sean,

Within a given GDALDataset opened on a VRT, if the VRT references 
several times the same source, only one GDALDataset will be opened for 
it, so you may benefit from the block cache mechanism (if it is 
triggered. VRTSource::IRasterIO() calls IRasterIO() on the source band, 
which doesn't necessarily always trigger block-based reading).  But if 
you open another VRT (or the same one), it will not share the same 
GDALDataset for sources that may be common with the first one, so no 
re-use of existing block cache. For network sources, the I/O cache at 
the /vsicurl/ level works however on filenames, not VSIFILE* instances, 
so you will save network reads


Even

Le 19/04/2024 à 16:48, Sean Gillies via gdal-dev a écrit :

Happy Friday, folks!

Are the source rasters of a VRT added to the block cache such that 
different VRTs using the same source can avoid reads from disk or the 
network? Or is it intended that the VSI cache covers this need instead?


Thanks,

--
Sean Gillies

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdal_contour: min value unexpected

[Even Rouault via gdal-dev]

Hi,


I can't imagine this is a bug that was never noticed before.
Why? If everybody followed that reasoning nobody will report and hardly 
any bug would be fixed. (and there are a number of known bugs that 
nobody has had time to fix either)

So I assume I'm doing something wrong. But I can't figure it out.


The best is to file a ticket with all the elements to reproduce 
(including a dataset)


Even

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Export PG to GPKG with foreign keys

[Even Rouault via gdal-dev]

Le 19/04/2024 à 10:25, Matteo Ghetta via gdal-dev a écrit :

Hi all,

I'm trying to export a PG table with several foreign keys to a 
geopackage.


What I tried so far are the native QGIS export and the QGIS Processing 
algorithm "Package layer" with the "Export related layers 
following..." option activated.


I'm checking ogr2ogr but I cannot see any option to include the export 
of the foreign keys. Is this option available in order to have a 
geopackage with the original table with all the other related tables?


In short, no

It might be possible to use the relationship API of 
https://gdal.org/api/gdaldataset_cpp.html#_CPPv4NK11GDALDataset20GetRelationshipNamesE12CSLConstList 
to build something around it. But adding foreign key constraint 
themselves would be tricky as it requires to rewrite the CREATE TABLE.


Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Potential geopackage rtree problem on MacOS ARM64

[Even Rouault via gdal-dev]

ok, I believe I've now a fix in https://github.com/OSGeo/gdal/pull/9700 
. Probably a subtle multi-threading issue related to different memory 
ordering between M1 and Intel CPUs (or just that it was easier to 
trigger on M1)


Le 16/04/2024 à 21:01, Even Rouault via gdal-dev a écrit :


Hi,

I've investigated that today, and I can quite reliably trigger a 
similar error with our existing tests on CI, but this is impossible to 
diagnose further without direct access to a machine where the error 
triggers (when simulating taking the the error code path on Linux, the 
fallback code works, whereas it doesn't on OSX Arm64, so there's 
something quite subtle happening here). So if someone has a OSX Arm64 
and is willing to give me ssh access to it, that would be welcome.


In the meantime, I've submitted 
https://github.com/OSGeo/gdal/pull/9685 which is just a workaround to 
disable (by default) the threaded RTree creation on that platform. It 
can be explicitly enabled by setting OGR_GPKG_ALLOW_THREADED_RTREE to YES


Even

Le 16/04/2024 à 08:13, Rahkonen Jukka via gdal-dev a écrit :


Hi,

It seems that at least two MacOS ARM64 users have faced a problem 
with the new, faster rtree creation method so maybe it is worth 
having a look. Obviously a big dataset and right hardware is needed 
for testing 
https://gis.stackexchange.com/questions/479958/how-to-fix-failed-to-prepare-sql-error-when-creating-gpkg-file-from-osm-extrac/479964#479964


-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Adding a CI "stale" workflow for pull requests without activity

[Even Rouault via gdal-dev]

Le 18/04/2024 à 23:54, Andrew C Aitchison via gdal-dev a écrit :

On Thu, 18 Apr 2024, Even Rouault via gdal-dev wrote:

I'm proposing in https://github.com/OSGeo/gdal/pull/9693 that we add 
a CI "stale" workflow for pull requests without activity. It is 
mostly a copy from QGIS similar workflow with the following 
changes:
- restrict the scope to pull requests only and not tickets (although 
we could potentially also do that for tickets that are in the 
"awaiting feedback" state)
- increase the period to tag a pull request as "stale" to 28 days 
without activity. The stale bot adds a message in the PR explaining 
that, and that they have an extra delay of 14 days to make it 
progress, otherwise it will be automatically closed. So basically a 
PR without activity for 6 weeks will be automatically closed. I think 
this is a fair delay.


The rationale for that is that it makes the cognitive load of PR 
reviewers lower by avoiding to have to read repeatedly a list of 
stale PRs, and it also encourages contributors to either make their 
work progress towards completion (or just make them realize that they 
don't have the bandwidth or motivation to make it further progress, 
which is also fine).


I assume this is for PRs awaiting response from the requester,
not PRs awaiting a response from a reviewer ?


I don't believe the bot can understand that subtlety. At least that's a 
way for the submitter to remember to ping for submitters to give a review


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Adding a CI "stale" workflow for pull requests without activity

[Even Rouault via gdal-dev]

Hi,

I'm proposing in https://github.com/OSGeo/gdal/pull/9693 that we add a 
CI "stale" workflow for pull requests without activity. It is mostly a 
copy from QGIS similar workflow with the following changes:
- restrict the scope to pull requests only and not tickets (although we 
could potentially also do that for tickets that are in the "awaiting 
feedback" state)
- increase the period to tag a pull request as "stale" to 28 days 
without activity. The stale bot adds a message in the PR explaining 
that, and that they have an extra delay of 14 days to make it progress, 
otherwise it will be automatically closed. So basically a PR without 
activity for 6 weeks will be automatically closed. I think this is a 
fair delay.


The rationale for that is that it makes the cognitive load of PR 
reviewers lower by avoiding to have to read repeatedly a list of stale 
PRs, and it also encourages contributors to either make their work 
progress towards completion (or just make them realize that they don't 
have the bandwidth or motivation to make it further progress, which is 
also fine).


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalwarp with RPCs causing bogus output

[Even Rouault via gdal-dev]

Hi,


I haven't done myself that exercice but I know that computing RPC values 
that are stable enough might be challenging, so perhaps the issue is 
related to that



A few other remarks:

- your gdalwarp command line refers to  RPC_DEM_SRS and 
RPC_DEM_MISSING_VALUE but doesn't include a RPC_DEM itself, hence those 
are likely to be non effective


- you generally don't want to use both RPC_HEIGHT and RPC_DEM. 
RPC_HEIGHT is essentially useful when you don't have a DEM available, 
and thus fallback taking an average elevation


- there's a subtlety regarding DEM. RPC reference for altitudes is WGS84 
ellipsoidal height, not orthometric/MSL altitude. But DEM values use 
orthometric/MSL altitude, hence a geoid correction must be applied. So 
you'd rather want to use RPC_DEM_SRS=EPSG:4326+5773 for example to use 
the EGM96 geoid (cf https://github.com/OSGeo/gdal/issues/3298). That 
said, misusing orthometric altitude vs ellipsoidal height generally 
accounts for small shifts, not big ones


- you could use gdaltransform with all your -to parameter and 
input_with_RPCs.tiff to check at least that the forward RPC 
transformation path works correctly (the one from longitude, latitude -> 
column, line). And with -i to check the inverse RPC transformer. The 
inverse RPC transformer can have a hard time converging in montainous 
areas or for images off-nadir



Even


Le 17/04/2024 à 23:30, Joseph McGlinchy via gdal-dev a écrit :

Hello,

I am attempting to implement georegistration through RPC. I have the 
following information I've used to calibrate the RPC coefficients, 
using all terms for numerator and denominator for both sample and line 
equations.


 *
image grid, stored in tiff format with no geo-information
associated with it, so it reflects the imaging orientation
 *
a 'ground' grid, which corresponds to the longitude/latitude
coordinates for each pixel determined from a line-of-sight vector
and imaging system coordinates where the pseudo-rays intersect the
WGS84 geoid
 *
a random sample of up to 4000 image coordinate / object space
coordinate pairs (I arrived at this number through trial and
error; using all pixels explodes RAM)
 *
elevation extracted at the longitude/latitude coordinates from a
DEM, in this case, SRTM, but have also tried using NASADEM


I am able to write out an image to EPSG:4326 by populating the RPC 
metadata and using the non-georeferenced image data. However, I am 
struggling to use |gdalwarp|​ in a reliable way to orthorectify that 
data, let alone writing it out in a way that 'bakes in' the 
georegistration with the RPCs whether that is in EPSG:4326 or the 
local UTM zone. I see strips of the image "ripped out" with odd curves 
in various places throughout the image.


The only way I've been able to use |gdalwarp|​ to write the image at 
all is with the following parameters (any DEM reference is to the SRTM 
DEM):


gdalwarp --config CPL_DEBUG ON -t_srs EPSG:32610 -rpc -to 
"RPC_DEM_SRS=+proj=longlat +datum=WGS84 +no_def" -to "RPC_HEIGHT=350" 
-to "RPC_DEM_MISSING_VALUE=0.001" -to "RPC_FOOTPRINT='POLYGON ((list 
of polygon coordinates comprising the long/lat grid))'" -to 
"RPC_MAX_ITERATIONS=101" input_with_RPCs.tiff output.tiff


This is the only configuration i can use to run |gdalwarp|​ 
successfully.  removing any single RPC_X tranformer option gives me 
bogus output. The RPC_HEIGHT value i specify above is not close to the 
mean or median elevation of the extent of my data; mean is ~195m and 
median is ~150m.


With the debug turned on, any other set of parameters gives me failed 
RPC convergence on several points. I am able to reproduce this 
regularly by specifying RPC_DEM=dem.tif, where dem.tif is the same 
data I used to extract elevation values when calibrating the RPCs. I 
am seeing normalized latitude and longitude values with magnitude > 1 
(I checked every location in the image, based on the metadata, the 
range is not outside of [-1,1]), as well as normalized altitude values 
with magnitude > 1 (there are some, not many, that have magnitude of 
1.75).


My workflow can be summarized as:

1.
load grids (image data, longitude, latitude)
2.
randomly sample up to 4000 points in image coordinates, object
coordinates
1.
assign z-value from SRTM DEM
2.
evaluate if any of the points are in NODATA areas of SRTM
(image is coastal, so there are NODATA areas for SRTM here),
if so, remove those and generate more points
3.
normalize coordinates of grids to be in [-1,1], recording offsets
and scale
4.
 calibrate RPC coefficients using all terms
5.
write out GeoTIFF with image grid for pixels, along with RPC
required metadata fields and CRS EPSG:4326


System information (please let me know if more is needed)
OS: Ubuntu 20.04 LTS (GNU/Linux 5.10.16.3-microsoft-standard-WSL2 
x86_64) (Windows Subsystem for Linux)

GDAL 3.6.0 (python)

Thank you in advance for 

Re: [gdal-dev] Hungarian Notation

[Even Rouault via gdal-dev]

Hi,

This is described in https://gdal.org/development/rfc/rfc8_devguide.html 
. I've submitted https://github.com/OSGeo/gdal/pull/9689 so it is going 
to be linked to 
https://gdal.org/development/dev_practices.html#making-changes-to-gdal


I'd probably agree that at my beginnings in GDAL, I might have felt 
unfamiliar with the Hungarian notation, but now I've fully internalized 
it, so I hardly notice it. It would be interesting to have opinions of 
other developers if they find it inconvenient/a hurdle, are neutral or 
like it.  The thing is that if we decide to no longer use it, the 
inconsistency of mixing free style naming and Hungarian notation might 
be problematic and make code confusing. We should probably aim at being 
consistent at least at the scope of a given function or file.


Even

Le 17/04/2024 à 15:48, Andrew Bell via gdal-dev a écrit :

Hi,

I was going to make some changes to some GDAL code and I generally 
want to follow along. I see that most of the code is still using 
Hungarian notation for identifier names. I haven't seen this anywhere 
else in like 20 years.


Is it OK to submit code without this naming convention or do we still 
think it's important. I didn't see an RFC or documentation that covers 
this, but there's lots so I may have missed it.


Thanks,

--
Andrew Bell
andrew.bell...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Potential geopackage rtree problem on MacOS ARM64

[Even Rouault via gdal-dev]

Hi,

I've investigated that today, and I can quite reliably trigger a similar 
error with our existing tests on CI, but this is impossible to diagnose 
further without direct access to a machine where the error triggers 
(when simulating taking the the error code path on Linux, the fallback 
code works, whereas it doesn't on OSX Arm64, so there's something quite 
subtle happening here). So if someone has a OSX Arm64 and is willing to 
give me ssh access to it, that would be welcome.


In the meantime, I've submitted https://github.com/OSGeo/gdal/pull/9685 
which is just a workaround to disable (by default) the threaded RTree 
creation on that platform. It can be explicitly enabled by setting 
OGR_GPKG_ALLOW_THREADED_RTREE to YES


Even

Le 16/04/2024 à 08:13, Rahkonen Jukka via gdal-dev a écrit :


Hi,

It seems that at least two MacOS ARM64 users have faced a problem with 
the new, faster rtree creation method so maybe it is worth having a 
look. Obviously a big dataset and right hardware is needed for testing 
https://gis.stackexchange.com/questions/479958/how-to-fix-failed-to-prepare-sql-error-when-creating-gpkg-file-from-osm-extrac/479964#479964


-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Printing a size_t variable on all github platforms ?

[Even Rouault via gdal-dev]

Andrew,

some hints at 
https://stackoverflow.com/questions/44382862/how-to-printf-a-size-t-without-warning-in-mingw-w64-gcc-7-1


Otherwise an alternative is to cast to uint64_t and use PRIu64

Even

Le 15/04/2024 à 19:49, Andrew C Aitchison via gdal-dev a écrit :


I am trying to print a size_t variable* with CPLDebug, but
am struggling to find a format that works for all the guthub builds.

The main problem is that the "build-windows-msys2-mingw" build
does not support the format option %zu

../../../frmts/vrc/VRC.cpp:159:47: error: unknown conversion type 
character 'z' in format [-Werror=format=]
  159 |  "VRC_png_read_data_fn(%p %p %zu) reached end 
of data",

  |   ^
https://github.com/andrew-aitchison/gdal/actions/runs/8691997363/job/23835554972 


lines 586-8.

Since size_t is usually an unsigned int on 64-bit systems and
an unsigned long on 32-bit systems ("build-windows-msys2-mingw"
and "Alpine, gcc 32-bit") I cannot use %u or %lu either,
without conditional compiling.
I don't see any helpful definition in  either

Am I missing a way of printing a size_t ?
I suppose I could cast it to an unsigned long (long?) and hope.

Thanks,

---  Pedantry

I'm ultimately trying to print "myVector.size()"
which is a "std::vector<...>::size_type", which
https://stackoverflow.com/questions/918567/size-t-vs-containersize-type
says *may* be be different from a size_t ... but
  https://en.cppreference.com/w/cpp/types/size_t
says that std::size_t is big enough (or the type is ill-formed).


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL DATA PATH - test

[Even Rouault via gdal-dev]

The following fails if GDAL_DATA is incorrect:

    srs = osr.SpatialReference()
   srs.SetStatePlane(403, 1)  # California III NAD83.


Le 15/04/2024 à 14:26, Paul Harwood via gdal-dev a écrit :

I have an interesting little problem.

I want to write (in the code using the API - not as a test script - 
actually in C#) a command that will fail if the GDAL data path is set 
incorrectly.


I am having difficulty finding a command that will fail if the path is 
incorrect.


I thought that some of the commands in OSR (like ImportFromPCI ) would 
fail since the data files are in the data folder - but when I try that 
with the GDAL-DATA configuration item set to a path that does not 
exists - they do not fail!


There are a lot of commands out there - and I do not want to have to 
try them all. Does anyone have a good idea about what should fail?


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] 3.9 release schedule

[Even Rouault via gdal-dev]

Hi,

here's the updated proposed 3.9 release schedule

- Monday April 22: feature freeze, creation of release/3.9 branch and 
issue a 3.9.0beta1


- Monday May 6th: issue of 3.9.0rc1

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] advice on python style - get file/dataset as bytes

[Even Rouault via gdal-dev]

Hi Michael,

I can't think of a better way with the current API.

There has been some work in progress in 
https://github.com/OSGeo/gdal/pull/8222 to make VSIFile accessible as a 
regular Python file, but this isn't merged


Even

Le 15/04/2024 à 01:48, Michael Sumner via gdal-dev a écrit :
Hi, I'm getting some pushback on my code style. I just want the raw 
bytes in-memory of a file in Python, for a manageable tiny dataset.


Is there anything especially wrong with the following?  It's 
essentially the same as the answer here:


https://lists.osgeo.org/pipermail/gdal-dev/2016-August/045030.html
   (I know it needs error checks, I just mean the bare style of how to 
get the task done).


temp_name = "/vsimem/some.tif"
from osgeo import gdal
gdal.UseExceptions()

## we do something with vrt
dsn = 
"vrt:///vsicurl/https://github.com/cran/rgl/raw/master/inst/textures/worldsmall.png?a_ullr=-180,90,180,-90=rgb 
"

ds = gdal.Open(dsn)
## write to tif format, but using MEM
gdal.Translate(temp_name, ds)

## now obtain those bytes from the GDAL virtual file
f = gdal.VSIFOpenL(temp_name, 'rb')
gdal.VSIFSeekL(f, 0, 2) # end
size = gdal.VSIFTellL(f)
gdal.VSIFSeekL(f, 0, 0) # begin

## the desired result is this
data = gdal.VSIFReadL(1, size, f)

## and clean up
gdal.VSIFCloseL(f)
gdal.Unlink(temp_name)

Cheers, Mike



--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How do I add a projection to proj 8?

[Even Rouault via gdal-dev]

Stephen,

there are 2 possiblities:

- you may reuse your modified /usr/share/proj/epsg file from PROJ.4. But 
in this case, the EPSG entries of proj.db will not be used, so you will 
use only legacy CRS and transformations


- or you add a custom entry in proj.db

For the later, the following will output SQL statements that should be 
put in the DB, but they need a bit of tweaking given that the logic that 
suggests the SQL statements try to set a "proper" CRS definition which 
lacks the PROJ.4 +over or +nadgrids=@null hack


projinfo --output-id EPSG:900914 -o SQL -q "+proj=merc +a=6371001 
+b=6371001 +lat_ts=0.0 +lon_0=0.0 +x_0=-4448 +y_0=-4448 +k=1.0 +units=m 
+over +nadgrids=@null +no_defs +type=crs"


With a bit of hand tuning, if you run the following statements in your 
proj.db,


INSERT INTO ellipsoid 
VALUES('EPSG','ELLPS_GEODETIC_DATUM_GEODETIC_CRS_900914','unknown','','IAU_2015','399',6371001,'EPSG','9001',NULL,6371001,0);
INSERT INTO geodetic_datum 
VALUES('EPSG','GEODETIC_DATUM_GEODETIC_CRS_900914','unknown using 
nadgrids=@null','','EPSG','ELLPS_GEODETIC_DATUM_GEODETIC_CRS_900914','EPSG','8901',NULL,NULL,NULL,NULL,NULL,0);
INSERT INTO usage 
VALUES('EPSG','USAGE_GEODETIC_DATUM_GEODETIC_CRS_900914','geodetic_datum','EPSG','GEODETIC_DATUM_GEODETIC_CRS_900914','PROJ','EXTENT_UNKNOWN','PROJ','SCOPE_UNKNOWN');
INSERT INTO geodetic_crs 
VALUES('EPSG','GEODETIC_CRS_900914','unknown','','geographic 
2D','EPSG','6424','EPSG','GEODETIC_DATUM_GEODETIC_CRS_900914',NULL,0);
INSERT INTO usage 
VALUES('EPSG','USAGE_GEODETIC_CRS_900914','geodetic_crs','EPSG','GEODETIC_CRS_900914','PROJ','EXTENT_UNKNOWN','PROJ','SCOPE_UNKNOWN');
INSERT INTO projected_crs VALUES('EPSG','900914','unknown 
(900914)','',NULL,NULL,'EPSG','GEODETIC_CRS_900914',NULL,NULL,'+proj=merc 
+a=6371001 +b=6371001 +lat_ts=0.0 +lon_0=0.0 +x_0=-4448 +y_0=-4448 
+k=1.0 +units=m +over +nadgrids=@null +no_defs',0);
INSERT INTO usage 
VALUES('EPSG','USAGE_PROJECTED_CRS_900914','projected_crs','EPSG','900914','PROJ','EXTENT_UNKNOWN','PROJ','SCOPE_UNKNOWN');


you'll get:

PROJ_DATA=/tmp bin/projinfo EPSG:900914
PROJ.4 string:
+proj=merc +a=6371001 +b=6371001 +lat_ts=0 +lon_0=0 +x_0=-4448 
+y_0=-4448 +k=1 +units=m +nadgrids=@null +wktext +no_defs +type=crs


WKT2:2019 string:
PROJCRS["unknown (900914)",
    BASEGEOGCRS["unknown",
    DATUM["unknown using nadgrids=@null",
    ELLIPSOID["unknown",6371001,0,
    LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
    ANGLEUNIT["degree",0.0174532925199433]]],
    CONVERSION["unknown",
    METHOD["Popular Visualisation Pseudo Mercator",
    ID["EPSG",1024]],
    PARAMETER["Latitude of natural origin",0,
    ANGLEUNIT["degree",0.0174532925199433],
    ID["EPSG",8801]],
    PARAMETER["Longitude of natural origin",0,
    ANGLEUNIT["degree",0.0174532925199433],
    ID["EPSG",8802]],
    PARAMETER["False easting",-4448,
    LENGTHUNIT["metre",1],
    ID["EPSG",8806]],
    PARAMETER["False northing",-4448,
    LENGTHUNIT["metre",1],
    ID["EPSG",8807]]],
    CS[Cartesian,2],
    AXIS["(E)",east,
    ORDER[1],
    LENGTHUNIT["metre",1]],
    AXIS["(N)",north,
    ORDER[2],
    LENGTHUNIT["metre",1]],
    ID["EPSG",900914]]

Even

Le 12/04/2024 à 23:24, Stephen Woodbridge via gdal-dev a écrit :

Hi all,

I've been gone for a while, but got called back to update a site I 
built and need to move from proj4 to proj 8 on Ubuntu 22.04. In the 
past I just added the following to /usr/share/proj/epsg


# HYCOM Mercator projection
<900914> +proj=merc +a=6371001 +b=6371001 +lat_ts=0.0 +lon_0=0.0 
+x_0=-4448 +y_0=-4448 +k=1.0 +units=m +over +nadgrids=@null +no_defs  <>


and was able to access it in gdal, mapserver, postgis, etc with 
"EPSG:900914"


How does one do that with the new system?

Thanks,
  -Steve W

 
	Virus-free.www.avast.com 
 



<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.5 is released

[Even Rouault via gdal-dev]

The corresponding Docker images are now available: 
https://github.com/OSGeo/gdal/tree/master/docker#images-of-releases


Le 04/04/2024 à 19:58, Even Rouault via gdal-dev a écrit :

Hi,

On behalf of the GDAL/OGR development team, I am pleased to announce 
the release of the GDAL/OGR 3.8.5 bug fix version. This is the last 
one planned in the 3.8.x series, with the forthcoming 3.9.0 released 
planned for the beginning of May.


Consult the release notes for the list of issues addressed:
    https://github.com/OSGeo/gdal/blob/v3.8.5/NEWS.md

The sources are available at:

    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.xz
    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.xz.md5
    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.xz.sig

or

    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.gz
    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.gz.md5
    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.gz.sig

or

    https://download.osgeo.org/gdal/3.8.5/gdal385.zip
    https://download.osgeo.org/gdal/3.8.5/gdal385.zip.md5
    https://download.osgeo.org/gdal/3.8.5/gdal385.zip.sig

Best regards,

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.8.5 is released

[Even Rouault via gdal-dev]

Hi,

On behalf of the GDAL/OGR development team, I am pleased to announce the 
release of the GDAL/OGR 3.8.5 bug fix version. This is the last one 
planned in the 3.8.x series, with the forthcoming 3.9.0 released planned 
for the beginning of May.


Consult the release notes for the list of issues addressed:
    https://github.com/OSGeo/gdal/blob/v3.8.5/NEWS.md

The sources are available at:

    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.xz
    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.xz.md5
    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.xz.sig

or

    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.gz
    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.gz.md5
    https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5.tar.gz.sig

or

    https://download.osgeo.org/gdal/3.8.5/gdal385.zip
    https://download.osgeo.org/gdal/3.8.5/gdal385.zip.md5
    https://download.osgeo.org/gdal/3.8.5/gdal385.zip.sig

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.8.5RC1 available, and motion to adopt it

[Even Rouault via gdal-dev]

Hi,

motion passed with +1 from PSC members EvenR, JukkaR, HowardB and JavierJS

Even

Le 02/04/2024 à 12:16, Even Rouault via gdal-dev a écrit :

Hi,

I have prepared a GDAL/OGR 3.8.5 release candidate. This is likely to 
be the
last bug fix release in the 3.8.x series, with 3.9.0 coming beginning 
of May.


Pick up an archive among the following ones (by ascending size):

  https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5rc1.tar.xz
  https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5rc1.tar.gz
  https://download.osgeo.org/gdal/3.8.5/gdal385rc1.zip

A snapshot of the gdalautotest suite is also available:

https://download.osgeo.org/gdal/3.8.5/gdalautotest-3.8.5rc1.tar.gz
  https://download.osgeo.org/gdal/3.8.5/gdalautotest-3.8.5rc1.zip

The NEWS file is here:

  https://github.com/OSGeo/gdal/blob/v3.8.5RC1/NEWS.md

Motion: adopt 3.8.5RC1 as final 3.8.5

Starting with my +1

Best regards,

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Unsupported HTTPAUTH value 'BEARER', ignored

[Even Rouault via gdal-dev]

Hi,

GDAL_HTTP_AUTH=BEARER support has been added in master / GDAL 3.9dev 
only (I've just fixed an erroneous statement about it being in 3.8)


In earlier versions, you can try setting 
GDAL_HTTP_HEADERS="Authentication: Bearer {your_token_here}"


Even

Le 04/04/2024 à 15:40, Michael Otto via gdal-dev a écrit :

Hello,

I am trying to pass an authentication for requests to WFS services. As 
a test, I have activated the logging of curl and have the errors 
output to a log file.


With HTTP-Basic authentication it seems to work with the following 
parameters (no errors are logged):

'Auth' is my ArrayList to use in java processbuilder.

Auth:[[--config, GDAL_HTTP_AUTH, BASIC, --config, GDAL_HTTP_USERPWD, 
username:password]]


But try this for Bearer :
Auth:[[--config, GDAL_HTTP_AUTH, BEARER, --config, GDAL_HTTP_BEARER, 
token]]


I get the following error in the error log:
Warning 1: Unsupported HTTPAUTH value 'BEARER', ignored.


What am I doing wrong and what can I do so that Bearer authentication 
is also accepted?

Can I use curl paramters directly instead?

Michael


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] discussion on xarray and the missing abstract-grid specification

[Even Rouault via gdal-dev]

Hi,

Not that I've a strong opinion on what GeoZarr/XArray should do, but yes 
1D coordinate arrays are a bit of a pain to deal with, when they 
actually just encode a regularly spaced grid. This is something I've hit 
in the bridge between the GDAL multidimensional API (roughly netCDF 
based) and the GDAL "classic" 2D raster API, when you want to expose a 
view of a multi-dim dataset as a classic one. There's this bit of logic 
in 
https://github.com/OSGeo/gdal/blob/master/gcore/gdalmultidim.cpp#L7334 
to guess if the 1D coordinate array is regularly spaced or not. The 
netCDF driver has a similar heuristics too. This is quite error prone, 
especially if Float32 coordinates are used. There it can be difficult to 
detect if it is a regularly spaced array but Float32 hasn't sufficient 
precision to fully encode it, or if it there are genuine irregularities 
in the spacing. For Float64, the precision issues are less a practical 
error, because comparing the expected value (assuming a regular spacing) 
with the one of the 1D coordinate array with a very small relative 
epsilon, like 1e-8, is sufficient.


Even


Le 03/04/2024 à 22:56, Michael Sumner via gdal-dev a écrit :
Here's an (ahem) extremely important discussion on the prospects for 
xarray to extend from the coordinates-only model (like that of netcdf) 
for geo reference:


https://discourse.pangeo.io/t/example-which-highlights-the-limitations-of-netcdf-style-coordinates-for-large-geospatial-rasters/

I'm heartened to see this recognized at this level. I think GDAL per 
se should feature in this discussion prominently, and not just the 
downstream Python packages that are mentioned.


For my input, I'll be highlighting examples of

-  degenerate rectilinear coordinates, and the problem of whether to 
assign a regular grid there (with a trivial lightweight Translate a la 
-a_ullr), or to define a new regular grid and push it through the Warp 
api, as a dataset that picks up or can be pointed to the 1D coordinate 
arrays. This has been the crux of the issue for me, I get to decisions 
where it's not clear what was intended or what should be done going 
forward.


- actual curvilinear coordinates, and the very very general power of 
that to resolve to a regular grid with very simple specification 
(specify some or nothing of target/extent/resolution/crs/dimension). 
Key problems here are when the coordinate arrays are not auto-detected 
(rare) or when the longitudes are unwrapped (less rare).


- that extent+dimension is complementary to the affine transform, and 
way more user-friendly way to work with rasters for the assign and/or 
target specification step (this is huge in R since the {raster} 
package ca. 2009 and very well supported by Warp and Translate). It's 
true that extent+dimension can't do shear params but it's a perfectly 
valid way to work and flipping from transform to extent is trivial 
numerically.


I hope some more voices from this community pitch in as well, very 
happy to discuss here or offline about any of this.  I'm not really a 
technical expert but I have a good overview of the landscape that GDAL 
sits in, and I'm making similar pitches for involvement in the R 
communities.


Cheers, Mike

--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] LAUNDER option for geopackages

[Even Rouault via gdal-dev]

Hi,

I've given this a crack at https://github.com/OSGeo/gdal/pull/9609

The newly introduced LAUNDER option for GeoPackage is indeed quite 
"extreme" compared to the existing ones, hence I've defaulted it to NO.


I've also added a LAUNDER_ASCII option to PG/PGDump to do something 
similar to GeoPackage LAUNDER.


Even

Le 03/04/2024 à 17:50, Rahkonen Jukka via gdal-dev a écrit :


Hi,

The GeoPackage standard recommends some laundering to be done by the 
clients:


Implementers should be aware of the fact that SQLite table names are 
not case


sensitive and that table names in sqlite_master and gpkg_extensions 
may not have


the same case. When searching for table name references, it is 
recommended to


transform table names to lower case with the lower() function. See the 
Abstract


Test Suite for an example.

If GeoNode does not have any other reason for its requirement than 
what you have pointed out, then maybe it is too strict.


Having launder in GPKG driver feels still reasonable. SQLite driver 
has it already https://gdal.org/drivers/vector/sqlite.html.



“LAUNDER=[YES/NO]: Defaults to YES. Controls whether layer and field 
names will be laundered for easier use in SQLite. Laundered names will 
be converted to lower case and some special characters(' - #) will be 
changed to underscores.”


It is either not documented or not implemented, but laundering 
identifiers to work without "quotas" would also require removing 
numbers from the beginning of the name. And at least % should be added 
to the list of special characters. But total laundering requires quite 
a lot https://www.sqlite.org/draft/tokenreq.html.


Laundering characters like äöå for GeoNode feels too much. Who would 
say into what characters to launder?


-Jukka Rahkonen-

*Lähettäjä:* gdal-dev  *Puolesta 
*Robert Hewlett via gdal-dev

*Lähetetty:* keskiviikko 3. huhtikuuta 2024 17.51
*Vastaanottaja:* gdal-dev@lists.osgeo.org
*Aihe:* [gdal-dev] LAUNDER option for geopackages

Hi,

Is there a LAUNDER for geopackages?

I ask this because at the following site:

https://www.geopackage.org/guidance/getting-started.html

Gives this specific guidance:

Note: *For maximum interoperability,* start your database identifiers 
(table names, column names, etc.) with a lowercase character and only 
use *lowercase characters*, numbers 0-9, and underscores (_).


Which then turned into a *constraint* in GeoNode (I could not upload a 
certain geopackage from the township of langley) where I eventually 
discovered in the celery logs this error:


ERROR/ForkPoolWorker-958] {'validation_code': 'RQ1', 
'validation_description': 'Layer names must start with a letter, and 
*valid char
acters are lowercase a-z*, numbers or underscores.', 'level': 'error', 
'locations': ['Error layer: Parcel_Attributes']}


I would love LAUNDER to be a more global option.

Upper for: Oracle and Shapefiles.

Lower for: Postgresql, Mysql, geopackages

Just wondering,

Rob


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Gdaladdo and external mask band

[Even Rouault via gdal-dev]

Hi Jukka,

I've realized this behaviour was actually documented in 
https://gdal.org/drivers/raster/gtiff.html#overviews-and-nodata-masks : 
"Internal overviews, external nodata mask: when running BuildOverviews() 
in update mode on the .tif file, only the overviews of the main bands 
will be generated. Overviews of the external .msk file must be 
explicitly generated by running BuildOverviews() on the .msk."


... but just after fixing it in https://github.com/OSGeo/gdal/pull/9603 
to do the expected thing: that is build overviews of the main bands and 
of the .msk


I think it is also high time to reconsider switching the default value 
of GDAL_TIFF_INTERNAL_MASK to YES : https://github.com/OSGeo/gdal/pull/9604


Even

Le 03/04/2024 à 10:56, Rahkonen Jukka via gdal-dev a écrit :


Hi,

I was making some tests with filegdb raster from Region 5 - State 
(usda.gov) 
, 
file 
https://usfs-public.box.com/shared/static/q96eev9gnsa9hw5bwpmwpmxerbdat0vi.zip
I converted the data into GeoTIFF, added overviews, and noticed, that 
the GeoTIFF is very slow to browse with QGIS at small scales. I 
noticed that it is because gdal_translate creates an external .msk 
mask band, and gdaladdo does not create overviews for the mask.


Is is somehow possible to create overviews for the .msk mask either 
automatically or manually? Or would it create a bad mask that would 
not match with the corresponding overviews of the data bands?


-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.8.5RC1 available, and motion to adopt it

[Even Rouault via gdal-dev]

Hi,

I have prepared a GDAL/OGR 3.8.5 release candidate. This is likely to be the
last bug fix release in the 3.8.x series, with 3.9.0 coming beginning of 
May.


Pick up an archive among the following ones (by ascending size):

  https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5rc1.tar.xz
  https://download.osgeo.org/gdal/3.8.5/gdal-3.8.5rc1.tar.gz
  https://download.osgeo.org/gdal/3.8.5/gdal385rc1.zip

A snapshot of the gdalautotest suite is also available:

https://download.osgeo.org/gdal/3.8.5/gdalautotest-3.8.5rc1.tar.gz
  https://download.osgeo.org/gdal/3.8.5/gdalautotest-3.8.5rc1.zip

The NEWS file is here:

  https://github.com/OSGeo/gdal/blob/v3.8.5RC1/NEWS.md

Motion: adopt 3.8.5RC1 as final 3.8.5

Starting with my +1

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] HDF5 and identified fields / primary dimension

[Even Rouault via gdal-dev]

Michael,


Warning 1: The dataset has several variables that could be identified 
as vector fields, but not all share the same primary dimension. 
Consequently they will be ignored.
Yes, the driver is super conservative/picky when trying to recognize a 
netCDF file as a vector layer, and its heuristics will return in error 
if there is any ambiguity.


I've seen similar cases in other files. I presume the driver could be 
updated to 1) choose the primary dimension and read the values while 
ignore others 2) user-specify the dimension to include, or 3) 
user-specify the fields to exclude


I guess option 2 could be reasonable as an open option

For that particular file, I see that the "feature_id" variable 
(corresponding to the "feature_id" dimension) has a cf_role = 
"timeseries_id" attribute, and that the global metadata has a 
featureType = "timeSeries" attribute. So given 
https://cfconventions.org/Data/cf-conventions/cf-conventions-1.11/cf-conventions.html#coordinates-metadata 
, this seems to be relatively standardized, and in that case the 
heuristics could be improve to recognize that the main dimension is 
feature_id (probably with a test that the size of the time dimension is 
1).  As far as I can see/remember, the vector layer support in netCDF 
was originally developed for the featureType=point and profile use cases 
, so some tuning for timeseries isn't unexpected


Or maybe if detecting that in the set of dimensions there is only one 
with > 1 sample and others ones are at 1, consider only the one with > 1 
sample


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] CVE-2024-3094 (aka "xz hackdoor") and GDAL

[Even Rouault via gdal-dev]

Hi,

TLDR: no specific reason to worry.

My longer analysis:

Those following the recent security news have certainly come across 
https://lwn.net/Articles/967180 or similar articles, and if you don't 
have and have been running a cutting edge Linux distribution recently, 
you *should* follow related security alerts emitted by your vendor.


liblzma (the library part of the xz software distribution) is an 
optional libtiff dependency for the TIFF LZMA codec, and an optional 
direct GDAL dependency since GDAL 1.8.0. Initially only used when 
building GDAL with its internal libtiff, and since GDAL 3.4.0, directly 
too as a Zarr codec.


Beyond activity in liblzma / xz being potentially suspicious with at 
least 2 years of "contributions" by malicious actor 
https://github.com/JiaT75, which makes the current headlines, it has 
also been found that they contributed to libarchive too since 2021 
(https://github.com/libarchive/libarchive/commits?author=JiaT75), and 
their latest commit https://github.com/libarchive/libarchive/pull/1609 
is in particular pointed as suspicious. As far as I can see, this commit 
is present in libarchive >= 3.6. I mention libarchive because it is an 
optional GDAL dependency since GDAL 3.7.0 for the /vsi7z/ and /vsirar/ 
virtual file systems.


We don't have vendored copies of liblzma or libarchive in the GDAL 
source code repository.


Regarding GDAL Docker images, here's the status of versions they ship:

- ghcr.io/osgeo/gdal:alpine-small-3.8.4 and 
ghcr.io/osgeo/gdal:alpine-normal-3.8.4:

xz-libs-5.4.3-r0
libarchive-3.7.2-r0

- ghcr.io/osgeo/gdal:ubuntu-full-3.8.4:
liblzma5:amd64    5.2.5-2ubuntu1
libarchive13:amd64    3.6.0-1ubuntu1

- ghcr.io/osgeo/gdal:ubuntu-small-3.8.4:
liblzma5:amd64  5.2.5-2ubuntu1

So those liblzma versions are not currently considered as affected. 
Regarding libarchive status and the impact of JiaT75's activity there, 
it seems to me we are yet in undecided territory and it wouldn't be 
surprising to see update of those libraries in the coming weeks, as 
exploration of past activity from that compromising entity is performed.


It is a good time to recall usual general best-practices:

- avoid opening datasets coming from unknown or untrusted sources

- if you do so, prefer doing that on dedicated workstations that don't 
have access to sensitive information


- you may reduce your potential exposure by disabling at build time 
optional dependencies you don't need.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] JPEG2000 / GMLJP2: parameters seem to be ignored (offset location)

[Even Rouault via gdal-dev]

This might be tricky, because JPEG2000 itself has a concept of a image 
space origin which is not necessarily (0,0) with the (XOSiz, YOSiz) 
fields of the SIZ marker, although most of the time it is 0,0.   I don't 
remember how the GDAL JPEG2000 drivers behave regarding that, if they 
ignore it (that is the GDAL 0,0 pixel is the one at (XOSiz,YOSiz)), of 
if the first valid pixel in GDAL image space is actually (XOSiz, YOSiz) 
and you have padding pixels between 0 and XOSiz/YOSiz.


You can get the XOSiz, YOSiz value with the 
https://github.com/OSGeo/gdal/blob/master/swig/python/gdal-utils/osgeo_utils/samples/dump_jp2.py 
script


Depending on how drivers behave, the combination of this JPEG2000 
imaging (XOSiz, YOSiz) concept with the gml:low shift from GMLJP2 might 
actually cancel and lead to the right result...


In any case data producers are calling for troubles by using any of 
those mechanisms...


Le 29/03/2024 à 09:45, Philippe Ghesquiere via gdal-dev a écrit :

Hi all,

I have some questions about GMLJP2 parameters concerning location.

*1) Description
*Some JP2K images have a pixel origin different than (0 ; 0) (DIMAP 
products for example).


In such cases, the GML header looks like like:

*7 7  <== Different than "0 0"*
    1000 1000


This origin coordinate should be taken into account for pixel location 
(See OGC 05-047r3  p93):


    
        
             Index position of the first grid post, which
             shall lie somwhere in the GridEnvelope. If absent, the 
startPoint is equal

             to the value of gridEnvelope::low from the grid definition.
        
    


It seems that this offset is not taken into account by GDAL location 
function.


*2) How to reproduce*
2.1) Data creation
Reference (no offset) image:
gdal_create -of JP2OpenJPEG -co GMLJP2=YES -outsize 100 100 -bands 3 
-burn 0 -burn 255 -burn 0 -a_srs epsg:32632 -a_ullr 50 500100 
500100 50 Offset_0.jp2


Image with offset:
sed s/"0 0"/"7 7"/g Offset_0.jp2 > Offset_7.jp2

2.2) Location test
echo "0 0" | gdaltransform Offset_0.jp2 Offset_7.jp2
0 0 0

*3) Question:*
Shouldn't we have a 7 pixels offset between the 2 images

Cheers
Philippe


The information in this e-mail is confidential. The contents may not 
be disclosed or used by anyone other than the addressee. Access to 
this e-mail by anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus 
immediately and delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or 
completeness of this e-mail as it has been sent over public networks. 
If you have any concerns over the content of this message or its 
Accuracy or Integrity, please contact Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated 
virus scanning software but you should take whatever measures you deem 
to be appropriate to ensure that this message and any attachments are 
virus free.


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Expiring token during paging on the WFS

[Even Rouault via gdal-dev]

Just added in 
https://github.com/OSGeo/gdal/commit/a6c3b0450994028a60cef854fbe6304910c7e277 
: "The content of the
file is not cached, and thus it is read again before issuing each HTTP 
request."


Le 28/03/2024 à 10:26, Michael Otto via gdal-dev a écrit :

Hello,

I have an important question about the use of a 'GDAL_HTTP_HEADER_FILE'.

If I request a WFS with "OGR_WFS_PAGING_ALLOWED=ON" and use a header 
file for authentication, will the header file be read with every new page?


I am wondering how I should deal with an expiring bearer token 
(validity 5min) with a very long lasting query (>5min).


Questions:
Does ogr2ogr query the header file again for each new page?
Can I possibly set the authentication for every single requested page?
Is there another way to deal with expiring tokens?

Michael

PS:
I use Java and start the apps via the ProcessBuilder.
If the header file is re-read for each page, I could update it 
accordingly (preferably without conflicts due to simultaneous reading 
and writing ;o) ).


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] About -oo -lco -dsco in docs

[Even Rouault via gdal-dev]

Salut Thomas,

There's a tension between being clear to users, and also aiming at being 
concise / not repeating us too much (we have already one thousand pages 
of doc to maintain), and if we need to repeat, find the technical way of 
doing it without actually having to copy the same text.


Probably a solution could be to have under each "Open options" 
paragraph, and introduction sentence like


"Consult :ref:`how to specify open options ` with command 
line utilities and the API." that links to a page explaining the concept 
and how to use it.


I was wondering if we'd just want to copy that sentence at all 
relevant places, or create a custom Sphinx extension for a directive 
that would do that. The later is probably overkill. Possibly a one-time 
simple Python script could automatically insert that at relevant places


Even


Le 26/03/2024 à 21:25, Thomas Gratier via gdal-dev a écrit :

Hi,

There are implicit rules when you read the OGR/GDAL docs in fact, in 
particular when you use `ogr2ogr` or `ogrinfo`


`-oo =` like "Open Options"
`-lco =` for "Layer Creation Option"
`-dsco =` for "DataSet Creation Option"

So, when you read for instance 
https://gdal.org/drivers/vector/csv.html, you will see there is a 
section "Open options" so you will know that you can use `-oo`. Also, 
you will see a "Layer Creation options" block so again, you will know 
it mean use `-lco`.


It's documented in "ogr2ogr" but you need to know the rule to make the 
link in your mind when reading the doc. Any idea to clarify the doc 
about the "tip".


An idea could be about adding in drivers docs "Open options" block a 
sentence like "When using the command line, use the flag `-oo 
=`" and do the same for "Layer creation options" block 
"When using the command line, use the flag `-lco =`"


Thoughts about the suggestion? Better idea? Not worth?

Regards,

Thomas Gratier

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Incorrect geolocation information using newer gdal versions

[Even Rouault via gdal-dev]

Yes, some versions ago the netCDF driver became much more stricter, 
expecting a strict respect of the netCDF CF conventions for axis names & 
attributes, to avoid false identification of non-georeferenced axis, 
which could cause issues.


The below debug message gives a hint to use the 
GDAL_NETCDF_IGNORE_XY_AXIS_NAME_CHECKS=YES to relax those checks.


Cf https://gdal.org/drivers/raster/netcdf.html#configuration-options

$ gdalinfo 2024032100.nc --config 
GDAL_NETCDF_IGNORE_XY_AXIS_NAME_CHECKS YES

[...]
Origin = (24500.000,465500.000)
Pixel Size = (1000.000,-1000.000)
[...]
Corner Coordinates:
Upper Left  (   24500.000,  465500.000)
Lower Left  (   24500.000,   14500.000)
Upper Right (  375500.000,  465500.000)
Lower Right (  375500.000,   14500.000)
Center  (  20.000,  24.000)

What is at least missing in that particular file is the 
Easting:standard_name = "projection_x_coordinate" and 
Northing:standard_name = "projection_y_coordinate" attributes.


Even

Le 26/03/2024 à 20:02, Andrew C Aitchison via gdal-dev a écrit :

On Tue, 26 Mar 2024, Cristhian Rivera via gdal-dev wrote:


Hi all,

I'm trying to debug an issue with a NetCDF file where in previous gdal
versions (up until 3.3.0) the geolocation was correctly identified by
gdalinfo, but in newer versions (>= 3.3.1) it is not.


I note that both versions output the warnings:

Warning 1: dimension #2 (Easting) is not a Longitude/X dimension.
Warning 1: dimension #1 (Northing) is not a Latitude/Y dimension.

gdalinfo --debug on 2024032100.nc
gives me more info:

Warning 1: dimension #2 (Easting) is not a Longitude/X dimension.
Warning 1: dimension #1 (Northing) is not a Latitude/Y dimension.
GDAL_netCDF:
=
SetProjectionFromVar( 65536, 4)
GDAL_netCDF: got grid_mapping crs
GDAL_netCDF: bIsGdalFile=0 bIsGdalCfFile=0 bSwitchedXY=0 bBottomUp=1
netCDF: Georeferencing ignored due to non-specific enough X axis name. 
Set GDAL_NETCDF_IGNORE_XY_AXIS_NAME_CHECKS=YES as configuration option 
to bypass this check
netCDF: Georeferencing ignored due to non-specific enough Y axis name. 
Set GDAL_NETCDF_IGNORE_XY_AXIS_NAME_CHECKS=YES as configuration option 
to bypass this check
GDAL_netCDF: bGotGeogCS=0 bGotCfSRS=0 bGotCfGT=0 bGotCfWktSRS=0 
bGotGdalSRS=0 bGotGdalGT=0

GDAL_netCDF: did not get geotransform from CF nor GDAL!
GDAL_netCDF: did not get projection from CF nor GDAL!
GDAL_netCDF: netcdf type=5 gdal type=6 signedByte=1
GDAL: GDALOpen(2024032100.nc, this=0x63c988f650a0) succeeds as 
netCDF.

GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(2024032100.nc, this=0x63c988f650a0)


In the attachments are the outputs of gdalinfo from the versions 
mentioned,
but the difference in both outputs is that newer versions cannot find 
the

"Origin" and "Pixel size", and the Corner coordinates differ a lot:

*Gdal 3.3.0:*


Origin = (24500.000,465500.000)
Pixel Size = (1000.000,-1000.000)




Corner Coordinates:

Upper Left  (   24500.000,  465500.000)
Lower Left  (   24500.000,   14500.000)
Upper Right (  375500.000,  465500.000)
Lower Right (  375500.000,   14500.000)
Center  (  20.000,  24.000)




= *Gdal 3.3.1*



Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  451.0)
Upper Right (  351.0,    0.0)
Lower Right (  351.0,  451.0)
Center  (  175.5,  225.5)



The files can also be found in the following drive repo if they are not
available in the mail:
https://drive.google.com/drive/folders/1zqV3n2SYUhOjgwxpYpChwqdNTiHAgNHj?usp=sharing 




I've tried checking the changelog of that version but it seems only 
like a

bug fix release.
Any ideas of what else I can test to identify the issue?




--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ogr2ogr, GMLAS driver, help

[Even Rouault via gdal-dev]

Hi,

It is very difficult to create manually the table & field structure that 
will be generated by the GMLAS driver. It is likely that the attribute 
name generated by the GMLAs driver is not the one you expect. First try 
ogr2ogr to a not-yet-existing PotsgreSQL table to see if data is 
correctly ingested (on a quick try on my side with a very simple 
example, it is)


Even

Le 26/03/2024 à 11:01, Bence Megyesi via gdal-dev a écrit :

Hello,

i am importing an xml with ogr2ogr using GMLAS driver to a PostgreSQL 
server. I am also using an .xsd file to correctly import and manage 
the imported xml in the server. This way i was able to import every 
element into the correct column based on the xsd, except one, a 
type="xsd:time" element. In the server i created a table list and 
their columns with types based on the xsd, but this "Uhrzeit" 
element's value wont get imported into the server. I checked the type 
in PostgreSQL, which is "time without time zone".
The value in the xml is HH:MM:SS formated so in theory it should work, 
but it wont.

My question is: what adjustments can i do with my ogr2ogr command
ogr2ogr_command = [
    'ogr2ogr', '-skipfailures', '-f', 'PostgreSQL', 
f'PG:host={host} port={port} dbname={dbname} password={password} 
user={user} schemas={schema}',

    f'GMLAS:{xml_file_path}',
    '-oo', f'XSD={xsd_file_path}',
    '-oo', 'REMOVE_UNUSED_LAYERS=YES', '-oo', 
'REMOVE_UNUSED_FIELDS=YES', '-append', '-forceNullable'

    ]
to be able to import this type of data too?

I hope i am sending my question to the right person. If this is not 
the case, kindly disregard my message.


Best regards,
Bence

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] USGS 3DEP (3D Elevation Program) - feet should be metres, how can I fix this?

[Even Rouault via gdal-dev]

Hi,

having looked at the file, I believe GDAL behaves correctly given the 
metadata in the file. This is clearly a unit type attached to the Band, 
and it should be corrected by the data producer to reflect what is 
really used in it


One way to correct it on your side is for example this little Python snippet

from osgeo import gdal
ds = gdal.Open('test.tif')
ds.GetRasterBand(1).SetUnitType('metre')

As the file is used in read-only mode, this will generate an auxiliary 
.aux.xml side car file overriding the unit.


You can also directly override the wrong metadata inside the TIFF with

ds = gdal.OpenEx('test.tif', open_options=['IGNORE_COG_LAYOUT_BREAK=YES'])
ds.GetRasterBand(1).SetUnitType('metre')

The IGNORE_COG_LAYOUT_BREAK=YES open option is because modifying TIFF 
metadata causes it to be rewritten at the end of the file, which 
"breaks" the COG convention. But that's still a perfectly valid GeoTIFF 
file, and will not change anything for local use of such file. You could 
reprocess that to a fully compliant COG with gdal_translate -of COG


Regarding how that is used by gdalwarp, the doc enhancement at 
https://github.com/OSGeo/gdal/pull/9527 captures my initial attempt at 
answering your questions.


Even


Le 22/03/2024 à 01:16, Conrad Bielski a écrit :

Just realised I should have put you also in CC

Here is my response message again.

Thanks Even,

you are correct about the tag number (I sent the incorrrect one).

I have provided a sample tif and some other descriptions of the issue 
here:
bad_geotiff - Google Drive 
<https://drive.google.com/drive/folders/1LVJqoKrnLRf6_5FpE7a5SrUGIWBPQmqa?usp=sharing>





bad_geotiff - Google Drive

<https://drive.google.com/drive/folders/1LVJqoKrnLRf6_5FpE7a5SrUGIWBPQmqa?usp=sharing> 



The three files there are the following:
1.  A bad tiff (this is slightly smaller; they are all pretty big).
2.  A GDAL info dump (survey feet is almost at the bottom)
3. A PDF with an explanation of the issue

thanks for your help.
Conrad


On Thursday, March 21, 2024 at 04:49:57 PM EDT, Even Rouault 
 wrote:




Le 21/03/2024 à 21:45, Conrad Bielski via gdal-dev a écrit :
Hello GDALers,

I have a question about reading USGS 3DEP (3D Elevation Program) data. 
Inside of this data, a GEOTIFF tag 42114 is provided which is causing 
problems with datum shifts.


There's no such thing as a GEOTIFF tag 42114.

The closest tags are:

#define TIFFTAG_GDAL_METADATA 42112
#define TIFFTAG_GDAL_NODATA 42113

It would be helpuful if you could provide the file, or the result of a 
"tiffdump -m 1000 the.tif" and "listgeo the.tif" on it.


And was the .tif.aux.xml provided with the TIFF, or computed by GDAL. 
Because the information might come from it too.





So when I use GDAL to compute the datum shifts, the tag is read and 
interprets that the DEM is showing elevation in 'feet' while the DEM 
is actually in metres. The DEMs are in fact in meter elevations and 
meter UTM horizontal coordinates. This is obviously erroneously 
integrated into the tag.


So my question is whether anyone has come across this issue and found 
a solution? Is there a way to edit the offending tag so that it is 
correctly interpreted as metres instead of feet? Other potential 
solutions so that GDAL interprets the elevation correctly?


Thanks in advance for your help. I provide below the output of 
gdalinfo which I hope could help with the offending info in bold.


Have a great day,
Conrad




Driver: GTiff/GeoTIFF
Files: 
I:\v3_us_3dep\asheville\ashevilleUSGS_1M_17_x34y392_NC_Phase5_2018_A18.tif

 
I:\v3_us_3dep\asheville\ashevilleUSGS_1M_17_x34y392_NC_Phase5_2018_A18.tif.aux.xml
Size is 10012, 10012
Coordinate System is:
PROJCRS["NAD83 / UTM zone 17N",
    BASEGEOGCRS["NAD83",
        DATUM["North American Datum 1983",
            ELLIPSOID["GRS 1980",6378137,298.257222101,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4269]],
    CONVERSION["UTM zone 17N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-81,
ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",50,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,

Re: [gdal-dev] USGS 3DEP (3D Elevation Program) - feet should be metres, how can I fix this?

[Even Rouault via gdal-dev]

Le 21/03/2024 à 21:45, Conrad Bielski via gdal-dev a écrit :

Hello GDALers,

I have a question about reading USGS 3DEP (3D Elevation Program) data. 
Inside of this data, a GEOTIFF tag 42114 is provided which is causing 
problems with datum shifts.


There's no such thing as a GEOTIFF tag 42114.

The closest tags are:

#define TIFFTAG_GDAL_METADATA 42112
#define TIFFTAG_GDAL_NODATA 42113

It would be helpuful if you could provide the file, or the result of a 
"tiffdump -m 1000 the.tif" and "listgeo the.tif" on it.


And was the .tif.aux.xml provided with the TIFF, or computed by GDAL. 
Because the information might come from it too.




So when I use GDAL to compute the datum shifts, the tag is read and 
interprets that the DEM is showing elevation in 'feet' while the DEM 
is actually in metres. The DEMs are in fact in meter elevations and 
meter UTM horizontal coordinates. This is obviously erroneously 
integrated into the tag.


So my question is whether anyone has come across this issue and found 
a solution? Is there a way to edit the offending tag so that it is 
correctly interpreted as metres instead of feet? Other potential 
solutions so that GDAL interprets the elevation correctly?


Thanks in advance for your help. I provide below the output of 
gdalinfo which I hope could help with the offending info in bold.


Have a great day,
Conrad




Driver: GTiff/GeoTIFF
Files: 
I:\v3_us_3dep\asheville\ashevilleUSGS_1M_17_x34y392_NC_Phase5_2018_A18.tif

 
I:\v3_us_3dep\asheville\ashevilleUSGS_1M_17_x34y392_NC_Phase5_2018_A18.tif.aux.xml
Size is 10012, 10012
Coordinate System is:
PROJCRS["NAD83 / UTM zone 17N",
    BASEGEOGCRS["NAD83",
        DATUM["North American Datum 1983",
            ELLIPSOID["GRS 1980",6378137,298.257222101,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4269]],
    CONVERSION["UTM zone 17N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-81,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",50,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["(E)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["(N)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    USAGE[
        SCOPE["Engineering survey, topographic mapping."],
        AREA["North America - between 84°W and 78°W - onshore and 
offshore. Canada - Nunavut; Ontario; Quebec. United States (USA) - 
Florida; Georgia; Kentucky; Maryland; Michigan; New York; North 
Carolina; Ohio; Pennsylvania; South Carolina; Tennessee; Virginia; 
West Virginia."],

        BBOX[23.81,-84,84,-78]],
    ID["EPSG",26917]]
Data axis to CRS axis mapping: 1,2
Origin = (339993.58705157042,3920005.81591943651)
Pixel Size = (1.000,-1.000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
  LAYOUT=COG
  PREDICTOR=3
Corner Coordinates:
Upper Left  (  339994.000, 3920006.000) ( 82d45'43.98"W, 35d24'38.21"N)
Lower Left  (  339994.000, 3909994.000) ( 82d45'36.92"W, 35d19'13.36"N)
Upper Right (  350006.000, 3920006.000) ( 82d39' 7.17"W, 35d24'43.82"N)
Lower Right (  350006.000, 3909994.000) ( 82d39' 0.55"W, 35d19'18.95"N)
Center      (  345000.000, 3915000.000) ( 82d42'22.15"W, 35d21'58.63"N)
Band 1 Block=512x512 Type=Float32, ColorInterp=Gray
  Min=669.509 Max=1604.949
  Minimum=669.509, Maximum=1604.949, Mean=988.295, StdDev=159.804
  NoData Value=-99
  Overviews: 5006x5006, 2503x2503, 1251x1251, 625x625, 312x312
*Unit Type: US survey foot*
  Metadata:
    STATISTICS_APPROXIMATE=YES
    STATISTICS_MAXIMUM=1604.9490966797
    STATISTICS_MEAN=988.29463693706
    STATISTICS_MINIMUM=669.50915527344
    STATISTICS_STDDEV=159.80361228109
    STATISTICS_VALID_PERCENT=100


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [EXTERNAL] [BULK] Re: Experience with slowness of free() on Windows with lots of allocations?

[Even Rouault via gdal-dev]

I've played with VirtualAlloc(NULL, SINGLE_ALLOC_SIZE, MEM_COMMIT | 
MEM_RESERVE, PAGE_READWRITE), and it does avoid the performance issue. 
However I see that VitualAlloc() allocates by chunks of 64 kB, so 
depending on the size of a block, it might cause significant waste of 
RAM, so that can't be used as a direct replacement of malloc().


My inclination would be to perhaps have an optional config option like 
GDAL_BLOCK_CACHE_USE_PRIVATE_HEAP that could be set, and when doing so 
it would use HeapCreate(0, 0, GDAL_CACHEMAX) to create a heap only used 
by the block cache. Not ideal, since that would reserve the whole 
GDAL_CACHEMAX (but for a large enough processing, you'll end up 
consuming it), but it has the advantage of not being extremely intrusive 
either... and could be easily ditched/replaced by something better in 
the future.


Regarding tcmalloc, I've had to use it on Linux too, but only on 
scenarios involving multithreading where it helps reducing RAM 
fragmentation: cf 
https://gdal.org/user/multithreading.html#ram-fragmentation-and-multi-threading 
. I've just tried quickly to use it on Windows to test it on the 
scenario, but didn't really manage to make it work. Even building it was 
challenging. Actually I tried https://github.com/gperftools/gperftools 
and I had to build from master since the latest tagged version doesn't 
build with CMake on Windows. But then nothing happens when linking 
tcmalloc_minimal.lib against my toy app. I probably missed something.


Anyway I don't really think we can force tcmalloc to be used in GDAL, as 
a library. Unless there would be a way to have its allocator to be 
optionnaly used at places that we control (ie explicitly call tc_malloc 
/ tc_free), and not replace the default malloc / free etc, which might 
be undesirable when GDAL is just a component of a larger application.


Disabling entirely the block cache (or setting it to a minimum value) is 
only a workable option for uncompressed formats, or if you use per-band 
blocks (INTERLEAVE=BAND in GTiff language) and not one block for all 
bands (INTERLEAVE=PIXEL), otherwise you'll pay multiple time the 
decompression.


Le 21/03/2024 à 14:38, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND 
APPLICATIONS INC] via gdal-dev a écrit :


+1. We use a variety of hand-rolled VirtualAlloc based (for basic 
tasks, a simple pointer bump, and for more elaborate needs, a ‘buddy’) 
allocators, some of which try to be smart about memory usage via 
de-committing regions.  In our work, we tend to disable the GDAL cache 
entirely and rely on the file system’s file cache instead, which is a 
simplification we can make but is surely untenable in general here.


*From: *gdal-dev  on behalf of Abel 
Pau via gdal-dev 

*Reply-To: *Abel Pau 
*Date: *Thursday, March 21, 2024 at 4:51 AM
*To: *"gdal-dev@lists.osgeo.org" 
*Subject: *[EXTERNAL] [BULK] Re: [gdal-dev] Experience with slowness 
of free() on Windows with lots of allocations?


*CAUTION:*This email originated from outside of NASA.  Please take 
care when clicking links or opening attachments. Use the "Report 
Message" button to report suspicious messages to the NASA SOC.




Hi Even,

you’re right. We also know that. When programming the driver I took it 
in consideration. Our solution is not rely on windows to make a good 
job with memory and we try to reuse as memory as possible instead of 
use calloc/free freely.


For instance, in the driver, for each feature I have to get or write 
the coordinates. I could do it every time I have to, so lots of times: 
create memory for reading, and then put them on the feature, and then 
free... so many times. What I do? When opening the layer I create some 
memory blocs of 250 Mb (due to the format itself) and I use that 
created memory to manage whatever I need. And when closing, I free it.


While doing that I observed that sometimes I have to use GDAL code 
that doesn’t take it in consideration (CPLRecode()for instance). 
Perhaps it could be improves as well.


Thanks for noticing that.

*De:*gdal-dev  *En nombre de *Javier 
Jimenez Shaw via gdal-dev

*Enviado el:* dijous, 21 de març de 2024 8:27
*Para:* Even Rouault 
*CC:* gdal dev 
*Asunto:* Re: [gdal-dev] Experience with slowness of free() on Windows 
with lots of allocations?


In my company we confirmed that "Windows heap allocation mechanism sucks."

Closing the application after using gtiff driver can take many seconds 
due to memory deallocations.


One workaround was to use tcmalloc. I will ask my colleagues more 
details next week.


On Thu, 21 Mar 2024, 01:55 Even Rouault via gdal-dev, 
 wrote:


Hi,

while investigating
https://github.com/OSGeo/gdal/issues/9510#issuecomment-2010950408,
I've
come to the conclusion that the Windows heap allocation mechanism
sucks.
Basically if you allocate a lot of heap regions of modest size with
malloc()/new[], the time spent when freeing them all with
c

Re: [gdal-dev] Compiling Poppler plugin under Windows (missing dependencies with GisInterrnals)

[Even Rouault via gdal-dev]

Philippe,





I guess, I did not initialize POPPLER_EXTRA_LIBRARIES:

  * is its type STRING ?


Yes


  * what separator should I use between library name ?


Semicolon ';'


I see this was added per 
https://github.com/OSGeo/gdal/commit/95ee1f855cd and 
https://github.com/OSGeo/gdal/commit/d69257df35c8ae3c283dce2077b7c49720f99663 
, although I'm a bit puzzled why gisinternals build recipe itself 
doesn't need it.


Feel free to submit a pull request to 
https://gdal.org/development/building_from_source.html to document that 
variable


Even


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Experience with slowness of free() on Windows with lots of allocations?

[Even Rouault via gdal-dev]

Hi,

while investigating 
https://github.com/OSGeo/gdal/issues/9510#issuecomment-2010950408, I've 
come to the conclusion that the Windows heap allocation mechanism sucks. 
Basically if you allocate a lot of heap regions of modest size with 
malloc()/new[], the time spent when freeing them all with corresponding 
free()/delete[] is excruciatingly slow (like ~ 10 seconds for ~ 80,000 
allocations). The slowness is clearly quadratic with the number of 
allocations. You only start noticing it with ~ 30,000 allocations. And 
interestingly, another condition for that slowness is that each 
individual allocation much be strictly greater than 4096 * 4 bytes. At 
exactly that value, perf is acceptable, but add one extra byte, and it 
suddenly drops. I suspect that there must be a threshold from which 
malloc() starts using VirtualAlloc() instead of the heap, which must 
involve slow system calls, instead of a user-land allocation mechanism.


Anyone has already hit that and found solutions? The only potential idea 
I found until now would be to use a private heap with HeapCreate() with 
a fixed maximum size, which is a bit problematic to adopt by default, 
basically that would mean that the size of GDAL_CACHEMAX would be 
consumed as soon as one use the block cache.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] palette file format doc ?

[Even Rouault via gdal-dev]

Le 21/03/2024 à 01:43, Michael Sumner a écrit :




> And, can index be *value* in any contexts?

If you use a raster with a signed data type, that could be negative
values (assuming I understand your question)



ah I see, arbitrary integer values map to a colour 0:(n-1) colours, 
match the ordered n values in the raster - that is "value" in the way 
I was thinking  ( I had in mind more general cases like interval 
ranges, but that's  another level up)


I might be entirely messing up the concept ... will check my take by 
experiment.


It was wrong about negative values. As this script ultimately writes a 
GDALColorTable, you must use >=0 indices.



--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [External] : Re: Python binding installation on windows

[Even Rouault via gdal-dev]

Le 20/03/2024 à 23:11, Fengting Chen a écrit :


After upgrading setuptools, the installation on windows worked. Just 
curious that why “GDAL-3.8.4-py3.6.egg-info” is created under the 
site-packages on windows, while I set up the PYTHON_ROOT to use python 
3.12 and clean up the build directory to rebuild everything.


On the other hand, on linux, when using python 3.12 for the build, the 
installation gave a lot of warnings like 
“/scratch/gdal/gdal-3.8.4/dist/lib/python3.12/site-packages/osgeo/ogr.py:5954: 
SyntaxWarning: invalid escape sequence '\*'” . It seems that this is 
the new warning from python 3.12. But I don’t see this warning in the 
windows build, which made me suspect if I was really building on 
windows using python 3.12.


Which SWIG version do you use? And can you paste line 5954 (and a few 
ones before and after) of your ogr.py ?


I can't reproduce locally any warning with SWIG 4.1.1 and python 3.12.0 
on my Linux. And on our CI with Fedora Rawhide and Python 3.12.2 and 
SWIG 4.2.1, there is no warning either:


cf https://github.com/OSGeo/gdal/actions/runs/8365025972/job/22901857183


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] palette file format doc ?

[Even Rouault via gdal-dev]

Michael,

Le 20/03/2024 à 21:38, Michael Sumner via gdal-dev a écrit :

Is the palette_file  .txt format documented?

https://gdal.org/programs/gdalattachpct.html

It's mentioned in a few utilities, and created by tests but I couldn't 
find an existing example or a description (I guessed, incorrectly at 
first, leaving out the index column). I


The initial attachpct.py script only worked with a GDAL raster as the 
source for the palette. The support for text formats, or qml, qlr seems 
to be a later addition by Idan Miara, and the best doc is probably the 
code: 
https://github.com/OSGeo/gdal/blob/master/swig/python/gdal-utils/osgeo_utils/auxiliary/color_palette.py 
. You may also have a look at autotest/pyscripts/test_gdal_utils.py 
which has tests for this color palette reading code. Looking a bit at 
it, I suspect the GDAL text format refered is the one actually used by 
gdaldem color-relief: 
https://gdal.org/programs/gdaldem.html#color-relief , which is itself 
derived from the one used by GRASS r.colors . But the Python code (only 
Python) and the one used by gdaldem (only C++) are different, so I 
wouldn't exclude there might be some differences.


| take it that it is 0-255 range values, but is index 0-based or 1?

0-based, as (unsigned) raster values are


And, can index be *value* in any contexts?


If you use a raster with a signed data type, that could be negative 
values (assuming I understand your question)


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Feature Request: New -oo for VRT

[Even Rouault via gdal-dev]

Hi,

assuming you use a Unix shell, and using the fact that the content of 
the VRT itself can be used as the datasource name, you can just do:


gdalinfo "$(sed 's/foo/bar/' my.vrt)"

Even

Le 20/03/2024 à 17:24, Scott via gdal-dev a écrit :
It would be nice to have an open option to substitute a string of text 
with some other value, treating .vrt something like a template. Such as:


   -oo REPLACE="'sourceString'='targetString'"

Thoughts? Thanks for listening!
Scott


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Enhanced GeoParquet bounding-box based spatial filtering in master

[Even Rouault via gdal-dev]

Hi,

FYI, for those interested in GeoParquet, 
https://github.com/OSGeo/gdal/pull/9185 has now been merged into master. 
This is only effective on files generated with the updated driver that 
adds a bounding box column, and maximum efficiency is reached when 
sorting the features with the SORT_BY_BBOX=YES layer creation option. 
The bbox column is only used currently when opening a single GeoParquet 
file (i.e. not using the "dataset" type of opening where you point to a 
hierarchy of parquet files). See the pull request for more details.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [External] : Re: Python binding installation on windows

[Even Rouault via gdal-dev]

"cmake --build . --target gdal_GEOR" should shorten a bit the build 
time. It will try to rebuild the main "GDAL" target, but if it is 
already up to date, that should be just a matter for the build system to 
check that no files need to be rebuilt.


Le 20/03/2024 à 17:00, Fengting Chen a écrit :


Thanks for the suggestion. I can upgrade the setuptools and try again. 
Another question: is it possible to only build a specific driver 
plugin without rebuilding the GDAL?


*From: *Even Rouault 
*Date: *Tuesday, March 19, 2024 at 4:44 PM
*To: *Fengting Chen , 
gdal-dev@lists.osgeo.org 
*Subject: *Re: [External] : Re: [gdal-dev] Python binding installation 
on windows


Le 19/03/2024 à 21:38, Fengting Chen a écrit :

I copied out the command from install_python_Release.cmake and ran
it manually. Then it worked. Not sure why the command not invoked.
I don’t see error from “cmake –build . –target install –config
Release”.

My setuptools is 40.6.2.

Not sure if that's the reason, but that's pretty obsolete. 
swig/python/pyproject.toml now requires setuptools >= 67




--
http://www.spatialys.com  
<https://urldefense.com/v3/__http:/www.spatialys.com__;!!ACWV5N9M2RV99hQ!N3nIB7d9Q9apKmPbtbMITxjtMJmtZqXchODLrjPOsu8tiqHXsnajFBCx-lbUaTurwnt1im6MokPguTmhAGhhBLWTNET6$>
My software is free, but my time generally not.


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] overview sizes for COG

[Even Rouault via gdal-dev]

Michael,

Hi, can we specify overview sizes exactly?


No, the BuildOverviews() interface onlys take an array of overview 
factors. It is up to the driver implementation to decide how it computes 
the overview size from the main raster size and overview factor.


The COG driver is a bit of a special case, since it automagically 
compute overviews if they don't exist in the source dataset, and doesn't 
take an explicit list of overview factors. It simply divides by two 
successively, with floor truncation: 
https://github.com/OSGeo/gdal/blob/master/frmts/gtiff/cogdriver.cpp#L1092


I have this odd grid that is 36000x17999, and I get consequently yucky 
overview sizes:


gdal_create -outsize 36000 17999 -ot Int8 -co SPARSE_OK=YES -a_srs 
EPSG:4326 -a_ullr 0 17999 0 36000 weird.tif

gdal_translate weird.tif cog.tif  -of COG
gdalinfo cog.tif | grep Overviews
#  Overviews: 18000x8999, 9000x4499, 4500x2249, 2250x1124, 1125x562, 
562x281, 281x14


Should I worry about this?


Probably not ? :-)

Can the sizes be specified via Translate? (I can see how they could be 
generated externally and bundled, but I'm looking for a way at this 
high level).
I guess someone could potentially add a OVERVIEW_SIZES=W1xH1,W2xH2,... 
creation option to the COG driver if that was really needed...


( Maybe I shouldn't even care about the sizes ... and maybe I should 
resample the grid to be a better overall size, but I consider that out 
of scope here.)


I'm comparing with other COG creators, e.g. odc which makes tidy 
overviews ( I will explore how it's getting done).


It looks like they might use (previous_size + 1) / 2 (or ceil rounding)

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdal_rasterize processed GTiff filesize question

[Even Rouault via gdal-dev]

Le 19/03/2024 à 22:56, Rahkonen Jukka via gdal-dev a écrit :


Hi,

I suppose that something similar than with gdalwarp 
https://trac.osgeo.org/gdal/wiki/UserDocs/GdalWarp#GeoTIFFoutput-coCOMPRESSisbroken 
happens but with gdal_rasterize I think there are no tricks that could 
help.


Actually the -optim switch may help: 
https://gdal.org/programs/gdal_rasterize.html#cmdoption-gdal_rasterize-optim


I suspect "-optim raster" is what you would want to optimize raster size

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [External] : Re: Python binding installation on windows

[Even Rouault via gdal-dev]

Le 19/03/2024 à 21:38, Fengting Chen a écrit :


I copied out the command from install_python_Release.cmake and ran it 
manually. Then it worked. Not sure why the command not invoked. I 
don’t see error from “cmake –build . –target install –config Release”.


My setuptools is 40.6.2.

Not sure if that's the reason, but that's pretty obsolete. 
swig/python/pyproject.toml now requires setuptools >= 67



--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Python binding installation on windows

[Even Rouault via gdal-dev]

Hi,

Le 19/03/2024 à 20:14, Fengting Chen via gdal-dev a écrit :


Hi, I was able to build the GDAL with python binding on without error 
on windows. However, “cmake --build . --target  install --config 
Release” command doesn’t install the python site-packages etc to the 
specified “CMAKE_INSTALL_PREFIX” directory. The rest of the GDAL 
components were installed properly.  Did I miss anything?


What I can tell is that .github/workflows/cmake_builds.yml lines 480 
"cmake --build $GITHUB_WORKSPACE/build --config Release --target 
install" seems to do the job on CI :


https://github.com/OSGeo/gdal/actions/runs/8346148650/job/22842755713

"""
[...]

copying build\lib.win-amd64-cpython-310\osgeo\gdal.py -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\gdalconst.py -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo


copying build\lib.win-amd64-cpython-310\osgeo\gdalnumeric.py -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\gdal_array.py -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\gnm.py -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\ogr.py -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\osr.py -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\_gdal.cp310-win_amd64.pyd 
-> D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying 
build\lib.win-amd64-cpython-310\osgeo\_gdalconst.cp310-win_amd64.pyd -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying 
build\lib.win-amd64-cpython-310\osgeo\_gdal_array.cp310-win_amd64.pyd -> 
D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\_gnm.cp310-win_amd64.pyd 
-> D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\_ogr.cp310-win_amd64.pyd 
-> D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo
copying build\lib.win-amd64-cpython-310\osgeo\_osr.cp310-win_amd64.pyd 
-> D:\\a\\gdal\\gdal\\install-gdal\Lib\site-packages\osgeo

[...]

"""

which setuptools version do you use? which logs do you get?

Perhaps also try running "python setup.py install" manually from the 
swig/python build directory


Even

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Censor area in tiles of aerial image

[Even Rouault via gdal-dev]

Why not just trying?

Demo:

$ gdal_create -outsize 10 10 -burn 255 test.tif -a_srs EPSG:4326 -a_ullr 
0 10 10 0
$ gdal_rasterize -burn 0 
'{"type":"Polygon","coordinates":[[[2,2],[2,4],[4,4],[4,2],[2,2]]]}' 
test.tif

$ gdal_translate test.tif /vsistdout/ -of aaigrid
ncols    10
nrows    10
xllcorner    0.
yllcorner    0.
cellsize 1.
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 0 0 255 255 255 255 255 255
255 255 0 0 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255

That said, I've attempted to clarifiy it pe 
https://github.com/OSGeo/gdal/commit/4377ce22ec8d26b76760e8c1e4a9ffa2bd240b7d


Le 19/03/2024 à 13:42, Carsten Lockenkötter via gdal-dev a écrit :

Hi Frank,

I have read about gdal_rasterize but it seems it works a bit different 
as i need it.
Gdal_rasterize converts a vector layer to a raster layer with specific 
dimensions and create a new file, like a mask.
It could be work for me yes, because i publish the raster files as 
image mosiac at the geoserver.
Maybe the new "mask" file overlays on me original files and the 
created wmts tiles of the geoserver could contain the mask.


I will try it, but is there another option to burn the vectorlayer 
into existing tiles?


Regards,
Carsten
Am 19.03.24, 00:14 schrieb Frank Warmerdam :

Carsten,

The gdal_rasterize command allows you to "burn in" polygons from
an OGR supported datasource into an existing raster.  If your
raster is a 3 band RGB file, you could use --burn 100 150 200 to
burn in the RGB value (100,150,200).   This will only work if the
raster format you are using supports update-in-place.

You would have to regenerate pyramids after this process -- they
are not automatically updated by GDAL when the "base layer" is
updated.

Best regards,
Frank


On Mon, Mar 18, 2024 at 5:34 PM Carsten Lockenkötter via gdal-dev
 wrote:

Hi,

is it possible to censor specific areas of an aerial image
using GDAL?

I have several smaller tiles that I've already transformed
into my desired coordinate system and generated internal pyramids.

Subsequently, I would like to censor certain areas based on
polygons (e.g., from a shapefile or an Oracle DB) (coloring
them grayish).

Set the color must be done after transforming coordinatesystem
and generating pyramids.

I usually use the compiled Windows binaries from
gisinternals.com .

Presumably, my plan doesn't work with that, right? At least I
haven't found anything in that direction.

I suppose this could be done with a Python, but I've never
worked with Python before.

Do I need to adjust the internal pyramids as well? Or do I
have to recreate them?

Could you please show me a brief example of how it could work,
so I have an approach?

I just need an idea of how to implement this and possibly some
tips on what else I need to consider.

Regards,

Carsten

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev



-- 
---+--

I set the clouds in motion - turn up   | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | USA: +1 650-701-7823
and watch the world go round - Rush    | CAN: +1 343-550-9984


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Censor area in tiles of aerial image

[Even Rouault via gdal-dev]

Thomas,

Le 19/03/2024 à 08:26, thomas bonfort via gdal-dev a écrit :
I have a side-question concerning the update-in-place behavior of the 
gtiff driver in this case: given that a compressed strile will nearly 
always be smaller after this update (due to better compression ratios 
on the uniform area), will libtiff overwrite the previous strile in 
place also, or will it be appended to the existing file (resulting in 
a larger final file size)?


If the updated tile/strip occupies a smaller size than its previous 
version, it will overwrite in place. Note however that it will not wipe 
out the bytes that are no longer used, which might potentially be an 
issue in the context of censoring. Thus I would recommend doing a final 
gdal_translate to get a "clean" version before distributing it.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev