Re: [gdal-dev] Multi-threaded GDALPolygonize C++

[Even Rouault]

Shayan,

make sure to read https://gdal.org/user/multithreading.html . I suspect 
you use the same GDAL Dataset or RasterBand object from multiple 
threads, which isn't allowed


Even

Le 26/06/2023 à 18:38, Shayan Ghofrany a écrit :

Good afternoon everyone,

I'm new to the mailing list, so if my question is not formatted 
correctly please let me know.


I am attempting to polygonize a raster file by first chunking it into 
smaller pieces and then using multi-threaded to polygonize those 
chunks simultaneously. I often come across Disk I/O errors which makes 
me think that what I want to do is not necessarily how GDALPolygonize 
should be used. I was wondering if anyone has attempted something 
similar and whether there are public examples available that I could 
look into.


Thanks for your time,

Shayan Ghofrany

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Codepage or UTF-8 in a metadata file Filename.aux in the Erdas Imagine format, describing a Filename.tif

[Even Rouault]

Mikael,

to my surprise, the HFA format is actually published at 
https://hexagongeospatial.fluidtopics.net/r/fH0o7KrMKUViXGUeoilQuA/5DlRUpslzb6NK6uTz98KSg 
. Not sure if it is "new" or had already been available. From a quick 
look, it doesn't mention anything about string encoding.


My intuition would be that the encoding would be whatever the one of the 
machine generated the file was, but perhaps that's a fixed one. You 
could potentially try to ask Hexagon support about that.


GDAL itself makes not that many assumptions about the encoding, although 
it tries to expose as UTF-8 as much as possible (and recode to UTF-8 
when it knows the source encoding), otherwise it will present strings as 
they are, hoping for the best. But language bindings might make stronger 
assumptions and indeed misbehave when UTF-8 is not encountered


Even

Le 26/06/2023 à 11:43, Mikael Rittri a écrit :


Hello list.

I have encountered a Filename.tif with an associated metadata file, 
Filename.aux. The .aux file can be understood by gdalinfo, which says


Driver: HFA/Erdas Imagine Images (.img)

Files: Filename.aux

   Filename.rrd

   Filename.rde

As I understand it, the .aux file is on an Erdas Imagine format 
intended to describe metadata for the Erdas .img format, but it can 
also be used to describe metadata for .tif files as in my case. (I 
have the Filename.rrd and the Filename.rde but not any Filename.img, 
so it is somewhat strange but useful that GDAL can read the .aux file 
directly).


Anyway, my question is: when the Filename.aux contains strings, in my 
case descriptions of terrain types represented by integers (part of a 
Raster Attribute Table), is there an established way to figure out 
whether the strings are stored in UTF-8, or if not, what codepage is 
used? In my case, the strings seem to be stored as 8-bit ASCII using 
the codepage 1252 (mainly for West-European alphabets), but GDAL seems 
to expect UTF-8 so the Swedish characters with diacritics become garbled.


I realize that if the .aux format is proprietary and has just been 
reverse-engineered, then maybe no-one knows the answer to this. But I 
am curious if anyone has had similar problems and maybe figured out a 
workaround. Or if there are any grounds to say that UTF-8 is mandatory 
in the .aux format, then my example file would be incorrect and that 
would also be useful to know.


Best regards,

Mikael Rittri

Carmenta Geospatial Technologies

Sweden

carmenta.com


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Can we do Delaunay with 3D points?

[Even Rouault]

Joaquim,


Looked at the GDAL tests and there it also uses only 2D (a 3D test is 
commented).


I assume you're speaking about the PolyhedralSurface / TIN tests in 
ogr_geom.py.  The reason is that the code (and in particularly GEOS) is 
not ready for those geometry types.


I am doing something wrong or delaunay in GDAL is only possible with 
with 2D points?


Quick testing shows that it works with a multipoint z with a recent GEOS 
master version  (but might work with older ones perhaps)


>>> g = ogr.CreateGeometryFromWkt("MULTIPOINT Z (0 0 10,0 1 10,1 1 12,1 
0 10)")


>>> triangulation = g.DelaunayTriangulation()

>>> triangulation.ExportToWkt()

'GEOMETRYCOLLECTION (POLYGON ((0 1 10,0 0 10,1 0 10,0 1 10)),POLYGON ((0 
1 10,1 0 10,1 1 12,0 1 10)))'


Even


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] 3D extent

[Even Rouault]

(re-adding the list)

Le 23/06/2023 à 11:34, B. De Mezzo a écrit :


Thanks!

I will try this sql request to handle this specific case.

Do you think it could be possible to have a OGR_L_GetExtent3D() 
function which returns, as a default implementation, the 
OGR_L_GetExtent() result and with 0.0 (or NaN) for the Z coordinates? 
Future works can re-implement the OGR_L_GetExtent3D driver function to 
fit the real data.


That's technically possible, but at bit odd. I'd say that while at it, 
it would be much better to have a default generic implementation in the 
base OGRLayer class that does the iteration, just like the 2D version at 
https://github.com/OSGeo/gdal/blob/master/ogr/ogrsf_frmts/generic/ogrlayer.cpp#L230 
. Driver specializations could indeed come afterwards


Benoit.

Le 23/06/2023 à 11:25, Even Rouault a écrit :


Benoit,

Indeed OGR_L_GetExtent() will never fill the Z fields, since it has 
no idea they are there. So a 3D specific function would be needed.


But in the case of GeoPackage, as many/most other formats, layer 
extent metadata doesn't include the Z component either. The generic 
option is to iterate over all features and use OGR_G_GetEnvelope3D() 
on their geometry.


In the specific case of GeoPackage, you could also issue a 
ExecuteSQL("SELECT MIN(ST_MinZ(geometry_column_name)), 
MAX(ST_MaxZ(geometry_column_name)) FROM layer_name") which should be 
faster on big layers


Even

Le 23/06/2023 à 11:14, B. De Mezzo a écrit :


Hi,

I was looking for a solution to retrieve a 3D extent (in my case 
from a gpkg). Currently, the OGR_L_GetExtent function uses an 
OGREnvelope and when I pass a OGREnvelope3D parameter, it does not 
fill the Z fields.


What should be the best approach to the minZ and maxZ for a layer?

Regards.


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

--
http://www.spatialys.com
My software is free, but my time generally not.


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] 3D extent

[Even Rouault]

Benoit,

Indeed OGR_L_GetExtent() will never fill the Z fields, since it has no 
idea they are there. So a 3D specific function would be needed.


But in the case of GeoPackage, as many/most other formats, layer extent 
metadata doesn't include the Z component either. The generic option is 
to iterate over all features and use OGR_G_GetEnvelope3D() on their 
geometry.


In the specific case of GeoPackage, you could also issue a 
ExecuteSQL("SELECT MIN(ST_MinZ(geometry_column_name)), 
MAX(ST_MaxZ(geometry_column_name)) FROM layer_name") which should be 
faster on big layers


Even

Le 23/06/2023 à 11:14, B. De Mezzo a écrit :


Hi,

I was looking for a solution to retrieve a 3D extent (in my case from 
a gpkg). Currently, the OGR_L_GetExtent function uses an OGREnvelope 
and when I pass a OGREnvelope3D parameter, it does not fill the Z fields.


What should be the best approach to the minZ and maxZ for a layer?

Regards.


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] A Java SWIG question

[Even Rouault]

Barry,

The file 
"${SWIG_INSTALL_PREFIX}/share/swig/{swig_version_here}/java/typemaps.i 
seems to be the place where the default mappings from unsigned long long 
to BigInteger are defined. I guess taking inspiration from those 
typemaps and overriding them could be the way to go.


An alternative I'm thinking of would be to modify MultiDimensional.i 
with something like the following (untested! just to give the idea) 
added towards its top


#ifdef SWIGJAVA

// For practical reason we want Java long to be used instead of default 
BigInteger. This approximation isn't much a problem since there is no 
real use case to be able to address values in the range [INT64_MAX, 
UINT64_MAX] for dimension sizes


typedef GInt64 MDimUInt64;

#else

typedef GUInt64 MDimUInt64;

#endif

and then replacing occurrences of GUInt64 (at least those dealing with 
dimension sizes) with MDimUInt64


Not super pretty admitedly

Even

Le 21/06/2023 à 19:34, Barry DeZonia a écrit :
Thanks Even. I will try to throw that together later. But I think my 
question is much more simple. I will show a small blurb below from 
swig/include/java/typemaps_java.i


Code I have now follows right here. This code might conflict with 
current code because I am wanting GUInt64s to go to longs and not 
BigIntegers throughout the Java api. Note the code I am at a loss as 
to how to write would be " Typemaps for GUInt64 * ". In other words a 
C array of GUInt64's which I'd like to translate back and forth to 
long[]'s. Any example code would be appreciated.


/***
 * Typemaps for GUInt64
 ***/

%typemap(in) (GUInt64) "$1 = $input;"
%typemap(out) (GUInt64) {
    /* %typemap(out) (GUInt64) */
    $result = result;
}
%typemap(jni) (GUInt64) "jlong"
%typemap(jtype) (GUInt64) "long"
%typemap(jstype) (GUInt64) "long"
%typemap(javain) (GUInt64) "$javainput"
%typemap(javaout) (GUInt64) {
    return $jnicall;
}

I tried a similar template for GUInt64* and made "jlong" into 
"jlongArray" and "long" into "long[]" in those three places above but 
I have no idea how to write the (javain) and (javaout) cases.


For the direction Java Array -> C Array, I guess taking inspiration from 
DEFINE_REGULAR_ARRAY_OUT(int64_t, jlong, SetLongArrayRegion); should work





On Wed, Jun 21, 2023 at 3:35 AM Even Rouault 
 wrote:


Barry,

it could help if you could point to a public branch in your github
fork
with your pending work

Even

Le 21/06/2023 à 07:58, Barry DeZonia a écrit :
> Hello,
>
> I am trying to work on the swig java api. I've made a lot of
progress
> and am running into one last error before my code completely
compiles.
> I am getting a swig complaint when passing a java long[] to a C/C++
> call that is expecting GUInt64*.
>
> Basically the error message is "long[] cannot be converted to
> SWIGTYPE_p_GUInt64". I've been reading and googling and I can't
figure
> out how to modify typemaps_java.i to accomplish this. I think it
is a
> simple thing but I am at a loss. Does anyone have any tips?
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
http://www.spatialys.com

My software is free, but my time generally not.


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] A Java SWIG question

[Even Rouault]

Barry,

it could help if you could point to a public branch in your github fork 
with your pending work


Even

Le 21/06/2023 à 07:58, Barry DeZonia a écrit :

Hello,

I am trying to work on the swig java api. I've made a lot of progress 
and am running into one last error before my code completely compiles. 
I am getting a swig complaint when passing a java long[] to a C/C++ 
call that is expecting GUInt64*.


Basically the error message is "long[] cannot be converted to 
SWIGTYPE_p_GUInt64". I've been reading and googling and I can't figure 
out how to modify typemaps_java.i to accomplish this. I think it is a 
simple thing but I am at a loss. Does anyone have any tips?


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ESRI Shapefile API missing

[Even Rouault]

Derek,

This is intended. GDAL build shouldn't leak/expose internal 
implementation details such as its embedded shapelib copy, which doesn't 
belong to the GDAL API. Either you use the OGR Shapefile driver through 
OGR API, or you link against external shapelib if you want to use 
shapelib directly


Note that https://github.com/OSGeo/gdal/pull/7973 is pending to fix the 
isolation of the embedded shapelib copy


Even

Le 21/06/2023 à 03:01, Derek Newhall a écrit :

Hi all,

We've just migrated from GDAL 3.5.1 to 3.7.0 which we build with 
(almost) all external dependencies as static libraries. However, we've 
noticed that when we try to compile our app against the resulting 
library the ESRI Shapefile API isn't available.


Severity Code Description
Error LNK2001 unresolved external symbol DBFClose
Error LNK2001 unresolved external symbol SHPOpen

Error LNK2001 unresolved external symbol SHPCreateObject
Error LNK2001 unresolved external symbol SHPClose

Looking at the .libs' symbols:

DUMPBIN /linkermember D:\gdal-3.5.1\build\gdal_i.lib | sls DBF

   1ED958 DBFOpen
   1ED958 __imp_DBFOpen
   1ED9BC DBFOpenLL
   1ED9BC __imp_DBFOpenLL
   1ED3D0 DBFCreate
   1ED3D0 __imp_DBFCreate
   1ED436 DBFCreateEx
   1ED436 __imp_DBFCreateEx

     1257 __imp_DBFWriteNULLAttribute
     1256 __imp_DBFWriteStringAttribute
     125B __imp_DBFWriteTuple

DUMPBIN /linkermember D:\gdal-3.7.0\build\gdal.lib | sls DBF

      DBF OSRGetAxesCount
      DBF __imp_OSRGetAxesCount

We build the DLL and apps with the following command in a .bat script 
(all library paths are valid and it compiles):


cmake -G "Visual Studio 16 2019" -DBUILD_APPS=ON -DBUILD_SHARED_LIBS=ON ^
-DCMAKE_BUILD_TYPE=RelWithDebInfo 
-DCMAKE_PREFIX_PATH=D:\gdal-3.7.0-build ^
-DCMAKE_CXX_FLAGS="-DCURL_STATICLIB -DXML_STATIC" 
-DGDAL_USE_EXTERNAL_LIBS=OFF ^

-DOGR_BUILD_OPTIONAL_DRIVERS=ON ^
-DPROJ_INCLUDE_DIR="%~dp0proj-6.3.1\src" ^
-DPROJ_LIBRARY_RELEASE="%~dp0proj-6.3.1\build_VS2017_x64\lib\RelWithDebInfo\proj.lib" 
^
-DGDAL_USE_CURL=ON -DCURL_INCLUDE_DIR="%~dp0libcurl\include" 
-DCURL_LIBRARY_RELEASE="%~dp0libcurl\lib\libcurl.lib" ^
-DGDAL_USE_ECW=ON -DECW_INCLUDE_DIR="%~dp0ERDAS ECW JPEG 2000 SDK 
5.5.0\Desktop Read-Only\include" -DECW_LIBRARY="%~dp0ERDAS ECW JPEG 
2000 SDK 5.5.0\Desktop Read-Only\lib\vc141\x64\NCSEcw.lib" ^

-DGDAL_USE_ODBC=ON ^
-DGDAL_USE_OPENSSL=ON -DOPENSSL_ROOT_DIR="%~dp0openssl-1.1.0f-vs2017" 
-DOPENSSL_USE_STATIC_LIBS=TRUE -DOPENSSL_MSVC_STATIC_RT=TRUE ^

-DGDAL_ENABLE_DRIVER_PDF_PLUGIN=ON ^
-DGDAL_USE_PDFIUM=ON 
-DPDFIUM_INCLUDE_DIR="%~dp0ExtractPDFium\install\include\pdfium" 
 -DPDFIUM_LIBRARY="%~dp0ExtractPDFium\install\lib\pdfium.lib" ^
-DGDAL_USE_SQLITE3=ON 
-DSQLite3_INCLUDE_DIR="%~dp0sqlite-amalgamation-3370200" 
 -DSQLite3_LIBRARY="%~dp0sqlite-amalgamation-3370200\sqlite3.lib" ^
-DGDAL_USE_XERCESC=ON -DXercesC_INCLUDE_DIR="%~dp0xerces-c-3.2.4\src" 
-DXercesC_LIBRARY="%~dp0xerces-c-3.2.4\build\src\Release\xerces-c_3.lib" ^
-DGDAL_USE_EXPAT=ON -DEXPAT_INCLUDE_DIR="%~dp0Expat\lib" 
-DEXPAT_LIBRARY="%~dp0Expat\RelWithDebInfo\libexpat.lib" ^

..

The build documentation at 
https://gdal.org/development/building_from_source.html says the ESRI 
Shapefile API cannot be disabled, so I'm wondering what's going on.  
Is this a bug, or did we just misconfigure something? This is our 
first time semi-successfully building with the new CMake system.


Derek Newhall


*Disclaimer*

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and 
others authorized to receive it. If you are not the recipient, you are 
hereby notified that any disclosure, copying, distribution or taking 
action in relation of the contents of this information is strictly 
prohibited and may be unlawful.



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] OGR CSV oddity

[Even Rouault]

yep, known oddity. Presumably out of caution, the CSV driver refused for 
years to open a CSV with a single column, so on the writing side, it 
adds this dummy comma. Since recently (3.7.0 I believe), the read side 
of the CSV accepts single column header without trailing comma, but the 
write side still generates one


Le 17/06/2023 à 17:54, Scott a écrit :
If I select a single column from a table, the header column name has a 
trailing comma. If I select more than 1 column it does not. Example:


ogr2ogr -f CSV -sql "select col1 from table limit 1" /vsistdout/ 
source.gpkg


Output:

col1,
123

Tested on 3.5.1 and 3.7

I've tried this on various column data types with the same result.

Pseudo suggestion:
header += header.length ? "," + colname : colname


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] libtiff 4.5.1 is released

[Even Rouault]

Le 14/06/2023 à 17:57, Jeff McKenna a écrit :
Thanks Even.  One question (for packagers), is if the tools you 
mentioned to be removed are set through -Dtiff-contrib correct?


This is still to be decided how we implement this. There's a discussion 
in https://gitlab.com/libtiff/libtiff/-/issues/580


"contrib" are even less supported/maintained stuff. We should probably 
not build them by default either.


Een

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] libtiff 4.5.1 is released

[Even Rouault]

Hi,

I've promoted rc3 as the final 4.5.1 release.

Read about this release at:
https://libtiff.gitlab.io/libtiff/releases/v4.5.1.html

Note the following warning:
This version will be the last one supporting most TIFF tools (except 
tiffinfo,

tiffdump, tiffcp and tiffset), whose maintenance will be discontinued, due
to the lack of contributors able to address reported security issues.
Starting with libtiff v4.6.0, their source code, at this time ,will still be
available in the source distribution, but they will no longer be built by
default, and issues related to them will no longer be accepted in the
libtiff bug tracker.

The release tarball can be downloaded at:

https://download.osgeo.org/libtiff/tiff-4.5.1.tar.gz
https://download.osgeo.org/libtiff/tiff-4.5.1.tar.xz
https://download.osgeo.org/libtiff/tiff-4.5.1.zip

Signatures:
https://download.osgeo.org/libtiff/tiff-4.5.1.tar.gz.sig
https://download.osgeo.org/libtiff/tiff-4.5.1.tar.xz.sig
https://download.osgeo.org/libtiff/tiff-4.5.1.zip.sig

The release is also available at:
https://gitlab.com/libtiff/libtiff/-/releases/v4.5.1

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Compatibility of GDAL binaries and GDAL jar files

[Even Rouault]

Le 13/06/2023 à 22:27, Dirk Stenger a écrit :

All,

I am using GDAL with an Java application and have a couple of 
questions regarding the compatibility of releases of the GDAL binaries 
and the GDAL jar files provided via Maven Central Repository [1].


The GDAL jar files provided via Maven Central Repository never contain 
a bug fix version (e.g. 3.5.0, 3.6.0, 3.7.0). The GDAL binaries, 
however, use bug fix versions (e.g. 3.6.4).
How do I match the versions? Should I, for example, use GDAL jar 3.6.0 
with GDAL binary 3.6.4? 


Yes, that should work

What is the release policy of the versions in Maven Central 
Repository, is there always a new jar release whenever there is a GDAL 
release of a minor version?
We generally don't release .jar for patch versions (the z in x.y.z), 
unless there would be a bugfix for the bindings themselves


Is there any compatibility scheme/matrix for using the GDAL jar file 
with GDAL binaries?
For example, can minor versions be combined (e.g. using GDAL jar 3.0.0 
with GDAL binary 3.6.4) or should there always be exact version matching?


You could likely using a GDAL jar of a lesser version than the binary, 
since the bindings rely on the C API/ABI which is stable within the same 
major version number (but the reverse would be true. GDAL jar of 3.6 
might use C symbols missing in libgdal < 3.6).


But such mixed version combinations are not tested by our CI : the CI 
only tests running the bindings against the same GDAL native version.


I'm not totally sure you can run a .jar against the 
libgdalalljni.so/.dll that would have been generated with a different 
version of SWIG though. I assume that if you remain with the same major 
version of SWIG that should work OK, but that's not tested either.


--

http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Minimum python version?

[Even Rouault]

We're still at 3.6

Le 13/06/2023 à 19:22, Kurt Schwehr a écrit :

Hi all,

What is the minimum python version for the GDAL python bindings? This 
is the only thing I found easily:


find . -type f | grep -v git | xargs grep 3 | grep python_version

cat autotest/requirements.txt
pytest>=6.0.0
pytest-sugar<=0.9.6; python_version < '3.7'
pytest-sugar; python_version >= '3.7'
pytest-env
lxml
jsonschema
filelock

I think the real answer is that it's 3.8 from what I see here:

swig/python/osgeo/__init__.py

# The following is a Sequence of tuples in the form of (gdal_version, 
python_version).
# Each line represents the minimum supported Python version of a given 
GDAL version.
# Introducing a new line for the next GDAL version will trigger a 
deprecation warning

# when importing osgeo from a Python version which will not be
# supported in the next version of GDAL.
gdal_version_and_min_supported_python_version = (
    ((0, 0), (0, 0)),
    ((1, 0), (2, 0)),
    ((2, 0), (2, 7)),
    ((3, 3), (3, 6)),
    # ((3, 4), (3, 7)),
    # ((3, 5), (3, 8)),
)

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Creation of a new driver from scratch

[Even Rouault]

Abel,

cf 
https://github.com/OSGeo/gdal/commit/bcd4ed241503a545c1dee2819849d54117ac4e78 
for a commit that adds a new vector driver.


See the additions in ogr/ogrsf_frmts/CMakeLists.txt , 
ogr/ogrsf_frmts/generic/ogrregisterall.cpp, 
ogr/ogrsf_frmts/ogrsf_frmts.h and frmts/drivers.ini


And similarly for a raster driver: 
https://github.com/OSGeo/gdal/commit/27203405bc33723163b3c9a66a7f8dbb82be3e9f


Even

Le 09/06/2023 à 12:11, Abel Pau a écrit :

Hi again.

Finally I managed to use CMake to create a Visual Studio sln. And it's 
beautiful see all drivers there.
But it's not crear for me what I have to modify (Cmakelist.txt,.. or something 
else) to include the new driver. I need it to be there to be able to debug.
I created a copy of, let's say, Shape and modified the Cmakelists.txt in an 
apropriate way... but I miss something because recompiling and rebuilding 
doesn't change anything and my driver is not in the project.

Anyone can make me a simple list of which files I have to modify?
THANKS in advance!!!

Abel.

-Mensaje original-----
De: Even Rouault 
Enviado el: divendres, 2 de juny de 2023 20:34
Para: Abel Pau 
CC: gdal-dev@lists.osgeo.org; Howard Butler 
Asunto: Re: [gdal-dev] Creation of a new driver from scratch



It is indeed desired that the code style of the contribution matches GDAL's 
style to some degree. GDAL provides a .clang-format file that might be helpful 
when you are editing to provide some automated conformance.

Cf https://gdal.org/development/dev_practices.html#commit-hooks also to install 
pre-commit hooks to ensure the formatting is OK & fix it if not.

https://gdal.org/development/rfc/rfc8_devguide.html also gives some hints on 
the general development practices. Generally try to stick with the conventions 
you observe in source code of drivers you take inspiration from.


You might be able to build a sln file using GDAL's CMake configuration, but 
there is no standard documentation about how to do this. Do know that most any 
configuration you provide must build and work with the standard GDAL CMake 
configuration which is typically invoked through the command line.

The source of "truth" will be the CMakeLists.txt file.

sln files are just a possible by-product of CMake that isn't stored in git.

You may just google "Cmake visual studio" as there's nothing GDAL specific 
regarding this topic. ==>
https://learn.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio?view=msvc-170
among other links that you'll get


· Once I have a Visual Studio solution (sln) to inspire myself, how 
should I proceed to create the driver? Is there any pre-existing pattern or 
template to follow?

Find an existing raster or vector driver in the project that matches MiraMon's 
vector or raster format and use it as a skeleton.


I would like to be able to debug it.

Familiarize yourself with GDAL's tests (a good place to learn is to look at 
GDAL's CI configuration), get tests running locally on your machine, and then 
write tests that provide test coverage for your raster or vector driver.

For vector drivers, the test_ogrsf utility that is built with GDAL is used by 
most unit tests of vector drivers to check their compliance w.r.t expectations 
of what a driver should do/not do. Not that of course this is only generic 
testing. You also need to add tests that check that the content you read from a 
test file is the one you expect.


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalinfo on large vrt takes a long time

[Even Rouault]

William,

it might be perhaps related to the GetMinimum() call done by gdalinfo. 
Cf https://trac.osgeo.org/gdal/ticket/5444


But normally it should only try to open the first source, and not all of 
them. At least that's what I could confirm on a quick testing. But I do 
see that the CanUseSourcesMinMaxImplementations() method will stat() 
sources whose filename looks like a local file (obviously if that's a 
mounted file system / vpn thing, it will not realize it is remote).


Try setting the VRT_MIN_MAX_FROM_SOURCES=NO environment variable / 
configuration option to see if that makes a difference. If it does, the 
CanUseSourcesMinMaxImplementations() logic should be modified to avoid 
doing those stat's().


If that's confirmed to be linked to GetMinmum(), you may also workaround 
the issue by doing a "gdalinfo -stats the.vrt" (from the server) to have 
statistics incorporated in the VRT, then GetMinimum() should be instant


Even

Le 09/06/2023 à 00:43, William Kyngesburye a écrit :

I'm writing a script that needs some info from a vrt raster, and one has 
thousands of files. When reading the vrt over the internet (vpn to our server) 
it takes a long time. I think it's looking at every file of the vrt.  What is 
gdal reading from the files that's not in the vrt itself? I used all the -no 
options and I'm not adding any other info options like checksums or stats. I 
just need basic info from the vrt that's in the vrt file.

-
William Kyngesburye



Don't Panic
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Two questions about Java API

[Even Rouault]

Sorry, missed the question

You need to create a typemap for "(int nList, GUIntBig* pList)" that can 
be mapped to an a long[] in Java. You probably don't care about the 
range of uint64_t (GUIntBig is a GDAL specific alias of uint64_t) that 
cannot be mapped to a int64_t, so no need to go with Biginteger[]


Such typemap should be similar to the existing "(int nList, int* pList)" one

SWIG indeed generates rather dummy SWIGTYPE_p_ types when there are 
missing typemaps. When such types appear in the Java API it means that a 
typemap is missing


Even


Le 07/06/2023 à 18:03, Barry DeZonia a écrit :
Thanks Even. I'll look at this later. But you only answered one of my 
questions. The type SWIGTYPE_p_GUIntBig is specified in the Java API 
for CreateAttribute(). No such thing is shown in the python tests you 
showed me. I would suspect because python has unlimited range integers 
and an array for them will work just fine, But this weird SWIG type in 
Java seems misplaced. What should the signature be? dimensions as 
lon[]? dimensions as Biginteger[]? And if so then why is the current 
signature so messy with this SWIG type. If it is supposed to be this 
weird SWIG type then how do I create it? The Java api javadoc does not 
show it has any useful methods.


On Wed, Jun 7, 2023 at 3:47 AM Even Rouault 
 wrote:


Barry,

you can't instantiate the objects directly. They are tied to an
underlying driver.

You could for example port this subset of those tests of the MEM
driver:

https://github.com/OSGeo/gdal/blob/master/autotest/gdrivers/memmultidim.py#L85

and

https://github.com/OSGeo/gdal/blob/master/autotest/gdrivers/memmultidim.py#L188

. Note that they test a bit more than just the API, but also the
implementation of the MEM driver, so you don't need to test all error
cases for example

Even

Le 07/06/2023 à 04:54, Barry DeZonia a écrit :
> Hello,
>
> I am trying to extend the Java API so that the MDArray code is more
> full featured. Right now I am working on extending Attribute to
return
> a Vector when Attribute::GetDimensions() is called.
Much of
> this support is missing and I am in the process of wiring it up.
I am
> starting to write test code to drive development and have two
> questions about my test code. I am pasting the code below with
> questions inlined.
>
> private static void testGetDimensions() {
>
>     // QUESTION 1:
>     // how do I create a Group programmatically?
>
>     Group group1 = new Group("universe1");  // NOPE
>
>     Group group2 = Group.CreateGroup("universe2");  // NOPE
>
>     // QUESTION 2:
>     // What is a SWIGTYPE_p_GUIntBig?
>     //  Is this type exposed in MDArray API as a flaw in the
current
> Java API?
>     //  See API docs for MDArray::CreateAttribute()
>
>     group1.CreateAttribute("jane",
>                                 1,
>                                 null,  // wants an array of
> SWIGTYPE_p_GUIntBig
> ExtendedDataType.Create(gdalconst.GDT_UInt16));
>
>     Attribute attribute = group.GetAttribute("jane");
>
>     Vector dims = attribute.GetDimensions();
> }
>
> So can anyone answer my two questions? Am I approaching this right?
> Thanks for any info.
>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
http://www.spatialys.com

My software is free, but my time generally not.


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Two questions about Java API

[Even Rouault]

Barry,

you can't instantiate the objects directly. They are tied to an 
underlying driver.


You could for example port this subset of those tests of the MEM driver: 
https://github.com/OSGeo/gdal/blob/master/autotest/gdrivers/memmultidim.py#L85 
and 
https://github.com/OSGeo/gdal/blob/master/autotest/gdrivers/memmultidim.py#L188 
. Note that they test a bit more than just the API, but also the 
implementation of the MEM driver, so you don't need to test all error 
cases for example


Even

Le 07/06/2023 à 04:54, Barry DeZonia a écrit :

Hello,

I am trying to extend the Java API so that the MDArray code is more 
full featured. Right now I am working on extending Attribute to return 
a Vector when Attribute::GetDimensions() is called. Much of 
this support is missing and I am in the process of wiring it up. I am 
starting to write test code to drive development and have two 
questions about my test code. I am pasting the code below with 
questions inlined.


private static void testGetDimensions() {

    // QUESTION 1:
    // how do I create a Group programmatically?

    Group group1 = new Group("universe1");  // NOPE

    Group group2 = Group.CreateGroup("universe2");  // NOPE

    // QUESTION 2:
    // What is a SWIGTYPE_p_GUIntBig?
    //  Is this type exposed in MDArray API as a flaw in the current 
Java API?

    //  See API docs for MDArray::CreateAttribute()

    group1.CreateAttribute("jane",
                                1,
                                null,  // wants an array of 
SWIGTYPE_p_GUIntBig

ExtendedDataType.Create(gdalconst.GDT_UInt16));

    Attribute attribute = group.GetAttribute("jane");

    Vector dims = attribute.GetDimensions();
}

So can anyone answer my two questions? Am I approaching this right? 
Thanks for any info.



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Java API: how to find dimensions of MDArray

[Even Rouault]

Le 03/06/2023 à 18:16, Barry DeZonia a écrit :
Ok, thanks. I will consider this for some of my future 
weekend-playtime-programming days. Yes, I'm a masochist.

great, we'd definitely welcome a few ones :-)


Are there other typemaps needed that would further strengthen the Java 
api support?


Basically most #if defined(SWIGPYTHON) in 
https://github.com/OSGeo/gdal/blob/master/swig/include/MultiDimensional.i 
<https://github.com/OSGeo/gdal/blob/master/swig/include/MultiDimensional.i#L159> 
around a method are an indication that a typemap is missing.


Like:

- 
https://github.com/OSGeo/gdal/blob/92f9cee9a9ed3f114d3e6a63d095ceb09ce85eec/swig/include/MultiDimensional.i#L210 
for CreateMDArray() (resolution of that one should be close to the 
existing "(int object_list_count, GDALRasterBandShadow **poObjects)" 
Java typemap)


- 
https://github.com/OSGeo/gdal/blob/92f9cee9a9ed3f114d3e6a63d095ceb09ce85eec/swig/include/MultiDimensional.i#L464 
for GetCoordinateVariables() (resolution of that one should be very 
close to the GetDimensions() one)


- 
https://github.com/OSGeo/gdal/blob/master/swig/include/MultiDimensional.i#L477 
for GetProcessingChunkSize() (resolution should be close to the existing 
"(int* pnCountOut, int** outErrorCodes)" java typemap


- etc..



On Sat, Jun 3, 2023 at 11:03 AM Even Rouault 
 wrote:



Le 03/06/2023 à 17:45, Barry DeZonia a écrit :

Thanks Even.

I might be able to find some time to do the SWIG work depending
upon the amount of development time needed. I am a long time
developer with experience in Java, C, and C++ (but have not used
SWIG before). Is it just a few signatures I need to write or do I
need to write full support for a GDALDimension class? Can you
comment on this?


Yes this is "just" a few signatures to map a C array of (existing)
GDALDimensionH objects to a vector of Java counterpart
org.gdal.gdal.Dimension objects. The org.gdal.gdal.Dimension class
already exists, with a "Dimension(long cPtr, boolean cMemoryOwn)"
constructor taking the value of the underlying C pointer as a Java
long. The cMemoryOwn boolean should probably be passed to true in
that context, since the C GDALDimensionH objects need to be freed
at the finalization of the Java Dimension object. This is really
about glueing stuff, but that said writing SWIG typemaps is a
whole adventure in itself... You may find relevant documentation
at
https://www.swig.org/Doc4.1/SWIGDocumentation.html#Java_typemaps
.  The hint I gave for the existing closest typemap should
hopefully put you on the right track, or at least as close as
possible.



Am I right in understanding that, as of the current Java
implementation, one really can't use MDArrays for much? Like if I
can't find the number of (... z planes, t steps, channels, etc.)
of data I will not be able to reconstruct the data?


Yes the multidim API is probably currently hardly usable outside
C, C++ and Python due to the lack of a few critical typemaps for
the other languages.

Even



On Sat, Jun 3, 2023 at 6:09 AM Even Rouault
 wrote:

Barry,

This method is indeed not available currently in the Java
bindings. It is only available currently in the Python
bindings (see

https://github.com/OSGeo/gdal/blob/master/swig/include/MultiDimensional.i#L159
) , since it requires writing a specific SWIG typemap for
each binding language when a method returns (or takes as
argument) a new non-primitive type such as here, with an
array of dimensions. The closest existing Java typemap I
found that can be used to take inspiration from is

https://github.com/OSGeo/gdal/blob/master/swig/include/java/typemaps_java.i#L235
but there would be changes to call the "Dimension(long cPtr,
boolean cMemoryOwn)" constructor.  Whether you want to try to
tackle that yourself or not, you may create a ticket about that

Even

Le 03/06/2023 à 09:09, Barry DeZonia a écrit :

On a related note is the Java api code in a public repo
somewhere? It would be helpful to look at that code
sometimes. (Like is GetDimensions() present in the Java code
but not exposed as a public method?)

On Sat, Jun 3, 2023 at 12:18 AM Barry DeZonia
 wrote:

Hi all,

I have access to an MDArray. I am trying to find its
dimensions. In the C++ API I can see that GDALMDArray
has a method called GetDimensions() to find the info I
need. But I am programming in Java and the Java API does
not show such a call for MDArray. Is there some way in
Java to find the info I need?


___
gdal-dev mailing l

Re: [gdal-dev] Java API: how to find dimensions of MDArray

[Even Rouault]

Le 03/06/2023 à 17:45, Barry DeZonia a écrit :

Thanks Even.

I might be able to find some time to do the SWIG work depending upon 
the amount of development time needed. I am a long time developer with 
experience in Java, C, and C++ (but have not used SWIG before). Is it 
just a few signatures I need to write or do I need to write full 
support for a GDALDimension class? Can you comment on this?


Yes this is "just" a few signatures to map a C array of (existing) 
GDALDimensionH objects to a vector of Java counterpart 
org.gdal.gdal.Dimension objects. The org.gdal.gdal.Dimension class 
already exists, with a "Dimension(long cPtr, boolean cMemoryOwn)" 
constructor taking the value of the underlying C pointer as a Java long. 
The cMemoryOwn boolean should probably be passed to true in that 
context, since the C GDALDimensionH objects need to be freed at the 
finalization of the Java Dimension object. This is really about glueing 
stuff, but that said writing SWIG typemaps is a whole adventure in 
itself... You may find relevant documentation at 
https://www.swig.org/Doc4.1/SWIGDocumentation.html#Java_typemaps .  The 
hint I gave for the existing closest typemap should hopefully put you on 
the right track, or at least as close as possible.




Am I right in understanding that, as of the current Java 
implementation, one really can't use MDArrays for much? Like if I 
can't find the number of (... z planes, t steps, channels, etc.) of 
data I will not be able to reconstruct the data?


Yes the multidim API is probably currently hardly usable outside C, C++ 
and Python due to the lack of a few critical typemaps for the other 
languages.


Even



On Sat, Jun 3, 2023 at 6:09 AM Even Rouault 
 wrote:


Barry,

This method is indeed not available currently in the Java
bindings. It is only available currently in the Python bindings
(see

https://github.com/OSGeo/gdal/blob/master/swig/include/MultiDimensional.i#L159
) , since it requires writing a specific SWIG typemap for each
binding language when a method returns (or takes as argument) a
new non-primitive type such as here, with an array of dimensions.
The closest existing Java typemap I found that can be used to take
inspiration from is

https://github.com/OSGeo/gdal/blob/master/swig/include/java/typemaps_java.i#L235
but there would be changes to call the "Dimension(long cPtr,
boolean cMemoryOwn)" constructor.  Whether you want to try to
tackle that yourself or not, you may create a ticket about that

Even

Le 03/06/2023 à 09:09, Barry DeZonia a écrit :

On a related note is the Java api code in a public repo
somewhere? It would be helpful to look at that code sometimes.
(Like is GetDimensions() present in the Java code but not exposed
as a public method?)

On Sat, Jun 3, 2023 at 12:18 AM Barry DeZonia
 wrote:

Hi all,

I have access to an MDArray. I am trying to find its
dimensions. In the C++ API I can see that GDALMDArray has a
method called GetDimensions() to find the info I need. But I
am programming in Java and the Java API does not show such a
call for MDArray. Is there some way in Java to find the info
I need?


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


-- 
http://www.spatialys.com

My software is free, but my time generally not.


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Java API: how to find dimensions of MDArray

[Even Rouault]

Barry,

This method is indeed not available currently in the Java bindings. It 
is only available currently in the Python bindings (see 
https://github.com/OSGeo/gdal/blob/master/swig/include/MultiDimensional.i#L159 
) , since it requires writing a specific SWIG typemap for each binding 
language when a method returns (or takes as argument) a new 
non-primitive type such as here, with an array of dimensions. The 
closest existing Java typemap I found that can be used to take 
inspiration from is 
https://github.com/OSGeo/gdal/blob/master/swig/include/java/typemaps_java.i#L235 
but there would be changes to call the "Dimension(long cPtr, boolean 
cMemoryOwn)" constructor.  Whether you want to try to tackle that 
yourself or not, you may create a ticket about that


Even

Le 03/06/2023 à 09:09, Barry DeZonia a écrit :
On a related note is the Java api code in a public repo somewhere? It 
would be helpful to look at that code sometimes. (Like is 
GetDimensions() present in the Java code but not exposed as a public 
method?)


On Sat, Jun 3, 2023 at 12:18 AM Barry DeZonia  wrote:

Hi all,

I have access to an MDArray. I am trying to find its dimensions.
In the C++ API I can see that GDALMDArray has a method called
GetDimensions() to find the info I need. But I am programming in
Java and the Java API does not show such a call for MDArray. Is
there some way in Java to find the info I need?


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Create a GDAL VRT for a 3D image that transposes the image dimensions

[Even Rouault]

You may use the multidimensional API for that : 
https://gdal.org/drivers/raster/vrt_multidimensional.html . Typically 
the  element is relevant. Note however that on-the-fly 
transposing can have bad performance, but for the relatively small image 
you mention, that might be OK. That said, you'd may be better using the 
MDim API ( https://gdal.org/tutorials/multidimensional_api_tut.html) 
directly to get a numpy array and use numpy native transpose capabilities.


Le 02/06/2023 à 18:30, Daniel Scheffler a écrit :

Hi!

I have an image with 500 rows, 1000 columns, and 2 bands in a NetCDF 
file which stores the data in the following scheme:

- dimension 1: rows
- dimension 2: bands
- dimension 3: columns

If I now do gdal.Open('filepath').GetRasterBand(1).ReadAsArray(), I 
get a numpy array in the shape 500x2. Is it somehow possible to create 
a GDAL VRT that accesses the data as rows/columns/bands instead of 
rows/bands/columns? If yes, how would I create such a VRT using the 
Python API?


Best,
Daniel


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Creation of a new driver from scratch

[Even Rouault]

It is indeed desired that the code style of the contribution matches GDAL's 
style to some degree. GDAL provides a .clang-format file that might be helpful 
when you are editing to provide some automated conformance.


Cf https://gdal.org/development/dev_practices.html#commit-hooks also to 
install pre-commit hooks to ensure the formatting is OK & fix it if not.


https://gdal.org/development/rfc/rfc8_devguide.html also gives some 
hints on the general development practices. Generally try to stick with 
the conventions you observe in source code of drivers you take 
inspiration from.




You might be able to build a sln file using GDAL's CMake configuration, but 
there is no standard documentation about how to do this. Do know that most any 
configuration you provide must build and work with the standard GDAL CMake 
configuration which is typically invoked through the command line.


The source of "truth" will be the CMakeLists.txt file.

sln files are just a possible by-product of CMake that isn't stored in git.

You may just google "Cmake visual studio" as there's nothing GDAL 
specific regarding this topic. ==> 
https://learn.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio?view=msvc-170 
among other links that you'll get





· Once I have a Visual Studio solution (sln) to inspire myself, how 
should I proceed to create the driver? Is there any pre-existing pattern or 
template to follow?

Find an existing raster or vector driver in the project that matches MiraMon's 
vector or raster format and use it as a skeleton.


I would like to be able to debug it.

Familiarize yourself with GDAL's tests (a good place to learn is to look at 
GDAL's CI configuration), get tests running locally on your machine, and then 
write tests that provide test coverage for your raster or vector driver.
For vector drivers, the test_ogrsf utility that is built with GDAL is 
used by most unit tests of vector drivers to check their compliance 
w.r.t expectations of what a driver should do/not do. Not that of course 
this is only generic testing. You also need to add tests that check that 
the content you read from a test file is the one you expect.


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Can block offsets ever be negative?

[Even Rouault]

Le 01/06/2023 à 20:39, Laurențiu Nicola via gdal-dev a écrit :

Hello,

Just a quick question: can block offsets (such as those in in 
GDALReadBlock or IRasterIO) ever be negative?


No, this isn't allowed.


Is there any format that supports that?

No


GDAL uses signed integers everywhere, which is fine, but the intention 
isn't always clear at times.

Doc enhancements to clarify that welcome


Thanks,
Laurentiu Nicola

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] OAPIF Unexpected Content-Type

[Even Rouault]

Denis,

You have to identify on which request this occurs

Run with CPL_DEBUG=ON CPL_CURL_VERBOSE=ON

This error is emitted when the server issues a response with a missing 
Content-Type header.


Le 01/06/2023 à 01:52, Denis Rykov a écrit :
I'm getting "ERROR 1: Unexpected Content-Type: (null)" in the middle 
of the ogrinfo output. What does this mean? I can adjust server 
configuration but it is not clear which endpoint is failing because 
apart from that message everything seems to be working fine.


$ ogrinfo OAPIF:http://localhost:8080/api/resource/6750/oapif/ layer_6209
INFO: Open of `OAPIF:http://localhost:8080/api/resource/6750/oapif/'
      using driver `OAPIF' successful.

Layer name: layer_6209
Metadata:
  TITLE=Test layer
Geometry: Point
ERROR 1: Unexpected Content-Type: (null)
Feature Count: 26
Extent: (0.00, 0.00) - (110.275300, 49.493019)
Layer SRS WKT:
GEOGCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
num: Real (0.0)
link: String (0.0)
wikipedia: String (0.0)

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [PROJ] Migrating to PROJ-9

[Even Rouault]

Le 30/05/2023 à 20:01, Nicolas Bellaiche a écrit :
>
>
> That's why I get:
>
>  echo 3356123.5400 1303218.3090 5247430.6050 | PROJ_DEBUG=3 
./bin/cs2cs IGNF:RGR92 +to IGNF:REUN47

>
> pj_open_lib(proj.ini): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/proj.ini) 
- succeeded

>
> pj_open_lib(proj.db): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/proj.db) 
- succeeded

>
>  instantiate_crs type: 10:
>
>  instantiate_crs type: 10:
>
> helmert: pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
>
> helmert: pj_ellipsoid - final:    ellps=GRS80
>
> helmert: Helmert parameters:
>
> helmert: x=  -789.99000  y=  627.33300  z=  89.68500
>
> helmert: rx= -0.60720  ry= -76.80190  rz= 10.56800
>
> helmert: s=  32.20830  exact=0  convention=position_vector
>
> helmert: dx=  0.0  dy=  0.0  dz=  0.0
>
> helmert: drx= 0.0  dry= 0.0  drz= 0.0
>
> helmert: ds=  0.0  t_epoch= 0.0
>
> helmert: Transformation parameters for observation t_obs=0 (t_epoch=0):
>
> helmert: x: -789.99
>
> helmert: y: 627.333
>
> helmert: z: 89.685
>
> helmert: s: 3.22083e-05
>
> helmert: rx: -2.94379e-06
>
> helmert: ry: -0.000372346
>
> helmert: rz: 5.12351e-05
>
> helmert: theta: 0
>
> helmert: Rotation Matrix:
>
> helmert:   |  1  -5.12351e-05  -0.000372346 |
>
> helmert:   |  5.12351e-05   1   2.94379e-06 |
>
> helmert:   |  0.000372346  -2.94379e-06   1 |
>
> 3353420.95 1304075.02 5248935.14
>
>
>
> =
>
>
> (base) nbellaiche@MAC1709W003-2 Release % echo 3356123.5400 
1303218.3090 5247430.6050 | PROJ_DEBUG=3 ./bin/cs2cs +init=IGNF:RGR92 
+to +init=IGNF:REUN47

>
> pj_open_lib(proj.ini): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/proj.ini) 
- succeeded

>
> pj_open_lib(proj.db): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/proj.db) 
- succeeded

>
> pj_open_lib(IGNF): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF) 
- succeeded

>
> get_init: searching cache for key: [IGNF:RGR92]
>
> pj_open_lib(IGNF): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF) 
- succeeded

>
> get_init: searching on in init files for [IGNF:RGR92]
>
> get_init_string: searching for section [RGR92] in init file [IGNF]
>
> pj_open_lib(IGNF): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF) 
- succeeded

>
> key=IGNF:RGR92, value: [title=Reseau geodesique Reunion 1992 
proj=geocent towgs84=0.,0.,0. a=6378137. 
rf=298.257222101 units=m no_defs]

>
> get_init: got [title=Reseau geodesique Reunion 1992 proj=geocent 
towgs84=0.,0.,0. a=6378137. rf=298.257222101 units=m 
no_defs], paralist[0,1]: [title=Reseau,geodesique]

>
>  instantiate_crs type: 19:
>
> pj_open_lib(IGNF): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF) 
- succeeded

>
> get_init: searching cache for key: [IGNF:REUN47]
>
> pj_open_lib(IGNF): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF) 
- succeeded

>
> get_init: searching on in init files for [IGNF:REUN47]
>
> get_init_string: searching for section [REUN47] in init file [IGNF]
>
> pj_open_lib(IGNF): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF) 
- succeeded

>
> key=IGNF:REUN47, value: [title=Reunion 1947 proj=geocent 
towgs84=789.5240,-626.4860,-89.9040,0.6006,76.7946,-10.5788,-32.324100 
a=6378388. rf=297.0 units=m no_defs]

>
> get_init: got [title=Reunion 1947 proj=geocent 
towgs84=789.5240,-626.4860,-89.9040,0.6006,76.7946,-10.5788,-32.324100 
a=6378388. rf=297.0 units=m no_defs], paralist[0,1]: 
[title=Reunion,1947]

>
>  instantiate_crs type: 19:
>
> pj_open_lib(IGNF): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF) 
- succeeded

>
> get_init: searching cache for key: [IGNF:RGR92]
>
> pj_open_lib(IGNF): call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF) 
- succeeded

>
> get_init: searching cache for key: [IGNF:REUN47]
>
> noop: pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
>
> noop: pj_ellipsoid - final:    ellps=GRS80
>
> 3356123.54 1303218.31 5247430.61
>
>
>
>

> Does it give a hint of what's wrong?


Yes, note the call 
fopen(/Users/nbellaiche/dev/sd-externs/macosx-clang/proj-9.1.1/nad/IGNF 
. I believe I mentioned in a early response that this IGNF file should 
no longer be used in modern PROJ versions. The IGNF file apparently uses 
the towgs84 parameter of the "REUNION-PITON DES NEIGES vers RGR92 
(RESEAU GEODESIQUE DE LA REUNION 1992)" transformation. That said, I'm 
not sure which one is the best as both the one used by current PROJ 
versions (RGR92 (RESEAU GEODESIQUE DE LA REUNION 1992) vers 
REUNION-PITON DES NEIGES)) and that one could apply...


--
http://www.spatialys.com
My software is free, but my 

Re: [gdal-dev] Improving details of the project build system and/or documentation

[Even Rouault]

PDAL had this penalty and then we did some profiling and saw that most of the cost was the dlopen, not the 
finding of the plugin via the filesystem. PDAL doesn't have a GDALOpen-like single entry method where drivers 
are all expected to be loaded and "tried" sequentially, however. We have a map that registers some 
properties of the "filename" that tells us which driver to actually dlopen, which is managed by 
libpdal.so. Users also have the ability to explicitly say "open this file with this driver at this 
dll/so location", which allows them to short cut the map.


yes my profiling also shows that it is essentially due to the dlopen() time.

Loading only the plugin you need, in an automated way, is not currently 
possible, because the Identify() method is in plugins. For simple 
identification by file extensions, we could imagine having some registry 
mapping extension(s) to plugin name, but the Identify() method can do 
much more complicated stuff





- offering the possibility of building just a driver as a plugin in a unit way 
would require people to build against the same GDAL headers as libgdal.

Does it really? If communication across the boundary of the plugin to GDAL is 
using public GDAL pointers/classes/methods that haven't changed in many 
releases, does the plugin version actually need to perfectly match the main 
library version?


Possibly 2 issues:

- you need to build a driver against the headers of the library you are 
going to run it against: even if the API remains compatible, you still 
need to build your plugins against GDAL headers of the same major.minor 
version due to ABI changes that occur almost at every feature release 
due to new virtual members or other such ABI incompatible change.


- you could possibly build driver FOO from GDAL 3.X sources against 
headers of GDAL 3.Y, but that may not be possible for all X,Y 
combinations due to some utility methods being added / changed / 
removed. For example in GDAL 3.6 we have remove the _GetProjectionRef() 
compatibility layer of GDAL 3.0. So if you build driver FOO from GDAL < 
3.6 against GDAL 3.6 headers, that isn't going to work (and probably the 
reverse)




For PDAL it hasn't, and we have used older binary plugins against newer main 
libraries with success. PDAL's interface in this regard is smaller, however, so 
the risk of changes causing problems are less. You could also wire in some 
explicit plugin versioning if you wanted a way to force a bump. I think if this 
were a community desire and priority, it could be done.


The ABI that plugins directly or indirectly depend on is quite large. I 
don't see how we could have a stable ABI, at least not without 
significant complication and bloat.


Requiring when building a standalone plugin that you use the same X.Y 
GDAL version for the plugin source and the GDAL headers doesn't seem a 
unreasonable requirement to me.



--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] WFS requests failing with /vsicurl/ against QGIS Server: missing Content-Length header

[Even Rouault]

Jorge,

Which error for which invocation of the WFS driver do you get ? Your 
below tries are already low level attempts.


e.g the below invocation works for me without any error with GDAL master 
(in "ghcr.io/osgeo/gdal:alpine-small-3.7.0" Docker image too)


ogr2ogr tmp.gpkg 
"WFS:https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS";


The WFS driver doesn't use /vsicurl/ but /vsicurl_streaming/ . The 
former requires HTTP GET Range request support, while the later doesn't.


and when using "ogrinfo http://something";, you are *not* using /vsicurl 
or /vsicurl_streaming. You are using the hidden HTTP driver that 
downloads locally the file before passing it to a driver.


I don't believe there is an issue with QGIS Server. It probably streams 
its output (didn't check though), so it cannot know in advance the 
Content-Length. The headers of the HEAD requests aren't particularly 
helpful too. Looking at the ones of the GET request, I see 
"Transfer-Encoding: chunked" for QGIS server, which is in line with a 
streamed output and OK


Even

Le 30/05/2023 à 00:23, Jorge Gustavo Rocha a écrit :

Hi,

I'm getting errors with WFS calls to QGIS Server using /vsicurl/. 
Against Geoserver I didn't notice this issue.


I can work on a solution, if the problem is on the QGIS Server side. I 
would love to have a second opinion on this.


What I have found:

1) The is no Content-Length: header in QGIS Server answer:

curl -L -I 
"https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS&version=2.0.0&request=GetFeature&typename=ponto_cotado&srsname=EPSG:5016";

HTTP/1.1 200 OK
Date: Mon, 29 May 2023 22:12:32 GMT
Server: Apache/2.4.41 (Ubuntu)
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, GET, OPTIONS, DELETE, PUT
Access-Control-Max-Age: 1000
Access-Control-Allow-Headers: x-requested-with, Content-Type, origin, 
authorization, accept, client-security-token

Content-Type: text/xml; subtype=gml/2.1.2; charset=utf-8

2) The is a Content-Length: header in Geoserver answer:

curl -L -I 
"https://wssig3.azores.gov.pt/geoserver/ows?service=WFS&request=GetFeature&typename=ef:EnvironmentalMonitoringFacility&version=2.0.0&srsname=EPSG:5015";

HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
Content-Type: application/gml+xml; version=3.2
Content-Disposition: inline; filename=features.xml
Content-Length: 494599
Server: Jetty(9.4.18.v20190429)

Example of testing calls:

This works (without vsicurl):
ogrinfo 
"https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS&version=2.0.0&request=GetFeature&typename=ponto_cotado&srsname=EPSG:5016";
INFO: Open of 
`https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS&version=2.0.0&request=GetFeature&typename=ponto_cotado&srsname=EPSG:5016'

  using driver `GML' successful.
1: ponto_cotado

This does not work:
ogrinfo 
"/vsicurl/https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS&version=2.0.0&request=GetFeature&typename=ponto_cotado&srsname=EPSG:5016"; 
--debug on
HTTP: libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 
libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib 
nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.14

VSICURL: HEAD did not provide file size. Retrying with GET
VSICURL: 
GetFileSize(https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS&version=2.0.0&request=GetFeature&typename=ponto_cotado&srsname=EPSG:5016): 
response_code=200, curl error msg=Failure writing output to destination

VSICURL: Request at offset 0, after end of file
VSICURL: Request at offset 0, after end of file
VSICURL: 
GetFileList(/vsicurl/https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin)

VSICURL: HEAD did not provide file size. Retrying with GET
VSICURL: 
GetFileSize(https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS&version=2.0.0&request=GetFeature&typename=ponto_cotado&srsname=EPSG:5011): 
response_code=200, curl error msg=Failure writing output to destination

VSICURL: Request at offset 0, after end of file
VSICURL: Request at offset 0, after end of file
VSICURL: Request at offset 0, after end of file
VSICURL: Request at offset 0, after end of file
ogrinfo failed - unable to open 
'/vsicurl/https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS&version=2.0.0&request=GetFeature&typename=ponto_cotado&srsname=EPSG:5016'.

DGNv8: Driver cleanup

Using docker, I can not use /vsicurl/ with WFS too:
docker run -ti --rm osgeo/gdal ogrinfo 
"/vsicurl/https://irig-madeira.pt/postgresql/drote/public/baram_2020/cgi-bin/qgis_mapserv.fcgi?service=WFS&version=2.0.0&request=GetFeature&typename=ponto_cotado&srsname=EPSG:5016"; 
--debug on
HTTP: libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 
libid

Re: [gdal-dev] Improving details of the project build system and/or documentation

[Even Rouault]

Now that the project has made it to CMake, I wonder if there is enthusiasm to 
break apart GDAL's build system to take advantage of its plugin capability in a 
default fashion to cut down on the massiveness of our default 
stuff-you-actually-want build situation. People rightly complain that 
libgdal.so links too much stuff, and in most cases, people don't need it all.

What if we were to default to having drivers be free-standing CMake projects 
that installed versioned plugin shared objects into known locations that could 
be activated at library load (or use) time? GDAL has this dynamic plugin 
capability right now, but it is not widely used, and build configuration is not 
so convenient for its usage. Plugins would allow each of these drivers to link 
to their esoteric dependencies without polluting the common library, and the 
situation we have now, where multiple drivers have similar sets of 
dependencies, could be handled in some kind of hierarchical fashion based on 
common dependencies.

For deployments like rasterio's wheels, external GDAL plugins could be loaded 
at runtime by the rasterio libgdal.so, and rasterio wouldn't need to be 
responsible for building a libgdal.so that provided kitchen sink capabilities. 
People could show up with their own driver plugins that included dependencies 
as needed.

GDAL's reputation of being hard to build and deploy is well earned and 
deserved. IMO, the CMake configuration has increased its build speed, system 
flexibility, and commonality on supported platforms, but it hasn't addressed a 
core challenge which is GDAL links to the world one DLL at a time. I don't know 
if the project wants to prioritize this topic relative to other issues, but we 
have some tools and resources available to us to improve the situation if so.


Some thoughts/questions regarding this:

- you definitely don't want to build all drivers (that can be built as 
plugins) as plugins. This works (this is actually my dev setup, to 
ensure that the core exports all the symbols needed for drivers built as 
plugins), but I've found there's a measurable penalty in doing so (300 
ms to load the 127 plugins on my system, ie each GDAL command line 
invokation will at least take 300 ms, which might be a significant 
penalty in some workflows) . So GDAL_ENABLE_PLUGINS_NO_DEPS=YES (cf 
https://gdal.org/development/building_from_source.html#cmdoption-arg-GDAL_ENABLE_PLUGINS_NO_DEPS-BOOL) 
is typically discouraged.  But 10 or so plugins should be fine.


- you can "emulate" building a single driver as plugin, by re-running a 
GDAL build with just the dependencies of that driver, and setting 
-DGDAL_ENABLE_DRIVER__PLUGIN=ON. The only build artifact of interest 
in that use case is the [gdal|ogr]_XXX.so/dll corresponding to that 
driver. The main inconvenience I can see with this current approach is 
that you have to pay the price for the core lib to be rebuilt, whereas 
with the free-standing CMake project, you'd point to the installed GDAL 
headers & lib


- by free-standing CMake project, what do you mean ? Couldn't that just 
the existing frmts//CMakeLists.txt file that would detect it is 
called as the top level CMakeLists.txt (probably by checking 
${CMAKE_CURRENT_SOURCE_DIR} == ${PROJECT_SOURCE_DIR}) and then change 
its logic to find libgdal and its dependencies ? And users would "cmake 
-S frmts/ -B plugin_" to configure just for that plugin.


- I don't understand the "where multiple drivers have similar sets of 
dependencies, could be handled in some kind of hierarchical fashion 
based on common dependencies". why is a special case needed if several 
drivers share the same dependencies ? Or maybe you were thinking to 
drivers depending on other drivers ? That situation can indeed happen 
currently, but when it happens, such drivers aren't allowed to be built 
as plugins. They have to be built with libgdal. Both because it makes 
the build system simpler, and also to avoid deployment complication 
where you'd have to know that gdal_.so links against gdal_.so.


- we would need to keep the current monolithic CMake project approach 
working, at the very least for people doing static builds, but even for 
people doing dynamic builds as it might be still be more practical if 
you can install all the dependencies at once. Having the standalone 
capability in addition would be an extra complexity, so we need to be 
sure it addresses real use cases, and perhaps restrict it to a few 
select drivers. Obvious candidates for standalone CMake projects would 
likely be for the few drivers that have proprietary dependencies (ECW, 
JP2KAK, MrSID, Oracle, ...) and aren't shipped by FOSS binary distributions.


- offering the possibility of building just a driver as a plugin in a 
unit way would require people to build against the same GDAL headers as 
libgdal. So people would have to download the same version of the GDAL 
sources as the one used by the libgdal provided by raster

Re: [gdal-dev] Improving details of the project build system and/or documentation

[Even Rouault]

Is MVT really a default driver?


You're right. It isn't. Sorry for the confusion


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Improving details of the project build system and/or documentation

[Even Rouault]

Hi Sean,


Using the CMake files as a source of truth helps. But they don't 
describe everything. For example, they don't explain that GDAL needs 
libhdf5 to be built with special features. That one stumped me for a 
while. I was seeing a build message like "HDF5 not detected (found 
version 1.12.1)". I finally found the clue in the vcpkg repo.


==> https://gdal.org/development/building_from_source.html#hdf5: "The 
HDF5 CXX library is needed for the KEA 
<https://gdal.org/drivers/raster/kea.html#raster-kea> driver."


That said, it isn't obvious to realize that detection of HDF5 fails 
because of that. That's a bit of a downside with CMake Find modules 
whose output isn't always very helpful.


Actually, I've changed the logic in 
https://github.com/OSGeo/gdal/pull/7851 to only check for HDF5 CXX if we 
have found libkea before. That reduces the requirements to what is 
really needed.




Also, the CMake files don't explain that the GDAL MBTiles driver 
depends on the OGR MVT driver.


Also fixed per https://github.com/OSGeo/gdal/pull/7851.  It is easy to 
miss capturing such dependencies to a driver that is built-in by default


Even




On Mon, May 22, 2023 at 8:53 AM Even Rouault 
 wrote:


Hi Sean,

I presume you got this because you defined
OGR_BUILD_OPTIONAL_DRIVERS=OFF which then cause
OGR_ENABLE_DRIVER_AVC to be set to OFF.

We already a quite overwhelming amount of documentation in
https://gdal.org/development/building_from_source.html about all
the existing variables, and I'm not sure adding an exhaustive list
of all the inter driver dependencies will be user-digestable and
could rot easily.

That said in https://github.com/OSGeo/gdal/pull/7806 I've tried to
improve the current error message with a hint for the reason for
the error and the (likely) reason for it to happen.

I've also added pointers in the doc page to the CMakeLists.txt
files where the dependencies are expressed. Hopefully people who
are in the business of making custom GDAL builds can make sense of
that.

Even

Le 20/05/2023 à 01:26, Sean Gillies a écrit :

Hi all,

I really appreciate the documentation at
https://gdal.org/development/building_from_source.html. But there
are gaps. For example, today I ran into

CMake Error at cmake/helpers/GdalDriverHelper.cmake:331 (message):
GDAL_ENABLE_DRIVER_AIGRID cannot be enabled because condition
OGR_ENABLE_DRIVER_AVC is not met. To ignore this error (but the
driver
will not be built), set the GDAL_IGNORE_FAILED_CONDITIONS variable

This dependence isn't documented. It's a bit frustrating to work
through missing directives one at a time when adding new drivers
to my build.

Is it possible for a driver's dependencies to be enabled when I
enable a driver?

If not, can we consider using GDAL maintenance funding and
resources on documenting the heck out of this system?




--
Sean Gillies

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Spatial view in GeoPackage

[Even Rouault]

Jukka,

currently the driver doesn't try to use the RTree of the base table (it 
could probably do it, but it would require first to check that both the 
FID column and geometry column of the same underlying table are available)


Jukka, Bo,

Regarding the use of Spatialite functions, I think the cleanest way is 
to define an extension to register such computed geometry column. That's 
what I've done in 
https://github.com/OSGeo/gdal/pull/7848/commits/8d7468f5342e3e9e54c11c913b8ca53f3effc4b3 
. It is the responsibility of the user creating the view to register the 
extension for its table, as show in 
https://github.com/OSGeo/gdal/blob/8d7468f5342e3e9e54c11c913b8ca53f3effc4b3/doc/source/drivers/vector/gpkg_spatialite_computed_geom_column.rst#extension-name-or-template


The 
https://github.com/OSGeo/gdal/blob/master/swig/python/gdal-utils/osgeo_utils/samples/validate_gpkg.py 
script (once updated by the PR) will use that to avoid erroring out.


Even

Le 28/05/2023 à 19:12, Rahkonen Jukka a écrit :


Hi,

Fixed for this use case, but what if user wants to use  ST_Tranform(… 
 in the view? Or something that may change the geometry type like 
ST_Intersection, ST_Buffer etc? In these cases I think that at least 
the usage of the spatial index of the main table should be disabled. 
But how could GDAL know when to skip the spatial index and when not? 
With a special column in gpkg_ogr_contents perhaps?


Somehow I do not feel comfortable. Maybe there should be a warning in

https://gdal.org/drivers/vector/gpkg.html#spatial-views that 
SpatiaLite functions should be used carefully in the views and such 
databases should be delivered only for known users, while views with 
uncomputed geometries should be safe and interoperable.


-Jukka Rahkonen-

*Lähettäjä:*gdal-dev  *Puolesta 
*Even Rouault

*Lähetetty:* sunnuntai 28. toukokuuta 2023 0.39
*Vastaanottaja:* Bo Victor Thomsen ; 
gdal-dev@lists.osgeo.org

*Aihe:* Re: [gdal-dev] Spatial view in GeoPackage

Bo Victor,

there was indeed a bug in the OGR GPKG driver preventing you to do 
what you wanted. Fixed per https://github.com/OSGeo/gdal/pull/7848


The issue was specific to the fact that the geometry column of the 
view is computed, and not just the simple selection of one of a table.


Note that such GeoPackage will probably only be readable by 
implementations that install Spatialite (or Spatialite compatible 
functions). It might also be prudent to wrap the result of ST_Multi() 
with AsGPB() to force creating a GeoPackage geometry blob instead of a 
Spatialite one. The OGR GPKG driver is tolerant to receiving both 
types of geometries, but technically the content should be a 
GeoPackage one.


Inserting into gpkg_ogr_contents is optional. Its purpose is to 
accelerate counting the number of features. But for layers < let's say 
100 k features, it probably doesn't impact performance much.


Even

Le 27/05/2023 à 22:08, Bo Victor Thomsen a écrit :

Hi list -

I'm working with making spatial views "visible" for QGIS, i.e. so
they look like a read-only table from the data source manager.

However, I have one problem:  QGIS doesn't recognize the crs for
the view. I've tracked the problem down to that *OGR* dosn't
recognize the crs for the view. Hence this mail...

What I've done:

*- creating the view: A union of parcels with the same owner
(CVR-number)
CREATE VIEW marker_2019_cvr AS
  SELECT "CVR", SUM(COALESCE(st_area(geom),0.0)) as area_m2,
count(*) as antal, st_multi(st_union(geom)) as geom from
marker_2019 group by 1;

-- insert or update gpkg_contents
INSERT OR REPLACE INTO

gpkg_contents(table_name,data_type,identifier,min_x,min_y,max_x,max_y,srs_id)
  SELECT
    'marker_2019_cvr' AS table_name,
    'features' AS data_type,
    'marker_2019_cvr' AS identifier,
    MIN(ST_MinX(geom)) AS min_x,
    MIN(ST_MinY(geom)) AS min_y,
    MAX(ST_MaxX(geom)) AS max_x,
    MAX(ST_MaxY(geom)) AS max_y,
    25832 AS srs_id -- same srs as the base layer for the view
  FROM marker_2019_cvr;

-- insert or update gpkg_geometry_columns
INSERT OR REPLACE INTO gpkg_geometry_columns ( table_name,
column_name , geometry_type_name, srs_id, z, m)
  VALUES ('marker_2019_cvr', 'geom', 'MULTIPOLYGON', 25832, 0, 0);

-- insert or update gpkg_ogr_columns (necessary ?)
INSERT OR REPLACE INTO gpkg_ogr_contents(table_name,feature_count)
  SELECT
    'marker_2019_cvr' AS table_name,
    count(*) AS feature_count
  FROM marker_2019_cvr;*

AFAIK, this is the necessary steps to make a spatial view visible
in a GeoPackage. And it works; sort of...

The QGIS datasource manager shows the view as a spatial table. I
can add it to the QGIS map, but... it can't find the CRS for the
l

Re: [gdal-dev] Spatial view in GeoPackage

[Even Rouault]

Bo Victor,

there was indeed a bug in the OGR GPKG driver preventing you to do what 
you wanted. Fixed per https://github.com/OSGeo/gdal/pull/7848


The issue was specific to the fact that the geometry column of the view 
is computed, and not just the simple selection of one of a table.


Note that such GeoPackage will probably only be readable by 
implementations that install Spatialite (or Spatialite compatible 
functions). It might also be prudent to wrap the result of ST_Multi() 
with AsGPB() to force creating a GeoPackage geometry blob instead of a 
Spatialite one. The OGR GPKG driver is tolerant to receiving both types 
of geometries, but technically the content should be a GeoPackage one.


Inserting into gpkg_ogr_contents is optional. Its purpose is to 
accelerate counting the number of features. But for layers < let's say 
100 k features, it probably doesn't impact performance much.


Even

Le 27/05/2023 à 22:08, Bo Victor Thomsen a écrit :


Hi list -

I'm working with making spatial views "visible" for QGIS, i.e. so they 
look like a read-only table from the data source manager.


However, I have one problem:  QGIS doesn't recognize the crs for the 
view. I've tracked the problem down to that *OGR* dosn't recognize the 
crs for the view. Hence this mail...


What I've done:

*- creating the view: A union of parcels with the same owner (CVR-number)
CREATE VIEW marker_2019_cvr AS
  SELECT "CVR", SUM(COALESCE(st_area(geom),0.0)) as area_m2, count(*) 
as antal, st_multi(st_union(geom)) as geom from marker_2019 group by 1;


-- insert or update gpkg_contents
INSERT OR REPLACE INTO 
gpkg_contents(table_name,data_type,identifier,min_x,min_y,max_x,max_y,srs_id)

  SELECT
    'marker_2019_cvr' AS table_name,
    'features' AS data_type,
    'marker_2019_cvr' AS identifier,
    MIN(ST_MinX(geom)) AS min_x,
    MIN(ST_MinY(geom)) AS min_y,
    MAX(ST_MaxX(geom)) AS max_x,
    MAX(ST_MaxY(geom)) AS max_y,
    25832 AS srs_id -- same srs as the base layer for the view
  FROM marker_2019_cvr;

-- insert or update gpkg_geometry_columns
INSERT OR REPLACE INTO gpkg_geometry_columns ( table_name, column_name 
, geometry_type_name, srs_id, z, m)

  VALUES ('marker_2019_cvr', 'geom', 'MULTIPOLYGON', 25832, 0, 0);

-- insert or update gpkg_ogr_columns (necessary ?)
INSERT OR REPLACE INTO gpkg_ogr_contents(table_name,feature_count)
  SELECT
    'marker_2019_cvr' AS table_name,
    count(*) AS feature_count
  FROM marker_2019_cvr;
*

AFAIK, this is the necessary steps to make a spatial view visible in a 
GeoPackage. And it works; sort of...


The QGIS datasource manager shows the view as a spatial table. I can 
add it to the QGIS map, but... it can't find the CRS for the layer and 
defaults to the projects CRS.


Digging a litle bit deeper: Using ogrinfo like this:

*ogrinfo -ro -so data_marker_vandloeb_soe.gpkg marker_2019_cvr*

shows the layers SRS WKT as (unknown).

Using the same command on the base layer *marker_2019 *gives the 
correct WKT.


25832 is the correct EPSG number for the CRS. And the definition 
exists in the *gpkg_spatial_ref_sys* table.


What am I doing wrong ? Or is it just "out of scope" for OGR ??

--
Med venlig hilsen / Best regards

Bo Victor Thomsen

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ogr.VectorTranslate to json appears to be incomplete

[Even Rouault]

Le 26/05/2023 à 19:40, Laurențiu Nicola via gdal-dev a écrit :

It's a bit strange, I can reproduce it when pasting into the REPL, but not when 
running from a file.


yes, the issue must come from the fact that gdal.VectorTranslate() 
returns a gdal.Dataset handle that is still active, and you could 
potentially add new features to it, hence the closing "] }" are emitted 
just when it is closed. In an interactive shell, for some reason the 
gdal.Dataset handle is not immediately closed when 
gdal.VectorTranslate() returns.


A safer way would be to explicitly do:

out_ds = gdal.VectorTranslate(...)

del out_ds

before getting the file content



Laurentiu

On Fri, May 26, 2023, at 20:27, Michael Smith wrote:

Well, a new install of gdal from conda-forge and now it works fine.
Sorry for the noise.

Mike


--

Michael Smith
Remote Sensing/GIS Center
US Army Corps of Engineers

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ogr.VectorTranslate to json appears to be incomplete

[Even Rouault]

Michael,

I've also just tried with 3.7.0 from conda-forge and this works fine for me:

I've just slightly modified your script to output more info:

from osgeo import ogr, gdal

ds = gdal.OpenEx('{ "type": "Polygon", "coordinates": [ [ [ 
179.828821134787177, 66.686572724877635 ], [ -179.718786846664386, 
66.695039153881993 ], [ -179.699204739767083, 66.515803857102057 ], [ 
179.851649370943306, 66.507409512673448 ], [ 179.828821134787177, 
66.686572724877635 ] ] ] }')
gdal.VectorTranslate("/vsimem/out.json", ds, format="GeoJSON", 
layerCreationOptions=["RFC7946=YES", "WRAPDATELINE=YES", "WRITE_BBOX=YES"])

f = gdal.VSIFOpenL("/vsimem/out.json", "rb")
content = gdal.VSIFReadL(1, 1, f).decode("UTF-8")
gdal.VSIFCloseL(f)
gdal.Unlink("/vsimem/out.json")
print(content)
print(len(content))

newds = ogr.Open(content)
lyr = newds.GetLayer(0)
f = lyr.GetNextFeature()
f.DumpReadable()

and it outputs:

{
"type": "FeatureCollection",
"bbox": [ 179.8288211, 66.5074095, -179.6992047, 66.6950392 ],
"features": [
{ "type": "Feature", "properties": { }, "bbox": [ 179.8288211, 
66.5074095, -179.6992047, 66.6950392 ], "geometry": { "type": 
"MultiPolygon", "coordinates": [ [ [ [ 180.0, 66.6897763 ], [ 
179.8288211, 66.6865727 ], [ 179.8516494, 66.5074095 ], [ 180.0, 
66.5101821 ], [ 180.0, 66.6897763 ] ] ], [ [ [ -179.7187868, 66.6950392 
], [ -180.0, 66.6897763 ], [ -180.0, 66.5101821 ], [ -179.6992047, 
66.5158039 ], [ -179.7187868, 66.6950392 ] ] ] ] } }

]
}

625
OGRFeature(OGRGeoJSON):0
  MULTIPOLYGON (((180.0 66.6897763,179.8288211 66.6865727,179.8516494 
66.5074095,180.0 66.5101821,180.0 66.6897763)),((-179.7187868 
66.6950392,-180 66.6897763,-180 66.5101821,-179.6992047 
66.5158039,-179.7187868 66.6950392)))


Even


Le 26/05/2023 à 15:12, Michael Smith a écrit :

I assume I am doing something incorrect here but I am taking an incoming 
geojson feature and attempting to split it on the dateline:

Using gdal 3.7.0 from conda-forge

ds = gdal.OpenEx('{ "type": "Polygon", "coordinates": [ [ [ 
179.828821134787177, 66.686572724877635 ], [ -179.718786846664386, 66.695039153881993 ], [ 
-179.699204739767083, 66.515803857102057 ], [ 179.851649370943306, 66.507409512673448 ], [ 
179.828821134787177, 66.686572724877635 ] ] ] }')
gdal.VectorTranslate("/vsimem/out.json", ds, format="GeoJSON", layerCreationOptions=["RFC7946=YES", 
"WRAPDATELINE=YES", "WRITE_BBOX=YES"])
f = gdal.VSIFOpenL("/vsimem/out.json", "rb")
content = gdal.VSIFReadL(1, 1, f).decode("UTF-8")
gdal.VSIFCloseL(f)
gdal.Unlink("/vsimem/out.json")

If I just do:

newds = ogr.Open(content)

I get RuntimeError: Failed to read GeoJSON data
It appears to be missing the closing “]}”.

If I do:
newds = ogr.Open(content+"]}")

Then it works fine.


--
Michael Smith
Remote Sensing/GIS Center
US Army Corps of Engineers



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Extracting Jpeg2000 image from nitf file

[Even Rouault]

Hi,

if you open such an image with debug traces (CPL_DEBUG=ON), you'll see 
something like


GDAL: GDALOpen(/vsisubfile/907_565,byte.ntf, this=) succeeds as 
JP2OpenJPEG


which means that the JPEG2000 file starts at byte 907 with a length of 
565 bytes.


Programatically from C/C++ , you could use the GDALGetOpenDatasets() 
methods after having opening the NITF file, and look for /vsisubfile/ files


Even

Le 26/05/2023 à 08:57, Tobby Moalem a écrit :

Hi,
For my understanding the nitf file is used as container, where 
compressed jpeg2000 can be saved inside the nitf file.
I wonder if there is a way to use gdal to extract the jpeg2000 part 
and save it separately (with extension of .j2c or other jpeg2000 
extensions) ?


Thanks in advance,
Tobby

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] gdalwarp: question about methods implemented when source grid is denser than target.

[Even Rouault]

Hi,

to briefly answer your question, for the methods bilinear, cubic, 
cubicspline and lanczos who have nominal kernel radius of respective 
values 1, 2, 2 and 3 (and thus a diameter double that size), the radius 
is extended by a factor source_size / dest_size to take into account the 
downsampling ( Cf lines 1188-1195 in alg/gdalwarpkernel.cpp), and the 
weights of the resampling kernel are evaluated with a scale factor too 
(cf lines 3713-3715 / 3741-3743)


Even

Le 24/05/2023 à 16:29, Ramiro Espada Guerrero a écrit :

Hi,

I would like to know if gdalwarp does any "average" regriding when the 
source grid is denser than the target grid regardless of the chosen 
method for the interpolation. And if there is any document where I can 
follow exactly the calculations. I tried to follow the code on github 
but it is pretty difficult to follow.


I've programmed my own bilinear interpolation function in fortran and 
I got smoother results when I applied -r bilinear with gdalwarp. So i 
wonder if there is a pre-average regriding subroutine that is not in 
the documentation page.


Thanks!





___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CSV driver inconsistent separator dealing to failures

[Even Rouault]

Moises,


In my humble opinion it makes sense to follow this approach if the 
separator is not clear, but in this case the first CSV line contains 
like 10 semicolons (the real separator) and just one comma. I believe 
that the actual behaviour could be improved adding some way of 
checking the most repeated separator. What do you think?


That seems reasonable. You may file an enhancement ticket about that, 
with the header of the CSV file on which you hit the issue as an 
illustration. We should also probably emit a CE_Warning visible even in 
production builds when such ambiguous situation arises.


Even



--
*Moises Calzado*

Support Engineer

+34671264286 | mcalz...@carto.com | CARTO 



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Improving details of the project build system and/or documentation

[Even Rouault]

Hi Sean,

I presume you got this because you defined 
OGR_BUILD_OPTIONAL_DRIVERS=OFF which then cause OGR_ENABLE_DRIVER_AVC to 
be set to OFF.


We already a quite overwhelming amount of documentation in 
https://gdal.org/development/building_from_source.html about all the 
existing variables, and I'm not sure adding an exhaustive list of all 
the inter driver dependencies will be user-digestable and could rot easily.


That said in https://github.com/OSGeo/gdal/pull/7806 I've tried to 
improve the current error message with a hint for the reason for the 
error and the (likely) reason for it to happen.


I've also added pointers in the doc page to the CMakeLists.txt files 
where the dependencies are expressed. Hopefully people who are in the 
business of making custom GDAL builds can make sense of that.


Even

Le 20/05/2023 à 01:26, Sean Gillies a écrit :

Hi all,

I really appreciate the documentation at 
https://gdal.org/development/building_from_source.html. But there are 
gaps. For example, today I ran into


CMake Error at cmake/helpers/GdalDriverHelper.cmake:331 (message):
GDAL_ENABLE_DRIVER_AIGRID cannot be enabled because condition
OGR_ENABLE_DRIVER_AVC is not met. To ignore this error (but the driver
will not be built), set the GDAL_IGNORE_FAILED_CONDITIONS variable

This dependence isn't documented. It's a bit frustrating to work 
through missing directives one at a time when adding new drivers to my 
build.


Is it possible for a driver's dependencies to be enabled when I enable 
a driver?


If not, can we consider using GDAL maintenance funding and resources 
on documenting the heck out of this system?


--
Sean Gillies

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Fwd: Transformation of image data between pixel/line and projected coordinates - using geolocation arrays, in both directions, without disk access

[Even Rouault]

Daniel,

You rightly spotted https://github.com/OSGeo/gdal/pull/6069 as the 
enabler for that capability.


if your existing target dataset has a geolocation array attached to do 
it, this should just be a matter of doing:


target_ds = gdal.Open( filename, gdal.GA_Update )

gdal.Warp(target_ds, source_ds, ... other options here ...)

If the target dataset doesn't have a geolocation array attached to it, 
you can point to an external one with the DST_GEOLOC_ARRAY tranformer option


gdal.Warp(target_ds, source_ds, 
transformerOptions=["DST_METHOD=GEOLOC_ARRAY", 
"DST_GEOLOC_ARRAY=/path/to/geoloc_dataset"], ... other options here ...)


Even

Le 16/05/2023 à 19:35, Daniel Scheffler a écrit :

Hi!

Some time ago, I asked for an inversion of gdal.Warp based on 
GEOLOCATION arrays (longitude/latitude). Back then, my question was: 
How can I transform an image with projected coordinates back to 
cartesian/image coordinates, given that a geolocation array tells GDAL 
where to put which pixel in the output?


In 11/2021, this was unfortunately not implemented yet. However, is 
seems like in the meantime someting like this has been added:

    - https://github.com/OSGeo/gdal/pull/5520
    - https://github.com/OSGeo/gdal/pull/6069
    - 
https://github.com/OSGeo/gdal/blob/c92b22d02c99eae0152f49595947fb3747ddc280/autotest/gcore/geoloc.py#L396


But I am not quite sure if that is what I want. If so, how would a 
Python implementation based on gdal.Warp look like? Is that documented 
somewhere?


Best,
Daniel



Am 30.11.2021 um 14:20 schrieb Daniel Scheffler:
Ok thanks, too bad that this is not implemented. I think the 
inversion of this transformation would be a nice feature to be added 
in GDAL. It would be very useful to me (especially if it is 
accessible via the Python bindings) and would ease the implementation 
in a processing pipeline for the upcoming EnMAP hyperspectral 
satellite. Should I open a feature request in the GDAL issue tracker 
on GitHub?



Am 30.11.2021 um 14:02 schrieb Even Rouault:



Le 30/11/2021 à 12:52, Daniel Scheffler a écrit :
Thanks a lot for taking the time, Even, I got the transformation 
from cartesian to projected coordinates to work in memory with the 
GTiff driver. With MEM, NUMPY or VRT it does not work because these 
formats are either not readable from /vsimem/ or don´t have a 
regular file path which is needed to set the X_DATASET and 
Y_DATASET keys in the GEOLOCATION metadata.
Ah indeed for X_DATASET/Y_DATASET, you can't use a MEM or NUMPY 
dataset. But a /vsimem/xxx dataset in another format should work.


Regarding the inversion of this transformation, i.e., from 
projected coordinates to pixel/line:
The logic of GDALCreateGenImgProjTransformer2() around 
https://github.com/OSGeo/gdal/blob/master/alg/gdaltransformer.cpp#L1825 
which is for the source dataset should be ported a few lines after 
for the target dataset.  Probably with a new transformer option to 
be able to point to an auxiliary dataset, such as a VRT one of 
your example, to extract the geolocation metadata items from it, 
that would be different from the target dataset itself, because, 
except perhaps for the netCDF case, most GDAL datasets that expose 
a GEOLOCATION metadata domain must be read-only.


I don´t completely get what you mean here. To me, this sounds like 
there might be a way to do the inverted transformation using the 
C-API of GDAL.

No, I meant there's some missing code to do that.
However, I am a Python developer and my C skills are a bit poor. Is 
there any way to use the Python bindings here?


Kind regards,
Daniel




Am 26.11.2021 um 12:38 schrieb Even Rouault:

Daniel,


I am trying to convert image data from cartesian/image 
coordinates to projected coordinates AND vice versa using 
geolocation arrays in GDAL. I have two questions:


 1. Since this transformation is part of a processing chain
implemented in Python, I try to transform the data directly
in-memory, i.e, without any disk access. This saves IO time
and avoids permission errors when trying to write temporary
data on Windows. How can this be done? I got correct results
with the code below, however, only when I temporarily write
the data to disk. I tried to write the data to /vsimem/ using
the MEM, GTiff and NUMPY drivers. However, gdal.Warp can´t
find the data there (FileNotFoundError). I think, also the
gdal.Transformer class might be useful and I found an
interesting thread on that here
<https://lists.osgeo.org/pipermail/gdal-dev/2012-January/031502.html>
and a related test in the GDAL autotest suite (here

<https://github.com/OSGeo/gdal/blob/master/autotest/alg/transformgeoloc.py>).
However, I can´t get it to work for my specific case.


There's no reason it won't work with a in-memory dataset.

If you use a MEM dataset, then you need to provide the dataset 
object itself as the input dataset of gdal.Wa

[gdal-dev] GDAL 3.7.0 is released

[Even Rouault]

Hi,

On behalf of the GDAL/OGR development team and community, I am pleased to
announce the release of GDAL/OGR 3.7.0. GDAL/OGR is a C++ geospatial data
access library for raster and vector file formats, databases and web 
services.

It includes bindings for several languages, and a variety of command line
tools.

   http://gdal.org/

The 3.7.0 release is a new feature release with the following highlights:

* [RFC 87](https://gdal.org/development/rfc/rfc87_signed_int8.html): Add 
GDT_Int8 support
* [RFC 88](https://gdal.org/development/rfc/rfc88_googletest.html): 
switch to GoogleTest framework for C++ tests
* [RFC 
89](https://gdal.org/development/rfc/rfc89_sql_logging_callback.html): 
SQL query logging callback
* [RFC 
90](https://gdal.org/development/rfc/rfc90_read_compressed_data.html): 
Direct access to compressed raster data
* [RFC 91](https://gdal.org/development/rfc/rfc91_dataset_close.html): 
GDALDataset::Close() method
* [RFC 93](https://gdal.org/development/rfc/rfc93_update_feature.html): 
OGRLayer::UpdateFeature() method
* [RFC 
94](https://gdal.org/development/rfc/rfc94_field_precision_width_metadata.html): 
Numeric fields width/precision metadata
* ogrinfo: make it accessible through a new GDALVectorInfo() C API call, 
and a -json switch
* Add read-only raster driver 
[NOAA_B](https://gdal.org/drivers/raster/noaa_b.html) to read NOAA 
GEOCON/NADCON5 .b grids
* Add read-only raster driver 
[NSIDCbin](https://gdal.org/drivers/raster/nsidcbin.html) for Sea Ice 
Concentrations
* Add read-only vector [GTFS](https://gdal.org/drivers/vector/gtfs.html) 
(General Transit Feed Specification) driver
* [TileDB](https://gdal.org/drivers/vector/tiledb.html): add read/write 
vector side
* Add support for [SOZip](https://sozip.org) (Seek Optimized ZIP) with 
enhanced /vsizip/ virtual file system and a new sozip utility
* [OpenFileGDB](https://gdal.org/drivers/raster/openfilegdb.html): add 
read-only support for raster datasets (.gdb v10)
* PNG: 1.7-2.0x speed-up in whole image decompression with libdeflate on 
Intel/AMD CPUs. Benefits to GPKG, MRF drivers
* [RFC 
69](https://gdal.org/development/rfc/rfc69_cplusplus_formatting.html): 
C++ code reformatting

* Code linting and security fixes
* Remove any traces of Rasdaman driver, now moved to 
OSGeo/gdal-extra-drivers repository (#4808)

* Bump of shared lib major version

More complete information on the new features and fixes in the 3.7.0 
release can be found at:


  https://github.com/OSGeo/gdal/blob/v3.7.0/NEWS.md

The release can be downloaded from:
  * https://download.osgeo.org/gdal/3.7.0/gdal370.zip - source as a zip
  * https://download.osgeo.org/gdal/3.7.0/gdal-3.7.0.tar.gz - source as 
.tar.gz
  * https://download.osgeo.org/gdal/3.7.0/gdal-3.7.0.tar.xz - source as 
.tar.xz
  * https://download.osgeo.org/gdal/3.7.0/gdalautotest-3.7.0.tar.gz - 
test suite
  * https://download.osgeo.org/gdal/3.7.0/gdal370doc.zip - 
documentation / website


Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: Adopt GDAL 3.7.0RC1 as 3.7.0 release

[Even Rouault]

Hi,

motion passed with +1 from PSC members HowardB, JukkaR, DanielM, 
MateuszL, NormanB and me.


Even

Le 08/05/2023 à 15:02, Even Rouault a écrit :

Hi,

Motion:

Adopt GDAL 3.7.0RC1 as 3.7.0 release

Starting with my +1

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] request limit on netcdf server ?

[Even Rouault]

Michael,

analysis and improvements in https://github.com/OSGeo/gdal/pull/7737

Even

Le 10/05/2023 à 09:03, Michael Sumner a écrit :

I'm having trouble with remote dsn from here:

https://esgf.nci.org.au/thredds/catalog/esgcet/cmip5/output1/CSIRO-BOM/ACCESS1-3/rcp85/day/ocean/day/r1i1p1/cmip5.output1.CSIRO-BOM.ACCESS1-3.rcp85.day.ocean.day.r1i1p1.v20120413.html#cmip5.output1.CSIRO-BOM.ACCESS1-3.rcp85.day.ocean.day.r1i1p1.v20120413

The support tells me I have been blocked for requests from my IP that 
exceed 300 in 5min. Apparently, daily you are cleared from the 
blocklist but with experiments today I've had two machines blocked 
(and now manually cleared). Still, it's hard to test :0


I've stripped down to this dsn, I thought limiting  the driver might 
limit the requests because it won't cascade down the driver 
identification (but I'm not sure if -if adds anymore than the 
"NETCDF:" is already doing). At any rate I need to target a subdataset 
so I need that syntax.


"vrt://NETCDF:\"/vsicurl/https://esgf.nci.org.au/thredds/fileServer/master/CMIP5/output1/CSIRO-BOM/ACCESS1-3/rcp85/day/ocean/day/r1i1p1/v20120413/tos/tos_day_ACCESS1-3_rcp85_r1i1p1_20560101-20651231.nc\ 
":tos?bands=1&if=NETCDF"


It takes quite a while to read one band, but the warper and RasterIO 
result is good.  I think the machine is now blocked again, I wasn't 
able to also run gdalinfo after warper and straight read.


I'm just doing standalone from-scratch opens, not with a persistent 
dataset - I assume that would reduce the request load, which I'll try 
in python when I have access again.


Questions:

1. Are there other things I can pursue to reduce my request load on 
the server?


2. Can this actually be faster?  RasterIO for one band 360x300 was 
minutes.  (I appreciate this might be better suited to mdim, but that 
seems similarly problematic in terms of requests and performance).


I've tried the config options   GDAL_NETCDF_VERIFY_DIMS / STRICT and 
GDAL_NETCDF_IGNORE_XY_AXIS_NAME_CHECK=/YES and that does seem to have 
helped the warper request but RasterIO was still slow.

.
This was on GDAL 3.8.0dev-414270a94b-dirty, released 2023/05/09 (debug 
build)


I see this repeated warning

Warning 1: dimension #2 (i) is not a Longitude/X dimension.
Warning 1: dimension #1 (j) is not a Latitude/Y dimension.
Warning 1: dimension #1 (i) is not a Longitude/X dimension.
Warning 1: dimension #0 (j) is not a Latitude/Y dimension.
Warning 1: dimension #1 (i) is not a Longitude/X dimension.
Warning 1: dimension #0 (j) is not a Latitude/Y dimension.

Thank you

--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Reading raster file

[Even Rouault]

Javier,

The error message comes from the XYZ driver which is confused and 
believes it might be for it, but this is definitely not a .xyz file


There are quite a lot of variations of ASCII grid formats. This one 
reminds me a bit of the IGN France ones that are handled by the 
https://github.com/OSGeo/gdal-extra-drivers/blob/master/src/ignfheightasciigrid/ignfheightasciigrid.cpp 
driver (note that it has been retired as out of tree), but I can see 
that this .gri file has min_lat, max_lat first whereas the French ones 
have min_lon, max_lon :-)


I've tried with 
https://www.isgeoid.polimi.it/Geoid/America/Argentina/public/GEOIDEAR16_20160419.isg, 
linked from 
https://www.isgeoid.polimi.it/Geoid/America/Argentina/argentina2016_g.html 
.  Hopefully it has the same content as the .gri file. I had to push in 
https://github.com/OSGeo/gdal/pull/7730 a fix to open it as the rounding 
of coordinates & resolution is quite rough in those .isg files.


with the fix:

$ gdalinfo GEOIDEAR16_20160419.isg -mm
Driver: ISG/International Service for the Geoid
Files: GEOIDEAR16_20160419.isg
Size is 1440, 2220
Coordinate System is:
GEOGCRS["WGS 84",
    DATUM["World Geodetic System 1984",
    ELLIPSOID["WGS 84",6378137,298.257223563,
    LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
    ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
    AXIS["geodetic latitude (Lat)",north,
    ORDER[1],
    ANGLEUNIT["degree",0.0174532925199433]],
    AXIS["geodetic longitude (Lon)",east,
    ORDER[2],
    ANGLEUNIT["degree",0.0174532925199433]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (-76.00985350006,-20.00873350002)
Pixel Size = (0.0166670,-0.0166670)
Metadata:
  MODEL_NAME=GEOIDEAR16
  MODEL_TYPE=gravimetric
Corner Coordinates:
Upper Left  ( -76.0098535, -20.0087335) ( 76d 0'35.47"W, 20d 0'31.44"S)
Lower Left  ( -76.0098535, -57.0094735) ( 76d 0'35.47"W, 57d 0'34.10"S)
Upper Right ( -52.0093735, -20.0087335) ( 52d 0'33.74"W, 20d 0'31.44"S)
Lower Right ( -52.0093735, -57.0094735) ( 52d 0'33.74"W, 57d 0'34.10"S)
Center  ( -64.0096135, -38.5091035) ( 64d 0'34.61"W, 38d30'32.77"S)
Band 1 Block=1440x1 Type=Float32, ColorInterp=Undefined
    Computed Min/Max=-6.007,45.203
  NoData Value=-
  Unit Type: meters


Note that CRS is hard coded to WGS 84 by the driver, and should be 
altered to something more relevant.


Even


Le 09/05/2023 à 17:21, Javier Jimenez Shaw a écrit :

Hi

I want to read the Argentinian Geoid model with GDAL.
It is an ascii raster floating-point file with just one band.
https://dnsg.ign.gob.ar/apps/geoidear/documentos/GEOIDE-Ar16.gri (22 MB)


The content is starts with:

   -57.000740   -20.016667   -76.00 -52.016187   
0.016667001   0.016667000


13.278 13.310 13.321 13.322 13.357 13.383 13.429 13.442
13.477 13.504 13.516 13.549 13.574 13.549 13.570 13.602
13.623 13.584 13.574 13.661 13.766 13.802 13.823 13.821
13.835 13.881 13.907 13.944 14.005 14.060 14.085 14.070
14.098 14.153 14.221 14.259 14.277 14.396 14.446 14.451
14.390 14.471 14.572 14.645 14.681 14.674 14.702 14.745
14.796 14.843 14.872 14.872 14.952 15.065 15.110 15.053
15.056 15.050 15.095 15.126 15.125 15.154 15.222 15.261
15.244 15.369 15.480 15.519 15.464 15.462 15.505 15.477

and finishes with

19.069 19.065 19.057 19.047 19.039 19.031 19.021 19.012
19.004 18.997 18.990 18.987 18.987 18.988 18.991 18.998
19.001 19.009 19.027 19.042 19.067 19.141 19.209 19.208
19.179 19.156 19.147 19.150 19.152 19.156 19.164 19.155

Total number of lines: 399602

It seems to have a width of 1440 and height of 2220.

gdalinfo says

$ gdalinfo GEOIDE-Ar16.gri
ERROR 1: Maximum number of characters allowed reached.
ERROR 1: Couldn't determine X spacing
gdalinfo failed - unable to open 'GEOIDE-Ar16.gri'.

I do not know which exact format it is. I can deduce the meaning of 
the values (specially knowing that it covers Argentina ;), but I would 
like to read it directly if possible.


Is that a know format for GDAL?

Thanks,
Javier.
.___ ._ ..._ .. . ._.  .___ .. __ . _. . __..  ...  ._ .__

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Integration of GDAL 3.5 into a custom Docker Image

[Even Rouault]

Dirk,



/[91mrsync: [sender] change_dir "lat-lon/gdal:debian-full/proj/x86_64" 
(in gdal-docker-cache) failed: No such file or directory (2)//
//[0m[91mrsync error: some files/attrs were not transferred (see 
previous errors) (code 23) at main.c(1819) [Receiver=3.2.3]//

//[0m[91mrsync: [Receiver] read error: Connection reset by peer (104)//
//[0mThe command '/bin/sh -c . /buildscripts/bh-set-envvars.sh && 
DESTDIR=/build_tmp_proj /buildscripts/bh-proj.sh  && 
LD_LIBRARY_PATH=/build_tmp_proj/usr/local/lib 
/build_tmp_proj/usr/local/bin/projsync --target-dir /tmp/proj_grids 
--all  && rm -rf /build_tmp_proj' returned a non-zero code: 23/


From the above it is not clear if the issue is with rsync or projsync.

rsync is just an optimization do have host caching of build artifacts 
when doing repeated builds. You could safely strip that off from the 
script, or just pass -t RSYNC_REMOTE="" in the docker build invokation.




- How long will GDAL 3.5 be supported? I did not find any maintenance 
schedule on the website.
We only support the branch of the last released feature version. So when 
3.7.0 will be officially approved (in a couple days), only 3.7.x will be 
supported, and until 3.8.0 will be released in ~6 months.
- Is the Docker build of release/3.5 branch still maintained or could 
it happen that e.g. a remote resource disappears meaning that the 
build does not work anymore?


The Docker build scripts tend to break regularly due to remote resources 
changing, typically with the Apache Arrow libraries. But you may also 
remove that if you don't need a build with everything.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Motion: Adopt GDAL 3.7.0RC1 as 3.7.0 release

[Even Rouault]

Hi,

Motion:

Adopt GDAL 3.7.0RC1 as 3.7.0 release

Starting with my +1

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

[Even Rouault]

ke sense?

Thanks so much for your help!

El mié, 3 may 2023 a las 15:10, Robert Hewlett
() escribió:

The .CSVT and .PRJ help to make a proper
geocsv dataset. Helps with QGIS And geopandas.
The column name that I use in the CSV is
usually geom and WKT shows up in the CSVT file
which seems to be a one line file that hints
at the data types in the CSV file.

I hope that makes sense.

CSVT
Integer, Integer,WKT

CSV
line_id,point_id,geom
1,1,"POINT(1000 1000)"

PRJ
EPSG:26910




On Wed, May 3, 2023, 05:23 Moises Calzado via
gdal-dev  wrote:

Hi Even,

Thanks so much for taking a look into that
one!

I have one doubt regarding the CSVT
content, as we're not really using it, but
it's required when using the GEOMETRY_NAME
layer creation option, as can be checked
in the CSV driver documentation:

 *

GEOMETRY_NAME=name (Starting with
GDAL 2.1): Name of geometry
column. Only used if
GEOMETRY=AS_WKT and
CREATE_CSVT=YES. Defaults to WKT

We really need this flag as we are
processing files that contain geometries
with different column names, and we always
want the same geometry name in the
generated output. Are we losing something
when using that flag to avoid this problem?
In my humble opinion, generating an
invalid CSV when using the -lco
CREATE_CSVT=YES looks like a bug for me,
as I can't see the reason why strings
containing line breaks can't be quoted.

    Could you please shed some light on this?

Looking forward to your reply,
Regards.

El mié, 3 may 2023 a las 14:00, Even
Rouault ()
escribió:

you didn't post to the list

Le 03/05/2023 à 13:49, Moises Calzado
a écrit :

Hi Even,

Thanks so much for taking a look into
that one!

I have one doubt regarding the CSVT
content, as we're not really using
it, but it's required when using the
GEOMETRY_NAME layer creation option,
as can be checked in the CSV driver
documentation:

 *

GEOMETRY_NAME=name (Starting
with GDAL 2.1): Name of
geometry column. Only used if
GEOMETRY=AS_WKT and
CREATE_CSVT=YES. Defaults to WKT

We really need this flag as we are
processing files that contain
geometries with different column
names, and we always want the same
geometry name in the generated
output. Are we losing something when
using that flag to avoid this problem?
In my humble opinion, generating an
invalid CSV when using the -lco
CREATE_CSVT=YES looks like a bug for
me, as I can't see the reason why
strings containing line breaks can't
    be quoted.

        Could you please shed some light on this?

Looking for

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

[Even Rouault]

Le 03/05/2023 à 14:22, Moises Calzado via gdal-dev a écrit :

Hi Even,

Thanks so much for taking a look into that one!

I have one doubt regarding the CSVT content, as we're not really using 
it, but it's required when using the GEOMETRY_NAME layer creation 
option, as can be checked in the CSV driver documentation:


 *

GEOMETRY_NAME=name (Starting with GDAL 2.1): Name of geometry
column. Only used if GEOMETRY=AS_WKT and CREATE_CSVT=YES.
Defaults to WKT

We really need this flag as we are processing files that contain 
geometries with different column names, and we always want the same 
geometry name in the generated output. Are we losing something when 
using that flag to avoid this problem?


The reason  for requiring CREASE_CSVT=YES is that when reading back a 
.csv without a .csvt the geometry column must be named WKT. Unless you 
specify the GEOM_POSSIBLE_NAMES open option (which must have been a 
later addition). That said it could be reasonable to relax that coupling 
and allow GEOMETRY_NAME without CREATE_CSVT=YES, with a warning in the 
doc about the consequence I just mentioned before


In my humble opinion, generating an invalid CSV when using the -lco 
CREATE_CSVT=YES looks like a bug for me,


Are you speaking about emitting the .prj and .csvt content when writing 
to /vsistdout ? Yes, I'd tend to agree they should not be emitted in 
that mode.


as I can't see the reason why strings containing line breaks can't be 
quoted.
I'm not following you about the issue with line breaks. In my previous 
message, I showed I didn't reproduce any issue: the CSV driver emits 
fields with double quotes, even when there are line breaks. Can you be 
more specific about what's wrong ? I don't see the connection with 
GEOMETRY_NAME.


Could you please shed some light on this?

Looking forward to your reply,
Regards.

El mié, 3 may 2023 a las 14:00, Even Rouault 
() escribió:


you didn't post to the list

Le 03/05/2023 à 13:49, Moises Calzado a écrit :

Hi Even,

Thanks so much for taking a look into that one!

I have one doubt regarding the CSVT content, as we're not really
using it, but it's required when using the GEOMETRY_NAME layer
creation option, as can be checked in the CSV driver documentation:

 *

GEOMETRY_NAME=name (Starting with GDAL 2.1): Name of
geometry column. Only used if GEOMETRY=AS_WKT and
CREATE_CSVT=YES. Defaults to WKT

We really need this flag as we are processing files that contain
geometries with different column names, and we always want the
same geometry name in the generated output. Are we losing
something when using that flag to avoid this problem?
In my humble opinion, generating an invalid CSV when using the
-lco CREATE_CSVT=YES looks like a bug for me, as I can't see the
reason why strings containing line breaks can't be quoted.

Could you please shed some light on this?

Looking forward to your reply,
Regards.

El sáb, 29 abr 2023 a las 15:44, Even Rouault
() escribió:

Moises,

as far as I can see with your example, the CSV driver behaves
"properly" in reading and writing of field values with line
breaks.

It follows the "Fields with embedded line breaks must be
quoted" rule of
https://en.wikipedia.org/wiki/Comma-separated_values

$ ogr2ogr out.csv /vsizip/dataframe.zip

$ cat out.csv
id,descriptio
"1",This is my third row
"2","this is
my string
"
"3",This is my third row

$ ogrinfo out.csv -al
INFO: Open of `out.csv'
  using driver `CSV' successful.

Layer name: out
Geometry: None
Feature Count: 3
Layer SRS WKT:
(unknown)
id: String (0.0)
descriptio: String (0.0)
OGRFeature(out):1
  id (String) = 1
  descriptio (String) = This is my third row

OGRFeature(out):2
  id (String) = 2
  descriptio (String) = this is
my string


OGRFeature(out):3
  id (String) = 3
  descriptio (String) = This is my third row

But in your example using /vsistdout/ and -lco
CREATE_CSVT=YES is going to result in an invalid CSV file
which will mix both the .csvt and .csv content

Even

Le 24/04/2023 à 13:34, Moises Calzado via gdal-dev a écrit :

Hello!

We're trying to convert a Shapefile into a CSV using ogr2ogr
and we're having some issues while dealing with some columns
that contain line breaks inside their values. If we have a
line with the following string, ogr2ogr detects that the
line break is a new line and it returns two lines.

Re: [gdal-dev] JPEG2000 change progression order without decoding

[Even Rouault]

I don't have in mind OSS to do that, but the kdu_transcode utility from 
Kakadu (proprietary) should be able to do it. Cf 
https://trac.osgeo.org/gdal/ticket/3295 / 
https://kakadusoftware.com/wp-content/uploads/Usage_Examples.txt


Le 03/05/2023 à 04:20, Tobby Moalem a écrit :
I wonder if there is a way to change the progression order of jpeg2000 
images, without the need of decoding it and then encode it again. For 
example if I have an image with RPCL progression and I want to convert 
it to LRCP. Is there a way to only move bits around (change the order 
of the file) instead of needing to transcode the image all over again 
(which may take a long time)?


Thanks in advance,
Tobby

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Deciding whether to use WFS paging after opening the datasource

[Even Rouault]

Craig,

Both options you propose are reasonable. I'd perhaps have a slight 
preference for option 2 (is my understanding correct that the user would 
be responsible to issue GetFeatureCount() manually before calling 
GetNextFeature() ? we don't want to systematically call 
GetFeatureCount() because it could be potentially a rather costly 
operation for some servers).


It could be enhanced with allowing a CHECK_WITH_HITS value for 
OGR_WFS_PAGING_ALLOWED (or perhaps through a new specific open option ?) 
that would automatically issue the feature count request (that could be 
useful for ogr2ogr types of scenario where the user can't easily cause 
ogr2ogr to issue GetFeatureCount())


Even

Le 03/05/2023 à 03:10, Craig de Stigter a écrit :

Hi folks

We're having trouble with OGR trying to fetch a response from this 
WFS2 URL: 
https://geo.irceline.be/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAMES=realtime:pm25_24hmean_station&COUNT=100&STARTINDEX=0 



The error from the server is

Cannot do natural order without a primary key, please add it or
specify a manual sort over existing attributes


This seems fair enough on the surface - we specified *STARTINDEX=0* 
but the server is unable to support paging for this dataset due to no 
natural sort order.


We'd happily disable paging for this dataset - it has a fairly low 
number of features - but we would prefer to implement a general 
purpose solution rather than special-case this particular dataset.


It would make sense to open the datasource, check the feature count by 
doing a 'hits' query (we're doing this anyway), and then use paging if 
there are more than 100 features - otherwise don't use paging.


However I can't find a way to accomplish this with GDAL. The 
*OGR_WFS_PAGING_ALLOWED* config var is only checked when the 
datasource is opened - so you can't open the datasource, check the 
feature count and then change the paging behaviour.


I can imagine either of these changes in GDAL itself might fix this:

1. GDAL could check *OGR_WFS_PAGING_ALLOWED* when actually first 
issuing GetFeature requests, rather than during datasource creation
2. GDAL could omit the pagination parameters if the feature count is 
already known and is less than the page size


How would you suggest we proceed?

--
Regards,
Craig

Platform Engineer
Koordinates

+64 21 256 9488  / koordinates.com 
 / @koordinates 


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.7.0 release candidate available

[Even Rouault]

Holger,

thanks for the feedback. Fixed per 
https://github.com/OSGeo/gdal/commit/b0039c648073713578bb0360f92a6c859a011832 
and backport to the 3.7 branch


I will probably not issue a new RC just because of that as those files 
aren't used by the code, so this will probably be only in 3.7.1


Even

Le 02/05/2023 à 19:57, Holger Jaekel a écrit :

Even,

Am 02.05.23 um 16:03 schrieb Even Rouault:

I have prepared a GDAL/OGR 3.7.0 release candidate.


The release candidate builds well on Alpine Linux Edge. I just observed
that the files gfs.xsd and gml_registry.xsd were mentioned in the news
but not copied to /usr/share/lib/gdal

Also the filename is ogrinfo_output.schema.json and not
ogrinfo_output.json.schema as it says in the news.

Best
  Holger
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.7.0 release candidate available

[Even Rouault]

Hi,

I have prepared a GDAL/OGR 3.7.0 release candidate.

NEWS at:

  https://github.com/OSGeo/gdal/blob/v3.7.0RC1/NEWS.md

Pick up an archive among the following ones (by ascending size):

  https://download.osgeo.org/gdal/3.7.0/gdal-3.7.0rc1.tar.xz
  https://download.osgeo.org/gdal/3.7.0/gdal-3.7.0rc1.tar.gz
  https://download.osgeo.org/gdal/3.7.0/gdal370rc1.zip

A snapshot of the gdalautotest suite is also available :

https://download.osgeo.org/gdal/3.7.0/gdalautotest-3.7.0rc1.tar.gz
  https://download.osgeo.org/gdal/3.7.0/gdalautotest-3.7.0rc1.zip

A snapshot of the documentation is at:

  https://download.osgeo.org/gdal/3.7.0/gdal370doc.zip

Note that the generated files for the Python bindings are no longer included
in the sources, so SWIG must be available at build time.

I'll call for a vote promoting it to next week if no serious problems 
are reported before.


The "release/3.7" branch has been created for all bug fixes related to 
3.7.x.

master is now targeting 3.8.0dev.

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Java Binding Versions in GDAL Docker Ubuntu Full

[Even Rouault]

Fixed per https://github.com/OSGeo/gdal/pull/7669

Le 01/05/2023 à 19:20, adamgutonski via gdal-dev a écrit :

Hello,

I am setting up a Docker container which copies from the GDAL 
ubuntu-full image, and I noticed that the Java bindings with 
ubuntu-full are for Java Version 11 only.  I am using Java Version 
17.  Are there any deprecated features or potential caveats to be 
aware of when using Java 17 and the GDAL bindings that have been 
compiled for 11?  I can't find any information online about this and I 
do not know how one would rebuild the container bindings.  Perhaps the 
bindings be updated in ubuntu-full in the future?


Thank you for any information you can provide.

Kind regards,
Adam Gutonski

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] +proj=wintri +over option works weel in gdalwarp but not in ogr2ogr

[Even Rouault]

Joaquim,

I've looked at that. Experimentally, it doesn't seem that the current 
maths for wintri in +over mode give reasonable results for longitudes > 
360deg in absolute value (and I'm not sure if there would be a natural 
extension possible. Far beyond my knowledge of the underlying maths).


gdalwarp does a little better by default because it couples the forward 
and inverse transformer to check the domain of validity and 
experimentally must find this limitation and will thus give up for 
source pixels whose longitude > 360 deg. For ogr2ogr / vector this is 
more complicated as reprojection of a single geometry  (here cl540.gpkg 
is a single polygon with longitudes in [-540,540] range). is a "all 
points succeed to reproject => OK" / "one single point fails to 
reproject => KO" type of situation, at least in the logic followed by 
OGRGeometry::transform() logic.


So while the following patch to PROJ is probably technically correct, it 
wouldn't help in practice


diff --git a/src/projections/aitoff.cpp b/src/projections/aitoff.cpp
index 9d060999f..cce319157 100644
--- a/src/projections/aitoff.cpp
+++ b/src/projections/aitoff.cpp
@@ -59,6 +59,11 @@ static PJ_XY aitoff_s_forward(PJ_LP lp, PJ *P) { /* 
Spheroidal, forward */

 struct pj_opaque *Q = static_cast(P->opaque);
 double c, d;

+    if( fabs(lp.lam) > 2 * M_PI ) {
+    proj_errno_set(
+    P, PROJ_ERR_COORD_TRANSFM_OUTSIDE_PROJECTION_DOMAIN);
+    return xy;
+    }
 c = 0.5 * lp.lam;
 d = acos(cos(lp.phi) * cos(c));
 if (d != 0.0) { /* basic Aitoff */

The only solution I can offer is that you must explictly clip your 
source geometry to the validity range of wintri with -clipsrc -360 -90 
360 90.


Even


Le 30/04/2023 à 03:11, Joaquim Manuel Freire Luís via gdal-dev a écrit :


Even, this is a kind of continuation of the subject that I brought up 
in https://github.com/OSGeo/gdal/issues/7644


The file size limitation of attachments doesn’t help to make this case 
easily reproducible, but the story is, I can make that rectangular map 
using gdalwarp and ogr2ogr that use the +over option and +proj=vandg


(see 
https://forum.generic-mapping-tools.org/t/best-projection-for-rectangular-world-map/3715/82?u=joaquim)


But when I try the same with +proj=wintry the gdalwarp op works well 
but the ogr2ogr doesn’t.


(see 
https://forum.generic-mapping-tools.org/t/best-projection-for-rectangular-world-map/3715/85?u=joaquim)


You can more less reproduce this case using this file

http://fct-gmt.ualg.pt/tmp/cl540.gpkg

ogr2ogr  -t_srs "+proj=wintri +over" cl540_wintri.gpkg cl540.gpkg

Why would that be in this case?

Joaquim


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GNU g++-13 and [[clang::fallthrough]]

[Even Rouault]

Andrew,

fixed per https://github.com/OSGeo/gdal/pull/7660 . Your patch proposal 
didn't work with older gcc versions that don't like the "::" in 
_has_cpp_attribute(clang::fallthrough). I've reworked & simplified it. 
I've verified it silences the warning with 23.04 + gcc 13


Even

Le 29/04/2023 à 17:00, Andrew C Aitchison a écrit :

On Sat, 29 Apr 2023, Andrew C Aitchison wrote:



Ubuntu 23-04 (Lunar) has g++ v13.0.1 (in the universe section,
this is not the default g++ compiler).
At least as curently shipped, this warns about the line
[[clang::fallthrough]];
(partly becuase with the default settings
[[fallthrough]];
is now needed to silence warnings where a fallthrough *is* intended).

Based on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109653#c3
I have attached a patch for ports/cpl_port.h


Sorry, the previous patch did not mauch gdal's usage of CPL_FALLTHROUGH.
Please use this patch instead.


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDALOpenShared: warning about "filename" vs "description"

[Even Rouault]

Hi Michael,

Fixed per https://github.com/OSGeo/gdal/pull/7659

Even

Le 24/04/2023 à 05:09, Michael Sumner a écrit :


When invoking  info on this vrt connection

gdalinfo 
"vrt://https://herbariumnsw-pds.s3-ap-southeast-2.amazonaws.com/images/NSW041500.jp2?a_ullr=0,9661,7180,0"; 



I see this warning

Warning 6: A dataset opened by GDALOpenShared should have the same 
filename 
(https://herbariumnsw-pds.s3-ap-southeast-2.amazonaws.com/images/NSW041500.jp2) 
and description (/vsimem/http_1/NSW041500.jp2)


The warning seems unreasonable given that we are quite far from a 
"filename" for a dsn? There's no warning on the bare url or with 
/vsicurl:


gdalinfo 
https://herbariumnsw-pds.s3-ap-southeast-2.amazonaws.com/images/NSW041500.jp2


gdalinfo 
 /vsicurl/https://herbariumnsw-pds.s3-ap-southeast-2.amazonaws.com/images/NSW041500.jp2


## GDAL version and provenance

Ubuntu 20.04

GDAL 3.7.0dev-717dcc0eed, released 2023/04/22 (debug build)




--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Ogr2ogr CSV driver not handling correctly line breaks inside columns

[Even Rouault]

Moises,

as far as I can see with your example, the CSV driver behaves "properly" 
in reading and writing of field values with line breaks.


It follows the "Fields with embedded line breaks must be quoted" rule of 
https://en.wikipedia.org/wiki/Comma-separated_values


$ ogr2ogr out.csv /vsizip/dataframe.zip

$ cat out.csv
id,descriptio
"1",This is my third row
"2","this is
my string
"
"3",This is my third row

$ ogrinfo out.csv -al
INFO: Open of `out.csv'
  using driver `CSV' successful.

Layer name: out
Geometry: None
Feature Count: 3
Layer SRS WKT:
(unknown)
id: String (0.0)
descriptio: String (0.0)
OGRFeature(out):1
  id (String) = 1
  descriptio (String) = This is my third row

OGRFeature(out):2
  id (String) = 2
  descriptio (String) = this is
my string


OGRFeature(out):3
  id (String) = 3
  descriptio (String) = This is my third row

But in your example using /vsistdout/ and -lco CREATE_CSVT=YES is going 
to result in an invalid CSV file which will mix both the .csvt and .csv 
content


Even

Le 24/04/2023 à 13:34, Moises Calzado via gdal-dev a écrit :

Hello!

We're trying to convert a Shapefile into a CSV using ogr2ogr and we're 
having some issues while dealing with some columns that contain line 
breaks inside their values. If we have a line with the following 
string, ogr2ogr detects that the line break is a new line and it 
returns two lines.


"this is my \n value"


That's the command that we're executing:

ogr2ogr -f CSV -skipfailures -makevalid /vsistdout/
/vsizip/shapefile.zip -simplify 0.1 -dim XY -t_srs EPSG:4326
-lco GEOMETRY=AS_WKT -lco GEOMETRY_NAME=geom -lco CREATE_CSVT=YES
> result.csv


Is this an expected behaviour, or is there any way to avoid this?
Sharing an example Shapefile so that you can try to reproduce that 
behaviour: 
https://drive.google.com/file/d/1gFqfTP02KTFoavJyyO-Ix05YwZB2tS24/view?usp=sharing


Thanks so much in advance,
Regards.

--
*Moises Calzado*

Support Engineer

+34671264286 | mcalz...@carto.com | CARTO 



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GRIB can't get nodata and cmake can't disable FileGDB

[Even Rouault]

Ivan,

(I see you've filed an issue for the GRIB nodata issue since.)




But I am getting a compilation error, somewhere on 
ogrsf_frmts/openfilegdb, and I cannot turn off that driver on cmake. 
The parameter -DGDAL_USE_FILEGDB=OFF seems to work fine.


For the OpenFileGDB driver, this should be 
-DOGR_ENABLE_DRIVER_OPENFILEGDB=OFF


-DGDAL_USE_FILEGDB=OFF is to disable the use of the proprietary SDK

I somehow recall someone had mentioned a building issue some time ago 
but can't find it. Anyway, I assume you may perhaps use an older 
compiler, like MSVC 2015 which we no longer test/support and that 
doesn't support some aspects of C++11. You may perhaps try to revert 
https://github.com/OSGeo/gdal/commit/b8076543f0fefb7106351297b845df2f8b4d8514 
alternatively, or better upgrade to something more recent


More generally when you report something, be very specific and post 
exact error messages.


Even

--

http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL client certificates: CURLOPT_SSLCERT/curl_easy_setopt missing?

[Even Rouault]

Stephan,

this should be addressed per https://github.com/OSGeo/gdal/pull/7657. 
I'd appreciate if you could do some runtime testing of this pull 
request, as it was only compile tested on my side.


There's no option to honor .curlrc, and it doesn't look trivial to add 
support for that given this is something specific to the "curl" binary 
itself, not the libcurl library (cf 
https://stackoverflow.com/questions/3698605/can-libcurl-be-configured-to-use-my-curlrc-when-used-from-php). 



You may get a similar functionality by setting the GDAL configuration 
options in the GDAL configuration file: 
https://gdal.org/user/configoptions.html#gdal-configuration-file


Even

Le 27/04/2023 à 08:36, Stephan Imfeld a écrit :

Hi

We are trying to serve some COG-Tiffs using a https server. Some of them 
require authentication. Accessing the data without or with basic authentication 
through gdal/qgis using /vsicurl/... works fine, but we have not found a way to 
use client certificates.

Using curl (7.81.0) directly (with
CURLOPT_SSLCERT/CURLOPT_SSLKEY/CURLOPT_KEYPASSWD/CURLOPT_SSLCERTTYPE)
works fine with client certificates enabled, even when using the options in a 
.curlrc file.

But it seems to be that libcurl does not honor .curlrc, and consequently we 
have not been able to get gdal working with the client certificate.

Maybe I overlooked something during the past few days, but I have not been able 
to get it working through gdal. Is there a way passing 
CURLOPT_SSLCERT/CURLOPT_SSLKEY/CURLOPT_KEYPASSWD/CURLOPT_SSLCERTTYPE (or using 
curl_easey_setup in a more general way) from gdal to libcurl? Any other ways to 
do it?

Regards,
Stephan

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] KML polygons with a label

[Even Rouault]

How do I get the LabelStyle correct?

What do you mean by "correct" ? Are you wishing to see the font, size 
and text attributes translated in LabelStyle ? As far as I can see in 
the KML spec, there is no direct equivalence for them.


There is a  element that could perhaps emulate some sort of 
scale, but this is an amplification factor, with 1 meaning nominal size. 
So how a font size of 12 in Arial or Helvetica translates as scale...


Even

--

http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.6.4 is released

[Even Rouault]

Hi,

On behalf of the GDAL/OGR development team, I am pleased to announce the 
release of the GDAL/OGR 3.6.4 bug fix version, the last one in the 3.6 
series


Consult the release notes for the list of issues addressed:

    https://github.com/OSGeo/gdal/blob/v3.6.4/NEWS.md

The sources are available at:

    https://download.osgeo.org/gdal/3.6.4/gdal-3.6.4.tar.xz
    https://download.osgeo.org/gdal/3.6.4/gdal-3.6.4.tar.gz
    https://download.osgeo.org/gdal/3.6.4/gdal364.zip

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: Adopt GDAL 3.6.4RC1 as 3.6.4 release

[Even Rouault]

Hi,

motion passed with +1 from PSC members JukkaR, HowardB and me.

Even

Le 19/04/2023 à 11:14, Even Rouault a écrit :

Hi,

Motion:

Adopt GDAL 3.6.4RC1 as 3.6.4 release

Starting with my +1

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Motion: Adopt GDAL 3.6.4RC1 as 3.6.4 release

[Even Rouault]

Hi,

Motion:

Adopt GDAL 3.6.4RC1 as 3.6.4 release

Starting with my +1

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.6.4 RC1 available

[Even Rouault]

Hi,

I have prepared a GDAL/OGR 3.6.4 release candidate.

Pick up an archive among the following ones (by ascending size):

  https://download.osgeo.org/gdal/3.6.4/gdal-3.6.4rc1.tar.xz
  https://download.osgeo.org/gdal/3.6.4/gdal-3.6.4rc1.tar.gz
  https://download.osgeo.org/gdal/3.6.4/gdal364rc1.zip

A snapshot of the gdalautotest suite is also available:

https://download.osgeo.org/gdal/3.6.4/gdalautotest-3.6.4rc1.tar.gz
  https://download.osgeo.org/gdal/3.6.4/gdalautotest-3.6.4rc1.zip

The NEWS file is here:

  https://github.com/OSGeo/gdal/blob/v3.6.4RC1/NEWS.md

I'll call for a vote promoting it to final before Wednesday if no
serious problems are reported before.

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Any timeframe for 3.7.0?

[Even Rouault]

Hi Nyall,

the plan is to have a release candidate out for beginning of May

Even

Le 17/04/2023 à 04:48, Nyall Dawson a écrit :

Hi list,

Is there any word on when the GDAL 3.7 release might be finalised?
(There's so much exciting changes in this release and I'm keen for
them to hit users!)


Nyall
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Mysterious master build break on Win11

[Even Rouault]

Joaquim,


Now, the weirdest part is that this happens in a new computer with 
Win11 but the GDAL repo dir was copied from another computer (that has 
Win10) and there this error does not show up. Both are built with same 
version of VS.


So you also copied a build directory with existing CMakeCache.txt and 
other generated files? I don't think this is a safe practice, unless 
your source & target computers are exactly the same.


What if you build from a clean build directory ?

Is there a M_LIB line in the CMakeCache.txt of the Win10 and Win11 
computers?


If you update to latest master (or cherry-pick 
https://github.com/OSGeo/gdal/commit/0a062f8ab132e22d1bace91081d606584b8e8d7c), 
run "cmake -DSHOW_DEPS_PER_TARGET=YES [...]", and look in the output 
messages lines starting with "Target [xxx]: links against" and 
containing m.lib . That should hopefully give a clue of which 
third-party library drag this m.lib dependency


Even

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] VRT file: Recursion detected

[Even Rouault]

Le 13/04/2023 à 11:02, Laurențiu Nicola via gdal-dev a écrit :

Hi,

Just a guess, but maybe your VRT is including itself?


That's indeed the purpose of this check (or detecting a VRT A including 
a VRT B including a VRT A, etc.). But the check is a bit tricky, and 
that might be just a bug in the particular situation. The best would be 
to open a ticket with a reproducer


Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Seeking functionality inverse of gdal_merge.py

[Even Rouault]

Jesse,

you may want to look at https://gdal.org/programs/gdal_retile.html

Even

Le 12/04/2023 à 20:41, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND 
APPLICATIONS INC] via gdal-dev a écrit :


Hi,

I’d like to take a large raster image and chop it into XY tiles.  I 
see gdal2tiles.py, but this is specific to web hosting and inherits 
untenable constraints (like byte only datatype).


Does such a program exist by the GDAL team, or do I need to implement 
this manually?  If the latter, I suppose I could modify the gdal2tiles 
program.



Best,

Jesse


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Problem with file source projection doing a vertical datum reprojection

[Even Rouault]

Le 12/04/2023 à 11:45, Michael Smith a écrit :

I guess I thought GTIFF_REPORT_COMPD_CS would only change the display
of the output, not change the interpretation of the SRS?

It is documented in https://gdal.org/drivers/raster/gtiff.html as:

"""GTIFF_REPORT_COMPD_CS : Can be set to TRUE to avoid stripping the 
vertical CRS of compound CRS when reading the SRS of a file. Does not 
affect the writing side. Default value : FALSE for GeoTIFF 1.0 files, or 
TRUE (starting with GDAL 3.1) for GeoTIFF 1.1 files."""


This seems to be a reasonably concise description of its effect (if 
you've suggestion to make it clearer, that's of course welcome). So this 
is not restricted to tuning display of the CRS in gdalinfo or other 
utilities, but really to what kind of CRS object the GeoTIFF returns to 
calling code.



--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Problem with file source projection doing a vertical datum reprojection

[Even Rouault]

Michael,

it is well possible this might have worked in the past due to slightly 
different behavior of GDAL and/or PROJ, but the current behavior seems 
consistent to me. You're trying to transform between 2 compound CRS, the 
source one with a unknown vertical datum and the target one with EGM2008 
height. There's nothing reasonable that PROJ can do regarding the 
vertical transformation in that situation, in particular it would be 
quite a bold assumption to assume that the source CRS is actually meant 
to be refering to ellipsoidal heights.  If you override with -s_srs 
EPSG:32617 (or drop REPORT_COMPD_CS=TRUE), then GDAL will auto-promote 
the source CRS to a 3D CRS with ellipsoidal heights, which could also be 
questionable from a purist point of view, but is the behaviour we have 
had for long now.


Even

Le 12/04/2023 à 10:36, Michael Smith a écrit :


I have a file with a projection shown, using REPORT_COMPD_CS=TRUE of:

COMPOUNDCRS["WGS 84 / UTM zone 17N + unknown",

    PROJCRS["WGS 84 / UTM zone 17N",

    BASEGEOGCRS["WGS 84",

    DATUM["World Geodetic System 1984",

    ELLIPSOID["WGS 84",6378137,298.257223563,

LENGTHUNIT["metre",1]]],

    PRIMEM["Greenwich",0,

ANGLEUNIT["degree",0.0174532925199433]],

    ID["EPSG",4326]],

    CONVERSION["UTM zone 17N",

    METHOD["Transverse Mercator",

    ID["EPSG",9807]],

    PARAMETER["Latitude of natural origin",0,

ANGLEUNIT["degree",0.0174532925199433],

    ID["EPSG",8801]],

    PARAMETER["Longitude of natural origin",-81,

ANGLEUNIT["degree",0.0174532925199433],

    ID["EPSG",8802]],

    PARAMETER["Scale factor at natural origin",0.9996,

    SCALEUNIT["unity",1],

    ID["EPSG",8805]],

    PARAMETER["False easting",50,

    LENGTHUNIT["metre",1],

    ID["EPSG",8806]],

    PARAMETER["False northing",0,

    LENGTHUNIT["metre",1],

    ID["EPSG",8807]]],

    CS[Cartesian,2],

    AXIS["easting",east,

    ORDER[1],

    LENGTHUNIT["metre",1]],

    AXIS["northing",north,

    ORDER[2],

    LENGTHUNIT["metre",1]],

    ID["EPSG",32617]],

    VERTCRS["unknown",

    VDATUM["unknown"],

    CS[vertical,1],

    AXIS["up",up,

    LENGTHUNIT["metre",1,

    ID["EPSG",9001]

When trying to gdalwarp to EGM2008 using gdalwarp -t_srs 
epsg:32617+3855, vertical datum reprojection does not occur unless I 
overwride the source srs with -s_srs epsg:32617. Using GDAL 3.6.2 and 
proj 9.1.1. With proj_debug, I can see that the PROJ_TRACE: 
vgridshift: is not occurring unless the srs is overridden. I’m fairly 
sure that this has worked in the past but I can’t say exactly which 
version. This is an older geotiff that was written about 7 years ago.


--

Michael Smith

US Army Corps / Remote Sensing GIS Center


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CloudSQLite - useless invention or not?

[Even Rouault]

Hi Jukka,

Thanks for sharing. This seems to be quite a recent addition (end of 
2019 for the start of the public activity), so probably somewhat a bit 
experimental, at least probably not as mature as core SQLite (also 
reading this comment from a likely early user of the project which 
questions some essential design aspects: 
https://sqlite.org/cloudsqlite/forumpost/3856ce738d). It is somewhat an 
hybrid solution between the SQLite we know (mostly for local usage) and 
a traditional server-based SQL database, but actually closer to the 
former than the later. It doesn't fully cover multi-user / concurrent 
scenarios, compared to to a traditional server-based SQL database, so 
I'm a bit skeptical one would want to use that for "enterprise" needs. 
For read-only scenarios, I'm not sure how better if would compare to 
accessing a regular SQLite3 database through /vsicurl/ , /vsis3/ etc. 
Probably not that much (the local deamon  that maintain a cached version 
of downloaded blocks might help in some scenarios though), but of course 
that would need to be tested. The "cloud" term somewhat brings the usual 
expectations of "magic" happening on a server (or many!), but 
CloudSQlite is really just about remote storage of the database, so all 
requests are still processed on the client. If you run a request without 
indices, you might end up downloading the whole database. Or a statement 
like "UPDATE table SET some_column = some_value" will definitely require 
downloading & re-uploading the whole table.


So I wouldn't say this is useless, as if it was designed, it is 
hopefully to serve some particular use cases, but I suspect they are 
quite niche and that existing well-known server-based database solutions 
are probably a better choice in the general case.


Even

Le 11/04/2023 à 21:55, Rahkonen Jukka a écrit :


Hi,

I got a link into 
https://sqlite.org/cloudsqlite/doc/trunk/www/index.wiki 
in a 
GeoPackage meeting today and I wonder if that could be useful for 
something. Any opinions?


-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] recent change?, issue with OGRGeometryFactory in R

[Even Rouault]

Michael,

There have indeed been recent changes in const correctness of a few 
methods returning or accepting a OGRSpatialReference* (cf 
https://github.com/OSGeo/gdal/pull/7381), including 
OGRGeometryFactory::createmFromWkt().


This requires a rebuild of code depending on the GDAL C++ API, as the 
GDAL ABI generally changes between feature versions.


Even

Le 01/04/2023 à 07:24, Michael Sumner a écrit :

Hi,

I'm here to ask for help finding where this issue is. If I use an 
earlier build it still works (these are R packages sf and terra that 
are well tested against previous releases).


Has something changed with the headers structure for 
OGRGeometryFactory perhaps? (I'm out of my depth for how to track this 
down).   If it's an issue on the R side I'll fix it there, but mostly 
I'm here to ask what has changed on the GDAL side to result in this 
symbol problem (or whatever it is I'm doing wrong).


using latest build  24afc75d3c3f2c85621e8b8d83f54a73b1d3ad33

I see

> library(sf)
Error: package or namespace load failed for ‘sf’ in dyn.load(file, 
DLLpath = DLLpath, ...):
 unable to load shared object 
'/usr/local/lib/R/site-library/sf/libs/sf.so':
  /usr/local/lib/R/site-library/sf/libs/sf.so: undefined symbol: 
_ZN18OGRGeometryFactory13createFromWktEPKcP19OGRSpatialReferencePP11OGRGeometry

> library(terra)
Error: package or namespace load failed for ‘terra’ in dyn.load(file, 
DLLpath = DLLpath, ...):
 unable to load shared object 
'/usr/local/lib/R/site-library/terra/libs/terra.so':
  /usr/local/lib/R/site-library/terra/libs/terra.so: undefined symbol: 
_ZN18OGRGeometryFactory13createFromWktEPKcP19OGRSpatialReferencePP11OGRGeometry


Using a prior build eca6f7bf17fe500ef4c0ba9793e93ea305bab1fc it works.

The offending code in terra is this (I think)

https://github.com/rspatial/terra/blob/ddcf84ce901a269ac13118d650efac16f89a8602/src/read_ogr.cpp#L694

In sf it is


https://github.com/r-spatial/sf/blob/main/src/gdal_read.cpp#L582
Thank you.

Cheers, Mike


--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Data corruption when using OpenFileGDB writing and SetFeature()

[Even Rouault]

Hi,

just to notify that we have had recently a few reports of users using 
the write side of the OpenFileGDB driver, and SetFeature() to update 
existing features.


This proved to cause data corruptions in some situations, sometimes with 
an error message on re-reading or when regenerating the spatial index, 
but not always. The same could also likely happen with DeleteFeature() + 
CreateFeature() scenarios, basically all situations where the driver 
tries to re-use a freed space by using the .freelist file where free 
spaces are recorded.


Creation of a new file or appending to a file whose features have never 
been edited or deleted is OK though. You may also workaround the bug by 
doing one SetFeature() operation at a time, closing the dataset, and 
deleting the .freelist files in the .gdb directory, before re-opening 
and doing the next edition, but that's likely to be unpractical & 
inefficient.


The bug has now been fixed in master and 3.6 branches, but affects 3.6.0 
to 3.6.3 releases. I'll issue a 3.6.4 in ~2 weeks.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL Docker Build fails

[Even Rouault]

Dirk,

Le 28/03/2023 à 21:11, Dirk Stenger a écrit :


All,

Currently, the GDAL Docker Build fails:

/~/git/gdal$ docker/ubuntu-full/build.sh --platform linux/amd64//
//...//
// ---> Running in c13d037d7761//
//Downloading cache...//
//rsync: [sender] change_dir "osgeo/gdal:ubuntu-full/proj/x86_64" (in 
gdal-docker-cache) failed: No such file or directory (2)//
//rsync error: some files/attrs were not transferred (see previous 
errors) (code 23) at main.c(1865) [Receiver=3.2.7]//

//rsync: [Receiver] read error: Connection reset by peer (104)//
//The command '/bin/sh -c . /buildscripts/bh-set-envvars.sh     && 
DESTDIR=/build_tmp_proj /buildscripts/bh-proj.sh      && 
LD_LIBRARY_PATH=/build_tmp_proj/usr/local/lib 
/build_tmp_proj/usr/local/bin/projsync --target-dir /tmp/proj_grids 
--all      && rm -rf /build_tmp_proj' returned a non-zero code: 23/


A couple of days ago, I was still able to build it (v3.5).

Make sure you update on latest git. If you're on the 3.5 branch, I'm not 
sure its build scripts will work properly with latest master


But given where the error occurs, I suspect this might be something 
related to downloading PROJ remote grids. There has been a new grid 
added today and the indexes refreshed. Perhaps there has been some 
out-of-sync issue with CloudFront. I suggest you just retry, a few times 
if needed (that just worked fine for me without any need for retry, but 
we might not hit the same CloudFront instances)


Even


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL sponsored maintenance report: Aug 2021-March 2023

[Even Rouault]

Hi,


After more than one year and a half of benefiting from the GDAL 
sponsored maintenance program, it is time to share with the community 
what work has been accomplished thanks to that sponsorship.  It is quite 
hard to summarize, as the amount of tasks reaches 1800 for a total of 
1313 hours. So lots of small tasks that make the daily reality of 
maintaining a project like GDAL.



Their global repartition is the following one:

 *

   Bug fixing: 51% of actions, 55% of time spent (including 177 bug
   fixes related to issues found by oss-fuzz)

 *

   Enhancements: 5% of actions, 15% of time spent

 *

   CMake related work: 5% of actions, 10% of time spent

 *

   Review of contributions: 20% of actions, 5% of time spent

 *

   Release management: 2% of actions, 4% of time spent

 *

   Bug triaging and analysis (without corresponding bug fix): 8% of
   actions, 3% of time spent

 *

   Maintenance of continuous integration setup: 3% of actions and time
   spent

 *

   Code enhancements/cleanups: 2% of actions and time spent

 *

   Documentation: 1% of actions and time spent

 *

   Other activities: mailing list discussions, bug reports to other
   projects, meetings, etc.


This work has benefited mostly to GDAL (80%), PROJ (15%), libtiff (2%), 
openjpeg (1%) and more marginally to other projects associated with GDAL 
such as Xerces-C, poppler, libgeotiff, cppcheck, netcdf, curl, libjxl 
and shapelib.



The CMake related work has been a major item of work for the mid 2021 - 
beginning of 2022 period. I cannot overstate the importance of the 
initial contribution of this work made by Hiroshi Miura who brought most 
of the initial material. That said, without the sponsored maintenance 
program which helped polishing and making it production ready for all 
environments supported by GDAL, this would likely have remained a 
out-of-tree project. I believe most stakeholders (contributors and 
users, at least the ones who build from source) are very satisfied with 
this transition from the historic build systems to a unified and more 
modern one, with consistent option naming. Building on Windows is in 
particular much easier nowadays, in particular when leveraging 
dependencies from distributions like vcpkg or Conda Forge. For GDAL 
developers, the new build system offers integration with IDEs and solves 
long standing annoyances like missing header dependency rules for 
partial rebuilds.



Although it doesn’t show up particularly in the above statistics, making 
sure that the continuous integration configurations remain “green” at 
all times is a constant source of attention. Given the number of 
environments tested and the number of dependencies of GDAL, there is 
hardly a week where some action is not needed in that area to make sure 
that the code builds and compiles cleanly, tests pass, etc. We can now 
track a few rolling distributions to detect as early as possible sources 
of incompatibilities with our dependencies, and act early: report issues 
to upstream if there are bugs, or do changes in our code & build scripts 
to take them into account.



Making sure that code contributions from others are reviewed in a timely 
manner is important for the contributor experience. The average delay 
for a submitted pull request (excluding mine) to be commented by me (or 
merged if no comment needed) is 22 hours, and the median time 3h20 
(statistics gathered on 601 pull requests since August 2021, source 
script at https://gist.github.com/rouault/0fbd37f59b8e93ae63761468a5600262)



In the category of regular tasks, one can also mention: updating the 
EPSG dataset in PROJ (and coordinating with IOGP when detecting issues), 
refreshing vendored of copies in the GDAL source tree (libtiff 
typically), making sure that new SQLite releases play nicely with PROJ 
which has some stressing SQL queries (we spot recently a performance 
regression in SQLite 3.41.0 and reported it to upstream)



The following RFCs have been implemented (not mentioning a few 
procedural ones) thanks to the sponsorship:


 *

   RFC 87: Signed int8 data type for raster
   

 *

   RFC 88: Use GoogleTest framework for C/C++ unit tests
   

 *

   RFC 90: Direct access to compressed raster data
   

 *

   RFC 91: GDALDataset::Close() method
   

 *

   RFC 93: OGRLayer::UpdateFeature() method
   


Other recent significant work includes enhancements in Python exception 
handling (in preparation for enabling them by default in a future 4.0 
release), and enabling them in the regression test suite and utilities.



The following feature releases have been issued: 3.4.0, 3.5.0 and 3.6.0. 
And the following bug fixes releases: 3.3.2, 3.3.3, 3

Re: [gdal-dev] Python bindings: enabling exceptions by default?

[Even Rouault]

Hi,

https://github.com/OSGeo/gdal/pull/7475 has now been merged into master. 
The UseExceptions()/DontUseExceptions() methods when invoked from one 
module also affect all other gdal, ogr, osr, gnm modules at once (that 
is now gdal.UseExceptions() is equivalent to gdal.UseExceptions() + 
ogr.UseExceptions() + osr.UseExceptions() + gnm.UseExceptions())


Even

Le 21/03/2023 à 18:21, Even Rouault a écrit :

Hi,

Given the feedback received, I've amended the pull request with an 
extra commit to disable exceptions by default, except in autotest/ and 
scripts. And emit a FutureWarning if exceptions have not been 
explicitly enabled or disabled. So this will test the 
exceptions-by-default mode in the GDAL code base without imposing it 
to external code, and warning such external code that behaviour will 
change in the future


Even

Le 19/03/2023 à 13:34, Even Rouault a écrit :

Hi,

I've prepared a pull request that does the above, but this raises a 
number of questions. See my longish comment at 
https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. 
Thoughts appreciated


Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: adopt RFC 94: Numeric fields width/precision metadata

[Even Rouault]

+1 Even

Le 23/03/2023 à 13:20, ElPaso a écrit :

Hi,


Motion: adopt RFC 94:  Numeric fields width/precision metadata

 https://github.com/OSGeo/gdal/pull/7264


Rendered text view:

https://github.com/OSGeo/gdal/blob/e19eda1db023abdaf2d24036aa2e167e99e35029/doc/source/development/rfc/rfc94_field_precision_width_metadata.rst 




Regards.



--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Python bindings: enabling exceptions by default?

[Even Rouault]

Hi,

Given the feedback received, I've amended the pull request with an extra 
commit to disable exceptions by default, except in autotest/ and 
scripts. And emit a FutureWarning if exceptions have not been explicitly 
enabled or disabled. So this will test the exceptions-by-default mode in 
the GDAL code base without imposing it to external code, and warning 
such external code that behaviour will change in the future


Even

Le 19/03/2023 à 13:34, Even Rouault a écrit :

Hi,

I've prepared a pull request that does the above, but this raises a 
number of questions. See my longish comment at 
https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. 
Thoughts appreciated


Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Python bindings: enabling exceptions by default?

[Even Rouault]

Does whether we call it 3.7 or 4.0 affect how long 3.6 is supported ?


I don't think Alan's comment would suggest that the next feature version 
planned for may would be called other than 3.7.0. IMHO there should be 
more substantial breaking changes to call it 4.0 than just what is 
discussed here. We have never talked about a 4.0 release up to now, 
either in its content or release schedule.


Until now, we have "only" maintained the last feature version during the 
development cycle of the next feature version, ie during 3.7 development 
phase, started after 3.6.0 release, and ending at 3.7.0, we issue point 
release of 3.6.z.  I see that when 3.0 was released we have issued 
bugfix releases of 3.0.z and 2.4.z (the latest one in the 2.y series) in 
parallel until 3.1.0 was released. I guess a similar strategy could 
apply for a 4.0 release (although the benefit/effort to have multiple 
several maintenance branches isn't obvious given than most packagers 
maintain just the last released version). Anyway, this derails from the 
topic of this thread, and could be re-discussed when we are at that point.


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL does not get correct GCPs from ESRI GeodataXform

[Even Rouault]

Filed as https://github.com/OSGeo/gdal/issues/7484 : At first sight the 
issue is that file uses TIFFTAG_RESOLUTIONUNIT=3 (pixels/cm), whereas 
the current code only deals with TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)


Le 20/03/2023 à 22:09, Rahkonen Jukka a écrit :


Hi,

I noticed this question in gis.stackexchange 
https://gis.stackexchange.com/questions/455609/warping-arcgis-9-2-georeferenced-tiffs-using-gdal-warp. 
It seems that GDAL cannot interpret right the ground control points 
which are stored into the aux.xml file. I can see that the support for 
GeodataXform has been improved a few times 
https://github.com/OSGeo/gdal/search?q=geodataxform&type=commits 
 but 
maybe this dataset (there is a link for downloading test data) has 
some previously untested sort of aux.xml. The GeoTIFF has wrong GCPs 
with small pixel values like (16.5886813769484,11.2421187314794) but I 
made also a test with “--config GDAL_GEOREF_SOURCES PAM” and even by 
creating a cleaned baseline tiff + PAM file but the GCPs looked still 
wrong.


-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Python bindings: enabling exceptions by default?

[Even Rouault]

Hi,

I've prepared a pull request that does the above, but this raises a 
number of questions. See my longish comment at 
https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. 
Thoughts appreciated


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Migration of GDAL Docker images from docker.io to ghcr.io

[Even Rouault]

Hi,

All open source projects with docker images hosted on docker.io have 
recently received a "Docker is sunsetting Free Team organizations" 
email, which indicates that they will be suspending our account on April 
14, 2023 unless we upgrade to a paid subscription (cf 
https://github.com/OSGeo/gdal/issues/7447). There's little interest in 
doing so (I've also tried to apply to their open source program, but 
there's some indication that it won't qualify for it), given that GitHub 
Container repository (ghcr.io) does the same job without the rate limit, 
so I've instead migrated the existing images to the ghcr.io.  You can 
see them at https://github.com/OSGeo/gdal/pkgs/container/gdal


So instead of osgeo/gdal:{tagname}, you must now use 
ghcr.io/osgeo/gdal:{tagname}


Our scripts and docs have been updated per 
https://github.com/OSGeo/gdal/pull/7463 . New images will no longer be 
pushed to the docker.io registry. Existing images remain for now, until 
the account gets suspended (or even if Docker back-pedals, I guess we'll 
eventually remove the *-latest tags to avoid people using outdated stuff 
without knowing)


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Difference between ogr2ogr and Python reprojection

[Even Rouault]

Nicolas,

ogr2ogr uses the OGRGeometryFactory::transformWithOptions() method that 
has logic to deal with poles, antimerdian, etc. whereas 
OGRGeometry::transform() does basic point-to-point reprojection


You can achieve what you want by using the ogr.GeomTransformer(ct) 
method that returns a ogr.GeomTransformer object that has a 
Transform(ogr.Geometry) method. See below test from GDAL's autotest suite:


def test_ogr_geomtransfomer_default():

    src = osr.SpatialReference()
    src.ImportFromEPSG(32660)

    dst = osr.SpatialReference()
    dst.SetAxisMappingStrategy(osr.OAMS_TRADITIONAL_GIS_ORDER)
    dst.ImportFromEPSG(4326)

    ct = osr.CoordinateTransformation(src, dst)
    geom = ogr.CreateGeometryFromWkt(
    "LINESTRING(832864.275023695 0,835092.849076364 0)"
    )
    transformer = ogr.GeomTransformer(ct)

    geom_dst = transformer.Transform(geom)
    assert (
    geom_dst.ExportToWkt()
    == "MULTILINESTRING ((179.99 0.0,180 0),(-180 0,-179.99 0.0))"
    )

Even

Le 14/03/2023 à 19:15, Dion-Degodez,Nicolas (ECCC) a écrit :


_Context_

We are trying to reproject a GeoJSON polygon to EPSG:4326.

In some cases, depending on whether we use the ogr2ogr command line or 
the equivalent in python, the resulting polygon is different.


In those situations, the command line gives us the appropriate 
polygon, whereas the python code gives us a distorted polygon.


_Question_

Any idea why there is a difference between the command line and the 
python results? Is there a mistake in our interpretation of the 
code/how we converted it in python?


_Sample GeoJSON_: https://pastebin.com/hUiVVjFv

Proj4: "+proj=stere +lat_0=90 +lat_ts=60 +lon_0=264 +x_0=0 +y_0=0 
+R=6371229 +units=m +no_defs"


_Command line that works_: ogr2ogr -s_srs "+proj=stere +lat_0=90 
+lat_ts=60 +lon_0=264 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs" 
-t_srs EPSG:4326 result_command_line.json geojson.json


Result: https://pasteboard.co/cTpZBRamEYv8.jpg

_Python code_ (we would expect the same output as the command line): 
https://pastebin.com/QgWDBL1W


Result: https://pasteboard.co/8iSv0vbnQlHw.jpg

Thank you!

Nicolas D.


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.6.3 is released

[Even Rouault]

Hi,

On behalf of the GDAL/OGR development team, I am pleased to announce the 
release of the GDAL/OGR 3.6.3 bug fix version.


Consult the release notes for the list of issues addressed:
  https://github.com/OSGeo/gdal/blob/v3.6.3/NEWS.md

The sources are available at:

    https://download.osgeo.org/gdal/3.6.3/gdal-3.6.3.tar.xz
    https://download.osgeo.org/gdal/3.6.3/gdal-3.6.3.tar.gz
    https://download.osgeo.org/gdal/3.6.3/gdal363.zip

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: Adopt GDAL 3.6.3RC1 as 3.6.3 release

[Even Rouault]

I declare this motion passed with +1 from PSC members HowardB, MateuszL, 
NormanB, JukkaR, TamasS and me.


Even

Le 09/03/2023 à 16:40, Even Rouault a écrit :
Same motion, just fixing the title to reflect the correction version 
number, as kindly pointed out.


Le 09/03/2023 à 16:33, Even Rouault a écrit :

Hi,

Motion:

Adopt GDAL 3.6.3RC1 as 3.6.3 release

Starting with my +1

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL - Python Bindings - CURL NETRC_FILE

[Even Rouault]

You're right, I looked too fast. I've added a GDAL_HTTP_NETRC_FILE 
option in https://github.com/OSGeo/gdal/pull/7435


Le 12/03/2023 à 16:11, Morgan Kerle a écrit :

HI Even,

Thanks for your reply. I had assume that GDAL_HTTP_NETRC set the 
option NETRC as in papszOptions which accepts "YES" or "NO" to 
indicate if ~/.netrc file or $NETRC should be searched for 
credentials. If this is the same option as CURLOPT_NETRC 
(https://curl.se/libcurl/c/CURLOPT_NETRC.html), then it does not allow 
the user to specify a location for the .netrc file. In any case, I 
attempted setting the GDAL_HTTP_NETRC via gdal.SetConfigOption(), but 
I am still getting the same curl output (Couldn't find host 
data.lpdaac.earthdatacloud.nasa.gov 
<http://data.lpdaac.earthdatacloud.nasa.gov> in the (nil) file; using 
defaults).


I assume (nil) file should be showing the file path supplied via 
GDAL_HTTP_NETRC?


Morgan

On Sun, 12 Mar 2023 at 10:58, Even Rouault 
 wrote:


Morgan,

The right name is GDAL_HTTP_NETRC :
https://gdal.org/api/cpl.html#_CPPv412CPLHTTPFetchPKc12CSLConstList

Even

GDAL_HTTP_NETRC

Le 12/03/2023 à 15:49, Morgan Kerle a écrit :

I am trying to use the /vsicurl/ file hander to access a remote
tiff file using basic auth. Is it possible to provide a alternate
location

for a .netrc file via the Python bindings? I have attempted to
use SetConfigOption("CPL_CURL_NETRC_FILE"
"abs/file/path/to/.netrc").

This appears to correctly set the CURL_NETRC_FILE option, as CURL
debugging output shows "Couldn't find host
data.lpdaac.earthdatacloud.nasa.gov
<http://data.lpdaac.earthdatacloud.nasa.gov> in the (nil) file;
using defaults". However, it appears the file is not found /
accessible by libcurl.

Any suggestions as to why this may be the case? I have already
confirmed that the file exists and is accessible, and running
CURL with the same parameters in bash works fine.

Regards,

Morgan

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


-- 
http://www.spatialys.com

My software is free, but my time generally not.


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL - Python Bindings - CURL NETRC_FILE

[Even Rouault]

Morgan,

The right name is GDAL_HTTP_NETRC : 
https://gdal.org/api/cpl.html#_CPPv412CPLHTTPFetchPKc12CSLConstList


Even

GDAL_HTTP_NETRC

Le 12/03/2023 à 15:49, Morgan Kerle a écrit :
I am trying to use the /vsicurl/ file hander to access a remote tiff 
file using basic auth. Is it possible to provide a alternate location


for a .netrc file via the Python bindings? I have attempted to use 
SetConfigOption("CPL_CURL_NETRC_FILE" "abs/file/path/to/.netrc").


This appears to correctly set the CURL_NETRC_FILE option, as CURL 
debugging output shows "Couldn't find host 
data.lpdaac.earthdatacloud.nasa.gov in the (nil) file; using 
defaults". However, it appears the file is not found / accessible by 
libcurl.


Any suggestions as to why this may be the case? I have already 
confirmed that the file exists and is accessible, and running CURL 
with the same parameters in bash works fine.


Regards,

Morgan

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Removal of Python SWIG python generated files from git

[Even Rouault]

To play devil's advocate, were there any (perceived) benefits to this
arrangement when originally introduced, other than not needing the
SWIG binary to compile Python bindings?


That predates the start of my involvement with GDAL, so my guess would 
be that this was just what you mention: for the sake of simplicity of 
people building GDAL, at a time where its only build requirement was the 
basic tools autoconf, make and g++. Nowadays, getting SWIG isn't harder 
than getting PROJ or any of the "optional" dependencies you generally 
want to make a reasonably feature complete GDAL build.


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Removal of Python SWIG python generated files from git

[Even Rouault]

Hi,

take this email as a mini-RFC regarding the changes in 
https://github.com/OSGeo/gdal/pull/7390 which remove the SWIG python 
generated files from git master, and thus make the SWIG binary a 
requirement to build the Python bindings.


Below, the text of the PR:

This removes a long-time oddity where the SWIG generated .py and .cpp 
files were stored in git, which was inconvenient for contributors, 
requiring them to have a precise SWIG version and enabling a specific 
CMake variable to resync the generated files to the source files, and 
causing confusion to occasional contributors who sometimes accidentally 
submitted changes to generated files instead of their source files. This 
was also different from the Java and CSharp bindings, which didn't have 
the generated files in git and required SWIG to be available to generate 
them.


Bonus of this move: in the process, due to some of our most up to date 
CI targets using SWIG 4.1, it enabled to fix 2 errors in SWIG syntax 
that were rejected by SWIG 4.1 (and hidden up to now due to using 4.0.2 
as our baseline)


This change shouldn't be a major hurdle for distributors, as most known 
binary distributions did regenerate the files anyway.


Note to people updating an existing git checkout: now the 
BUILD_JAVA/CSHARP/PYTHON_BINDINGS CMake variables are strictly honored. 
They are now initialized to ON only if all build requirements are 
available (SWIG and Python for the python bindings), and if set to ON 
but a requirement is missing, CMake fails. This might cause issues for 
people for example not having the requirements for the Java or CSharp 
bindings, but with an existing CMakeCache.txt with them set. Fix: run 
"cmake -UBUILD_JAVA_BINDINGS -UBUILD_CHSARP_BINDINGS 
-UBUILD_PYTHON_BINDINGS"


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Motion: Adopt GDAL 3.6.3RC1 as 3.6.3 release

[Even Rouault]

Same motion, just fixing the title to reflect the correction version 
number, as kindly pointed out.


Le 09/03/2023 à 16:33, Even Rouault a écrit :

Hi,

Motion:

Adopt GDAL 3.6.3RC1 as 3.6.3 release

Starting with my +1

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Motion: Adopt GDAL 3.6.3RC1 as 3.6.1 release

[Even Rouault]

Hi,

Motion:

Adopt GDAL 3.6.3RC1 as 3.6.3 release

Starting with my +1

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] OpenFileGDB: add read-only support for raster datasets

[Even Rouault]

Hi,

for those not actively monitoring github activity, I just want to point 
at https://github.com/OSGeo/gdal/pull/7389 that adds read access to 
raster layers of File
Geodatabases (.gdb directories) created by ArcGIS 10 and above (which is 
a capability that didn't even exist in the closed source FileGDB SDK)


Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ERROR 1: No inverse operation

[Even Rouault]

The file is indeed fine and was recognized in earlier GDAL versions (<= 
3.4). Bug fixed per https://github.com/OSGeo/gdal/pull/7382



Le 08/03/2023 à 14:22, Micha Silver a écrit :


Inline...


On 08/03/2023 14:33, Even Rouault wrote:


Ah, the issue here is that the GeoTIFF keys in the .tif partially 
define the projection (or possibly the .tif.aux.xml contain this 
invalid WKT). But METHOD["unnamed"] means that the projection method 
is unknown, so it is logical that GDAL/PROJ can't compute geographic 
coordinates from projected coordinates.



The output of "listgeo HiBar_21022022_A1.tif" might be useful to 
further diagnose.




I can't see any problem here:


listgeo HiBar_21022022_A1.tif
Geotiff_Information:
  Version: 1
  Key_Revision: 1.0
  Tagged_Information:
 ModelTiepointTag (2,3):
0 0 0
202245.240786848  421915.720555977  0
 ModelPixelScaleTag (1,3):
0.125 0.125 1
 End_Of_Tags.
  Keyed_Information:
 GTModelTypeGeoKey (Short,1): ModelTypeProjected
 GTRasterTypeGeoKey (Short,1): RasterPixelIsArea
 GTCitationGeoKey (Ascii,26): "PCS Name = Israel_TM_Grid"
 GeographicTypeGeoKey (Short,1): Code-4141 (Israel 1993)
 GeogAngularUnitsGeoKey (Short,1): Angular_Degree
 GeogEllipsoidGeoKey (Short,1): Ellipse_GRS_1980
 GeogSemiMajorAxisGeoKey (Double,1): 6378137
 GeogSemiMinorAxisGeoKey (Double,1): 6356752.31414036
 GeogInvFlatteningGeoKey (Double,1): 298.257222101
 GeogTOWGS84GeoKey (Double,7): -48  55   52
0    0    0
0
 ProjectedCSTypeGeoKey (Short,1): Code-2039 (Israel 1993 / Israeli 
TM Grid)

 PCSCitationGeoKey (Ascii,15): "Israel_TM_Grid"
 ProjLinearUnitsGeoKey (Short,1): Linear_Meter
 End_Of_Keys.
  End_Of_Geotiff.

PCS = 2039 (Israel 1993 / Israeli TM Grid)
Projection = 18204 (Israeli TM)
Projection Method: CT_TransverseMercator
  ProjNatOriginLatGeoKey: 31.734394 ( 31d44' 3.82"N)
  ProjNatOriginLongGeoKey: 35.204517 ( 35d12'16.26"E)
  ProjScaleAtNatOriginGeoKey: 1.07
  ProjFalseEastingGeoKey: 219529.584000 m
  ProjFalseNorthingGeoKey: 626907.39 m
GCS: 4141/Israel 1993
Datum: 6141/Israel 1993
Ellipsoid: 7019/GRS 1980 (6378137.00,6356752.31)
Prime Meridian: 8901/Greenwich (0.00/  0d 0' 0.00"E)
TOWGS84: -48,55,52,0,0,0,0
Projection Linear Units: 9001/metre (1.00m)

Corner Coordinates:
Upper Left    (  202245.241,  421915.721)  ( 35d 1'32.11"E, 29d53' 
7.03"N)
Lower Left    (  202245.241,  418790.721)  ( 35d 1'32.29"E, 
29d51'25.54"N)
Upper Right   (  205370.241,  421915.721)  ( 35d 3'28.57"E, 29d53' 
7.17"N)
Lower Right   (  205370.241,  418790.721)  ( 35d 3'28.72"E, 
29d51'25.68"N)

Center    (  203807.741,  420353.221)  ( 35d 2'30.42"E, 29d52'16.36"N)

Then, after:


gdal_translate -a_srs EPSG:2039 HiBar_21022022_A1.tif 
HiBar_21022022_A1b.tif



The ERROR goes away and I get:


gdalinfo HiBar_21022022_A1b.tif
Driver: GTiff/GeoTIFF
Files: HiBar_21022022_A1b.tif
Size is 25000, 25000
Coordinate System is:
PROJCRS["Israel 1993 / Israeli TM Grid",
   BASEGEOGCRS["Israel 1993",
   DATUM["Israel 1993",
   ELLIPSOID["GRS 1980",6378137,298.257222101,
   LENGTHUNIT["metre",1]]],
   PRIMEM["Greenwich",0,
   ANGLEUNIT["degree",0.0174532925199433]],
   ID["EPSG",4141]],
   CONVERSION["Israeli TM",
   METHOD["Transverse Mercator",
   ID["EPSG",9807]],
   PARAMETER["Latitude of natural origin",31.734393611,
   ANGLEUNIT["degree",0.0174532925199433],
   ID["EPSG",8801]],
   PARAMETER["Longitude of natural origin",35.204516944,
   ANGLEUNIT["degree",0.0174532925199433],
   ID["EPSG",8802]],
   PARAMETER["Scale factor at natural origin",1.067,
   SCALEUNIT["unity",1],
   ID["EPSG",8805]],
   PARAMETER["False easting",219529.584,
   LENGTHUNIT["metre",1],
   ID["EPSG",8806]],
   PARAMETER["False northing",626907.39,
   LENGTHUNIT["metre",1],
   ID["EPSG",8807]]],
   CS[Cartesian,2],
   AXIS["(E)",east,
   ORDER[1],
   LENGTHUNIT["metre",1]],
   AXIS["(N)",north,
   ORDER[2],
   LENGTHUNIT["metre",1]],
   USAGE[
   SCOPE["Cadastre, engineering survey, topographic mapping (large 
and medium scale)."],

   AREA["Israel - onshore; Palestine Territory - onshore."],
   BBOX[29.45,34.17,33.28,35.69

Re: [gdal-dev] ERROR 1: No inverse operation

[Even Rouault]

Ah, the issue here is that the GeoTIFF keys in the .tif partially define 
the projection (or possibly the .tif.aux.xml contain this invalid WKT). 
But METHOD["unnamed"] means that the projection method is unknown, so it 
is logical that GDAL/PROJ can't compute geographic coordinates from 
projected coordinates.



The output of "listgeo HiBar_21022022_A1.tif" might be useful to further 
diagnose.



You likely need to gdal_translate -a_srs EPSG:2039 to do something 
useful with that file, and report to the data producer that the CRS 
encoding is broken



Le 08/03/2023 à 13:18, Micha Silver a écrit :


Hi Even:


Here's the full gdalinfo, with the reported CRS:


gdalinfo HiBar_21022022_A1.tif
Driver: GTiff/GeoTIFF
Files: HiBar_21022022_A1.tif
  HiBar_21022022_A1.tif.aux.xml
Size is 25000, 25000
Coordinate System is:
PROJCRS["Israel 1993 / Israeli TM Grid",
   BASEGEOGCRS["WGS 84",
   DATUM["World Geodetic System 1984",
   ELLIPSOID["WGS 84",6378137,298.257223563,
   LENGTHUNIT["metre",1,
   ID["EPSG",9001,
   PRIMEM["Greenwich",0,
   ANGLEUNIT["degree",0.0174532925199433,
   ID["EPSG",9122,
   CONVERSION["unnamed",
   METHOD["unnamed"]],
   CS[Cartesian,2],
   AXIS["(E)",east,
   ORDER[1],
   LENGTHUNIT["metre",1,
   ID["EPSG",9001]]],
   AXIS["(N)",north,
   ORDER[2],
   LENGTHUNIT["metre",1,
   ID["EPSG",9001
Data axis to CRS axis mapping: 1,2
Origin = (202245.240786847687559,421915.720555976789910)
Pixel Size = (0.125,-0.125)
Metadata:
 AREA_OR_POINT=Area
Image Structure Metadata:
 INTERLEAVE=PIXEL
Corner Coordinates:
ERROR 1: No inverse operation
Upper Left  (  202245.241,  421915.721)
ERROR 1: No inverse operation
Lower Left  (  202245.241,  418790.721)
ERROR 1: No inverse operation
Upper Right (  205370.241,  421915.721)
ERROR 1: No inverse operation
Lower Right (  205370.241,  418790.721)
ERROR 1: No inverse operation
Center  (  203807.741,  420353.221)

and proj version:


proj
Rel. 9.1.1, December 1st, 2022


(a newly installed debian 12)


Thanks,

Micha


On 08/03/2023 13:47, Even Rouault wrote:


Micha,


which CRS is reported and which PROJ version do you use ? It is 
presumably one projection method for which PROJ only implements the 
forward conversion, that is from geographic to projected coordinates, 
but not the reverse way



Even


Le 08/03/2023 à 09:37, Micha Silver a écrit :


I received some Geotiff files, and gdalinfo shows:


Corner Coordinates:
ERROR 1: No inverse operation
Upper Left  (  205370.241,  418790.721)
ERROR 1: No inverse operation
Lower Left  (  205370.241,  415665.721)
ERROR 1: No inverse operation
Upper Right (  206198.866,  418790.721)
ERROR 1: No inverse operation
Lower Right (  206198.866,  415665.721)
ERROR 1: No inverse operation
Center  (  205784.553,  417228.221)

What does that mean?


gdalinfo --version
GDAL 3.6.2, released 2023/01/02

Thanks


--
Micha Silver
Remote Sensing Lab, Sde Boker
Ben Gurion University
+972-523-665918

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

--
http://www.spatialys.com
My software is free, but my time generally not.

--
Micha Silver
Remote Sensing Lab, Sde Boker
Ben Gurion University
+972-523-665918


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ERROR 1: No inverse operation

[Even Rouault]

Micha,


which CRS is reported and which PROJ version do you use ? It is 
presumably one projection method for which PROJ only implements the 
forward conversion, that is from geographic to projected coordinates, 
but not the reverse way



Even


Le 08/03/2023 à 09:37, Micha Silver a écrit :


I received some Geotiff files, and gdalinfo shows:


Corner Coordinates:
ERROR 1: No inverse operation
Upper Left  (  205370.241,  418790.721)
ERROR 1: No inverse operation
Lower Left  (  205370.241,  415665.721)
ERROR 1: No inverse operation
Upper Right (  206198.866,  418790.721)
ERROR 1: No inverse operation
Lower Right (  206198.866,  415665.721)
ERROR 1: No inverse operation
Center  (  205784.553,  417228.221)

What does that mean?


gdalinfo --version
GDAL 3.6.2, released 2023/01/02

Thanks


--
Micha Silver
Remote Sensing Lab, Sde Boker
Ben Gurion University
+972-523-665918

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] question about python installation

[Even Rouault]

Michael,

My experience is that understanding how and where to install Python 
stuff is utterly difficult. And this is even more complicated on 
Debian/Ubuntu that have many patches regarding sys.path and distutils, 
plus the fact that distutils is deprecated and going to be removed in 
Python 3.12, without a clear replacement. Just look at all the horrible 
logic in swig/python/CMakeLists.txt from line 428 to 504 + 
swig/python/trimmedsysconfig.py + swig/python/install_python.cmake.in.  
Perhaps there's a better way...


GDAL hopefully does the right thing for a typical distribution 
installation where CMAKE_INSTALL_PREFIX=/usr . For other prefixes, such 
as the implicit /usr/local prefix, you're on your own, and you will 
likely have to set the PYTHONPATH environment variable to point to 
/usr/local/lib/python3/dist-packages


GDAL_PYTHON_INSTALL_PREFIX will not help you here (and I'm not sure to 
remember what the actual use case for it was)


Even

Le 08/03/2023 à 02:19, Michael Sumner a écrit :
Hello, apologies as this is not so much a GDAL but a python question, 
but I'm interested in what others do, hopefully I'm missing a key step 
that's not hacky :)


I made a gist to record the details:

https://gist.github.com/mdsumner/526af876cfddaa5ff245ab376b3cec84

The crux is,  GDAL has placed the python osgeo lib files into 
'/usr/local/lib/python3/dist-packages', but python is only looking in 
'/usr/local/lib/python3.10/dist-packages' (as seen in sys.path).


Is there a config step with cmake, or a subsequent step with python 
config that I should be doing?


Is it GDAL_PYTHON_INSTALL_PREFIX ?

Thank you.


--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.6.3 RC1 available

[Even Rouault]

Hi,

I have prepared a GDAL/OGR 3.6.3 release candidate.

Pick up an archive among the following ones (by ascending size):

  https://download.osgeo.org/gdal/3.6.3/gdal-3.6.3rc1.tar.xz
  https://download.osgeo.org/gdal/3.6.3/gdal-3.6.3rc1.tar.gz
  https://download.osgeo.org/gdal/3.6.3/gdal363rc1.zip

A snapshot of the gdalautotest suite is also available :

https://download.osgeo.org/gdal/3.6.3/gdalautotest-3.6.3rc1.tar.gz
  https://download.osgeo.org/gdal/3.6.3/gdalautotest-3.6.3rc1.zip

The NEWS file is here :

  https://github.com/OSGeo/gdal/blob/v3.6.3RC1/NEWS.md

I'll call for a vote promoting it to final before end of week if no
serious problems are reported before.

Best regards,

Even

--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] getting SRS units from GetAttrValue("UNIT", 0)?

[Even Rouault]

Le 07/03/2023 à 17:08, Edzer Pebesma a écrit :

Thanks, Even!

I was after the units of the horizontal coordinates; GetLinearUnits() 
returns "unknown" for EPSG:4326 and, according to the docs returns the 
units of the vertical coordinate in 3D CRS (didn't check). So what I 
now use is GetAttrValue("UNIT", 0) (which returns "degree" for 
EPSG:4326"), and if that returns NULL, use GetLinearUnit(&unit) if its 
value differs from "unknown".


Try:

if( srs->IsGeographic() )
   srs->GetAngularUnits(&unit);
else

  srs->GetLinearUnits(&unit);


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] getting SRS units from GetAttrValue("UNIT", 0)?

[Even Rouault]

Edzer,

if you change your code to use GetLinearUnits() it will work:

#include 
#include 
#include 

int main()
{
    OGRSpatialReference *srs = new OGRSpatialReference;
    int epsg = 3031;
    srs->importFromEPSG(epsg);
    const char* unit = "";
    srs->GetLinearUnits(&unit);
    std::cout << epsg << ":" << unit << std::endl;
    const char *proj4s = "+proj=eqearth";
    srs->importFromProj4(proj4s);
    unit = "";
    srs->GetLinearUnits(&unit);
    std::cout << proj4s << ":" << unit << std::endl;
    return 0;
}


If you wonder why GetAttrValue("UNIT", 0) works sometimes and not not 
for others, this is a bit tricky. When querying WKT nodes directly like 
with GetAttrValue(), GDAL works internally with a WKT1 representation, 
for backward compatibility as a lot of code inside GDAL (and potentially 
outside) queries with WKT1 node names. The issue here is that eqearth is 
a new projection method that didn't exist in GDAL <= 2.4, so GDAL 
fallbacks to WKT2 representation, and it's not the UNIT keyword in WKT2 
but LENGTHUNIT.


Using GetLinearUnits() is more robust, as independent from the WKT1/WKT2 
representation


Even


Le 07/03/2023 à 11:38, Edzer Pebesma a écrit :
I switched getting coordinate units from the expanded proj4string 
representation to GetAttrValue("UNIT", 0), but failed; try


#include 
#include 
#include 

int main()
{
OGRSpatialReference *srs = new OGRSpatialReference;
int epsg = 3031;
srs->importFromEPSG(epsg);
std::cout << epsg << ":" << srs->GetAttrValue("UNIT",0) << std::endl;
const char *proj4s = "+proj=eqearth";
srs->importFromProj4(proj4s);
const char *u = srs->GetAttrValue("UNIT",0);
std::cout << proj4s << ":" << (u ? u : "NULL") << std::endl;
return 0;
}

which gives

3031:metre
+proj=eqearth:NULL

whereas the proj4string expansion of "+proj=eqearth" is "+proj=eqearth 
+lon_0=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs", so has the units.


What do I miss here? What is the recommended way to get coordinate units?


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] building against external tiff/geotiff sources ?

[Even Rouault]

Hi,



currently some interesting features (for example jxl compression in
tiff) requires one to build against internal tiff/geotiff - after
looking at the code, from my understanding this is because the gtiff
driver reaches to internal tiff functions not present in the public API,
which (to me) is a valid reason.  Afaict the bundled copies dont diverge
from upstream, since even maintains both sides in parallel.
The JXL codec is the only functionality that currently requires using 
the internal copy of libtiff, not because there's anything special in 
it, but because TIFF codecs require access to private headers of 
libtiff. The reason for the JXL codec being in GDAL only is that it 
makes it easier for early adopters to assess adequacy of this codec for 
GDAL purposes without requiring first to have it accepted in libtiff 
(there was a similar situation a few years ago with the LERC codec). The 
medium/long term plan is to migrate it to libtiff, presumably once 
libjxl has reached 1.0 and API stability.


Thus, two questions:
- in the long run, are there plans to allow building against external
   libtiff/libgeotiff libraries/packages for all features 'restricted' to
the internal copies of those (eg stop relying on internal funcs, or put
those in the public API), or that'd be too much a maintainance hell ?
Cf above. Relying on internal libtiff funcs for the JXL codec is by 
design/compulsory.


- would it be possible to allow building against external
   libtiff/libgeotiff _sources_ ? this way i'd still be able to apply
fixes to one place, but i also suppose that'll require a tight version
dependency on those..


Everything is possible, but I don't think we want to complicate the 
build system with that extra possibility.


So yes packages that follow rules of not using vendored libraries can't 
use the JXL codec for now. Hopefully that's just a transient state.


Even

--

http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: adopt RFC 93: OGRLayer::UpdateFeature() method

[Even Rouault]

Hi

I declare this motion passed with +1 from PSC members Jukka, MateuszL 
and me.


Even

Le 22/02/2023 à 10:19, Even Rouault a écrit :

Hi,

Motion: adopt RFC 93: OGRLayer::UpdateFeature() method

https://github.com/OSGeo/gdal/pull/7249

Starting with my +1

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev