Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows failed at linking

[Even Rouault via gdal-dev]

Le 14/03/2024 à 19:27, Fengting Chen a écrit :


I included this fix but it doesn’t help. I was building the georaster 
driver as a plugin but the linker failed at building gdal library. So 
the fix doesn’t seem to be related to the problem.


You can't build the driver as a plugin with internal libjpeg. CMake 
should prevent you from doing that because of


https://github.com/OSGeo/gdal/blob/master/frmts/georaster/CMakeLists.txt#L6

But I now remember (thanks git blame!) that during the defered driver 
loading RFC development, I found that the condition to check that was 
broken. This was fixed in 
https://github.com/OSGeo/gdal/commit/adb0fd695253cf9fb91bee46309e6c66b25e8d65



Building Custom Rule C:/fechen/gdal-3.8.4/CMakeLists.txt

vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_src(struct 
jpeg_decompress_struct *,struct VSIVirtualHandle *)"


(?jpeg_vsiio_src@@YAXPEAUjpeg_decompress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]


vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_dest(struct 
jpeg_compress_struct *,struct VSIVirtualHandle *)" 
(?jpeg_vsiio_dest@@YAXPEAUjpeg_compress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]


Creating library C:/fechen/gdal-3.8.4/build/Debug/gdald.lib and object 
C:/fechen/gdal-3.8.4/build/Debug/gdald.exp


C:\fechen\gdal-3.8.4\build\Debug\gdald.dll : fatal error LNK1169: one 
or more multiply defined symbols found 
[C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]


*From: *Even Rouault 
*Date: *Thursday, March 14, 2024 at 11:52 AM
*To: *Fengting Chen , 
gdal-dev@lists.osgeo.org 
*Subject: *Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows 
failed at linking


Hi,

possibly already fixed in master with commit 
https://github.com/OSGeo/gdal/commit/849226ce1a9 



Even

Le 14/03/2024 à 16:47, Fengting Chen a écrit :

After upgrading the windows build environment to VS2022, I got the
following linker error:

vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_src(struct
jpeg_decompress_struct *,struct VSIVirtualHandle *)"
(?jpeg_vsiio_src@@YAXPEAUjpeg_decompress_struct@@PEAUVSIVirtualHandle@@@Z)
already defined in vsidataio.obj
[C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]

vsidataio.obj : error LNK2005: "void __cdecl
jpeg_vsiio_dest(struct jpeg_compress_struct *,struct
VSIVirtualHandle *)"
(?jpeg_vsiio_dest@@YAXPEAUjpeg_compress_struct@@PEAUVSIVirtualHandle@@@Z)
already defined in vsidataio.obj
[C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]

Creating library C:/fechen/gdal-3.8.4/build/Debug/gdald.lib and
object C:/fechen/gdal-3.8.4/build/Debug/gdald.exp

C:\fechen\gdal-3.8.4\build\Debug\gdald.dll : fatal error LNK1169:
one or more multiply defined symbols found
[C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]

I set “GDAL_USE_JPEG_INTERNAL”  with “ON”. Any suggestions?

Thanks!

*From: *Even Rouault 

*Date: *Tuesday, March 12, 2024 at 12:56 PM
*To: *Fengting Chen 
, gdal-dev@lists.osgeo.org
 
*Subject: *Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on
windows failed at linking

Not the answer you'll probably want to hear, but:

- Use an up-to-date compiler

- Use package distributions like vcpkg or conda-forge that already
brings all the dependencies "for free"

- It seems you mix builds of different compilers (VS vs mingw).
That's a bit asking for troubles ;-)

Le 12/03/2024 à 17:48, Fengting Chen via gdal-dev a écrit :

Resend with more information:

I am trying to build GDAL 3.8.4 on windows under VS2015. The
build failed at linking stage that a bunch of external symbols
not found, specifically the symbols in libproj, libcurl. I
checked the dll library stub .lib for libproj and libcurl
using “dumpbin” and found the symbols in the library a little
different, for example the linker error shows
“__imp__curl_slist_append” not found, the symbol output from
the “dumpbin” is  “__imp_curl_slist_append” .

Any idea how to fix this?

Thanks!

*From: *gdal-dev 
 on behalf of
Fengting Chen via gdal-dev 

*Date: *Monday, March 11, 2024 at 4:50 PM
*To: *gdal-dev@lists.osgeo.org 

*Subject: *[External] : [gdal-dev] GDAL 3.8.4 build on windows
failed at linking

GDAL 3.8.4 build on Windows from VS2015 x64 command prompt:
“cmake –build .” 

Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows failed at linking

[Fengting Chen via gdal-dev]

I included this fix but it doesn’t help. I was building the georaster driver as 
a plugin but the linker failed at building gdal library. So the fix doesn’t 
seem to be related to the problem.

Building Custom Rule C:/fechen/gdal-3.8.4/CMakeLists.txt
vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_src(struct 
jpeg_decompress_struct *,struct VSIVirtualHandle *)"
(?jpeg_vsiio_src@@YAXPEAUjpeg_decompress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]
vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_dest(struct 
jpeg_compress_struct *,struct VSIVirtualHandle *)" 
(?jpeg_vsiio_dest@@YAXPEAUjpeg_compress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]
 Creating library C:/fechen/gdal-3.8.4/build/Debug/gdald.lib and object 
C:/fechen/gdal-3.8.4/build/Debug/gdald.exp
C:\fechen\gdal-3.8.4\build\Debug\gdald.dll : fatal error LNK1169: one or more 
multiply defined symbols found [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]

From: Even Rouault 
Date: Thursday, March 14, 2024 at 11:52 AM
To: Fengting Chen , gdal-dev@lists.osgeo.org 

Subject: Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows failed at 
linking

Hi,

possibly already fixed in master with commit 
https://github.com/OSGeo/gdal/commit/849226ce1a9

Even
Le 14/03/2024 à 16:47, Fengting Chen a écrit :
After upgrading the windows build environment to VS2022, I got the following 
linker error:

vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_src(struct 
jpeg_decompress_struct *,struct VSIVirtualHandle *)" 
(?jpeg_vsiio_src@@YAXPEAUjpeg_decompress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]
vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_dest(struct 
jpeg_compress_struct *,struct VSIVirtualHandle *)" 
(?jpeg_vsiio_dest@@YAXPEAUjpeg_compress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]
 Creating library C:/fechen/gdal-3.8.4/build/Debug/gdald.lib and object 
C:/fechen/gdal-3.8.4/build/Debug/gdald.exp
C:\fechen\gdal-3.8.4\build\Debug\gdald.dll : fatal error LNK1169: one or more 
multiply defined symbols found [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]

I set “GDAL_USE_JPEG_INTERNAL”  with “ON”. Any suggestions?

Thanks!

From: Even Rouault 

Date: Tuesday, March 12, 2024 at 12:56 PM
To: Fengting Chen , 
gdal-dev@lists.osgeo.org 

Subject: Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows failed at 
linking

Not the answer you'll probably want to hear, but:

- Use an up-to-date compiler

- Use package distributions like vcpkg or conda-forge that already brings all 
the dependencies "for free"

- It seems you mix builds of different compilers (VS vs mingw). That's a bit 
asking for troubles ;-)
Le 12/03/2024 à 17:48, Fengting Chen via gdal-dev a écrit :
Resend with more information:

I am trying to build GDAL 3.8.4 on windows under VS2015. The build failed at 
linking stage that a bunch of external symbols not found, specifically the 
symbols in libproj, libcurl. I checked the dll library stub .lib for libproj 
and libcurl using “dumpbin” and found the symbols in the library a little 
different, for example the linker error shows “__imp__curl_slist_append” not 
found, the symbol output from the “dumpbin” is  “__imp_curl_slist_append” .

Any idea how to fix this?

Thanks!

From: gdal-dev 
 on 
behalf of Fengting Chen via gdal-dev 

Date: Monday, March 11, 2024 at 4:50 PM
To: gdal-dev@lists.osgeo.org 

Subject: [External] : [gdal-dev] GDAL 3.8.4 build on windows failed at linking
GDAL 3.8.4 build on Windows from VS2015 x64 command prompt: “cmake –build .” 
throws the following error (there are more similar errors):

cpl_vsil_az.obj : error LNK2001: unresolved external symbol 
__imp__curl_slist_append [E:\gdal-3.8.4\build\GDAL.vcxproj]

cmake options has:

set (GDAL_USE_CURLON   CACHE  BOOL "" FORCE)
set (CURL_INCLUDE_DIR "E:/SDK/curl-8.6.0_1-win64-mingw/include" 
CACHE PATH "" FORCE)
set (CURL_LIBRARY_RELEASE 
"E:/SDK/curl-8.6.0_1-win64-mingw/bin/libcurl-x64.lib" CACHE FILEPATH "" FORCE)

libcurl-x64.lib is the DLL stub. Using dumpbin on libcurl-x64.lib shows that 
“__imp_curl_slist_append” is available. But the linker error is 
“__imp__curl_slist_append” not found. There is an extra “_” in it.

Any idea why this error occurred?

Thanks!

Re: [gdal-dev] Hanging (or very slow) when ogr2ogr appending to PostGIS table

[Even Rouault via gdal-dev]

(re-adding list)

Le 14/03/2024 à 18:58, Robin Wilson a écrit :

Hi Even,

Perfect - adding —config PG_USE_COPY YES solved it. I’ll write a blog 
post about this that hopefully will be found by someone with the same 
issue in future.
You may also suggest documentation enhancement to 
https://gdal.org/drivers/vector/pg.html . There's the convenient "Edit 
on GitHub" button at the top right


For context, is there a particular reason why this is only enabled by 
default when you’re creating a new table?


Yes, this is related to unique identifiers. At table creation, one can 
make the reasonable expectation that no other process will concurrently 
update the table since it didn't exist before. When updating/appending, 
that hypothesis is less likely, hence it is safer to fallback to the 
slower INSERT mode. I presume in the Azure context that you have a 
client->server->client round-trip at each feature insertion.




For what it’s worth, I was running this on GDAL 3.8.3 - I checked it 
was recent, but forgot to include it in my first email.


Thanks,

Robin



--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Hanging (or very slow) when ogr2ogr appending to PostGIS table

[Even Rouault via gdal-dev]

Robin,

- output of "gdalinfo --version" ?

- try adding ``--config PG_USE_COPY YES`` to  the command line where you 
append to the existing table. Cf 
https://gdal.org/drivers/vector/pg.html#configuration-options (copy mode 
is enabled by default at table creation, but not in append scenarios)


- if the above didn't help, try also setting ``--config 
OGR2OGR_USE_ARROW_API NO`` (please report if that makes a difference)


Even

Le 14/03/2024 à 18:28, Robin Wilson via gdal-dev a écrit :

Hi,

I’ve been using ogr2ogr to import vector datasets into PostGIS tables.

I originally tried this using a local Postgres server, and ran the 
following commands to import one file into a PostGIS table, and then 
append the contents of another file to the same table:


ogr2ogr --debug ON -f PostgreSQL PG:”host=localhost user=postgres 
password=blah dbname=test_db" buildings1.gpkg -nln ogr_test
ogr2ogr -append -update --debug ON -f PostgreSQL PG:”host=localhost 
user=postgres password=blah dbname=test_db" buildings2.gpkg -nln ogr_test


This works fine, and both commands run quickly.

I then tried the same thing, but with a PostGIS server hosted on 
Azure. Obviously I expect things to take longer when accessing across 
the internet, but this time the first command completed quickly, but 
the second command seems to hang. Looking at the debug output, it shows:


GPKG: GeoPackage v1.2.0
GDAL: GDALOpen(buildings2.gpkg, this=0x131011800) succeeds as GPKG.
PG: Client encoding: 'UTF8'
PG: PostGIS schema: 'public'
PG: PostgreSQL version string : 'PostgreSQL 15.4 on 
x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 
7.5.0, 64-bit'

PG: PostGIS version string : '3.3 USE_GEOS=1 USE_PROJ=1 USE_STATS=1'
GDAL: GDALOpen(PG:, this=0x140f25850) succeeds as 
PostgreSQL.

PG: Primary key name (FID): fid, type : int4
PG: Using column 'fid' as FID for table 'ogr_test'
OGR2OGR: Using WriteArrowBatch()

and then nothing else.

I’m intrigued as to why it seems to hang, and what it is doing, or 
trying to do. I’ve tried adding buildings2.gpkg to a new table on the 
Azure PostGIS server and it completes very quickly, so it’s not just 
that buildings2 is larger and takes a long time to upload. Similarly, 
merging the two buildings files with ogrmerge.py and then running the 
ogr2ogr command to import to PostGIS also works, and runs quickly.


I’m also intrigued as to why this only seems to be happening with the 
Azure server - is there some configuration option I need to set? I’ve 
tried connecting as the ‘postgres’ root user, so it shouldn’t be a 
permissions issue.


Any ideas or suggestions of things to try?

Any help gratefully received,

Best regards,

Robin

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Hanging (or very slow) when ogr2ogr appending to PostGIS table

[Robin Wilson via gdal-dev]

Hi,

I’ve been using ogr2ogr to import vector datasets into PostGIS tables.

I originally tried this using a local Postgres server, and ran the following 
commands to import one file into a PostGIS table, and then append the contents 
of another file to the same table:

ogr2ogr --debug ON -f PostgreSQL PG:”host=localhost user=postgres password=blah 
dbname=test_db" buildings1.gpkg -nln ogr_test
ogr2ogr -append -update --debug ON -f PostgreSQL PG:”host=localhost 
user=postgres password=blah dbname=test_db" buildings2.gpkg -nln ogr_test

This works fine, and both commands run quickly.

I then tried the same thing, but with a PostGIS server hosted on Azure. 
Obviously I expect things to take longer when accessing across the internet, 
but this time the first command completed quickly, but the second command seems 
to hang. Looking at the debug output, it shows:

GPKG: GeoPackage v1.2.0
GDAL: GDALOpen(buildings2.gpkg, this=0x131011800) succeeds as GPKG.
PG: Client encoding: 'UTF8'
PG: PostGIS schema: 'public'
PG: PostgreSQL version string : 'PostgreSQL 15.4 on x86_64-pc-linux-gnu, 
compiled by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit'
PG: PostGIS version string : '3.3 USE_GEOS=1 USE_PROJ=1 USE_STATS=1'
GDAL: GDALOpen(PG:, this=0x140f25850) succeeds as PostgreSQL.
PG: Primary key name (FID): fid, type : int4
PG: Using column 'fid' as FID for table 'ogr_test'
OGR2OGR: Using WriteArrowBatch()

and then nothing else.

I’m intrigued as to why it seems to hang, and what it is doing, or trying to 
do. I’ve tried adding buildings2.gpkg to a new table on the Azure PostGIS 
server and it completes very quickly, so it’s not just that buildings2 is 
larger and takes a long time to upload. Similarly, merging the two buildings 
files with ogrmerge.py and then running the ogr2ogr command to import to 
PostGIS also works, and runs quickly.

I’m also intrigued as to why this only seems to be happening with the Azure 
server - is there some configuration option I need to set? I’ve tried 
connecting as the ‘postgres’ root user, so it shouldn’t be a permissions issue.

Any ideas or suggestions of things to try?

Any help gratefully received,

Best regards,

Robin___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] There's something strange about two tests that are very similar

[Abel Pau via gdal-dev]

Thanks Even,
your feedback helped a lot.

De: Even Rouault 
Enviado el: dijous, 14 de març de 2024 10:13
Para: Abel Pau ; gdal-dev@lists.osgeo.org
Asunto: Re: [gdal-dev] There's something strange about two tests that are very 
similar


gdb is not going to help for test_ogr_basic_10. As you can see in the test, it 
"runexternal", that is it forks a child process, so gdb isn't going to debug 
this child process by default.

The easiest is to run "test_ogrsf -all_drivers" directly / under gdb / under 
valgrind

Actually, I've just tried that on my system. Nothing under Valgrind, but the 
execution time was quite slow once it reaches the Miramon driver, and breaking 
under gdb, I see the following

INFO: MiraMonVector: Creation of /foo/test.pol should have failed.
INFO: MiraMonVector: CreateFeature() at line 780 succeeded but failed to assign 
FID to feature.
^C--Type  for more, q to quit, c to continue without paging--

Thread 1 "test_ogrsf" received signal SIGINT, Interrupt.
__memset_avx2_erms () at 
../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:151
151../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: Aucun fichier 
ou dossier de ce type.
(gdb) bt
#0  __memset_avx2_erms () at 
../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:151
#1  0x75e40ed5 in CPLCalloc (nCount=1, nSize=104857600) at 
/home/even/gdal/MiraMon-Vector-driver/port/cpl_conv.cpp:164
#2  0x76f39a33 in MMMoveFromFileToFile (pSrcFile=0x55bcd870, 
pDestFile=0x562b3b90, nOffset=0x561f3990)
at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2693
#3  0x76f388fb in MMCloseArcLayer (hMiraMonLayer=0x561e4f90) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2150
#4  0x76f38dc5 in MMCloseLayer (hMiraMonLayer=0x561e4f90) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2283
#5  0x76f2d955 in OGRMiraMonLayer::~OGRMiraMonLayer 
(this=0x561d5aa0, __in_chrg=) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:535
#6  0x76f2de22 in OGRMiraMonLayer::~OGRMiraMonLayer 
(this=0x561d5aa0, __in_chrg=) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:619
#7  0x76f2af7a in OGRMiraMonDataSource::~OGRMiraMonDataSource 
(this=0x56265cb0, __in_chrg=)
at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp:50
#8  0x76f2b014 in OGRMiraMonDataSource::~OGRMiraMonDataSource 
(this=0x56265cb0, __in_chrg=)
at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp:57
#9  0x773d61a3 in GDALClose (hDS=0x56265cb0) at 
/home/even/gdal/MiraMon-Vector-driver/gcore/gdaldataset.cpp:4128
#10 0xe167 in TestCreateLayer (poDriver=0x5564e3f0, 
eGeomType=wkbMultiPoint) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:977
#11 0xec61 in TestCreate (poDriver=0x5564e3f0, 
bFromAllDrivers=1) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:1094
#12 0xbd56 in ThreadFunctionInternal (psContext=0x7fffdb60) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:325
#13 0xbc26 in ThreadFunction (user_data=0x7fffdb60) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:283
#14 0xba62 in main (nArgc=2, papszArgv=0x556521b0) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:233

Actually looking at your MMMoveFromFileToFile(), I see it is allocating a 
working buffer of 100 MB. That's much larger than what you really need. A 
buffer of 64 KB or so would be equally performant. This function might also 
leak the buffer at line 2705.

"INFO: MiraMonVector: Creation of /foo/test.pol should have failed." also 
indicates some logic error in your driver.

and

$ gdb --args ogr2ogr /foo/test.pol autotest/ogr/data/poly.shp

crashes at

ERROR 4: Error pMMPolygonLayer->pF: Cannot open file /foo/test.pol.

Program received signal SIGSEGV, Segmentation fault.
MMAddArcRecordToMMDB (hMiraMonLayer=0x7fffebbe48f0, hMMFeature=0x7fffebc030d0, 
nElemCount=0, pArcHeader=0x55af3940) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:6602
6602 pMMArcLayer->MMAdmDB.pMMBDXP->nRecords 
+ 1))
(gdb) bt
#0  MMAddArcRecordToMMDB (hMiraMonLayer=0x7fffebbe48f0, 
hMMFeature=0x7fffebc030d0, nElemCount=0, pArcHeader=0x55af3940)
at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:6602
#1  0x76f3ccd6 in MMCreateFeaturePolOrArc 
(hMiraMonLayer=0x7fffebbe48f0, hMMFeature=0x7fffebc030d0) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:4112
#2  0x76f3de4e in MMAddFeature (hMiraMonLayer=0x7fffebbe48f0, 
hMiraMonFeature=0x7fffebc030d0) at 

Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows failed at linking

[Even Rouault via gdal-dev]

Hi,

possibly already fixed in master with commit 
https://github.com/OSGeo/gdal/commit/849226ce1a9


Even

Le 14/03/2024 à 16:47, Fengting Chen a écrit :


After upgrading the windows build environment to VS2022, I got the 
following linker error:


vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_src(struct 
jpeg_decompress_struct *,struct VSIVirtualHandle *)" 
(?jpeg_vsiio_src@@YAXPEAUjpeg_decompress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]


vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_dest(struct 
jpeg_compress_struct *,struct VSIVirtualHandle *)" 
(?jpeg_vsiio_dest@@YAXPEAUjpeg_compress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]


Creating library C:/fechen/gdal-3.8.4/build/Debug/gdald.lib and object 
C:/fechen/gdal-3.8.4/build/Debug/gdald.exp


C:\fechen\gdal-3.8.4\build\Debug\gdald.dll : fatal error LNK1169: one 
or more multiply defined symbols found 
[C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]


I set “GDAL_USE_JPEG_INTERNAL”  with “ON”. Any suggestions?

Thanks!

*From: *Even Rouault 
*Date: *Tuesday, March 12, 2024 at 12:56 PM
*To: *Fengting Chen , 
gdal-dev@lists.osgeo.org 
*Subject: *Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows 
failed at linking


Not the answer you'll probably want to hear, but:

- Use an up-to-date compiler

- Use package distributions like vcpkg or conda-forge that already 
brings all the dependencies "for free"


- It seems you mix builds of different compilers (VS vs mingw). That's 
a bit asking for troubles ;-)


Le 12/03/2024 à 17:48, Fengting Chen via gdal-dev a écrit :

Resend with more information:

I am trying to build GDAL 3.8.4 on windows under VS2015. The build
failed at linking stage that a bunch of external symbols not
found, specifically the symbols in libproj, libcurl. I checked the
dll library stub .lib for libproj and libcurl using “dumpbin” and
found the symbols in the library a little different, for example
the linker error shows “__imp__curl_slist_append” not found, the
symbol output from the “dumpbin” is  “__imp_curl_slist_append” .

Any idea how to fix this?

Thanks!

*From: *gdal-dev 
 on behalf of Fengting
Chen via gdal-dev 

*Date: *Monday, March 11, 2024 at 4:50 PM
*To: *gdal-dev@lists.osgeo.org 

*Subject: *[External] : [gdal-dev] GDAL 3.8.4 build on windows
failed at linking

GDAL 3.8.4 build on Windows from VS2015 x64 command prompt: “cmake
–build .” throws the following error (there are more similar errors):

cpl_vsil_az.obj : error LNK2001: unresolved external symbol
__imp__curl_slist_append [E:\gdal-3.8.4\build\GDAL.vcxproj]

cmake options has:

set (GDAL_USE_CURL    ON   CACHE  BOOL "" FORCE)

set (CURL_INCLUDE_DIR "E:/SDK/curl-8.6.0_1-win64-mingw/include"
CACHE PATH "" FORCE)

set (CURL_LIBRARY_RELEASE
"E:/SDK/curl-8.6.0_1-win64-mingw/bin/libcurl-x64.lib" CACHE
FILEPATH "" FORCE)

libcurl-x64.lib is the DLL stub. Using dumpbin on libcurl-x64.lib
shows that “__imp_curl_slist_append” is available. But the linker
error is “__imp__curl_slist_append” not found. There is an extra
“_” in it.

Any idea why this error occurred?

Thanks!



___

gdal-dev mailing list

gdal-dev@lists.osgeo.org

https://lists.osgeo.org/mailman/listinfo/gdal-dev  


--
http://www.spatialys.com  

My software is free, but my time generally not.


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows failed at linking

[Fengting Chen via gdal-dev]

After upgrading the windows build environment to VS2022, I got the following 
linker error:

vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_src(struct 
jpeg_decompress_struct *,struct VSIVirtualHandle *)" 
(?jpeg_vsiio_src@@YAXPEAUjpeg_decompress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]
vsidataio.obj : error LNK2005: "void __cdecl jpeg_vsiio_dest(struct 
jpeg_compress_struct *,struct VSIVirtualHandle *)" 
(?jpeg_vsiio_dest@@YAXPEAUjpeg_compress_struct@@PEAUVSIVirtualHandle@@@Z) 
already defined in vsidataio.obj [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]
 Creating library C:/fechen/gdal-3.8.4/build/Debug/gdald.lib and object 
C:/fechen/gdal-3.8.4/build/Debug/gdald.exp
C:\fechen\gdal-3.8.4\build\Debug\gdald.dll : fatal error LNK1169: one or more 
multiply defined symbols found [C:\fechen\gdal-3.8.4\build\GDAL.vcxproj]

I set “GDAL_USE_JPEG_INTERNAL”  with “ON”. Any suggestions?

Thanks!

From: Even Rouault 
Date: Tuesday, March 12, 2024 at 12:56 PM
To: Fengting Chen , gdal-dev@lists.osgeo.org 

Subject: Re: [gdal-dev] FW: [External] : GDAL 3.8.4 build on windows failed at 
linking

Not the answer you'll probably want to hear, but:

- Use an up-to-date compiler

- Use package distributions like vcpkg or conda-forge that already brings all 
the dependencies "for free"

- It seems you mix builds of different compilers (VS vs mingw). That's a bit 
asking for troubles ;-)
Le 12/03/2024 à 17:48, Fengting Chen via gdal-dev a écrit :
Resend with more information:

I am trying to build GDAL 3.8.4 on windows under VS2015. The build failed at 
linking stage that a bunch of external symbols not found, specifically the 
symbols in libproj, libcurl. I checked the dll library stub .lib for libproj 
and libcurl using “dumpbin” and found the symbols in the library a little 
different, for example the linker error shows “__imp__curl_slist_append” not 
found, the symbol output from the “dumpbin” is  “__imp_curl_slist_append” .

Any idea how to fix this?

Thanks!

From: gdal-dev 
 on 
behalf of Fengting Chen via gdal-dev 

Date: Monday, March 11, 2024 at 4:50 PM
To: gdal-dev@lists.osgeo.org 

Subject: [External] : [gdal-dev] GDAL 3.8.4 build on windows failed at linking
GDAL 3.8.4 build on Windows from VS2015 x64 command prompt: “cmake –build .” 
throws the following error (there are more similar errors):

cpl_vsil_az.obj : error LNK2001: unresolved external symbol 
__imp__curl_slist_append [E:\gdal-3.8.4\build\GDAL.vcxproj]

cmake options has:

set (GDAL_USE_CURLON   CACHE  BOOL "" FORCE)
set (CURL_INCLUDE_DIR "E:/SDK/curl-8.6.0_1-win64-mingw/include" 
CACHE PATH "" FORCE)
set (CURL_LIBRARY_RELEASE 
"E:/SDK/curl-8.6.0_1-win64-mingw/bin/libcurl-x64.lib" CACHE FILEPATH "" FORCE)

libcurl-x64.lib is the DLL stub. Using dumpbin on libcurl-x64.lib shows that 
“__imp_curl_slist_append” is available. But the linker error is 
“__imp__curl_slist_append” not found. There is an extra “_” in it.

Any idea why this error occurred?

Thanks!





___

gdal-dev mailing list

gdal-dev@lists.osgeo.org

https://lists.osgeo.org/mailman/listinfo/gdal-dev

--

http://www.spatialys.com

My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] There's something strange about two tests that are very similar

[Even Rouault via gdal-dev]

gdb is not going to help for test_ogr_basic_10. As you can see in the 
test, it "runexternal", that is it forks a child process, so gdb isn't 
going to debug this child process by default.


The easiest is to run "test_ogrsf -all_drivers" directly / under gdb / 
under valgrind


Actually, I've just tried that on my system. Nothing under Valgrind, but 
the execution time was quite slow once it reaches the Miramon driver, 
and breaking under gdb, I see the following


INFO: MiraMonVector: Creation of /foo/test.pol should have failed.
INFO: MiraMonVector: CreateFeature() at line 780 succeeded but failed to 
assign FID to feature.

^C--Type  for more, q to quit, c to continue without paging--

Thread 1 "test_ogrsf" received signal SIGINT, Interrupt.
__memset_avx2_erms () at 
../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:151
151    ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: Aucun 
fichier ou dossier de ce type.

(gdb) bt
#0  __memset_avx2_erms () at 
../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:151
#1  0x75e40ed5 in CPLCalloc (nCount=1, nSize=104857600) at 
/home/even/gdal/MiraMon-Vector-driver/port/cpl_conv.cpp:164
#2  0x76f39a33 in MMMoveFromFileToFile (pSrcFile=0x55bcd870, 
pDestFile=0x562b3b90, nOffset=0x561f3990)
    at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2693
#3  0x76f388fb in MMCloseArcLayer (hMiraMonLayer=0x561e4f90) 
at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2150
#4  0x76f38dc5 in MMCloseLayer (hMiraMonLayer=0x561e4f90) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2283
#5  0x76f2d955 in OGRMiraMonLayer::~OGRMiraMonLayer 
(this=0x561d5aa0, __in_chrg=) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:535
#6  0x76f2de22 in OGRMiraMonLayer::~OGRMiraMonLayer 
(this=0x561d5aa0, __in_chrg=) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:619
#7  0x76f2af7a in OGRMiraMonDataSource::~OGRMiraMonDataSource 
(this=0x56265cb0, __in_chrg=)
    at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp:50
#8  0x76f2b014 in OGRMiraMonDataSource::~OGRMiraMonDataSource 
(this=0x56265cb0, __in_chrg=)
    at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp:57
#9  0x773d61a3 in GDALClose (hDS=0x56265cb0) at 
/home/even/gdal/MiraMon-Vector-driver/gcore/gdaldataset.cpp:4128
#10 0xe167 in TestCreateLayer (poDriver=0x5564e3f0, 
eGeomType=wkbMultiPoint) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:977
#11 0xec61 in TestCreate (poDriver=0x5564e3f0, 
bFromAllDrivers=1) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:1094
#12 0xbd56 in ThreadFunctionInternal 
(psContext=0x7fffdb60) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:325
#13 0xbc26 in ThreadFunction (user_data=0x7fffdb60) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:283
#14 0xba62 in main (nArgc=2, papszArgv=0x556521b0) at 
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:233


Actually looking at your MMMoveFromFileToFile(), I see it is allocating 
a working buffer of 100 MB. That's much larger than what you really 
need. A buffer of 64 KB or so would be equally performant. This function 
might also leak the buffer at line 2705.


"INFO: MiraMonVector: Creation of /foo/test.pol should have failed." 
also indicates some logic error in your driver.


and

$ gdb --args ogr2ogr /foo/test.pol autotest/ogr/data/poly.shp

crashes at

ERROR 4: Error pMMPolygonLayer->pF: Cannot open file /foo/test.pol.

Program received signal SIGSEGV, Segmentation fault.
MMAddArcRecordToMMDB (hMiraMonLayer=0x7fffebbe48f0, 
hMMFeature=0x7fffebc030d0, nElemCount=0, pArcHeader=0x55af3940) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:6602

6602 pMMArcLayer->MMAdmDB.pMMBDXP->nRecords + 1))
(gdb) bt
#0  MMAddArcRecordToMMDB (hMiraMonLayer=0x7fffebbe48f0, 
hMMFeature=0x7fffebc030d0, nElemCount=0, pArcHeader=0x55af3940)
    at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:6602
#1  0x76f3ccd6 in MMCreateFeaturePolOrArc 
(hMiraMonLayer=0x7fffebbe48f0, hMMFeature=0x7fffebc030d0) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:4112
#2  0x76f3de4e in MMAddFeature (hMiraMonLayer=0x7fffebbe48f0, 
hMiraMonFeature=0x7fffebc030d0) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:4598
#3  0x76f31787 in OGRMiraMonLayer::MMWriteGeometry 
(this=0x7fffebbc6010) at 
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:1718
#4  0x76f30e01 in OGRMiraMonLayer::MMProcessGeometry 

Re: [gdal-dev] There's something strange about two tests that are very similar

[Abel Pau via gdal-dev]

Sorry,
I merged from my phone and I think I did it wrong.

At this time actions are going well except the same that didn’t work at the 
beginning of this message. The MacOS one.
Merge remote-tracking branch 'upstream/master' into MiraMon-Vector-dr… · 
AbelPau/gdal@fa6f9e0 
(github.com)

Your proposal of rebasing didn’t solved the problem in macOS.

So, I’ll try to get an image of a mac and compile, but when I tried to use a 
Virtual Box it was extremly slow.

Thanks for any input to try to get some call stack with the error described 
below.
Using gdb I obtain
ogr/ogr_basic_test.py ✓ 
100% ██

Results (7.72s):
   1 passed
[Inferior 1 (process 15792) exited normally]

And the same with Valgrind.
Thanks!

__ test_ogr_basic_10 ___
246442:
246542:
 def test_ogr_basic_10():
246642:
246742:
 import test_cli_utilities
246842:
246942:
 if test_cli_utilities.get_test_ogrsf_path() is None:
247042:
 pytest.skip()
247142:
247242:
 ret = gdaltest.runexternal(
247342:
 test_cli_utilities.get_test_ogrsf_path() + " -all_drivers"
247442:
 )
247542:
247642:
 > assert "INFO" in ret
247742:
 E AssertionError: assert 'INFO' in '\nERROR ret code = -4'
247842:
247942:
 /Users/runner/work/gdal/gdal/build/autotest/ogr/ogr_basic_test.py:454: 
AssertionError


De: gdal-dev  En nombre de Abel Pau via 
gdal-dev
Enviado el: dijous, 14 de març de 2024 7:44
Para: Even Rouault ; gdal-dev@lists.osgeo.org
Asunto: Re: [gdal-dev] There's something strange about two tests that are very 
similar

Hi,
I rebased yesterday, after your message, but same error appeared and some more.
I rebased al 7.00 again and it seems that there are again some errors in linux 
builds.

I assume it's because of lasts code updates. I'll wait until something stable 
to rebase again.

Enviat des de l'Outlook per a l'Android

From: Even Rouault 
mailto:even.roua...@spatialys.com>>
Sent: Thursday, March 14, 2024 12:15:38 AM
To: Abel Pau mailto:a@creaf.uab.cat>>; 
gdal-dev@lists.osgeo.org 
mailto:gdal-dev@lists.osgeo.org>>
Subject: Re: [gdal-dev] There's something strange about two tests that are very 
similar


try rebasing on top of latest master. It looks like the errors are only those 
fixed per 
https://github.com/OSGeo/gdal/commit/6703d3071de7155d320a39a580f27230428dcaca

--

http://www.spatialys.com

My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] There's something strange about two tests that are very similar

[Abel Pau via gdal-dev]

Hi,
I rebased yesterday, after your message, but same error appeared and some more.
I rebased al 7.00 again and it seems that there are again some errors in linux 
builds.

I assume it's because of lasts code updates. I'll wait until something stable 
to rebase again.

Enviat des de l'Outlook per a l'Android

From: Even Rouault 
Sent: Thursday, March 14, 2024 12:15:38 AM
To: Abel Pau ; gdal-dev@lists.osgeo.org 

Subject: Re: [gdal-dev] There's something strange about two tests that are very 
similar


try rebasing on top of latest master. It looks like the errors are only those 
fixed per 
https://github.com/OSGeo/gdal/commit/6703d3071de7155d320a39a580f27230428dcaca

--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev