Re: [gdal-dev] Testing the driver
Even, you are right! [cid:image002.png@01DA745F.C8585C00] So, the driver only creates a folder if it’s the last one of a path (with no extension) because this is the place where to write a set of files (a layer). So, thanks again! At this time, if I’m not wrong only there is one red light Merge remote-tracking branch 'upstream/master' into MiraMon-Vector-dr… · AbelPau/gdal@3f03aad (github.com)<https://github.com/AbelPau/gdal/actions/runs/8244678121> I a macOs... I’ll try to figure it out :) De: Even Rouault Enviado el: dilluns, 11 de març de 2024 17:56 Para: Abel Pau ; gdal-dev@lists.osgeo.org Asunto: Re: [gdal-dev] Testing the driver ok, I've looked a bit at the code So I believe the issue is that - https://github.com/OSGeo/gdal/blob/master/apps/test_ogrsf.cpp#L1064 forms a "/foo/test.pol" filename for the miramon driver and call Create() with it - and then https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L207 "osPath = CPLGetPath(pszRootName);" sets "/foo/" as osPath, which is then passed to VSIMkdir(), and causing the following mess I suspect the VSIMkdir() call should only be done in the branch https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L211 where pszRootName has an empty extension. Even Le 11/03/2024 à 17:30, Abel Pau a écrit : Hi, going back to the misterious failing test_tiff_write_133 I am not able to understand what is wrong with that. It seems that it would be failing when writing to an unauthorized path. But instead of that it is not failing (and this is the failure, not failing when it should fail). # Test writing into a non authorized file ds = gdaltest.tiff_drv.Create( "/foo/bar", 1024, 1000, 3, options=["STREAMABLE_OUTPUT=YES", "BLOCKYSIZE=1"] ) assert ds is None So, there is anything I am missing about that? The actions that informs that is re-runed in debug mode: Merge from base · AbelPau/gdal@3ec4655 (github.com)<https://github.com/AbelPau/gdal/actions/runs/8234751437/job/22520988179> FAILED gcore/tiff_write.py::test_tiff_write_133 - AssertionError: assert > is None It’s not None X-( Perhaps I, as Even said, I created a path before? But how to delete that? I don’t use anymore: VSIMkdirRecursive(). I use VSIMkdir() Perhaps if the destination folder doesn’t exist I should NOT create it and return a FAILURE? De: Abel Pau <mailto:a@creaf.uab.cat> Enviado el: dimecres, 6 de març de 2024 16:24 Para: Abel Pau <mailto:a@creaf.uab.cat>; Even Rouault <mailto:even.roua...@spatialys.com>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: RE: [gdal-dev] Testing the driver Hi, It seems nothing changes. I understand that the environment is new and the execution is not related with the last one. Here there are 5 tests that fail.. Any idea of what can be happening? They are very unrelated Bye VSIMkdirRecursive() · AbelPau/gdal@646b98b (github.com)<https://github.com/AbelPau/gdal/actions/runs/8172351502/job/22342474513> De: gdal-dev mailto:gdal-dev-boun...@lists.osgeo.org>> En nombre de Abel Pau via gdal-dev Enviado el: dimecres, 6 de març de 2024 13:52 Para: Even Rouault mailto:even.roua...@spatialys.com>>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: Re: [gdal-dev] Testing the driver Ok, I ‘ve changed that. Let’s see if it’s the problem. It’s all so delicate :) Thanks again! De: Even Rouault mailto:even.roua...@spatialys.com>> Enviado el: dimecres, 6 de març de 2024 13:36 Para: Abel Pau mailto:a@creaf.uab.cat>>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: Re: [gdal-dev] Testing the driver Le 06/03/2024 à 13:14, Abel Pau a écrit : Hi Even, I finally discovered the error. It was the fixture. In the wrong place. Now I’m creating the test. I hope finish it soon. On the other hand, in my actions tab: Merge branch 'OSGeo:master' into master · AbelPau/gdal@0249b6d (github.com)<https://github.com/AbelPau/gdal/actions/runs/8169099745/job/22332488002> There are some tiff failures, but nothing on my hand about tiff. == FAILURES === 36: _ test_tiff_write_133 _ 36: 36: def test_tiff_write_133(): Do you know what it can be? There are sometimes random failures, but here it fails on both the build-windows-msys2-mingw and build-windows-conda configs . I would suspect this might be a side effect of a previous run of the Miramon driver by another test with an invalid filename such as /foo/bar. Actually I see that test_ogrsf tries to create a /foo/test file. And https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L219 does a VSIMkdirRecursive(), so it must
Re: [gdal-dev] Testing the driver
ok, I've looked a bit at the code
So I believe the issue is that
- https://github.com/OSGeo/gdal/blob/master/apps/test_ogrsf.cpp#L1064
forms a "/foo/test.pol" filename for the miramon driver and call
Create() with it
- and then
https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L207
"osPath = CPLGetPath(pszRootName);" sets "/foo/" as osPath, which is
then passed to VSIMkdir(), and causing the following mess
I suspect the VSIMkdir() call should only be done in the branch
https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L211
where pszRootName has an empty extension.
Even
Le 11/03/2024 à 17:30, Abel Pau a écrit :
Hi,
going back to the misterious failing test_tiff_write_133 I am not able
to understand what is wrong with that.
It seems that it would be failing when writing to an unauthorized
path. But instead of that it is not failing (and this is the failure,
not failing when it should fail).
# Test writing into a non authorized file
ds = gdaltest.tiff_drv.Create(
"/foo/bar", 1024, 1000, 3, options=["STREAMABLE_OUTPUT=YES",
"BLOCKYSIZE=1"]
)
assert ds is None
So, there is anything I am missing about that?
The actions that informs that is re-runed in debug mode:
Merge from base · AbelPau/gdal@3ec4655 (github.com)
<https://github.com/AbelPau/gdal/actions/runs/8234751437/job/22520988179>
FAILED gcore/tiff_write.py::test_tiff_write_133 - AssertionError:
assert 'GDALDatasetShadow *' at 0x06FEDE30> > is None
It’s not None X-(
Perhaps I, as Even said, I created a path before? But how to delete
that? I don’t use anymore: VSIMkdirRecursive(). I use VSIMkdir()
*Perhaps if the destination folder doesn’t exist I should NOT create
it and return a FAILURE?*
*De:*Abel Pau
*Enviado el:* dimecres, 6 de març de 2024 16:24
*Para:* Abel Pau ; Even Rouault
; gdal-dev@lists.osgeo.org
*Asunto:* RE: [gdal-dev] Testing the driver
Hi,
It seems nothing changes. I understand that the environment is new and
the execution is not related with the last one.
Here there are 5 tests that fail..
Any idea of what can be happening?
They are very unrelated
Bye VSIMkdirRecursive() · AbelPau/gdal@646b98b (github.com)
<https://github.com/AbelPau/gdal/actions/runs/8172351502/job/22342474513>
*De:*gdal-dev *En nombre de *Abel
Pau via gdal-dev
*Enviado el:* dimecres, 6 de març de 2024 13:52
*Para:* Even Rouault ;
gdal-dev@lists.osgeo.org
*Asunto:* Re: [gdal-dev] Testing the driver
Ok, I ‘ve changed that. Let’s see if it’s the problem.
It’s all so delicate :)
Thanks again!
*De:*Even Rouault
*Enviado el:* dimecres, 6 de març de 2024 13:36
*Para:* Abel Pau ; gdal-dev@lists.osgeo.org
*Asunto:* Re: [gdal-dev] Testing the driver
Le 06/03/2024 à 13:14, Abel Pau a écrit :
Hi Even,
I finally discovered the error. It was the fixture. In the wrong
place.
Now I’m creating the test.
I hope finish it soon.
On the other hand,
in my actions tab: Merge branch 'OSGeo:master' into master ·
AbelPau/gdal@0249b6d (github.com)
<https://github.com/AbelPau/gdal/actions/runs/8169099745/job/22332488002>
There are some tiff failures, but nothing on my hand about tiff.
== FAILURES
===
36: _ test_tiff_write_133
_
36:
36: def test_tiff_write_133():
Do you know what it can be?
There are sometimes random failures, but here it fails on both the
build-windows-msys2-mingw and build-windows-conda configs . I would
suspect this might be a side effect of a previous run of the Miramon
driver by another test with an invalid filename such as /foo/bar.
Actually I see that test_ogrsf tries to create a /foo/test file.
And
https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L219
does a VSIMkdirRecursive(), so it must create a "/foo" directory. I
would recommend against using VSIMkdirRecursive() in a driver. You
might use VSIMkdir() to create the latest level of directory, but
creating the whole hiearchy is granting too much power to a driver.
Even
*De:*Even Rouault
<mailto:even.roua...@spatialys.com>
*Enviado el:* dimecres, 6 de març de 2024 13:09
*Para:* Abel Pau
<mailto:a@creaf.uab.cat>; gdal-dev@lists.osgeo.org
*Asunto:* Re: [gdal-dev] Testing the driver
Hi,
I don't see anything wrong. I've tried that on my native Linux
build and the test_ogr_miramon_vector_1() is found. Does "pytest
autotest/ogr/ogr_basic_test.py" work?*
Note: you don't need the try / except in your test case unless
you'd need to some particular cleanup, but that's not the case
here.
Re: [gdal-dev] Testing the driver
Hi, going back to the misterious failing test_tiff_write_133 I am not able to understand what is wrong with that. It seems that it would be failing when writing to an unauthorized path. But instead of that it is not failing (and this is the failure, not failing when it should fail). # Test writing into a non authorized file ds = gdaltest.tiff_drv.Create( "/foo/bar", 1024, 1000, 3, options=["STREAMABLE_OUTPUT=YES", "BLOCKYSIZE=1"] ) assert ds is None So, there is anything I am missing about that? The actions that informs that is re-runed in debug mode: Merge from base · AbelPau/gdal@3ec4655 (github.com)<https://github.com/AbelPau/gdal/actions/runs/8234751437/job/22520988179> FAILED gcore/tiff_write.py::test_tiff_write_133 - AssertionError: assert > is None It’s not None X-( Perhaps I, as Even said, I created a path before? But how to delete that? I don’t use anymore: VSIMkdirRecursive(). I use VSIMkdir() Perhaps if the destination folder doesn’t exist I should NOT create it and return a FAILURE? De: Abel Pau Enviado el: dimecres, 6 de març de 2024 16:24 Para: Abel Pau ; Even Rouault ; gdal-dev@lists.osgeo.org Asunto: RE: [gdal-dev] Testing the driver Hi, It seems nothing changes. I understand that the environment is new and the execution is not related with the last one. Here there are 5 tests that fail.. Any idea of what can be happening? They are very unrelated Bye VSIMkdirRecursive() · AbelPau/gdal@646b98b (github.com)<https://github.com/AbelPau/gdal/actions/runs/8172351502/job/22342474513> De: gdal-dev mailto:gdal-dev-boun...@lists.osgeo.org>> En nombre de Abel Pau via gdal-dev Enviado el: dimecres, 6 de març de 2024 13:52 Para: Even Rouault mailto:even.roua...@spatialys.com>>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: Re: [gdal-dev] Testing the driver Ok, I ‘ve changed that. Let’s see if it’s the problem. It’s all so delicate :) Thanks again! De: Even Rouault mailto:even.roua...@spatialys.com>> Enviado el: dimecres, 6 de març de 2024 13:36 Para: Abel Pau mailto:a@creaf.uab.cat>>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: Re: [gdal-dev] Testing the driver Le 06/03/2024 à 13:14, Abel Pau a écrit : Hi Even, I finally discovered the error. It was the fixture. In the wrong place. Now I’m creating the test. I hope finish it soon. On the other hand, in my actions tab: Merge branch 'OSGeo:master' into master · AbelPau/gdal@0249b6d (github.com)<https://github.com/AbelPau/gdal/actions/runs/8169099745/job/22332488002> There are some tiff failures, but nothing on my hand about tiff. == FAILURES === 36: _ test_tiff_write_133 _ 36: 36: def test_tiff_write_133(): Do you know what it can be? There are sometimes random failures, but here it fails on both the build-windows-msys2-mingw and build-windows-conda configs . I would suspect this might be a side effect of a previous run of the Miramon driver by another test with an invalid filename such as /foo/bar. Actually I see that test_ogrsf tries to create a /foo/test file. And https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L219 does a VSIMkdirRecursive(), so it must create a "/foo" directory. I would recommend against using VSIMkdirRecursive() in a driver. You might use VSIMkdir() to create the latest level of directory, but creating the whole hiearchy is granting too much power to a driver. Even De: Even Rouault <mailto:even.roua...@spatialys.com> Enviado el: dimecres, 6 de març de 2024 13:09 Para: Abel Pau <mailto:a@creaf.uab.cat>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: Re: [gdal-dev] Testing the driver Hi, I don't see anything wrong. I've tried that on my native Linux build and the test_ogr_miramon_vector_1() is found. Does "pytest autotest/ogr/ogr_basic_test.py" work?* Note: you don't need the try / except in your test case unless you'd need to some particular cleanup, but that's not the case here. pytest handles test failures nicely Even Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit : Hi again, after solving some issues I used WSL (Windows subsystem Linux) to create an environment where I am able to run tests. I run the cmake inside build folder in the environment. It’s slow but finally it finish. After cmake --build . --target install all is ready to be tested. I create a simple test ogr_miramon_vector.py (see the code below) to prove that it’s reliable. I run: pytest autotest/ogr/ogr_miramon_vector.py and: apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest autotest/ogr/ogr_miramon_vector.py Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, pytest-sugar
Re: [gdal-dev] Testing the driver
I hate that. I never use "long" exactly for that problem. When the size is important, uint64_t and friends are very useful, and well defined. On Fri, 8 Mar 2024 at 10:16, Abel Pau via gdal-dev wrote: > FINALLY! > > I discovered the problem. > > > > I was using “unsigned long” as 4 bytes variable. On windows it’s like that. > > > > BUT on linux this kind of variable has a 8-bytes size. > > > > This caused a mess in linux (but not in wondows). > > So, mistery closed! > > > > Thanks you all. > > > > *De:* gdal-dev *En nombre de *Abel Pau > via gdal-dev > *Enviado el:* dijous, 7 de març de 2024 23:42 > *Para:* Even Rouault ; Andrew C Aitchison < > and...@aitchison.me.uk>; Daniel Evans > *CC:* 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) < > gdal-dev@lists.osgeo.org> > *Asunto:* Re: [gdal-dev] Testing the driver > > > > Hi, > > It’s in debug mode, yes, as you suggested. > > > > yes, it crashes in the same way. Same big number. > > > > It crashes because of the variable is not read from the file (this > variable should be 1). > > But debugging in windows (where not crashes) this variable is set when > opening the layer, and then I suppose has the value corrupted at some point. > > So I am going to put my old fashion printf on. > > > > Thanks again! > > > > > > > > *De:* Even Rouault > *Enviado el:* dijous, 7 de març de 2024 23:32 > *Para:* Abel Pau ; Andrew C Aitchison < > and...@aitchison.me.uk>; Daniel Evans > *CC:* 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) < > gdal-dev@lists.osgeo.org> > *Asunto:* Re: [gdal-dev] Testing the driver > > > > > > At #10 we can see the variable nNum set to a non-sense megabignumber. > > Is it on a -DCMAKE_BUILD_TYPE=Debug build ? Otherwise stack traces and > variable content in the debugger might look like garbage because of > optimizations. > > If it is a debug build, then there's likely some memory corruption or > uninitialized variable in your code. Valgrind is generally a great tool to > spot that. Otherwise add good old printf() statements to track down where > the corruption starts. And given your Python sample code, I assume you > could more easily reproduce the issue in a pure native context, just > running "ogrinfo -al -q > autotest/ogr/data/miramon/Polygons/SimplePolygons/SimplePolFile.pol" (if it > doesn't crash, try running it under gdb or valgrind) > > -- > > http://www.spatialys.com > > My software is free, but my time generally not. > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
FINALLY! I discovered the problem. I was using “unsigned long” as 4 bytes variable. On windows it’s like that. BUT on linux this kind of variable has a 8-bytes size. This caused a mess in linux (but not in wondows). So, mistery closed! Thanks you all. De: gdal-dev En nombre de Abel Pau via gdal-dev Enviado el: dijous, 7 de març de 2024 23:42 Para: Even Rouault ; Andrew C Aitchison ; Daniel Evans CC: 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) Asunto: Re: [gdal-dev] Testing the driver Hi, It’s in debug mode, yes, as you suggested. yes, it crashes in the same way. Same big number. It crashes because of the variable is not read from the file (this variable should be 1). But debugging in windows (where not crashes) this variable is set when opening the layer, and then I suppose has the value corrupted at some point. So I am going to put my old fashion printf on. Thanks again! De: Even Rouault mailto:even.roua...@spatialys.com>> Enviado el: dijous, 7 de març de 2024 23:32 Para: Abel Pau mailto:a@creaf.uab.cat>>; Andrew C Aitchison mailto:and...@aitchison.me.uk>>; Daniel Evans mailto:daniel.fred.ev...@gmail.com>> CC: 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>) mailto:gdal-dev@lists.osgeo.org>> Asunto: Re: [gdal-dev] Testing the driver At #10 we can see the variable nNum set to a non-sense megabignumber. Is it on a -DCMAKE_BUILD_TYPE=Debug build ? Otherwise stack traces and variable content in the debugger might look like garbage because of optimizations. If it is a debug build, then there's likely some memory corruption or uninitialized variable in your code. Valgrind is generally a great tool to spot that. Otherwise add good old printf() statements to track down where the corruption starts. And given your Python sample code, I assume you could more easily reproduce the issue in a pure native context, just running "ogrinfo -al -q autotest/ogr/data/miramon/Polygons/SimplePolygons/SimplePolFile.pol" (if it doesn't crash, try running it under gdb or valgrind) -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
Hi, It’s in debug mode, yes, as you suggested. yes, it crashes in the same way. Same big number. It crashes because of the variable is not read from the file (this variable should be 1). But debugging in windows (where not crashes) this variable is set when opening the layer, and then I suppose has the value corrupted at some point. So I am going to put my old fashion printf on. Thanks again! De: Even Rouault Enviado el: dijous, 7 de març de 2024 23:32 Para: Abel Pau ; Andrew C Aitchison ; Daniel Evans CC: 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) Asunto: Re: [gdal-dev] Testing the driver At #10 we can see the variable nNum set to a non-sense megabignumber. Is it on a -DCMAKE_BUILD_TYPE=Debug build ? Otherwise stack traces and variable content in the debugger might look like garbage because of optimizations. If it is a debug build, then there's likely some memory corruption or uninitialized variable in your code. Valgrind is generally a great tool to spot that. Otherwise add good old printf() statements to track down where the corruption starts. And given your Python sample code, I assume you could more easily reproduce the issue in a pure native context, just running "ogrinfo -al -q autotest/ogr/data/miramon/Polygons/SimplePolygons/SimplePolFile.pol" (if it doesn't crash, try running it under gdb or valgrind) -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
At #10 we can see the variable nNum set to a non-sense megabignumber. Is it on a -DCMAKE_BUILD_TYPE=Debug build ? Otherwise stack traces and variable content in the debugger might look like garbage because of optimizations. If it is a debug build, then there's likely some memory corruption or uninitialized variable in your code. Valgrind is generally a great tool to spot that. Otherwise add good old printf() statements to track down where the corruption starts. And given your Python sample code, I assume you could more easily reproduce the issue in a pure native context, just running "ogrinfo -al -q autotest/ogr/data/miramon/Polygons/SimplePolygons/SimplePolFile.pol" (if it doesn't crash, try running it under gdb or valgrind) -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
Hi,
I think that I found the way to debug and the trace makes me ask a question
that can be interesting to everybody who uses python and it's not an expert,
like me.
In my test I do:
ds = gdal.OpenEx("data/miramon/Polygons/SimplePolygons/SimplePolFile.pol")
lyr = ds.GetLayer(0)
assert lyr is not None, "Failed to get layer"
assert lyr.GetFeatureCount() == 3
assert lyr.GetGeomType() == ogr.wkbPolygon
f = lyr.GetNextFeature()
And at that moment there is a backtrace available:
#9 0x74b3d86c in CPLRealloc (pData=0x0, nNewSize=667227037326010464)
at /gdal/port/cpl_conv.cpp:280
#10 0x75cb638a in MMResizeMiraMonPolygonArcs (pFID=0x56456358,
nMax=0x56456348,
nNum=4653387708260263558, nIncr=0, nProposedMax=0) at
/gdal/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:4464
#11 0x75cc33e4 in MMGetMultiPolygonCoordinates
(hMiraMonLayer=0x56446fd0, i_pol=1, flag_z=0)
at /gdal/ogr/ogrsf_frmts/miramon/mm_rdlayr.c:353
#12 0x75cc3dfb in MMGetGeoFeatureFromVector
(hMiraMonLayer=0x56446fd0, i_elem=1)
at /gdal/ogr/ogrsf_frmts/miramon/mm_rdlayr.c:614
#13 0x75ca6d63 in OGRMiraMonLayer::GetFeature (this=0x56419300,
nFeatureId=1)
at /gdal/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:860
#14 0x75ca6598 in OGRMiraMonLayer::GetNextRawFeature
(this=0x56419300)
at /gdal/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:696
#15 0x75cac1df in
OGRGetNextFeatureThroughRaw::GetNextFeature
(this=0x56419300)
at /gdal/ogr/ogrsf_frmts/ogrsf_frmts.h:455
#16 0x75cac186 in OGRMiraMonLayer::GetNextFeature (this=0x56419300)
at /gdal/ogr/ogrsf_frmts/miramon/ogrmiramon.h:109
#17 0x760e860b in OGR_L_GetNextFeature (hLayer=0x56419300)
At #10 we can see the variable nNum set to a non-sense megabignumber.
This variable is set during Opening...
But my question is: The opening is in ds =
gdal.OpenEx("data/miramon/Polygons/SimplePolygons/SimplePolFile.pol")
The variable is stored in a C++ class member (in a old fashion C struct) but
available that it's filled in the opening.
After that, lyr = ds.GetLayer(0) is called.
And after that when lyr.GetNextFeature() is called... is this nNum set before
still alive?
Or perhaps I should read every time I Get a Next Feature?
Thanks for answering this question!
Thanks Daniel for your pacience.
-Mensaje original-
De: Andrew C Aitchison
Enviado el: dijous, 7 de març de 2024 1:01
Para: Daniel Evans
CC: Abel Pau ; 'gdal-dev@lists.osgeo.org'
(gdal-dev@lists.osgeo.org)
Asunto: Re: [gdal-dev] Testing the driver
On Wed, 6 Mar 2024, Daniel Evans via gdal-dev wrote:
> Is it worth moving this in-depth discussion to a PR or similar for the
> new driver?
>
> My thinking is that a lengthy discussion on memory leak detection
> techniques in C++, how to run tests in Python, etc., aren't topics
> relevant to most GDAL mailing list subscribers.
Maybe so, but how to develop gdal is surely on-topic for gdal-dev ?
I"m still following and learning.
--
Andrew C. Aitchison Kendal, UK
and...@aitchison.me.uk
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
On Wed, 6 Mar 2024, Daniel Evans via gdal-dev wrote: Is it worth moving this in-depth discussion to a PR or similar for the new driver? My thinking is that a lengthy discussion on memory leak detection techniques in C++, how to run tests in Python, etc., aren't topics relevant to most GDAL mailing list subscribers. Maybe so, but how to develop gdal is surely on-topic for gdal-dev ? I"m still following and learning. -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
Is it worth moving this in-depth discussion to a PR or similar for the new driver? My thinking is that a lengthy discussion on memory leak detection techniques in C++, how to run tests in Python, etc., aren't topics relevant to most GDAL mailing list subscribers. Cheers, Daniel On Wed, 6 Mar 2024, 16:54 Abel Pau via gdal-dev, wrote: > I was using ubuntu 20.04 and it doesn’t work there. > > Trying to install 22.04 but I have some issues (again). > > > > > > > > > > *De:* Even Rouault > *Enviado el:* dimecres, 6 de març de 2024 17:12 > *Para:* Abel Pau ; gdal-dev@lists.osgeo.org > *Asunto:* Re: [gdal-dev] Testing the driver > > > > PYTHONMALLOC=malloc gdb --args python3 -m pytest > autotest/ogr/ogr_miramon_vector.py > > PYTHONMALLOC=malloc valgrind python3 -m pytest > autotest/ogr/ogr_miramon_vector.py > > Le 06/03/2024 à 16:02, Abel Pau via gdal-dev a écrit : > > Hi again, > > anyone have any advice for debuging into python code? > I am using Pdb but it’s a little confusing. > > Any experience? > > Thanks > > > > > > apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest > autotest/ogr/ogr_miramon_vector.py > > Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, > pytest-sugar 1.0.0) > > benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False > min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10 > warmup=False warmup_iterations=10) > > GDAL Build Info: > > PAM_ENABLED: YES > > OGR_ENABLED: YES > > CURL_ENABLED: YES > > CURL_VERSION: 7.68.0 > > GEOS_ENABLED: YES > > GEOS_VERSION: 3.8.0-CAPI-1.13.1 > > PROJ_BUILD_VERSION: 6.3.1 > > PROJ_RUNTIME_VERSION: 6.3.1 > > COMPILER: GCC 9.4.0 > > GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data may > be skipped) > > GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped) > > rootdir: /mnt/d/GitHub-repository/gdal/build/autotest > > configfile: pytest.ini > > plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3 > > collected 3 items > > > > ogr/ogr_miramon_vector.py ✓✓ > > > 67% > ██▋ > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > PDB set_trace (IO-capturing turned off) > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > > /mnt/
Re: [gdal-dev] Testing the driver
I was using ubuntu 20.04 and it doesn’t work there. Trying to install 22.04 but I have some issues (again). De: Even Rouault Enviado el: dimecres, 6 de març de 2024 17:12 Para: Abel Pau ; gdal-dev@lists.osgeo.org Asunto: Re: [gdal-dev] Testing the driver PYTHONMALLOC=malloc gdb --args python3 -m pytest autotest/ogr/ogr_miramon_vector.py PYTHONMALLOC=malloc valgrind python3 -m pytest autotest/ogr/ogr_miramon_vector.py Le 06/03/2024 à 16:02, Abel Pau via gdal-dev a écrit : Hi again, anyone have any advice for debuging into python code? I am using Pdb but it’s a little confusing. Any experience? Thanks apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest autotest/ogr/ogr_miramon_vector.py Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, pytest-sugar 1.0.0) benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=10) GDAL Build Info: PAM_ENABLED: YES OGR_ENABLED: YES CURL_ENABLED: YES CURL_VERSION: 7.68.0 GEOS_ENABLED: YES GEOS_VERSION: 3.8.0-CAPI-1.13.1 PROJ_BUILD_VERSION: 6.3.1 PROJ_RUNTIME_VERSION: 6.3.1 COMPILER: GCC 9.4.0 GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data may be skipped) GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped) rootdir: /mnt/d/GitHub-repository/gdal/build/autotest configfile: pytest.ini plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3 collected 3 items ogr/ogr_miramon_vector.py ✓✓ 67%██▋ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > /mnt/d/GitHub-repository/gdal/build/autotest/ogr/ogr_miramon_vector.py(132)test_ogr_miramon_simple_polygon() -> f = lyr.GetNextFeature() (Pdb) s --Call-- > /mnt/d/GitHub-repository/gdal/build/swig/python/osgeo/ogr.py(1654)GetNextFeature() -> def GetNextFeature(self, *args) -> "OGRFeatureShadow *": (Pdb) s > /mnt/d/GitHub-repository/gdal/build/swig/python/osgeo/ogr.py(1668)GetNextFeature() -> return _ogr.Layer_GetNextFeature(self, *args) (Pdb) s ERROR 2: CPLRealloc(): Out of memory allocating 667227037326010464 bytes. Fatal Python error: Aborted Current thread 0x7f67c7140740 (most recent call first): File "/mnt/d/GitHub-repository/gdal/build/swig/python/osgeo/ogr.py", line 1668 in GetNextFeature File "/mnt/d/GitHub-repository/gdal/build/autotest/ogr/ogr_miramon_vector.py", line 132 in test_ogr_miramon_simple_polygon File "/usr/local/lib/python3.8/dist-packages/_pytest/python.py", line 194 in pytest_pyfunc_call File "/usr/local/lib/python3.8/dist-packages/pluggy/_callers.py", line 102 in _multicall File "/usr/local/lib/python3.8/dist-packages/pluggy/_manager.py", line 119 in _hookexec File "/usr/local/lib/py
Re: [gdal-dev] Testing the driver
usr/local/lib/python3.8/dist-packages/pluggy/_hooks.py", line
501 in __call__
File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line
352 in pytest_runtestloop
File "/usr/local/lib/python3.8/dist-packages/pluggy/_callers.py", line
102 in _multicall
File "/usr/local/lib/python3.8/dist-packages/pluggy/_manager.py", line
119 in _hookexec
File "/usr/local/lib/python3.8/dist-packages/pluggy/_hooks.py", line
501 in __call__
File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line
327 in _main
File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line
273 in wrap_session
File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line
320 in pytest_cmdline_main
File "/usr/local/lib/python3.8/dist-packages/pluggy/_callers.py", line
102 in _multicall
File "/usr/local/lib/python3.8/dist-packages/pluggy/_manager.py", line
119 in _hookexec
File "/usr/local/lib/python3.8/dist-packages/pluggy/_hooks.py", line
501 in __call__
File
"/usr/local/lib/python3.8/dist-packages/_pytest/config/__init__.py",
line 175 in main
File
"/usr/local/lib/python3.8/dist-packages/_pytest/config/__init__.py",
line 198 in console_main
File "/usr/local/bin/pytest", line 8 in
Aborted
*De:*gdal-dev *En nombre de *Abel
Pau via gdal-dev
*Enviado el:* dimecres, 6 de març de 2024 15:08
*Para:* Even Rouault ;
gdal-dev@lists.osgeo.org
*Asunto:* Re: [gdal-dev] Testing the driver
About the question: . Does "pytest autotest/ogr/ogr_basic_test.py" work?*
The answer is YES.
ogr/ogr_basic_test.py
✓✓100%
██
Results (5.98s):
66 passed
*De:*Even Rouault
*Enviado el:* dimecres, 6 de març de 2024 13:09
*Para:* Abel Pau ; gdal-dev@lists.osgeo.org
*Asunto:* Re: [gdal-dev] Testing the driver
Hi,
I don't see anything wrong. I've tried that on my native Linux build
and the test_ogr_miramon_vector_1() is found. Does "pytest
autotest/ogr/ogr_basic_test.py" work?*
Note: you don't need the try / except in your test case unless you'd
need to some particular cleanup, but that's not the case here. pytest
handles test failures nicely
Even
Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit :
Hi again,
after solving some issues I used WSL (Windows subsystem Linux) to
create an environment where I am able to run tests.
I run the cmake inside build folder in the environment. It’s slow
but finally it finish. After cmake --build . --target install all
is ready to be tested.
I create a simple test ogr_miramon_vector.py (see the code below)
to prove that it’s reliable.
I run:
pytest autotest/ogr/ogr_miramon_vector.py
and:
apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest
autotest/ogr/ogr_miramon_vector.py
Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2,
pytest-sugar 1.0.0)
benchmark: 4.0.0 (defaults: timer=time.perf_counter
disable_gc=False min_rounds=5 min_time=0.05 max_time=1.0
calibration_precision=10 warmup=False warmup_iterations=10)
GDAL Build Info:
PAM_ENABLED: YES
OGR_ENABLED: YES
CURL_ENABLED: YES
CURL_VERSION: 7.68.0
GEOS_ENABLED: YES
GEOS_VERSION: 3.8.0-CAPI-1.13.1
PROJ_BUILD_VERSION: 6.3.1
PROJ_RUNTIME_VERSION: 6.3.1
COMPILER: GCC 9.4.0
GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded
data may be skipped)
GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be
skipped)
rootdir: /mnt/d/GitHub-repository/gdal/build/autotest
configfile: pytest.ini
plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3
*collected 0 items*
My questions is why it seems it’s not working?
Thanks!
The test:
-
import os
import gdaltest
import ogrtest
import pytest
from osgeo import gdal, ogr, osr
pytestmark = pytest.mark.require_driver("MiraMonVector")
###
@pytest.fixture(scope="module", autouse=True)
def init():
with gdaltest.config_option("CPL_DEBUG", "ON"):
yield
###########
# basic test
def test_ogr_miramon_vector_1():
try:
ds =
gdal.OpenEx("data/miramon/Points/SimplePoints/SimplePointsFile.pnt")
lyr = ds.GetLayer(0)
assert lyr is not None, "Failed to get layer"
assert lyr.GetFeatureCount() == 3
assert lyr.GetGeomType() == ogr.wkbPoint
f = lyr.GetNextFeature()
assert f.GetFID() == 0
assert f.GetGeometryRef().ExportToWkt() == "POINT (513.49 848.81)"
Re: [gdal-dev] Testing the driver
Hi,
It seems nothing changes. I understand that the environment is new and the
execution is not related with the last one.
Here there are 5 tests that fail..
Any idea of what can be happening?
They are very unrelated
Bye VSIMkdirRecursive() · AbelPau/gdal@646b98b
(github.com)<https://github.com/AbelPau/gdal/actions/runs/8172351502/job/22342474513>
De: gdal-dev En nombre de Abel Pau via
gdal-dev
Enviado el: dimecres, 6 de març de 2024 13:52
Para: Even Rouault ; gdal-dev@lists.osgeo.org
Asunto: Re: [gdal-dev] Testing the driver
Ok, I ‘ve changed that. Let’s see if it’s the problem.
It’s all so delicate :)
Thanks again!
De: Even Rouault mailto:even.roua...@spatialys.com>>
Enviado el: dimecres, 6 de març de 2024 13:36
Para: Abel Pau mailto:a@creaf.uab.cat>>;
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Asunto: Re: [gdal-dev] Testing the driver
Le 06/03/2024 à 13:14, Abel Pau a écrit :
Hi Even,
I finally discovered the error. It was the fixture. In the wrong place.
Now I’m creating the test.
I hope finish it soon.
On the other hand,
in my actions tab: Merge branch 'OSGeo:master' into master ·
AbelPau/gdal@0249b6d
(github.com)<https://github.com/AbelPau/gdal/actions/runs/8169099745/job/22332488002>
There are some tiff failures, but nothing on my hand about tiff.
== FAILURES ===
36: _ test_tiff_write_133
_
36:
36: def test_tiff_write_133():
Do you know what it can be?
There are sometimes random failures, but here it fails on both the
build-windows-msys2-mingw and build-windows-conda configs . I would suspect
this might be a side effect of a previous run of the Miramon driver by another
test with an invalid filename such as /foo/bar. Actually I see that test_ogrsf
tries to create a /foo/test file.
And
https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L219
does a VSIMkdirRecursive(), so it must create a "/foo" directory. I would
recommend against using VSIMkdirRecursive() in a driver. You might use
VSIMkdir() to create the latest level of directory, but creating the whole
hiearchy is granting too much power to a driver.
Even
De: Even Rouault <mailto:even.roua...@spatialys.com>
Enviado el: dimecres, 6 de març de 2024 13:09
Para: Abel Pau <mailto:a@creaf.uab.cat>;
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Asunto: Re: [gdal-dev] Testing the driver
Hi,
I don't see anything wrong. I've tried that on my native Linux build and the
test_ogr_miramon_vector_1() is found. Does "pytest
autotest/ogr/ogr_basic_test.py" work?*
Note: you don't need the try / except in your test case unless you'd need to
some particular cleanup, but that's not the case here. pytest handles test
failures nicely
Even
Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit :
Hi again,
after solving some issues I used WSL (Windows subsystem Linux) to create an
environment where I am able to run tests.
I run the cmake inside build folder in the environment. It’s slow but finally
it finish. After cmake --build . --target install all is ready to be tested.
I create a simple test ogr_miramon_vector.py (see the code below) to prove that
it’s reliable.
I run:
pytest autotest/ogr/ogr_miramon_vector.py
and:
apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest
autotest/ogr/ogr_miramon_vector.py
Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, pytest-sugar
1.0.0)
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False
min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10
warmup=False warmup_iterations=10)
GDAL Build Info:
PAM_ENABLED: YES
OGR_ENABLED: YES
CURL_ENABLED: YES
CURL_VERSION: 7.68.0
GEOS_ENABLED: YES
GEOS_VERSION: 3.8.0-CAPI-1.13.1
PROJ_BUILD_VERSION: 6.3.1
PROJ_RUNTIME_VERSION: 6.3.1
COMPILER: GCC 9.4.0
GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data may be
skipped)
GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped)
rootdir: /mnt/d/GitHub-repository/gdal/build/autotest
configfile: pytest.ini
plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3
collected 0 items
My questions is why it seems it’s not working?
Thanks!
The test:
-
import os
import gdaltest
import ogrtest
import pytest
from osgeo import gdal, ogr, osr
pytestmark = pytest.mark.require_driver("MiraMonVector")
###
@pytest.fixture(scope="module", autouse=True)
def init():
with gdaltest.config_option("CPL_DEBUG", "ON"):
yield
###
# basic test
def test_ogr_miramon_vector_1():
try:
ds =
Re: [gdal-dev] Testing the driver
File "/usr/local/lib/python3.8/dist-packages/pluggy/_hooks.py", line 501 in __call__ File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 263 in File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 342 in from_call File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 262 in call_runtest_hook File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 223 in call_and_report File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 134 in runtestprotocol File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 115 in pytest_runtest_protocol File "/usr/local/lib/python3.8/dist-packages/pluggy/_callers.py", line 102 in _multicall File "/usr/local/lib/python3.8/dist-packages/pluggy/_manager.py", line 119 in _hookexec File "/usr/local/lib/python3.8/dist-packages/pluggy/_hooks.py", line 501 in __call__ File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 352 in pytest_runtestloop File "/usr/local/lib/python3.8/dist-packages/pluggy/_callers.py", line 102 in _multicall File "/usr/local/lib/python3.8/dist-packages/pluggy/_manager.py", line 119 in _hookexec File "/usr/local/lib/python3.8/dist-packages/pluggy/_hooks.py", line 501 in __call__ File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 327 in _main File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 273 in wrap_session File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 320 in pytest_cmdline_main File "/usr/local/lib/python3.8/dist-packages/pluggy/_callers.py", line 102 in _multicall File "/usr/local/lib/python3.8/dist-packages/pluggy/_manager.py", line 119 in _hookexec File "/usr/local/lib/python3.8/dist-packages/pluggy/_hooks.py", line 501 in __call__ File "/usr/local/lib/python3.8/dist-packages/_pytest/config/__init__.py", line 175 in main File "/usr/local/lib/python3.8/dist-packages/_pytest/config/__init__.py", line 198 in console_main File "/usr/local/bin/pytest", line 8 in Aborted De: gdal-dev En nombre de Abel Pau via gdal-dev Enviado el: dimecres, 6 de març de 2024 15:08 Para: Even Rouault ; gdal-dev@lists.osgeo.org Asunto: Re: [gdal-dev] Testing the driver About the question: . Does "pytest autotest/ogr/ogr_basic_test.py" work?* The answer is YES. ogr/ogr_basic_test.py ✓✓ 100% ██ Results (5.98s): 66 passed De: Even Rouault mailto:even.roua...@spatialys.com>> Enviado el: dimecres, 6 de març de 2024 13:09 Para: Abel Pau mailto:a@creaf.uab.cat>>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: Re: [gdal-dev] Testing the driver Hi, I don't see anything wrong. I've tried that on my native Linux build and the test_ogr_miramon_vector_1() is found. Does "pytest autotest/ogr/ogr_basic_test.py" work?* Note: you don't need the try / except in your test case unless you'd need to some particular cleanup, but that's not the case here. pytest handles test failures nicely Even Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit : Hi again, after solving some issues I used WSL (Windows subsystem Linux) to create an environment where I am able to run tests. I run the cmake inside build folder in the environment. It’s slow but finally it finish. After cmake --build . --target install all is ready to be tested. I create a simple test ogr_miramon_vector.py (see the code below) to prove that it’s reliable. I run: pytest autotest/ogr/ogr_miramon_vector.py and: apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest autotest/ogr/ogr_miramon_vector.py Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, pytest-sugar 1.0.0) benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=10) GDAL Build Info: PAM_ENABLED: YES OGR_ENABLED: YES CURL_ENABLED: YES CURL_VERSION: 7.68.0 GEOS_ENABLED: YES GEOS_VERSION: 3.8.0-CAPI-1.13.1 PROJ_BUILD_VERSION: 6.3.1 PROJ_RUNTIME_VERSION: 6.3.1 COMPILER: GCC 9.4.0 GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data may be skipped) GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped) rootdir: /mnt/d/GitHub-repository/gdal/build/autotest configfile: pytest.ini plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3 collected 0 items My questions is why it seems it’s not working? Thanks! The test: ---
Re: [gdal-dev] Testing the driver
About the question: . Does "pytest autotest/ogr/ogr_basic_test.py" work?*
The answer is YES.
ogr/ogr_basic_test.py
✓✓
100% ██
Results (5.98s):
66 passed
De: Even Rouault
Enviado el: dimecres, 6 de març de 2024 13:09
Para: Abel Pau ; gdal-dev@lists.osgeo.org
Asunto: Re: [gdal-dev] Testing the driver
Hi,
I don't see anything wrong. I've tried that on my native Linux build and the
test_ogr_miramon_vector_1() is found. Does "pytest
autotest/ogr/ogr_basic_test.py" work?*
Note: you don't need the try / except in your test case unless you'd need to
some particular cleanup, but that's not the case here. pytest handles test
failures nicely
Even
Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit :
Hi again,
after solving some issues I used WSL (Windows subsystem Linux) to create an
environment where I am able to run tests.
I run the cmake inside build folder in the environment. It’s slow but finally
it finish. After cmake --build . --target install all is ready to be tested.
I create a simple test ogr_miramon_vector.py (see the code below) to prove that
it’s reliable.
I run:
pytest autotest/ogr/ogr_miramon_vector.py
and:
apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest
autotest/ogr/ogr_miramon_vector.py
Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, pytest-sugar
1.0.0)
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False
min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10
warmup=False warmup_iterations=10)
GDAL Build Info:
PAM_ENABLED: YES
OGR_ENABLED: YES
CURL_ENABLED: YES
CURL_VERSION: 7.68.0
GEOS_ENABLED: YES
GEOS_VERSION: 3.8.0-CAPI-1.13.1
PROJ_BUILD_VERSION: 6.3.1
PROJ_RUNTIME_VERSION: 6.3.1
COMPILER: GCC 9.4.0
GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data may be
skipped)
GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped)
rootdir: /mnt/d/GitHub-repository/gdal/build/autotest
configfile: pytest.ini
plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3
collected 0 items
My questions is why it seems it’s not working?
Thanks!
The test:
-
import os
import gdaltest
import ogrtest
import pytest
from osgeo import gdal, ogr, osr
pytestmark = pytest.mark.require_driver("MiraMonVector")
###
@pytest.fixture(scope="module", autouse=True)
def init():
with gdaltest.config_option("CPL_DEBUG", "ON"):
yield
###
# basic test
def test_ogr_miramon_vector_1():
try:
ds =
gdal.OpenEx("data/miramon/Points/SimplePoints/SimplePointsFile.pnt")
lyr = ds.GetLayer(0)
assert lyr is not None, "Failed to get layer"
assert lyr.GetFeatureCount() == 3
assert lyr.GetGeomType() == ogr.wkbPoint
f = lyr.GetNextFeature()
assert f.GetFID() == 0
assert f.GetGeometryRef().ExportToWkt() == "POINT (513.49 848.81)"
assert f.GetField("ID_GRAFIC") == "0"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "1"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "2"
ds = None
except Exception as e:
pytest.fail(f"Test failed with exception: {e}")
De: Even Rouault <mailto:even.roua...@spatialys.com>
Enviado el: divendres, 9 de febrer de 2024 11:48
Para: Abel Pau <mailto:a@creaf.uab.cat>;
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Asunto: Re: [gdal-dev] Testing the driver
Abel,
Le 09/02/2024 à 10:55, Abel Pau via gdal-dev a écrit :
Hi,
I am at the lasts steps before pulling a request about the MiraMon driver.
I need to write some documentation and formalize the tests.
After that, I’ll do the pull request to github.
I'd suggest first before issuing the pull request that you push to your fork on
github and look at the Actions tab. That will allow you to fix a lot of things
on your side, before issuing the PR itself
I am a little confused about the testing. I can use pytest or ctest, right?
Which is the favourite? Are there any changes from the official documentation?
ctest is just the CMake way of launching the test suite. It will execute C++
tests of autotest/cpp directly, and for tests written in python will launch
"pytest autotest/X" for each directory.
"ctest --test-dir $build_dir -R autotest_ogr -V" will just run all the
autotest/ogr tests, which can be quite long already.
To test your own development,
Re: [gdal-dev] Testing the driver
Ok, I ‘ve changed that. Let’s see if it’s the problem.
It’s all so delicate :)
Thanks again!
De: Even Rouault
Enviado el: dimecres, 6 de març de 2024 13:36
Para: Abel Pau ; gdal-dev@lists.osgeo.org
Asunto: Re: [gdal-dev] Testing the driver
Le 06/03/2024 à 13:14, Abel Pau a écrit :
Hi Even,
I finally discovered the error. It was the fixture. In the wrong place.
Now I’m creating the test.
I hope finish it soon.
On the other hand,
in my actions tab: Merge branch 'OSGeo:master' into master ·
AbelPau/gdal@0249b6d
(github.com)<https://github.com/AbelPau/gdal/actions/runs/8169099745/job/22332488002>
There are some tiff failures, but nothing on my hand about tiff.
== FAILURES ===
36: _ test_tiff_write_133
_
36:
36: def test_tiff_write_133():
Do you know what it can be?
There are sometimes random failures, but here it fails on both the
build-windows-msys2-mingw and build-windows-conda configs . I would suspect
this might be a side effect of a previous run of the Miramon driver by another
test with an invalid filename such as /foo/bar. Actually I see that test_ogrsf
tries to create a /foo/test file.
And
https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L219
does a VSIMkdirRecursive(), so it must create a "/foo" directory. I would
recommend against using VSIMkdirRecursive() in a driver. You might use
VSIMkdir() to create the latest level of directory, but creating the whole
hiearchy is granting too much power to a driver.
Even
De: Even Rouault <mailto:even.roua...@spatialys.com>
Enviado el: dimecres, 6 de març de 2024 13:09
Para: Abel Pau <mailto:a@creaf.uab.cat>;
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Asunto: Re: [gdal-dev] Testing the driver
Hi,
I don't see anything wrong. I've tried that on my native Linux build and the
test_ogr_miramon_vector_1() is found. Does "pytest
autotest/ogr/ogr_basic_test.py" work?*
Note: you don't need the try / except in your test case unless you'd need to
some particular cleanup, but that's not the case here. pytest handles test
failures nicely
Even
Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit :
Hi again,
after solving some issues I used WSL (Windows subsystem Linux) to create an
environment where I am able to run tests.
I run the cmake inside build folder in the environment. It’s slow but finally
it finish. After cmake --build . --target install all is ready to be tested.
I create a simple test ogr_miramon_vector.py (see the code below) to prove that
it’s reliable.
I run:
pytest autotest/ogr/ogr_miramon_vector.py
and:
apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest
autotest/ogr/ogr_miramon_vector.py
Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, pytest-sugar
1.0.0)
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False
min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10
warmup=False warmup_iterations=10)
GDAL Build Info:
PAM_ENABLED: YES
OGR_ENABLED: YES
CURL_ENABLED: YES
CURL_VERSION: 7.68.0
GEOS_ENABLED: YES
GEOS_VERSION: 3.8.0-CAPI-1.13.1
PROJ_BUILD_VERSION: 6.3.1
PROJ_RUNTIME_VERSION: 6.3.1
COMPILER: GCC 9.4.0
GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data may be
skipped)
GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped)
rootdir: /mnt/d/GitHub-repository/gdal/build/autotest
configfile: pytest.ini
plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3
collected 0 items
My questions is why it seems it’s not working?
Thanks!
The test:
-
import os
import gdaltest
import ogrtest
import pytest
from osgeo import gdal, ogr, osr
pytestmark = pytest.mark.require_driver("MiraMonVector")
###
@pytest.fixture(scope="module", autouse=True)
def init():
with gdaltest.config_option("CPL_DEBUG", "ON"):
yield
###
# basic test
def test_ogr_miramon_vector_1():
try:
ds =
gdal.OpenEx("data/miramon/Points/SimplePoints/SimplePointsFile.pnt")
lyr = ds.GetLayer(0)
assert lyr is not None, "Failed to get layer"
assert lyr.GetFeatureCount() == 3
assert lyr.GetGeomType() == ogr.wkbPoint
f = lyr.GetNextFeature()
assert f.GetFID() == 0
assert f.GetGeometryRef().ExportToWkt() == "POINT (513.49 848.81)"
assert f.GetField("ID_GRAFIC") == "0"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "1"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == &qu
Re: [gdal-dev] Testing the driver
Le 06/03/2024 à 13:14, Abel Pau a écrit :
Hi Even,
I finally discovered the error. It was the fixture. In the wrong place.
Now I’m creating the test.
I hope finish it soon.
On the other hand,
in my actions tab: Merge branch 'OSGeo:master' into master ·
AbelPau/gdal@0249b6d (github.com)
<https://github.com/AbelPau/gdal/actions/runs/8169099745/job/22332488002>
There are some tiff failures, but nothing on my hand about tiff.
== FAILURES
===
36: _ test_tiff_write_133
_
36:
36: def test_tiff_write_133():
Do you know what it can be?
There are sometimes random failures, but here it fails on both the
build-windows-msys2-mingw and build-windows-conda configs . I would
suspect this might be a side effect of a previous run of the Miramon
driver by another test with an invalid filename such as /foo/bar.
Actually I see that test_ogrsf tries to create a /foo/test file.
And
https://github.com/AbelPau/gdal/blob/master/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp#L219
does a VSIMkdirRecursive(), so it must create a "/foo" directory. I
would recommend against using VSIMkdirRecursive() in a driver. You might
use VSIMkdir() to create the latest level of directory, but creating the
whole hiearchy is granting too much power to a driver.
Even
*De:*Even Rouault
*Enviado el:* dimecres, 6 de març de 2024 13:09
*Para:* Abel Pau ; gdal-dev@lists.osgeo.org
*Asunto:* Re: [gdal-dev] Testing the driver
Hi,
I don't see anything wrong. I've tried that on my native Linux build
and the test_ogr_miramon_vector_1() is found. Does "pytest
autotest/ogr/ogr_basic_test.py" work?*
Note: you don't need the try / except in your test case unless you'd
need to some particular cleanup, but that's not the case here. pytest
handles test failures nicely
Even
Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit :
Hi again,
after solving some issues I used WSL (Windows subsystem Linux) to
create an environment where I am able to run tests.
I run the cmake inside build folder in the environment. It’s slow
but finally it finish. After cmake --build . --target install all
is ready to be tested.
I create a simple test ogr_miramon_vector.py (see the code below)
to prove that it’s reliable.
I run:
pytest autotest/ogr/ogr_miramon_vector.py
and:
apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest
autotest/ogr/ogr_miramon_vector.py
Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2,
pytest-sugar 1.0.0)
benchmark: 4.0.0 (defaults: timer=time.perf_counter
disable_gc=False min_rounds=5 min_time=0.05 max_time=1.0
calibration_precision=10 warmup=False warmup_iterations=10)
GDAL Build Info:
PAM_ENABLED: YES
OGR_ENABLED: YES
CURL_ENABLED: YES
CURL_VERSION: 7.68.0
GEOS_ENABLED: YES
GEOS_VERSION: 3.8.0-CAPI-1.13.1
PROJ_BUILD_VERSION: 6.3.1
PROJ_RUNTIME_VERSION: 6.3.1
COMPILER: GCC 9.4.0
GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded
data may be skipped)
GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be
skipped)
rootdir: /mnt/d/GitHub-repository/gdal/build/autotest
configfile: pytest.ini
plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3
*collected 0 items*
My questions is why it seems it’s not working?
Thanks!
The test:
-
import os
import gdaltest
import ogrtest
import pytest
from osgeo import gdal, ogr, osr
pytestmark = pytest.mark.require_driver("MiraMonVector")
###
@pytest.fixture(scope="module", autouse=True)
def init():
with gdaltest.config_option("CPL_DEBUG", "ON"):
yield
###
# basic test
def test_ogr_miramon_vector_1():
try:
ds =
gdal.OpenEx("data/miramon/Points/SimplePoints/SimplePointsFile.pnt")
lyr = ds.GetLayer(0)
assert lyr is not None, "Failed to get layer"
assert lyr.GetFeatureCount() == 3
assert lyr.GetGeomType() == ogr.wkbPoint
f = lyr.GetNextFeature()
assert f.GetFID() == 0
assert f.GetGeometryRef().ExportToWkt() == "POINT (513.49 848.81)"
assert f.GetField("ID_GRAFIC") == "0"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "1"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "2"
ds = None
except Exception as e:
pytest.fail(f"Test failed with exception: {e}")
*De:*Even Rouault
Re: [gdal-dev] Testing the driver
Hi Even,
I finally discovered the error. It was the fixture. In the wrong place.
Now I’m creating the test.
I hope finish it soon.
On the other hand,
in my actions tab: Merge branch 'OSGeo:master' into master ·
AbelPau/gdal@0249b6d
(github.com)<https://github.com/AbelPau/gdal/actions/runs/8169099745/job/22332488002>
There are some tiff failures, but nothing on my hand about tiff.
== FAILURES ===
36: _ test_tiff_write_133
_
36:
36: def test_tiff_write_133():
Do you know what it can be?
De: Even Rouault
Enviado el: dimecres, 6 de març de 2024 13:09
Para: Abel Pau ; gdal-dev@lists.osgeo.org
Asunto: Re: [gdal-dev] Testing the driver
Hi,
I don't see anything wrong. I've tried that on my native Linux build and the
test_ogr_miramon_vector_1() is found. Does "pytest
autotest/ogr/ogr_basic_test.py" work?*
Note: you don't need the try / except in your test case unless you'd need to
some particular cleanup, but that's not the case here. pytest handles test
failures nicely
Even
Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit :
Hi again,
after solving some issues I used WSL (Windows subsystem Linux) to create an
environment where I am able to run tests.
I run the cmake inside build folder in the environment. It’s slow but finally
it finish. After cmake --build . --target install all is ready to be tested.
I create a simple test ogr_miramon_vector.py (see the code below) to prove that
it’s reliable.
I run:
pytest autotest/ogr/ogr_miramon_vector.py
and:
apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest
autotest/ogr/ogr_miramon_vector.py
Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, pytest-sugar
1.0.0)
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False
min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10
warmup=False warmup_iterations=10)
GDAL Build Info:
PAM_ENABLED: YES
OGR_ENABLED: YES
CURL_ENABLED: YES
CURL_VERSION: 7.68.0
GEOS_ENABLED: YES
GEOS_VERSION: 3.8.0-CAPI-1.13.1
PROJ_BUILD_VERSION: 6.3.1
PROJ_RUNTIME_VERSION: 6.3.1
COMPILER: GCC 9.4.0
GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data may be
skipped)
GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped)
rootdir: /mnt/d/GitHub-repository/gdal/build/autotest
configfile: pytest.ini
plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3
collected 0 items
My questions is why it seems it’s not working?
Thanks!
The test:
-
import os
import gdaltest
import ogrtest
import pytest
from osgeo import gdal, ogr, osr
pytestmark = pytest.mark.require_driver("MiraMonVector")
###
@pytest.fixture(scope="module", autouse=True)
def init():
with gdaltest.config_option("CPL_DEBUG", "ON"):
yield
###
# basic test
def test_ogr_miramon_vector_1():
try:
ds =
gdal.OpenEx("data/miramon/Points/SimplePoints/SimplePointsFile.pnt")
lyr = ds.GetLayer(0)
assert lyr is not None, "Failed to get layer"
assert lyr.GetFeatureCount() == 3
assert lyr.GetGeomType() == ogr.wkbPoint
f = lyr.GetNextFeature()
assert f.GetFID() == 0
assert f.GetGeometryRef().ExportToWkt() == "POINT (513.49 848.81)"
assert f.GetField("ID_GRAFIC") == "0"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "1"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "2"
ds = None
except Exception as e:
pytest.fail(f"Test failed with exception: {e}")
De: Even Rouault
<mailto:%3ceven.roua...@spatialys.com%3e>
Enviado el: divendres, 9 de febrer de 2024 11:48
Para: Abel Pau <mailto:a@creaf.uab.cat>;
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Asunto: Re: [gdal-dev] Testing the driver
Abel,
Le 09/02/2024 à 10:55, Abel Pau via gdal-dev a écrit :
Hi,
I am at the lasts steps before pulling a request about the MiraMon driver.
I need to write some documentation and formalize the tests.
After that, I’ll do the pull request to github.
I'd suggest first before issuing the pull request that you push to your fork on
github and look at the Actions tab. That will allow you to fix a lot of things
on your side, before issuing the PR itself
I am a little confused about the testing. I can use pytest or ctest, right?
Which is the favourite? Are there any changes from the official documentation?
ctest is just the CMake way of launching the test suite. It will execute C++
tests of autotest/cpp directly, and f
Re: [gdal-dev] Testing the driver
Hi,
I don't see anything wrong. I've tried that on my native Linux build and
the test_ogr_miramon_vector_1() is found. Does "pytest
autotest/ogr/ogr_basic_test.py" work?*
Note: you don't need the try / except in your test case unless you'd
need to some particular cleanup, but that's not the case here. pytest
handles test failures nicely
Even
Le 05/03/2024 à 22:28, Abel Pau via gdal-dev a écrit :
Hi again,
after solving some issues I used WSL (Windows subsystem Linux) to
create an environment where I am able to run tests.
I run the cmake inside build folder in the environment. It’s slow but
finally it finish. After cmake --build . --target install all is ready
to be tested.
I create a simple test ogr_miramon_vector.py (see the code below) to
prove that it’s reliable.
I run:
pytest autotest/ogr/ogr_miramon_vector.py
and:
apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest
autotest/ogr/ogr_miramon_vector.py
Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2,
pytest-sugar 1.0.0)
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False
min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10
warmup=False warmup_iterations=10)
GDAL Build Info:
PAM_ENABLED: YES
OGR_ENABLED: YES
CURL_ENABLED: YES
CURL_VERSION: 7.68.0
GEOS_ENABLED: YES
GEOS_VERSION: 3.8.0-CAPI-1.13.1
PROJ_BUILD_VERSION: 6.3.1
PROJ_RUNTIME_VERSION: 6.3.1
COMPILER: GCC 9.4.0
GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data
may be skipped)
GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped)
rootdir: /mnt/d/GitHub-repository/gdal/build/autotest
configfile: pytest.ini
plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3
*collected 0 items*
My questions is why it seems it’s not working?
Thanks!
The test:
-
import os
import gdaltest
import ogrtest
import pytest
from osgeo import gdal, ogr, osr
pytestmark = pytest.mark.require_driver("MiraMonVector")
###
@pytest.fixture(scope="module", autouse=True)
def init():
with gdaltest.config_option("CPL_DEBUG", "ON"):
yield
###
# basic test
def test_ogr_miramon_vector_1():
try:
ds = gdal.OpenEx("data/miramon/Points/SimplePoints/SimplePointsFile.pnt")
lyr = ds.GetLayer(0)
assert lyr is not None, "Failed to get layer"
assert lyr.GetFeatureCount() == 3
assert lyr.GetGeomType() == ogr.wkbPoint
f = lyr.GetNextFeature()
assert f.GetFID() == 0
assert f.GetGeometryRef().ExportToWkt() == "POINT (513.49 848.81)"
assert f.GetField("ID_GRAFIC") == "0"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "1"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "2"
ds = None
except Exception as e:
pytest.fail(f"Test failed with exception: {e}")
*De:*Even Rouault
*Enviado el:* divendres, 9 de febrer de 2024 11:48
*Para:* Abel Pau ; gdal-dev@lists.osgeo.org
*Asunto:* Re: [gdal-dev] Testing the driver
Abel,
Le 09/02/2024 à 10:55, Abel Pau via gdal-dev a écrit :
Hi,
I am at the lasts steps before pulling a request about the MiraMon
driver.
I need to write some documentation and formalize the tests.
After that, I’ll do the pull request to github.
I'd suggest first before issuing the pull request that you push to
your fork on github and look at the Actions tab. That will allow you
to fix a lot of things on your side, before issuing the PR itself
I am a little confused about the testing. I can use pytest or
ctest, right? Which is the favourite? Are there any changes from
the official documentation?
ctest is just the CMake way of launching the test suite. It will
execute C++ tests of autotest/cpp directly, and for tests written in
python will launch "pytest autotest/X" for each directory.
"ctest --test-dir $build_dir -R autotest_ogr -V" will just run all
the autotest/ogr tests, which can be quite long already.
To test your own development, you may have a more pleasant experience
by directly running just the tests for your driver with something like
"pytest autotest/ogr/ogr_miramon.py" (be careful on Windows, the
content of $build_dir/autotest is copied from $source_dir/autotest
each time "cmake" is run, so if you edit your test .py file directly
in the build directory, be super careful of not accidentally losing
your work, and make sure to copy its content to the source directory
first. That's admittedly an annoying point of the current test setup
on Windows, compared to Unix where we use symbolic links)
after setting the environment to have PYTHONPATH point to something
lik
Re: [gdal-dev] Testing the driver
Hi again,
after solving some issues I used WSL (Windows subsystem Linux) to create an
environment where I am able to run tests.
I run the cmake inside build folder in the environment. It’s slow but finally
it finish. After cmake --build . --target install all is ready to be tested.
I create a simple test ogr_miramon_vector.py (see the code below) to prove that
it’s reliable.
I run:
pytest autotest/ogr/ogr_miramon_vector.py
and:
apau@ABEL2:/mnt/d/GitHub-repository/gdal/build$ pytest
autotest/ogr/ogr_miramon_vector.py
Test session starts (platform: linux, Python 3.8.10, pytest 8.0.2, pytest-sugar
1.0.0)
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False
min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10
warmup=False warmup_iterations=10)
GDAL Build Info:
PAM_ENABLED: YES
OGR_ENABLED: YES
CURL_ENABLED: YES
CURL_VERSION: 7.68.0
GEOS_ENABLED: YES
GEOS_VERSION: 3.8.0-CAPI-1.13.1
PROJ_BUILD_VERSION: 6.3.1
PROJ_RUNTIME_VERSION: 6.3.1
COMPILER: GCC 9.4.0
GDAL_DOWNLOAD_TEST_DATA: undefined (tests relying on downloaded data may be
skipped)
GDAL_RUN_SLOW_TESTS: undefined (tests marked as "slow" will be skipped)
rootdir: /mnt/d/GitHub-repository/gdal/build/autotest
configfile: pytest.ini
plugins: benchmark-4.0.0, sugar-1.0.0, env-1.1.3
collected 0 items
My questions is why it seems it’s not working?
Thanks!
The test:
-
import os
import gdaltest
import ogrtest
import pytest
from osgeo import gdal, ogr, osr
pytestmark = pytest.mark.require_driver("MiraMonVector")
###
@pytest.fixture(scope="module", autouse=True)
def init():
with gdaltest.config_option("CPL_DEBUG", "ON"):
yield
###
# basic test
def test_ogr_miramon_vector_1():
try:
ds =
gdal.OpenEx("data/miramon/Points/SimplePoints/SimplePointsFile.pnt")
lyr = ds.GetLayer(0)
assert lyr is not None, "Failed to get layer"
assert lyr.GetFeatureCount() == 3
assert lyr.GetGeomType() == ogr.wkbPoint
f = lyr.GetNextFeature()
assert f.GetFID() == 0
assert f.GetGeometryRef().ExportToWkt() == "POINT (513.49 848.81)"
assert f.GetField("ID_GRAFIC") == "0"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "1"
f = lyr.GetNextFeature()
assert f.GetField("ID_GRAFIC") == "2"
ds = None
except Exception as e:
pytest.fail(f"Test failed with exception: {e}")
De: Even Rouault
Enviado el: divendres, 9 de febrer de 2024 11:48
Para: Abel Pau ; gdal-dev@lists.osgeo.org
Asunto: Re: [gdal-dev] Testing the driver
Abel,
Le 09/02/2024 à 10:55, Abel Pau via gdal-dev a écrit :
Hi,
I am at the lasts steps before pulling a request about the MiraMon driver.
I need to write some documentation and formalize the tests.
After that, I’ll do the pull request to github.
I'd suggest first before issuing the pull request that you push to your fork on
github and look at the Actions tab. That will allow you to fix a lot of things
on your side, before issuing the PR itself
I am a little confused about the testing. I can use pytest or ctest, right?
Which is the favourite? Are there any changes from the official documentation?
ctest is just the CMake way of launching the test suite. It will execute C++
tests of autotest/cpp directly, and for tests written in python will launch
"pytest autotest/X" for each directory.
"ctest --test-dir $build_dir -R autotest_ogr -V" will just run all the
autotest/ogr tests, which can be quite long already.
To test your own development, you may have a more pleasant experience by
directly running just the tests for your driver with something like "pytest
autotest/ogr/ogr_miramon.py" (be careful on Windows, the content of
$build_dir/autotest is copied from $source_dir/autotest each time "cmake" is
run, so if you edit your test .py file directly in the build directory, be
super careful of not accidentally losing your work, and make sure to copy its
content to the source directory first. That's admittedly an annoying point of
the current test setup on Windows, compared to Unix where we use symbolic links)
after setting the environment to have PYTHONPATH point to something like
$build_dir/swig/python/Release or $build_dir/swig/python/Debug (I believe
you're on Windows?). If you look at the first lines output by the above "ctest
--test-dir $build_dir -R autotest_ogr -V" invokation, you'll actually see the
PYTHONPATH value to specify.
You also need to first install pytest and other testing dependencies with:
python -m pip install autotest/requireme
Re: [gdal-dev] Testing the driver
On Sun, 18 Feb 2024, Even Rouault wrote: Le 10/02/2024 à 18:34, Andrew C Aitchison via gdal-dev a écrit : On Sat, 10 Feb 2024, Even Rouault via gdal-dev wrote: To test your own development, you may have a more pleasant experience by directly running just the tests for your driver with something like "pytest autotest/ogr/ogr_miramon.py" (be careful on Windows, the content of $build_dir/autotest is copied from $source_dir/autotest each time "cmake" is run, so if you edit your test .py file directly in the build directory, be super careful of not accidentally losing your work, and make sure to copy its content to the source directory first. That's admittedly an annoying point of the current test setup on Windows, compared to Unix where we use symbolic links) Actually Dan figured out it was possible to run directly the tests against your test file from the source directory, and not the one that is copied. Cf https://github.com/OSGeo/gdal/pull/9224 So you can actually do from the build directory: pytest -c autotest/pytest.ini ../autotest/ogr/ogr_gpx.py Recent tarball releases have cmake/template/pytest.ini.in but how do I genenerate autotest/pytest.ini ? It is generated by cmake Ah. If I do: tar Jxf gdal-3.8.4rc1.tar.xz tar zxf gdalautotest-3.8.4rc1.tar.gz ln -s ../gdalautotest-3.8.4 gdal-3.8.4/autotest cd gdal-3.8.4 cmake -B build build/autotest/pytest.ini is generated. That link is not what I had expected - ln -s ../gdalautotest-3.8.4 gdal-3.8.4/gdalautotest would have been more obvious to me, and I did try running cmake inside gdalautotest-3.8.4 first. -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
Le 10/02/2024 à 18:34, Andrew C Aitchison via gdal-dev a écrit : On Sat, 10 Feb 2024, Even Rouault via gdal-dev wrote: To test your own development, you may have a more pleasant experience by directly running just the tests for your driver with something like "pytest autotest/ogr/ogr_miramon.py" (be careful on Windows, the content of $build_dir/autotest is copied from $source_dir/autotest each time "cmake" is run, so if you edit your test .py file directly in the build directory, be super careful of not accidentally losing your work, and make sure to copy its content to the source directory first. That's admittedly an annoying point of the current test setup on Windows, compared to Unix where we use symbolic links) Actually Dan figured out it was possible to run directly the tests against your test file from the source directory, and not the one that is copied. Cf https://github.com/OSGeo/gdal/pull/9224 So you can actually do from the build directory: pytest -c autotest/pytest.ini ../autotest/ogr/ogr_gpx.py Recent tarball releases have cmake/template/pytest.ini.in but how do I genenerate autotest/pytest.ini ? It is generated by cmake ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
On Sat, 10 Feb 2024, Even Rouault via gdal-dev wrote: To test your own development, you may have a more pleasant experience by directly running just the tests for your driver with something like "pytest autotest/ogr/ogr_miramon.py" (be careful on Windows, the content of $build_dir/autotest is copied from $source_dir/autotest each time "cmake" is run, so if you edit your test .py file directly in the build directory, be super careful of not accidentally losing your work, and make sure to copy its content to the source directory first. That's admittedly an annoying point of the current test setup on Windows, compared to Unix where we use symbolic links) Actually Dan figured out it was possible to run directly the tests against your test file from the source directory, and not the one that is copied. Cf https://github.com/OSGeo/gdal/pull/9224 So you can actually do from the build directory: pytest -c autotest/pytest.ini ../autotest/ogr/ogr_gpx.py Recent tarball releases have cmake/template/pytest.ini.in but how do I genenerate autotest/pytest.ini ? -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
To test your own development, you may have a more pleasant experience by directly running just the tests for your driver with something like "pytest autotest/ogr/ogr_miramon.py" (be careful on Windows, the content of $build_dir/autotest is copied from $source_dir/autotest each time "cmake" is run, so if you edit your test .py file directly in the build directory, be super careful of not accidentally losing your work, and make sure to copy its content to the source directory first. That's admittedly an annoying point of the current test setup on Windows, compared to Unix where we use symbolic links) Actually Dan figured out it was possible to run directly the tests against your test file from the source directory, and not the one that is copied. Cf https://github.com/OSGeo/gdal/pull/9224 So you can actually do from the build directory: pytest -c autotest/pytest.ini ../autotest/ogr/ogr_gpx.py -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Testing the driver
Abel, Le 09/02/2024 à 10:55, Abel Pau via gdal-dev a écrit : Hi, I am at the lasts steps before pulling a request about the MiraMon driver. I need to write some documentation and formalize the tests. After that, I’ll do the pull request to github. I'd suggest first before issuing the pull request that you push to your fork on github and look at the Actions tab. That will allow you to fix a lot of things on your side, before issuing the PR itself I am a little confused about the testing. I can use pytest or ctest, right? Which is the favourite? Are there any changes from the official documentation? ctest is just the CMake way of launching the test suite. It will execute C++ tests of autotest/cpp directly, and for tests written in python will launch "pytest autotest/X" for each directory. "ctest --test-dir $build_dir -R autotest_ogr -V" will just run all the autotest/ogr tests, which can be quite long already. To test your own development, you may have a more pleasant experience by directly running just the tests for your driver with something like "pytest autotest/ogr/ogr_miramon.py" (be careful on Windows, the content of $build_dir/autotest is copied from $source_dir/autotest each time "cmake" is run, so if you edit your test .py file directly in the build directory, be super careful of not accidentally losing your work, and make sure to copy its content to the source directory first. That's admittedly an annoying point of the current test setup on Windows, compared to Unix where we use symbolic links) after setting the environment to have PYTHONPATH point to something like $build_dir/swig/python/Release or $build_dir/swig/python/Debug (I believe you're on Windows?). If you look at the first lines output by the above "ctest --test-dir $build_dir -R autotest_ogr -V" invokation, you'll actually see the PYTHONPATH value to specify. You also need to first install pytest and other testing dependencies with: python -m pip install autotest/requirements.txt There is a minimal test to create? A maximal test suite, you mean ;-) You should aim for a "reasonable" coverage of the code you wrote. Aiming to test the nominal code paths of your driver is desirable (testing the error cases generally requires a lot more effort). Can you recommend me some driver that tests things like: ·Read a point/arc/polygon layer from some format (gml,kml, gpckg,..) and assert the number of readed objectes ·Read a point layer and assert some points (3d included) and some of the fields values ·The same with arcs and polygons ·Create some layer from the own format to anothers and compare the results with some “good” results. ·Create multiple layers from one outer format (like gpx) and verify the name of the created files... You don't necessarily need to use other formats. It is actually better if the tests of a format don't depend too much on other formats, to keep things isolated. To test the read part of your driver, add a autotest/ogr/data/miramon directory with *small* test files, ideally at most a few KB each to keep the size of the GDAL repository reasonable, and a few features in each is often enough to unit test, with different type of geometries, attributes, and use the OGR Python API to open the file and iterate over its layers and features to check their content. Those files should have ideally be produced by the Miramon software and not by the writing side of your driver, to check the interoperability of your driver with a "reference" software. For the write site of the driver, you can for example run gdal.VectorTranslate(dest, source) on those files, and use again the test function to validate that the read side of your driver likes what the write site has produced. An alternative is also to do a binary comparison of the file generated by your driver with a reference test file stored in for example autotest/ogr/data/miramon/ref_output. But this may be sometimes a fragile approach if the output of your driver might change in the future (would require regenerating the reference test files). I'd suggest your test suite also has a test that runs the "test_ogrsf" command line utility which is a kind of compliance test suite which checks a number of expectations for a driver, like that GetFeatureCount() returns the same number as iterating with GetNextFeature(), etc etc It is difficult to point at a "reference" test suite, as all drivers have their particularities and may need specific tests. Potential sources of inspirations: - autotest/ogr/ogr_gtfs.py . Shows very simple testing of the read side of a driver, and includes a test_ogrsf test - autotest/ogr/ogr_csv.py has examples where the writing side of the driver is checked by opening the output file and checking that some strings are present in it (only easily doable with text based formats) - autotest/ogr/ogr_openfilegdb_write.py . Extensive testing of the writing si
