Re: [gdal-dev] Python bindings: enabling exceptions by default?
Hi, https://github.com/OSGeo/gdal/pull/7475 has now been merged into master. The UseExceptions()/DontUseExceptions() methods when invoked from one module also affect all other gdal, ogr, osr, gnm modules at once (that is now gdal.UseExceptions() is equivalent to gdal.UseExceptions() + ogr.UseExceptions() + osr.UseExceptions() + gnm.UseExceptions()) Even Le 21/03/2023 à 18:21, Even Rouault a écrit : Hi, Given the feedback received, I've amended the pull request with an extra commit to disable exceptions by default, except in autotest/ and scripts. And emit a FutureWarning if exceptions have not been explicitly enabled or disabled. So this will test the exceptions-by-default mode in the GDAL code base without imposing it to external code, and warning such external code that behaviour will change in the future Even Le 19/03/2023 à 13:34, Even Rouault a écrit : Hi, I've prepared a pull request that does the above, but this raises a number of questions. See my longish comment at https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. Thoughts appreciated Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Python bindings: enabling exceptions by default?
Hi, Given the feedback received, I've amended the pull request with an extra commit to disable exceptions by default, except in autotest/ and scripts. And emit a FutureWarning if exceptions have not been explicitly enabled or disabled. So this will test the exceptions-by-default mode in the GDAL code base without imposing it to external code, and warning such external code that behaviour will change in the future Even Le 19/03/2023 à 13:34, Even Rouault a écrit : Hi, I've prepared a pull request that does the above, but this raises a number of questions. See my longish comment at https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. Thoughts appreciated Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Python bindings: enabling exceptions by default?
Does whether we call it 3.7 or 4.0 affect how long 3.6 is supported ? I don't think Alan's comment would suggest that the next feature version planned for may would be called other than 3.7.0. IMHO there should be more substantial breaking changes to call it 4.0 than just what is discussed here. We have never talked about a 4.0 release up to now, either in its content or release schedule. Until now, we have "only" maintained the last feature version during the development cycle of the next feature version, ie during 3.7 development phase, started after 3.6.0 release, and ending at 3.7.0, we issue point release of 3.6.z. I see that when 3.0 was released we have issued bugfix releases of 3.0.z and 2.4.z (the latest one in the 2.y series) in parallel until 3.1.0 was released. I guess a similar strategy could apply for a 4.0 release (although the benefit/effort to have multiple several maintenance branches isn't obvious given than most packagers maintain just the last released version). Anyway, this derails from the topic of this thread, and could be re-discussed when we are at that point. -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Python bindings: enabling exceptions by default?
On Mon, 20 Mar 2023, Alan Snow wrote: I think that this is a good change. However, I recommend making this change associated with the GDAL 4.0 release as it is likely going to break existing codebases. Does whether we call it 3.7 or 4.0 affect how long 3.6 is supported ? -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Python bindings: enabling exceptions by default?
Folks, As long as there is a clear path to disabling python exception handling fairly cleanly I can live with it. Like Kurt, we have a large code base using GDAL mostly without exceptions enabled for GDAL. If there is going to be a change, I also agree 4.0 is the time to do it. Best regards, Frank On Mon, Mar 20, 2023 at 4:07 PM Kurt Schwehr wrote: > We are heavy users of the swig bindings. We have some Fiona users and are > only just now in the process of starting to use rasterio. We have only 3 > instances of calling UseExceptions. Turning on UseExceptions > immediately blew up a bunch of my tests that were making incorrect > assumptions. Nothing major, but I haven't looked for more than a couple > seconds for trouble. > > +1 for switching to have it enabled by default. I'm not sure if it would > be better with 3.7.0 or 4.0.0. > > On Mon, Mar 20, 2023 at 10:58 AM Howard Butler wrote: > >> >> >> > On Mar 19, 2023, at 7:34 AM, Even Rouault >> wrote: >> > >> > Hi, >> > >> > I've prepared a pull request that does the above, but this raises a >> number of questions. See my longish comment at >> https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. >> Thoughts appreciated >> >> First off, thank you for doing this. >> >> Speaking for myself, pretty much all of the ogr.py/gdal.py code I've >> written for the past 15 years has gdal.UseExceptions() after the "from >> osgeo import gdal" import. The reasons why we had to have this 15 years ago >> were reasonable, but now it is just confusing boilerplate. It's time to >> ditch it. >> >> It has been a regret that we didn't clean up the secondary effects of >> UseExceptions/DontUseExceptions a long time ago. I'm sure the original sin >> was a result of my mucking with the SWIG bindings to get something that >> kind of worked and giving up after that. I'm glad your PR addresses this >> and makes things better for other handler-aware code that is often mixed >> together with gdal.py such as Fiona/rasterio, QGIS bindings, or PDAL stuff >> (in my case). >> >> As for enabling them by default at our next 3.7 release ... one one hand, >> it has been fifteen years already and the project has been telegraphing >> through the options that this could/should change. On the other, GDAL is >> *very* conservative with API evolution, and making UseExceptions default >> behavior in the Python bindings breaks existing code that wasn't doing >> gdal.UseExceptions already. The nice thing about activating this behavior >> is that it causes it to throw an exception and tell the user about >> something that was otherwise silently failing, so the remedy in the case of >> most of these breakages is going to be clear. >> >> I would vote that we go for enabling UseExceptions by default for GDAL >> 3.7 based on the following assumptions: >> * The group that will feel the largest impact by the change is GDAL's own >> autotest suite >> * Exceptions are the Python Way, and default behavior of GDAL's bindings >> is out of step >> * Tons of user-land code is already doing UseExceptions, as it is a >> practice that is recommended in many tutorials and books and SE posts >> * The largest portion of GDAL-using Python developers are using Rasterio >> or Fiona, not gdal.py and ogr.py, and they would not be impacted by these >> changes (and maybe be impacted positively as the ticket describes). >> >> Are you a GDAL Python bindings user? Would you be impacted by this policy >> change? Please chime in. >> >> Howard >> ___ >> gdal-dev mailing list >> gdal-dev@lists.osgeo.org >> https://lists.osgeo.org/mailman/listinfo/gdal-dev >> > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | +1 650-701-7823 and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Python bindings: enabling exceptions by default?
We are heavy users of the swig bindings. We have some Fiona users and are only just now in the process of starting to use rasterio. We have only 3 instances of calling UseExceptions. Turning on UseExceptions immediately blew up a bunch of my tests that were making incorrect assumptions. Nothing major, but I haven't looked for more than a couple seconds for trouble. +1 for switching to have it enabled by default. I'm not sure if it would be better with 3.7.0 or 4.0.0. On Mon, Mar 20, 2023 at 10:58 AM Howard Butler wrote: > > > > On Mar 19, 2023, at 7:34 AM, Even Rouault > wrote: > > > > Hi, > > > > I've prepared a pull request that does the above, but this raises a > number of questions. See my longish comment at > https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. Thoughts > appreciated > > First off, thank you for doing this. > > Speaking for myself, pretty much all of the ogr.py/gdal.py code I've > written for the past 15 years has gdal.UseExceptions() after the "from > osgeo import gdal" import. The reasons why we had to have this 15 years ago > were reasonable, but now it is just confusing boilerplate. It's time to > ditch it. > > It has been a regret that we didn't clean up the secondary effects of > UseExceptions/DontUseExceptions a long time ago. I'm sure the original sin > was a result of my mucking with the SWIG bindings to get something that > kind of worked and giving up after that. I'm glad your PR addresses this > and makes things better for other handler-aware code that is often mixed > together with gdal.py such as Fiona/rasterio, QGIS bindings, or PDAL stuff > (in my case). > > As for enabling them by default at our next 3.7 release ... one one hand, > it has been fifteen years already and the project has been telegraphing > through the options that this could/should change. On the other, GDAL is > *very* conservative with API evolution, and making UseExceptions default > behavior in the Python bindings breaks existing code that wasn't doing > gdal.UseExceptions already. The nice thing about activating this behavior > is that it causes it to throw an exception and tell the user about > something that was otherwise silently failing, so the remedy in the case of > most of these breakages is going to be clear. > > I would vote that we go for enabling UseExceptions by default for GDAL 3.7 > based on the following assumptions: > * The group that will feel the largest impact by the change is GDAL's own > autotest suite > * Exceptions are the Python Way, and default behavior of GDAL's bindings > is out of step > * Tons of user-land code is already doing UseExceptions, as it is a > practice that is recommended in many tutorials and books and SE posts > * The largest portion of GDAL-using Python developers are using Rasterio > or Fiona, not gdal.py and ogr.py, and they would not be impacted by these > changes (and maybe be impacted positively as the ticket describes). > > Are you a GDAL Python bindings user? Would you be impacted by this policy > change? Please chime in. > > Howard > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Python bindings: enabling exceptions by default?
I think that this is a good change. However, I recommend making this change associated with the GDAL 4.0 release as it is likely going to break existing codebases. On Mon, Mar 20, 2023, 12:58 PM Howard Butler wrote: > > > > On Mar 19, 2023, at 7:34 AM, Even Rouault > wrote: > > > > Hi, > > > > I've prepared a pull request that does the above, but this raises a > number of questions. See my longish comment at > https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. Thoughts > appreciated > > First off, thank you for doing this. > > Speaking for myself, pretty much all of the ogr.py/gdal.py code I've > written for the past 15 years has gdal.UseExceptions() after the "from > osgeo import gdal" import. The reasons why we had to have this 15 years ago > were reasonable, but now it is just confusing boilerplate. It's time to > ditch it. > > It has been a regret that we didn't clean up the secondary effects of > UseExceptions/DontUseExceptions a long time ago. I'm sure the original sin > was a result of my mucking with the SWIG bindings to get something that > kind of worked and giving up after that. I'm glad your PR addresses this > and makes things better for other handler-aware code that is often mixed > together with gdal.py such as Fiona/rasterio, QGIS bindings, or PDAL stuff > (in my case). > > As for enabling them by default at our next 3.7 release ... one one hand, > it has been fifteen years already and the project has been telegraphing > through the options that this could/should change. On the other, GDAL is > *very* conservative with API evolution, and making UseExceptions default > behavior in the Python bindings breaks existing code that wasn't doing > gdal.UseExceptions already. The nice thing about activating this behavior > is that it causes it to throw an exception and tell the user about > something that was otherwise silently failing, so the remedy in the case of > most of these breakages is going to be clear. > > I would vote that we go for enabling UseExceptions by default for GDAL 3.7 > based on the following assumptions: > * The group that will feel the largest impact by the change is GDAL's own > autotest suite > * Exceptions are the Python Way, and default behavior of GDAL's bindings > is out of step > * Tons of user-land code is already doing UseExceptions, as it is a > practice that is recommended in many tutorials and books and SE posts > * The largest portion of GDAL-using Python developers are using Rasterio > or Fiona, not gdal.py and ogr.py, and they would not be impacted by these > changes (and maybe be impacted positively as the ticket describes). > > Are you a GDAL Python bindings user? Would you be impacted by this policy > change? Please chime in. > > Howard > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Python bindings: enabling exceptions by default?
> On Mar 19, 2023, at 7:34 AM, Even Rouault wrote: > > Hi, > > I've prepared a pull request that does the above, but this raises a number of > questions. See my longish comment at > https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. Thoughts > appreciated First off, thank you for doing this. Speaking for myself, pretty much all of the ogr.py/gdal.py code I've written for the past 15 years has gdal.UseExceptions() after the "from osgeo import gdal" import. The reasons why we had to have this 15 years ago were reasonable, but now it is just confusing boilerplate. It's time to ditch it. It has been a regret that we didn't clean up the secondary effects of UseExceptions/DontUseExceptions a long time ago. I'm sure the original sin was a result of my mucking with the SWIG bindings to get something that kind of worked and giving up after that. I'm glad your PR addresses this and makes things better for other handler-aware code that is often mixed together with gdal.py such as Fiona/rasterio, QGIS bindings, or PDAL stuff (in my case). As for enabling them by default at our next 3.7 release ... one one hand, it has been fifteen years already and the project has been telegraphing through the options that this could/should change. On the other, GDAL is *very* conservative with API evolution, and making UseExceptions default behavior in the Python bindings breaks existing code that wasn't doing gdal.UseExceptions already. The nice thing about activating this behavior is that it causes it to throw an exception and tell the user about something that was otherwise silently failing, so the remedy in the case of most of these breakages is going to be clear. I would vote that we go for enabling UseExceptions by default for GDAL 3.7 based on the following assumptions: * The group that will feel the largest impact by the change is GDAL's own autotest suite * Exceptions are the Python Way, and default behavior of GDAL's bindings is out of step * Tons of user-land code is already doing UseExceptions, as it is a practice that is recommended in many tutorials and books and SE posts * The largest portion of GDAL-using Python developers are using Rasterio or Fiona, not gdal.py and ogr.py, and they would not be impacted by these changes (and maybe be impacted positively as the ticket describes). Are you a GDAL Python bindings user? Would you be impacted by this policy change? Please chime in. Howard ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Python bindings: enabling exceptions by default?
Hi, I've prepared a pull request that does the above, but this raises a number of questions. See my longish comment at https://github.com/OSGeo/gdal/pull/7475#issuecomment-1475239852. Thoughts appreciated Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
