Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Alexander Mohr]

actually, even more importantly the reason to drop this is because it is wrong 
on linux.  When building gdal it will link to a specific version of libproj (ex 
libproj.so.3), with the current implementation, at runtime it will try to load 
libproj.so.  This does not existing in non -dev packages on purpose because it 
expects you to have a hard dependency to a versioned library so you won't 
accidentally load a version that's incompatible.

If a dynamic load is to be performed, it needs to remember the version that it 
was linked against, and not try to load the dev version.

Furthermore, the only reason to dynamically load the library is if it may not 
be available, but there currently is no fallback, you get an error that it 
can't load libproj.so.

So in fact the current implementation is broken in two ways.  It should just 
have a hard linker dependency against libproj.so.XX and if needed a flag that 
will cause all the methods to either not be available, or throw an exception if 
it was built w/o libproj support.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Ivan Lucena]

Hi Howard,


Yep. It is a license issue but I am not going to get into the details.


But apart from that, lets think about other scenarios:


An application is using GDAL and Proj4 and someone decide to update GDAL to get 
some bug fixes.


But GDAL is build *without* static Proj4 and therefore is blind to the presence 
of Proj4 shared library.


That would be a problem. Right?


Anyway. If the change makes sense and it is for the better. And it is for a new 
release (no backport).


+0 ( I can't vote anyway)


Best regards,


Ivan




From: gdal-dev <gdal-dev-boun...@lists.osgeo.org> on behalf of Howard Butler 
<how...@hobu.co>
Sent: Sunday, May 7, 2017 9:47:26 AM
To: Frank Warmerdam
Cc: gdal dev
Subject: Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

I'm +1 on this. I too found it confusing that Proj.4 worked in this way and no 
other libraries did in GDAL.


> I my case, I *cannot* distribute proj4 into my GDAL build and I *need* to 
> have the options to let the user decided if they want to add proj4 shared 
> libraries.

Ivan,

I don't understand the scenario here. It isn't a licensing thing to prevent you 
from distributing Proj.4, and what do you do in the case of all of the other 
shared libraries that are linked at compile time that GDAL uses?

Howard

>
> On May 7, 2017, at 3:01 AM, Frank Warmerdam <warmer...@pobox.com> wrote:
>
> Even,
>
> I'm +0 on this change.
>
> On the one hand, I *like* the fact that the dlopen() approach means
> that adding libproj.so after the fact means that GDAL starts
> supporting coordinate system conversion without a rebuild.  On the
> other hand, it isn't clear why we do this only for libproj.
>
> I have some appreciation for Ivan's stated concerns with regard to
> regression, but I don't think that should hold back cleanup in major
> new versions.
>
> Best regards,
> Frank
>
>
> On Sat, May 6, 2017 at 12:09 PM, Even Rouault
> <even.roua...@spatialys.com> wrote:
>> Ivan,
>>
>>
>>
>>> I understand the good intention of cleaning up code but that should not
>>
>>> remove functionality. You are breaking backward compatibility. What if
>>
>>> someone updates GDAL in some installation, proj4 is there and it will not
>>
>>> going to be used?
>>
>>
>>
>> Well that would be documented in the release notes... There are a number of
>> breaks in each new release. That's not really different. And the way of
>> solving in the case you mention is rather simple.
>>
>>
>>
>>> I my case, I *cannot* distribute proj4 into my GDAL build
>>
>>
>>
>> I've the feeling you're abusing -1 for a reason which you mentionned
>> privately to me but in my opinion isn't really valid for the rest of the
>> community. But this is not an important enough topic for me to fight about
>> and create useless tensions among contributors.
>>
>>
>>
>> So I'm dropping the ball on this. For posterity, my patch is at:
>>
>> https://trac.osgeo.org/gdal/ticket/6882
>>
>>
>>
>>
>>
>> Even
>>
>>
>>
>> --
>>
>> Spatialys - Geospatial professional services
>>
>> http://www.spatialys.com
>>
>>
>> ___
>> gdal-dev mailing list
>> gdal-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> --
> ---+--
> I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
> light and sound - activate the windows |
> and watch the world go round - Rush| Geospatial Software Developer
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Howard Butler]

I'm +1 on this. I too found it confusing that Proj.4 worked in this way and no 
other libraries did in GDAL.


> I my case, I *cannot* distribute proj4 into my GDAL build and I *need* to 
> have the options to let the user decided if they want to add proj4 shared 
> libraries.

Ivan,

I don't understand the scenario here. It isn't a licensing thing to prevent you 
from distributing Proj.4, and what do you do in the case of all of the other 
shared libraries that are linked at compile time that GDAL uses?

Howard

> 
> On May 7, 2017, at 3:01 AM, Frank Warmerdam  wrote:
> 
> Even,
> 
> I'm +0 on this change.
> 
> On the one hand, I *like* the fact that the dlopen() approach means
> that adding libproj.so after the fact means that GDAL starts
> supporting coordinate system conversion without a rebuild.  On the
> other hand, it isn't clear why we do this only for libproj.
> 
> I have some appreciation for Ivan's stated concerns with regard to
> regression, but I don't think that should hold back cleanup in major
> new versions.
> 
> Best regards,
> Frank
> 
> 
> On Sat, May 6, 2017 at 12:09 PM, Even Rouault
>  wrote:
>> Ivan,
>> 
>> 
>> 
>>> I understand the good intention of cleaning up code but that should not
>> 
>>> remove functionality. You are breaking backward compatibility. What if
>> 
>>> someone updates GDAL in some installation, proj4 is there and it will not
>> 
>>> going to be used?
>> 
>> 
>> 
>> Well that would be documented in the release notes... There are a number of
>> breaks in each new release. That's not really different. And the way of
>> solving in the case you mention is rather simple.
>> 
>> 
>> 
>>> I my case, I *cannot* distribute proj4 into my GDAL build
>> 
>> 
>> 
>> I've the feeling you're abusing -1 for a reason which you mentionned
>> privately to me but in my opinion isn't really valid for the rest of the
>> community. But this is not an important enough topic for me to fight about
>> and create useless tensions among contributors.
>> 
>> 
>> 
>> So I'm dropping the ball on this. For posterity, my patch is at:
>> 
>> https://trac.osgeo.org/gdal/ticket/6882
>> 
>> 
>> 
>> 
>> 
>> Even
>> 
>> 
>> 
>> --
>> 
>> Spatialys - Geospatial professional services
>> 
>> http://www.spatialys.com
>> 
>> 
>> ___
>> gdal-dev mailing list
>> gdal-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
> 
> 
> 
> -- 
> ---+--
> I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
> light and sound - activate the windows |
> and watch the world go round - Rush| Geospatial Software Developer
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Frank Warmerdam]

Even,

I'm +0 on this change.

On the one hand, I *like* the fact that the dlopen() approach means
that adding libproj.so after the fact means that GDAL starts
supporting coordinate system conversion without a rebuild.  On the
other hand, it isn't clear why we do this only for libproj.

I have some appreciation for Ivan's stated concerns with regard to
regression, but I don't think that should hold back cleanup in major
new versions.

Best regards,
Frank


On Sat, May 6, 2017 at 12:09 PM, Even Rouault
 wrote:
> Ivan,
>
>
>
>> I understand the good intention of cleaning up code but that should not
>
>> remove functionality. You are breaking backward compatibility. What if
>
>> someone updates GDAL in some installation, proj4 is there and it will not
>
>> going to be used?
>
>
>
> Well that would be documented in the release notes... There are a number of
> breaks in each new release. That's not really different. And the way of
> solving in the case you mention is rather simple.
>
>
>
>> I my case, I *cannot* distribute proj4 into my GDAL build
>
>
>
> I've the feeling you're abusing -1 for a reason which you mentionned
> privately to me but in my opinion isn't really valid for the rest of the
> community. But this is not an important enough topic for me to fight about
> and create useless tensions among contributors.
>
>
>
> So I'm dropping the ball on this. For posterity, my patch is at:
>
> https://trac.osgeo.org/gdal/ticket/6882
>
>
>
>
>
> Even
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev



-- 
---+--
I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
light and sound - activate the windows |
and watch the world go round - Rush| Geospatial Software Developer
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Ivan Lucena]

-1


We have the options to build drivers against static and dynamic libraries of 
its SDKs, like HDF4,5 and openjpeg for example, using the same 
"--with-driver-name"


Why not keep that same option for proj4?


--with-proj4=-library-path


I have projects where I let user to decide if they want to download and install 
proj4 while using GDAL and I would prefer to keep that option.


Maybe with a compiler option we can basically "remove" the dynamic load when 
static linked proj4 was used. Config can translate the --with-proj4 into a C++ 
define, right?


Regards,

Ivan



From: gdal-dev <gdal-dev-boun...@lists.osgeo.org> on behalf of Kurt Schwehr 
<schw...@gmail.com>
Sent: Saturday, May 6, 2017 11:20:34 AM
To: Even Rouault
Cc: gdal dev
Subject: Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

+1

On May 6, 2017 4:58 AM, "Even Rouault" 
<even.roua...@spatialys.com<mailto:even.roua...@spatialys.com>> wrote:

Hi,



Currently the default mode of linking GDAL with proj.4 is to use dynamic 
loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the reason 
for that was that it could have make it easier to use an alternate projection 
engine, but apparently nobody cared enough to plug a new one, and it could be 
done with standard linking. One downside of the current mechanism (besides code 
complication) is that it requires to list the exact library name of proj4 in 
GDAL source code, which can change depending on the soname of proj4. And this 
can cause subtle issues like

https://trac.osgeo.org/gdal/ticket/6881



So I'd suggest just keeping standard linking mechanism, and renaming the 
current --with-static-proj4 configure flag as --with-proj4, as the current name 
is confusing, and making it the default behaviour.



Any thoughts ?



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Andrew C Aitchison]

On Sat, 6 May 2017, Even Rouault wrote:


I'm intending to drop support for proj.4 < 4.8 too to do some cleanups


Red Hat / Centos / Scientific Linux 6 use proj4 v4.8.x
so I have no problem with this.

--
Andrew C. Aitchison Cambridge, UK
and...@aitchison.me.uk


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Ivan Lucena]

HI Even,


I understand the good intention of cleaning up code but that should not remove 
functionality. You are breaking backward compatibility. What if someone updates 
GDAL in some installation, proj4 is there and it will not going to be used?


I my case, I *cannot* distribute proj4 into my GDAL build and I *need* to have 
the options to let the user decided if they want to add proj4 shared libraries.


Regards,

Ivan






From: Even Rouault <even.roua...@spatialys.com>
Sent: Saturday, May 6, 2017 1:35:18 PM
To: Ivan Lucena
Cc: Kurt Schwehr; gdal dev
Subject: Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?


On samedi 6 mai 2017 17:04:33 CEST Ivan Lucena wrote:

> -1

>

>

> We have the options to build drivers against static and dynamic libraries of

> its SDKs, like HDF4,5 and openjpeg for example, using the same

> "--with-driver-name"

>

>

> Why not keep that same option for proj4?



I think you are confusing things. For a good reason since the 
--with-static-proj4 name is confusing. --with-static-proj4 can link against 
libproj.a or libproj.so depending on which is available, which is classic build 
time linking. My suggestion is to keep this as the only option to use proj.4, 
and rename it --with-proj4 for more clarity and consistency



When this option is not specified, proj4 is optionaly loaded through 
dlopen()/LoadLibrary(), which is runtime linking and what I'm proposing to drop 
for simplicity.



HDF4, 5, openjpeg are only supported through build time linking.



>

>

> --with-proj4=-library-path

>

>

> I have projects where I let user to decide if they want to download and

> install proj4 while using GDAL and I would prefer to keep that option.

>



Well, that seem to be a rather particular use case. Given the small size of 
libproj regarding libgdal, you could always build against proj.



After my cleanup and removal of pre-4.8 support, ogrct.cpp is now down to 1110 
lines against 1311 before, and much easier to follow. And that save testing 
behaviour in --with-static-proj4 and --without-static-proj4 cases



>

> Maybe with a compiler option we can basically "remove" the dynamic load when

> static linked proj4 was used.



That's already the case. If currently you use --with-static-proj4, dynamic 
loading at runtime is disabled.



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Even Rouault]

Ivan,

> I understand the good intention of cleaning up code but that should not
> remove functionality. You are breaking backward compatibility. What if
> someone updates GDAL in some installation, proj4 is there and it will not
> going to be used?

Well that would be documented in the release notes... There are a number of 
breaks in each 
new release. That's not really different. And the way of solving in the case 
you mention is 
rather simple.

> I my case, I *cannot* distribute proj4 into my GDAL build

I've the feeling you're abusing -1 for a reason which you mentionned privately 
to me but in 
my opinion isn't really valid for the rest of the community. But this is not an 
important 
enough topic for me to fight about and create useless tensions among 
contributors.

So I'm dropping the ball on this. For posterity, my patch is at:
https://trac.osgeo.org/gdal/ticket/6882


Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Even Rouault]

On samedi 6 mai 2017 17:04:33 CEST Ivan Lucena wrote:
> -1
> 
> 
> We have the options to build drivers against static and dynamic libraries of
> its SDKs, like HDF4,5 and openjpeg for example, using the same
> "--with-driver-name"
> 
> 
> Why not keep that same option for proj4?

I think you are confusing things. For a good reason since the 
--with-static-proj4 name is 
confusing. --with-static-proj4 can link against libproj.a or libproj.so 
depending on which is 
available, which is classic build time linking. My suggestion is to keep this 
as the only option 
to use proj.4, and rename it --with-proj4 for more clarity and consistency

When this option is not specified, proj4 is optionaly loaded through 
dlopen()/LoadLibrary(), 
which is runtime linking and what I'm proposing to drop for simplicity.

HDF4, 5, openjpeg are only supported through build time linking. 

> 
> 
> --with-proj4=-library-path
> 
> 
> I have projects where I let user to decide if they want to download and
> install proj4 while using GDAL and I would prefer to keep that option.
> 

Well, that seem to be a rather particular use case. Given the small size of 
libproj regarding 
libgdal, you could always build against proj.

After my cleanup and removal of pre-4.8 support, ogrct.cpp is now down to 1110 
lines 
against 1311 before, and much easier to follow. And that save testing behaviour 
in --with-
static-proj4 and --without-static-proj4 cases

> 
> Maybe with a compiler option we can basically "remove" the dynamic load when
> static linked proj4 was used.

That's already the case. If currently you use --with-static-proj4, dynamic 
loading at runtime is 
disabled.

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Zachary Flamig]

+1

I’ve been tricked by this several times where I think I have gdal successfully 
compiled and then go to reproject something only to find out it can’t :-(

Zac


> On May 6, 2017, at 10:20 AM, Kurt Schwehr  wrote:
> 
> +1
> 
> On May 6, 2017 4:58 AM, "Even Rouault"  > wrote:
> Hi,
>  
> Currently the default mode of linking GDAL with proj.4 is to use dynamic 
> loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the 
> reason for that was that it could have make it easier to use an alternate 
> projection engine, but apparently nobody cared enough to plug a new one, and 
> it could be done with standard linking. One downside of the current mechanism 
> (besides code complication) is that it requires to list the exact library 
> name of proj4 in GDAL source code, which can change depending on the soname 
> of proj4. And this can cause subtle issues like
> https://trac.osgeo.org/gdal/ticket/6881 
> 
>  
> So I'd suggest just keeping standard linking mechanism, and renaming the 
> current --with-static-proj4 configure flag as --with-proj4, as the current 
> name is confusing, and making it the default behaviour.
>  
> Any thoughts ?
>  
> Even
>  
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com 
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org 
> https://lists.osgeo.org/mailman/listinfo/gdal-dev 
> 
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Even Rouault]

On samedi 6 mai 2017 13:58:06 CEST Even Rouault wrote:
> Hi,
> 
> Currently the default mode of linking GDAL with proj.4 is to use dynamic
> loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the
> reason for that was that it could have make it easier to use an alternate
> projection engine, but apparently nobody cared enough to plug a new one,
> and it could be done with standard linking. One downside of the current
> mechanism (besides code complication) is that it requires to list the exact
> library name of proj4 in GDAL source code, which can change depending on
> the soname of proj4. And this can cause subtle issues like
> https://trac.osgeo.org/gdal/ticket/6881
> 
> So I'd suggest just keeping standard linking mechanism, and renaming the
> current --with- static-proj4 configure flag as --with-proj4, as the current
> name is confusing, and making it the default behaviour.
> 
> Any thoughts ?

I'm intending to drop support for proj.4 < 4.8 too to do some cleanups

> 
> Even


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Kurt Schwehr]

+1

On May 6, 2017 4:58 AM, "Even Rouault"  wrote:

> Hi,
>
>
>
> Currently the default mode of linking GDAL with proj.4 is to use dynamic
> loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the
> reason for that was that it could have make it easier to use an alternate
> projection engine, but apparently nobody cared enough to plug a new one,
> and it could be done with standard linking. One downside of the current
> mechanism (besides code complication) is that it requires to list the exact
> library name of proj4 in GDAL source code, which can change depending on
> the soname of proj4. And this can cause subtle issues like
>
> https://trac.osgeo.org/gdal/ticket/6881
>
>
>
> So I'd suggest just keeping standard linking mechanism, and renaming the
> current --with-static-proj4 configure flag as --with-proj4, as the current
> name is confusing, and making it the default behaviour.
>
>
>
> Any thoughts ?
>
>
>
> Even
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

[Even Rouault]

Hi,

Currently the default mode of linking GDAL with proj.4 is to use dynamic 
loading mechanism 
(dlopen on Unix, LoadLibary on Windows). I believe the reason for that was that 
it could have 
make it easier to use an alternate projection engine, but apparently nobody 
cared enough to 
plug a new one, and it could be done with standard linking. One downside of the 
current 
mechanism (besides code complication) is that it requires to list the exact 
library name of 
proj4 in GDAL source code, which can change depending on the soname of proj4. 
And this can 
cause subtle issues like
https://trac.osgeo.org/gdal/ticket/6881

So I'd suggest just keeping standard linking mechanism, and renaming the 
current --with-
static-proj4 configure flag as --with-proj4, as the current name is confusing, 
and making it the 
default behaviour.

Any thoughts ?

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev