Re: Runtime loader issue

2018-05-10 Thread Steve Kargl 
On Wed, May 09, 2018 at 04:45:51PM -0700, Steve Kargl wrote:
> In review PR 228007, it came to my attention some individuals are
> mis-characterizing a FreeBSD loader issue as "gfortran's FreeBSD
> issue".  See

It seems we've had the same discussion 2 years ago.  My time flies.

ttps://lists.freebsd.org/pipermail/freebsd-ports/2016-August/104384.html

-- 
Steve
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Runtime loader issue

2018-05-10 Thread Wojciech Puchar 

just another - out of thousands - example that shared libraries are one
of the worst thing invented in computing.

Maybe except of single system wide shared library with constant interface.

On Thu, 10 May 2018, Gleb Popov wrote:


On Thu, May 10, 2018 at 2:45 AM, Steve Kargl <
s...@troutmask.apl.washington.edu> wrote:


In review PR 228007, it came to my attention some individuals are
mis-characterizing a FreeBSD loader issue as "gfortran's FreeBSD
issue".  See

https://lists.freebsd.org/pipermail/freebsd-fortran/2018-May/000124.html

The problem can be summarized by the following

% gfortran7 -o z h.f90
% ./z
/lib/libgcc_s.so.1: version GCC_4.8.0 required by \
  /usr/local/lib/gcc7/libgfortran.so.4 not found

gfortran7 is installed from ports/lang/gcc7.  This is not
a "gfortran's FreeBSD issue".  This is a FreeBSD loader issue.

Specifically, there is a shared library name clash.

% ldconfig -r | grep gcc_
 6:-lgcc_s.1 => /lib/libgcc_s.so.1
   716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1

% ldd z
z:
   libgfortran.so.4 => /usr/local/lib/gcc7/libgfortran.so.4 (0x200645000)
   libm.so.5 => /lib/libm.so.5 (0x200a17000)
   libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x200a4b000)
   libquadmath.so.0 => /usr/local/lib/gcc7/libquadmath.so.0 (0x200a63000)
   libc.so.7 => /lib/libc.so.7 (0x200ca3000)

So, the runtime loader finds 6 instead of 716, tries to link,
fails, and issues an error message.  There are a number ways to
fix this issue.

1) By far, the best solution would be to stop hijacking the libgcc
   name in libraries installed on FreeBSD that are not related to
   actual GCC software.

% ls -l /lib/libgcc* /usr/lib/libgcc*
(trimming lines)
/lib/libgcc_s.so.1
/usr/lib/libgcc.a@ -> libcompiler_rt.a
/usr/lib/libgcc_eh.a
/usr/lib/libgcc_eh_p.a
/usr/lib/libgcc_p.a@ -> libcompiler_rt_p.a
/usr/lib/libgcc_s.so@ -> ../../lib/libgcc_s.so.1

   Why not use libcompiler_rt_s.so.1 (or libclang_s.so.1)?  Yes, I'm
   aware that clang does not work on all archs and the ancient gcc
   lives on.

2) Given the expected push back againt solution 1), this solution
   proposes bumping the library version for /lib/libgcc_s.so.1 from
   1 to some larger value, say, 10.  It is unlikely that GCC will
   bump its shared library number anytime soon.  GCC bumped it from
   0 to 1 some 16 years ago.

   https://gcc.gnu.org/viewcvs/gcc?view=revision=43316

   This solution, however, papers over the general problem with
   name clashes.

3) This solution is to actually fix the runtime loader.  If an error
   occurs with loading a shared library, then iterate over the entries
   in the hints file to check to see if another hint would satisfy
   the linking.  Here, instead of issuing the above error, the loader
   would find entry 716, and load the correct libgcc_s.so.1.

   Admittedly, I haven't looked to see how difficult this solution
   would be.

4) Bump the shared library number of the individual ports.  As a proof
   of concept, I've done this with ports/lang/gcc6.

% cat /usr/ports/lang/gcc6/files/patch-t-slibgcc
--- libgcc/config/t-slibgcc.orig2018-05-08 12:47:42.334495000 -0700
+++ libgcc/config/t-slibgcc 2018-05-08 12:45:26.872312000 -0700
@@ -20,7 +20,7 @@

 SHLIB_EXT = .so
 SHLIB_SOLINK = @shlib_base_name@.so
-SHLIB_SOVERSION = 1
+SHLIB_SOVERSION = 2
 SHLIB_SONAME = @shlib_base_name@.so.$(SHLIB_SOVERSION)
 SHLIB_MAP = @shlib_map_file@
 SHLIB_OBJS = @shlib_objs@

% ldconfig -r | grep gcc_
 6:-lgcc_s.1 => /lib/libgcc_s.so.1
   716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1
   766:-lgcc_s.2 => /usr/local/lib/gcc6/libgcc_s.so.2

% gfortran6 -o z h.f90
% ./z
 hello
% ldd z
z:
   libgfortran.so.3 => /usr/local/lib/gcc6/libgfortran.so.3 (0x200645000)
   libm.so.5 => /lib/libm.so.5 (0x20096c000)
   libgcc_s.so.2 => /usr/local/lib/gcc6/libgcc_s.so.2 (0x2009a)
   libquadmath.so.0 => /usr/local/lib/gcc7/libquadmath.so.0 (0x200bb7000)
   libc.so.7 => /lib/libc.so.7 (0x200df7000)

   This works for this particular name conflict.  Hopefully, FreeBSD
   never needs to bump /lib/libgcc_s.so.1 to /lib/libgcc_s.so.2.  This,
   however, introduces an incompatibility with what is actually distributed
   by GCC.

Finally, can people stop referring to the above error as
"gfortran's FreeBSD issue".  This is a FreeBSD runtime loader issue.



Our libgcc also lacks some functionality compared to the original one:
https://reviews.freebsd.org/D11482



--
Steve
___
freebsd-hack...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"



__

Re: Runtime loader issue

2018-05-10 Thread Steve Kargl 
On Thu, May 10, 2018 at 06:21:51PM +0200, Tijl Coosemans wrote:
> On Wed, 9 May 2018 16:45:51 -0700 Steve Kargl 
>  wrote:
> > 
> > So, the runtime loader finds 6 instead of 716, tries to link, 
> > fails, and issues an error message.  There are a number ways to
> > fix this issue.
> > 
> > 1) By far, the best solution would be to stop hijacking the libgcc
> >name in libraries installed on FreeBSD that are not related to
> >actual GCC software.
> > 
> > % ls -l /lib/libgcc* /usr/lib/libgcc*
> > (trimming lines)
> > /lib/libgcc_s.so.1
> > /usr/lib/libgcc.a@ -> libcompiler_rt.a
> > /usr/lib/libgcc_eh.a
> > /usr/lib/libgcc_eh_p.a
> > /usr/lib/libgcc_p.a@ -> libcompiler_rt_p.a
> > /usr/lib/libgcc_s.so@ -> ../../lib/libgcc_s.so.1
> > 
> >Why not use libcompiler_rt_s.so.1 (or libclang_s.so.1)?  Yes, I'm
> >aware that clang does not work on all archs and the ancient gcc
> >lives on.
> > 
> > 2) Given the expected push back againt solution 1), this solution
> >proposes bumping the library version for /lib/libgcc_s.so.1 from
> >1 to some larger value, say, 10.  It is unlikely that GCC will
> >bump its shared library number anytime soon.  GCC bumped it from
> >0 to 1 some 16 years ago.
> > 
> >https://gcc.gnu.org/viewcvs/gcc?view=revision=43316
> > 
> >This solution, however, papers over the general problem with
> >name clashes.
> > 
> 
> libgcc_s.so implements the _Unwind_* API to handle stack unwinding.  This
> code is shared between all compilers and languages because the stack can
> contain frames from different compilers and languages.  So, you cannot
> change the name or version of the library.

Well, yes one could change the name and/or shared library number.
All those other tools would then need to be adapted to look for
a renamed or number-bumped library.  Yes, painful.

> I've been testing the attached patch in combination with the ports tree
> patch from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228046.
> 
> The patch makes three changes to /etc/rc.d/ldconfig:
> 1) Use 'sort -r' so /usr/local/lib/gcc7 appears before /usr/local/lib/gcc6.
> 2) Move hardcoded paths like /lib and /usr/lib to /etc/defaults/rc.conf
>so the order relative to other paths can be configured.
> 3) Change the order of ldconfig_local_dirs and ldconfig_paths so /lib and
>/usr/lib appear last.  This brings rc.d/ldconfig in line with compilers
>and rtld(1) which also search /lib and /usr/lib last.

This appears to work around the problem with libgcc_s.so.1. 
It would be a welcomed solution to so-called "gfortran's
FreeBSD issue", but is does not solve the problem with name
clashes.  It is possible to have two independent libraries
for independent projects where no shuffling of order in hints
will work.

-- 
Steve
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Runtime loader issue

2018-05-10 Thread Diane Bruce 
On Thu, May 10, 2018 at 08:15:22AM -0700, Steve Kargl wrote:
> On Thu, May 10, 2018 at 10:24:52AM -0400, Diane Bruce wrote:
> > On Thu, May 10, 2018 at 10:46:31AM +0300, Gleb Popov wrote:
> > > On Thu, May 10, 2018 at 2:45 AM, Steve Kargl <
> > > s...@troutmask.apl.washington.edu> wrote:
> > > 
...
> > 
> > Yep.
> > See https://wiki.freebsd.org/libgcc%20problem
> > 
> 
> Interesting page.  I was aware that in the past you tried working
> out a solution to the problem.  I did not realize you docoumented
> the issue.  A few corrections to your wiki page.
> 
> 1) The correct spelling of the name of the language is Fortran (with a
>capital F).  It has been the official standard spelling since Fortran
>90.

Fixed

> 
> 2) You have "... to always support quad math (*8) and ...".  Quad
>precision in gfortran is REAL(16) (the REAL*16 notation is nonstandard).

I think I got it right this time...
> 
> 3) "subsitute" is normally spelled with an extra 't'. :-)

OOOps ;) fixed
 
> > Very ;)
> 
> Just started reading the source code.  Don't scare the unwary. :-)

;)

> 
> -- 
> Steve

Diane
-- 
- d...@freebsd.org d...@db.net http://www.db.net/~db
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Runtime loader issue

2018-05-10 Thread Tijl Coosemans 
On Wed, 9 May 2018 16:45:51 -0700 Steve Kargl 
<s...@troutmask.apl.washington.edu> wrote:
> In review PR 228007, it came to my attention some individuals are
> mis-characterizing a FreeBSD loader issue as "gfortran's FreeBSD
> issue".  See
>  
> https://lists.freebsd.org/pipermail/freebsd-fortran/2018-May/000124.html
> 
> The problem can be summarized by the following 
> 
> % gfortran7 -o z h.f90
> % ./z
> /lib/libgcc_s.so.1: version GCC_4.8.0 required by \
>   /usr/local/lib/gcc7/libgfortran.so.4 not found
> 
> gfortran7 is installed from ports/lang/gcc7.  This is not 
> a "gfortran's FreeBSD issue".  This is a FreeBSD loader issue.
> 
> Specifically, there is a shared library name clash.
> 
> % ldconfig -r | grep gcc_
>  6:-lgcc_s.1 => /lib/libgcc_s.so.1
>716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1
> 
> % ldd z
> z:
>libgfortran.so.4 => /usr/local/lib/gcc7/libgfortran.so.4 (0x200645000)
>libm.so.5 => /lib/libm.so.5 (0x200a17000)
>libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x200a4b000)
>libquadmath.so.0 => /usr/local/lib/gcc7/libquadmath.so.0 (0x200a63000)
>libc.so.7 => /lib/libc.so.7 (0x200ca3000)
> 
> So, the runtime loader finds 6 instead of 716, tries to link, 
> fails, and issues an error message.  There are a number ways to
> fix this issue.
> 
> 1) By far, the best solution would be to stop hijacking the libgcc
>name in libraries installed on FreeBSD that are not related to
>actual GCC software.
> 
> % ls -l /lib/libgcc* /usr/lib/libgcc*
> (trimming lines)
> /lib/libgcc_s.so.1
> /usr/lib/libgcc.a@ -> libcompiler_rt.a
> /usr/lib/libgcc_eh.a
> /usr/lib/libgcc_eh_p.a
> /usr/lib/libgcc_p.a@ -> libcompiler_rt_p.a
> /usr/lib/libgcc_s.so@ -> ../../lib/libgcc_s.so.1
> 
>Why not use libcompiler_rt_s.so.1 (or libclang_s.so.1)?  Yes, I'm
>aware that clang does not work on all archs and the ancient gcc
>lives on.
> 
> 2) Given the expected push back againt solution 1), this solution
>proposes bumping the library version for /lib/libgcc_s.so.1 from
>1 to some larger value, say, 10.  It is unlikely that GCC will
>bump its shared library number anytime soon.  GCC bumped it from
>0 to 1 some 16 years ago.
> 
>https://gcc.gnu.org/viewcvs/gcc?view=revision=43316
> 
>This solution, however, papers over the general problem with
>name clashes.
> 
> 3) This solution is to actually fix the runtime loader.  If an error
>occurs with loading a shared library, then iterate over the entries
>in the hints file to check to see if another hint would satisfy
>the linking.  Here, instead of issuing the above error, the loader
>would find entry 716, and load the correct libgcc_s.so.1.
> 
>Admittedly, I haven't looked to see how difficult this solution
>would be.
>
> 4) Bump the shared library number of the individual ports.  As a proof
>of concept, I've done this with ports/lang/gcc6.
> 
> % cat /usr/ports/lang/gcc6/files/patch-t-slibgcc 
> --- libgcc/config/t-slibgcc.orig2018-05-08 12:47:42.334495000 -0700
> +++ libgcc/config/t-slibgcc 2018-05-08 12:45:26.872312000 -0700
> @@ -20,7 +20,7 @@
>  
>  SHLIB_EXT = .so
>  SHLIB_SOLINK = @shlib_base_name@.so
> -SHLIB_SOVERSION = 1
> +SHLIB_SOVERSION = 2
>  SHLIB_SONAME = @shlib_base_name@.so.$(SHLIB_SOVERSION)
>  SHLIB_MAP = @shlib_map_file@
>  SHLIB_OBJS = @shlib_objs@
> 
> % ldconfig -r | grep gcc_
>  6:-lgcc_s.1 => /lib/libgcc_s.so.1
>716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1
>766:-lgcc_s.2 => /usr/local/lib/gcc6/libgcc_s.so.2
> 
> % gfortran6 -o z h.f90
> % ./z
>  hello
> % ldd z
> z:
>libgfortran.so.3 => /usr/local/lib/gcc6/libgfortran.so.3 (0x200645000)
>libm.so.5 => /lib/libm.so.5 (0x20096c000)
>libgcc_s.so.2 => /usr/local/lib/gcc6/libgcc_s.so.2 (0x2009a)
>libquadmath.so.0 => /usr/local/lib/gcc7/libquadmath.so.0 (0x200bb7000)
>libc.so.7 => /lib/libc.so.7 (0x200df7000)
> 
>This works for this particular name conflict.  Hopefully, FreeBSD
>never needs to bump /lib/libgcc_s.so.1 to /lib/libgcc_s.so.2.  This,
>however, introduces an incompatibility with what is actually distributed
>by GCC.
> 
> Finally, can people stop referring to the above error as
> "gfortran's FreeBSD issue".  This is a FreeBSD runtime loader issue.

libgcc_s.so implements the _Unwind_* API to handle stack unwinding.  This
code is shared between all compilers and languages because the stack can
contain frames from different compilers and languages.  So, you cannot
change the name or version of the library.

I

Re: Runtime loader issue

2018-05-10 Thread Steve Kargl 
On Thu, May 10, 2018 at 10:24:52AM -0400, Diane Bruce wrote:
> On Thu, May 10, 2018 at 10:46:31AM +0300, Gleb Popov wrote:
> > On Thu, May 10, 2018 at 2:45 AM, Steve Kargl <
> > s...@troutmask.apl.washington.edu> wrote:
> > 
> > > In review PR 228007, it came to my attention some individuals are
> > > mis-characterizing a FreeBSD loader issue as "gfortran's FreeBSD
> > > issue".  See
> 
> Indeed. I've tried to make it clear it is a name conflict with libgcc
> in my own bug reports on the subject.
> 
> > >
> > > https://lists.freebsd.org/pipermail/freebsd-fortran/2018-May/000124.html
> > >
> > > The problem can be summarized by the following
> ...
> > > gfortran7 is installed from ports/lang/gcc7.  This is not
> > > a "gfortran's FreeBSD issue".  This is a FreeBSD loader issue.
> > >
> > > Specifically, there is a shared library name clash.
> > >
> > > % ldconfig -r | grep gcc_
> > >  6:-lgcc_s.1 => /lib/libgcc_s.so.1
> > >716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1
> 
> Yep.
> See https://wiki.freebsd.org/libgcc%20problem
> 

Interesting page.  I was aware that in the past you tried working
out a solution to the problem.  I did not realize you docoumented
the issue.  A few corrections to your wiki page.

1) The correct spelling of the name of the language is Fortran (with a
   capital F).  It has been the official standard spelling since Fortran
   90.

2) You have "... to always support quad math (*8) and ...".  Quad
   precision in gfortran is REAL(16) (the REAL*16 notation is nonstandard).

3) "subsitute" is normally spelled with an extra 't'. :-)


> > > So, the runtime loader finds 6 instead of 716, tries to link,
> > > fails, and issues an error message.  There are a number ways to
> > > fix this issue.
> > >
> > > 1) By far, the best solution would be to stop hijacking the libgcc
> > >name in libraries installed on FreeBSD that are not related to
> > >actual GCC software.
> 
> Agreed, however this has the side effect of meaning conflicts with libraries
> between clang and gcc libs. Notably gfortran and flang use different
> conventions for I/O :(
> 
> See http://people.FreeeBSD.org/~db/fortran_libs.txt
> 

Page doesn't appear to exist?  If I go to 

https://people.freebsd.org/homepage.html

you're not listed.

> > >Why not use libcompiler_rt_s.so.1 (or libclang_s.so.1)?  Yes, I'm
> > >aware that clang does not work on all archs and the ancient gcc
> > >lives on.
> > >
> 
> I've argued for this as well and frankly I still think it is the best
> solution all around. 
> 
> > > 2) Given the expected push back againt solution 1), this solution
> > >proposes bumping the library version for /lib/libgcc_s.so.1 from
> > >1 to some larger value, say, 10.  It is unlikely that GCC will
> > >bump its shared library number anytime soon.  GCC bumped it from
> > >0 to 1 some 16 years ago.
> > >
> > >https://gcc.gnu.org/viewcvs/gcc?view=revision=43316
> > >
> > >This solution, however, papers over the general problem with
> > >name clashes.
> 
> Yep. I know work is slowly being done to modernise our libgcc already
> but the toolchain folks are busy...
> 
> > >
> > > 3) This solution is to actually fix the runtime loader.  If an error
> > >occurs with loading a shared library, then iterate over the entries
> > >in the hints file to check to see if another hint would satisfy
> > >the linking.  Here, instead of issuing the above error, the loader
> > >would find entry 716, and load the correct libgcc_s.so.1.
> 
> This breaks if the bad libgcc is already linked. We'd have to rip
> out the original bindings at run time, then re-bind to a new libgcc. 
> I looked at the rtld code months ago. Nasty and silly.
> 
> > >
> > >Admittedly, I haven't looked to see how difficult this solution
> > >would be.
> 
> Very ;)

Just started reading the source code.  Don't scare the unwary. :-)

-- 
Steve
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Runtime loader issue

2018-05-10 Thread Diane Bruce 
On Thu, May 10, 2018 at 10:46:31AM +0300, Gleb Popov wrote:
> On Thu, May 10, 2018 at 2:45 AM, Steve Kargl <
> s...@troutmask.apl.washington.edu> wrote:
> 
> > In review PR 228007, it came to my attention some individuals are
> > mis-characterizing a FreeBSD loader issue as "gfortran's FreeBSD
> > issue".  See

Indeed. I've tried to make it clear it is a name conflict with libgcc
in my own bug reports on the subject.

> >
> > https://lists.freebsd.org/pipermail/freebsd-fortran/2018-May/000124.html
> >
> > The problem can be summarized by the following
...
> > gfortran7 is installed from ports/lang/gcc7.  This is not
> > a "gfortran's FreeBSD issue".  This is a FreeBSD loader issue.
> >
> > Specifically, there is a shared library name clash.
> >
> > % ldconfig -r | grep gcc_
> >  6:-lgcc_s.1 => /lib/libgcc_s.so.1
> >716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1

Yep.
See https://wiki.freebsd.org/libgcc%20problem

> >
> > So, the runtime loader finds 6 instead of 716, tries to link,
> > fails, and issues an error message.  There are a number ways to
> > fix this issue.
> >
> > 1) By far, the best solution would be to stop hijacking the libgcc
> >name in libraries installed on FreeBSD that are not related to
> >actual GCC software.

Agreed, however this has the side effect of meaning conflicts with libraries
between clang and gcc libs. Notably gfortran and flang use different
conventions for I/O :(

See http://people.FreeeBSD.org/~db/fortran_libs.txt

> >
> >Why not use libcompiler_rt_s.so.1 (or libclang_s.so.1)?  Yes, I'm
> >aware that clang does not work on all archs and the ancient gcc
> >lives on.
> >

I've argued for this as well and frankly I still think it is the best
solution all around. 

> > 2) Given the expected push back againt solution 1), this solution
> >proposes bumping the library version for /lib/libgcc_s.so.1 from
> >1 to some larger value, say, 10.  It is unlikely that GCC will
> >bump its shared library number anytime soon.  GCC bumped it from
> >0 to 1 some 16 years ago.
> >
> >https://gcc.gnu.org/viewcvs/gcc?view=revision=43316
> >
> >This solution, however, papers over the general problem with
> >name clashes.

Yep. I know work is slowly being done to modernise our libgcc already
but the toolchain folks are busy...

> >
> > 3) This solution is to actually fix the runtime loader.  If an error
> >occurs with loading a shared library, then iterate over the entries
> >in the hints file to check to see if another hint would satisfy
> >the linking.  Here, instead of issuing the above error, the loader
> >would find entry 716, and load the correct libgcc_s.so.1.

This breaks if the bad libgcc is already linked. We'd have to rip
out the original bindings at run time, then re-bind to a new libgcc. 
I looked at the rtld code months ago. Nasty and silly.


> >
> >Admittedly, I haven't looked to see how difficult this solution
> >would be.


Very ;)

> >
> > 4) Bump the shared library number of the individual ports.  As a proof
> >of concept, I've done this with ports/lang/gcc6.
> >
...
> >
> > Finally, can people stop referring to the above error as
> > "gfortran's FreeBSD issue".  This is a FreeBSD runtime loader issue.

Yes, please. I tracked it down to libgcc months ago, made my findings
generally available and yet people are still calling it a libgfortran problem.

> >
> 
> Our libgcc also lacks some functionality compared to the original one:
> https://reviews.freebsd.org/D11482

Yes.


Diane
-- 
- d...@freebsd.org d...@db.net http://www.db.net/~db
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Runtime loader issue

2018-05-10 Thread Gleb Popov 
On Thu, May 10, 2018 at 2:45 AM, Steve Kargl <
s...@troutmask.apl.washington.edu> wrote:

> In review PR 228007, it came to my attention some individuals are
> mis-characterizing a FreeBSD loader issue as "gfortran's FreeBSD
> issue".  See
>
> https://lists.freebsd.org/pipermail/freebsd-fortran/2018-May/000124.html
>
> The problem can be summarized by the following
>
> % gfortran7 -o z h.f90
> % ./z
> /lib/libgcc_s.so.1: version GCC_4.8.0 required by \
>   /usr/local/lib/gcc7/libgfortran.so.4 not found
>
> gfortran7 is installed from ports/lang/gcc7.  This is not
> a "gfortran's FreeBSD issue".  This is a FreeBSD loader issue.
>
> Specifically, there is a shared library name clash.
>
> % ldconfig -r | grep gcc_
>  6:-lgcc_s.1 => /lib/libgcc_s.so.1
>716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1
>
> % ldd z
> z:
>libgfortran.so.4 => /usr/local/lib/gcc7/libgfortran.so.4 (0x200645000)
>libm.so.5 => /lib/libm.so.5 (0x200a17000)
>libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x200a4b000)
>libquadmath.so.0 => /usr/local/lib/gcc7/libquadmath.so.0 (0x200a63000)
>libc.so.7 => /lib/libc.so.7 (0x200ca3000)
>
> So, the runtime loader finds 6 instead of 716, tries to link,
> fails, and issues an error message.  There are a number ways to
> fix this issue.
>
> 1) By far, the best solution would be to stop hijacking the libgcc
>name in libraries installed on FreeBSD that are not related to
>actual GCC software.
>
> % ls -l /lib/libgcc* /usr/lib/libgcc*
> (trimming lines)
> /lib/libgcc_s.so.1
> /usr/lib/libgcc.a@ -> libcompiler_rt.a
> /usr/lib/libgcc_eh.a
> /usr/lib/libgcc_eh_p.a
> /usr/lib/libgcc_p.a@ -> libcompiler_rt_p.a
> /usr/lib/libgcc_s.so@ -> ../../lib/libgcc_s.so.1
>
>Why not use libcompiler_rt_s.so.1 (or libclang_s.so.1)?  Yes, I'm
>aware that clang does not work on all archs and the ancient gcc
>lives on.
>
> 2) Given the expected push back againt solution 1), this solution
>proposes bumping the library version for /lib/libgcc_s.so.1 from
>1 to some larger value, say, 10.  It is unlikely that GCC will
>bump its shared library number anytime soon.  GCC bumped it from
>0 to 1 some 16 years ago.
>
>https://gcc.gnu.org/viewcvs/gcc?view=revision=43316
>
>This solution, however, papers over the general problem with
>name clashes.
>
> 3) This solution is to actually fix the runtime loader.  If an error
>occurs with loading a shared library, then iterate over the entries
>in the hints file to check to see if another hint would satisfy
>the linking.  Here, instead of issuing the above error, the loader
>would find entry 716, and load the correct libgcc_s.so.1.
>
>Admittedly, I haven't looked to see how difficult this solution
>would be.
>
> 4) Bump the shared library number of the individual ports.  As a proof
>of concept, I've done this with ports/lang/gcc6.
>
> % cat /usr/ports/lang/gcc6/files/patch-t-slibgcc
> --- libgcc/config/t-slibgcc.orig2018-05-08 12:47:42.334495000 -0700
> +++ libgcc/config/t-slibgcc 2018-05-08 12:45:26.872312000 -0700
> @@ -20,7 +20,7 @@
>
>  SHLIB_EXT = .so
>  SHLIB_SOLINK = @shlib_base_name@.so
> -SHLIB_SOVERSION = 1
> +SHLIB_SOVERSION = 2
>  SHLIB_SONAME = @shlib_base_name@.so.$(SHLIB_SOVERSION)
>  SHLIB_MAP = @shlib_map_file@
>  SHLIB_OBJS = @shlib_objs@
>
> % ldconfig -r | grep gcc_
>  6:-lgcc_s.1 => /lib/libgcc_s.so.1
>716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1
>766:-lgcc_s.2 => /usr/local/lib/gcc6/libgcc_s.so.2
>
> % gfortran6 -o z h.f90
> % ./z
>  hello
> % ldd z
> z:
>libgfortran.so.3 => /usr/local/lib/gcc6/libgfortran.so.3 (0x200645000)
>libm.so.5 => /lib/libm.so.5 (0x20096c000)
>libgcc_s.so.2 => /usr/local/lib/gcc6/libgcc_s.so.2 (0x2009a)
>libquadmath.so.0 => /usr/local/lib/gcc7/libquadmath.so.0 (0x200bb7000)
>libc.so.7 => /lib/libc.so.7 (0x200df7000)
>
>This works for this particular name conflict.  Hopefully, FreeBSD
>never needs to bump /lib/libgcc_s.so.1 to /lib/libgcc_s.so.2.  This,
>however, introduces an incompatibility with what is actually distributed
>by GCC.
>
> Finally, can people stop referring to the above error as
> "gfortran's FreeBSD issue".  This is a FreeBSD runtime loader issue.
>

Our libgcc also lacks some functionality compared to the original one:
https://reviews.freebsd.org/D11482


> --
> Steve
> ___
> freebsd-hack...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
>
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Runtime loader issue

2018-05-09 Thread Steve Kargl 
In review PR 228007, it came to my attention some individuals are
mis-characterizing a FreeBSD loader issue as "gfortran's FreeBSD
issue".  See
 
https://lists.freebsd.org/pipermail/freebsd-fortran/2018-May/000124.html

The problem can be summarized by the following 

% gfortran7 -o z h.f90
% ./z
/lib/libgcc_s.so.1: version GCC_4.8.0 required by \
  /usr/local/lib/gcc7/libgfortran.so.4 not found

gfortran7 is installed from ports/lang/gcc7.  This is not 
a "gfortran's FreeBSD issue".  This is a FreeBSD loader issue.

Specifically, there is a shared library name clash.

% ldconfig -r | grep gcc_
 6:-lgcc_s.1 => /lib/libgcc_s.so.1
   716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1

% ldd z
z:
   libgfortran.so.4 => /usr/local/lib/gcc7/libgfortran.so.4 (0x200645000)
   libm.so.5 => /lib/libm.so.5 (0x200a17000)
   libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x200a4b000)
   libquadmath.so.0 => /usr/local/lib/gcc7/libquadmath.so.0 (0x200a63000)
   libc.so.7 => /lib/libc.so.7 (0x200ca3000)

So, the runtime loader finds 6 instead of 716, tries to link, 
fails, and issues an error message.  There are a number ways to
fix this issue.

1) By far, the best solution would be to stop hijacking the libgcc
   name in libraries installed on FreeBSD that are not related to
   actual GCC software.

% ls -l /lib/libgcc* /usr/lib/libgcc*
(trimming lines)
/lib/libgcc_s.so.1
/usr/lib/libgcc.a@ -> libcompiler_rt.a
/usr/lib/libgcc_eh.a
/usr/lib/libgcc_eh_p.a
/usr/lib/libgcc_p.a@ -> libcompiler_rt_p.a
/usr/lib/libgcc_s.so@ -> ../../lib/libgcc_s.so.1

   Why not use libcompiler_rt_s.so.1 (or libclang_s.so.1)?  Yes, I'm
   aware that clang does not work on all archs and the ancient gcc
   lives on.

2) Given the expected push back againt solution 1), this solution
   proposes bumping the library version for /lib/libgcc_s.so.1 from
   1 to some larger value, say, 10.  It is unlikely that GCC will
   bump its shared library number anytime soon.  GCC bumped it from
   0 to 1 some 16 years ago.

   https://gcc.gnu.org/viewcvs/gcc?view=revision=43316

   This solution, however, papers over the general problem with
   name clashes.

3) This solution is to actually fix the runtime loader.  If an error
   occurs with loading a shared library, then iterate over the entries
   in the hints file to check to see if another hint would satisfy
   the linking.  Here, instead of issuing the above error, the loader
   would find entry 716, and load the correct libgcc_s.so.1.

   Admittedly, I haven't looked to see how difficult this solution
   would be.
   
4) Bump the shared library number of the individual ports.  As a proof
   of concept, I've done this with ports/lang/gcc6.

% cat /usr/ports/lang/gcc6/files/patch-t-slibgcc 
--- libgcc/config/t-slibgcc.orig2018-05-08 12:47:42.334495000 -0700
+++ libgcc/config/t-slibgcc 2018-05-08 12:45:26.872312000 -0700
@@ -20,7 +20,7 @@
 
 SHLIB_EXT = .so
 SHLIB_SOLINK = @shlib_base_name@.so
-SHLIB_SOVERSION = 1
+SHLIB_SOVERSION = 2
 SHLIB_SONAME = @shlib_base_name@.so.$(SHLIB_SOVERSION)
 SHLIB_MAP = @shlib_map_file@
 SHLIB_OBJS = @shlib_objs@

% ldconfig -r | grep gcc_
 6:-lgcc_s.1 => /lib/libgcc_s.so.1
   716:-lgcc_s.1 => /usr/local/lib/gcc7/libgcc_s.so.1
   766:-lgcc_s.2 => /usr/local/lib/gcc6/libgcc_s.so.2

% gfortran6 -o z h.f90
% ./z
 hello
% ldd z
z:
   libgfortran.so.3 => /usr/local/lib/gcc6/libgfortran.so.3 (0x200645000)
   libm.so.5 => /lib/libm.so.5 (0x20096c000)
   libgcc_s.so.2 => /usr/local/lib/gcc6/libgcc_s.so.2 (0x2009a)
   libquadmath.so.0 => /usr/local/lib/gcc7/libquadmath.so.0 (0x200bb7000)
   libc.so.7 => /lib/libc.so.7 (0x200df7000)

   This works for this particular name conflict.  Hopefully, FreeBSD
   never needs to bump /lib/libgcc_s.so.1 to /lib/libgcc_s.so.2.  This,
   however, introduces an incompatibility with what is actually distributed
   by GCC.

Finally, can people stop referring to the above error as
"gfortran's FreeBSD issue".  This is a FreeBSD runtime loader issue.

-- 
Steve
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"