Re: plmerge core dumps...

[Riccardo Mottola]

Hi!

On 13/02/2020 10:44, Andreas Fink wrote:

This is not a problem of gnustep but of objectiveC.
Yes its a pain that the standard linker doesnt work. But it is what it 
is. until someone fixes the standard linker we can't do much.
This is mainly given by the fact that modern ObjC support in gcc is 
still missing. clang does much better here but there where still some 
bugs in clang before version 8 which can get you into trouble.


So golden rule is clang >= 8.0 and linker = ld.gold and it will work.



this is very strange, since I on OpenBSD with clang 8, I had to force 
bfd to get something usable! Just the opposite...


LDFLAGS=-fuse-ld=bfd

Riccardo

Re: plmerge core dumps...

[David Chisnall]

On 13/02/2020 16:51, Richard Frith-Macdonald wrote:

Thanks for that information.

My (limited) understanding of -r is that it's for incremental linking, where 
the file produced by the linker can itself be used to be linked into an 
executable.


Correct.  The v2 ABI depends on a couple of interesting features:

1. The linker creates __start_{section} and __stop_{section} symbols 
within each linked output that give the range of a section.
2. Some COMDAT merging so that the function that calls the Objective-C 
load function is deduplicated and so is the pointer to it in the .ctors 
or .init section (which causes the run-time linker to call it) and the 
data structure that it passes to the linker.



So is the problem then in the generation of the subproject rather than in the 
final linking?


It appears as if BFD ld is resolving one or both of these too early, so 
the set of sections provided in the final link does not include the ones 
from the subproject.  I haven't looked too closely which part is failing.



If so, perhaps we could solve this by altering gnustep-make to add 
'-fuse-ld=gold' to the command line used to build a subproject (and perhaps 
when building a bundle)?


If GNUstep Make had the conventional configure/build split, then I'd 
recommend adding a check that the linker was BFD and aborting if so.  I 
don't want to enforce gold, because LLD also works fine (as does 
LINK.EXE or LLD-LINK.EXE on Windows).  On many platforms, LLD is the 
default linker, it is only GNU/Linux where the default linker does not work.


David

Re: plmerge core dumps...

[Richard Frith-Macdonald]

> On 13 Feb 2020, at 14:15, David Chisnall  wrote:
> 
> On 13/02/2020 10:35, Richard Frith-Macdonald wrote:
>> Is it a bug in the linker?
> 
> It is, but it is a bug that shows up *only* with ld -r.  LLD did not support 
> ld -r for a long time and that made no difference to over 90% of the things 
> in the FreeBSD ports collection.  It is a very rarely used corner case for 
> the linker and so likely to be very poorly tested (it also has quite poorly 
> defined semantics, which is part of the problem).
> 
> GNUstep Make uses this for 'subprojects'.  We hit this early because -base 
> Additions is a subproject.  These are compiled as .o files and then linked to 
> form a single .o file using ld -r.  Most modern build systems would typically 
> just pass the list of .o files to the parent project to link, but GNUstep 
> Make tries to create a single subproject.o.
> 
> It is good that -base tests this corner case of -make, because it would 
> otherwise hit external users, but I am not convinced by the value of this 
> option in GNUstep Make overall (though, as I have said before, I am also not 
> convinced of the value of investing time and engineering resources in 
> maintaining a custom build system that does not compose well with any other 
> project's build system).

Thanks for that information.

My (limited) understanding of -r is that it's for incremental linking, where 
the file produced by the linker can itself be used to be linked into an 
executable. 

So is the problem then in the generation of the subproject rather than in the 
final linking?

If so, perhaps we could solve this by altering gnustep-make to add 
'-fuse-ld=gold' to the command line used to build a subproject (and perhaps 
when building a bundle)? 

Re: plmerge core dumps...

[Johannes Brakensiek]

Hey guys,

On 13 Feb 2020, at 15:31, Gregory Casamento wrote:


Another question.  How do i switch to ld.gold?


as I’ve written multiple times, these links describe how to get a 
working setup (and how to use ld.gold):


http://wiki.gnustep.org/index.php/Building_GNUstep_under_Debian_FreeBSD
https://github.com/plaurent/gnustep-build/

It works, but using libobjc2 runtime version 2.0 has the issues 
described in other mails about loading bundles/frameworks (no problems 
when using runtime version 1.9). Where these issues derive from we are 
still examining.


Regards
Johannes

Re: plmerge core dumps...

[David Chisnall]

-fuse-ld=gold (or -fuse-ld=lld for lld).

LLD is the system linker on most *NIX platforms (Android, *BSD).  On 
platforms where BFD is the default, gold has been part of the binutils 
package that installs BFD for quite a long time and so the -fuse-ld= 
flag will instruct gcc or clang to select it.


David

On 13/02/2020 14:31, Gregory Casamento wrote:
An observation.  I only see this on subsequent installs.  The first 
always goes ok.


Another question.  How do i switch to ld.gold?

On Thu, Feb 13, 2020, 9:16 AM David Chisnall > wrote:


On 13/02/2020 10:35, Richard Frith-Macdonald wrote:
 > Is it a bug in the linker?

It is, but it is a bug that shows up *only* with ld -r.  LLD did not
support ld -r for a long time and that made no difference to over
90% of
the things in the FreeBSD ports collection.  It is a very rarely used
corner case for the linker and so likely to be very poorly tested (it
also has quite poorly defined semantics, which is part of the problem).

GNUstep Make uses this for 'subprojects'.  We hit this early because
-base Additions is a subproject.  These are compiled as .o files and
then linked to form a single .o file using ld -r.  Most modern build
systems would typically just pass the list of .o files to the parent
project to link, but GNUstep Make tries to create a single subproject.o.

It is good that -base tests this corner case of -make, because it would
otherwise hit external users, but I am not convinced by the value of
this option in GNUstep Make overall (though, as I have said before,
I am
also not convinced of the value of investing time and engineering
resources in maintaining a custom build system that does not compose
well with any other project's build system).

David

Re: plmerge core dumps...

[Gregory Casamento]

An observation.  I only see this on subsequent installs.  The first always
goes ok.

Another question.  How do i switch to ld.gold?

On Thu, Feb 13, 2020, 9:16 AM David Chisnall 
wrote:

> On 13/02/2020 10:35, Richard Frith-Macdonald wrote:
> > Is it a bug in the linker?
>
> It is, but it is a bug that shows up *only* with ld -r.  LLD did not
> support ld -r for a long time and that made no difference to over 90% of
> the things in the FreeBSD ports collection.  It is a very rarely used
> corner case for the linker and so likely to be very poorly tested (it
> also has quite poorly defined semantics, which is part of the problem).
>
> GNUstep Make uses this for 'subprojects'.  We hit this early because
> -base Additions is a subproject.  These are compiled as .o files and
> then linked to form a single .o file using ld -r.  Most modern build
> systems would typically just pass the list of .o files to the parent
> project to link, but GNUstep Make tries to create a single subproject.o.
>
> It is good that -base tests this corner case of -make, because it would
> otherwise hit external users, but I am not convinced by the value of
> this option in GNUstep Make overall (though, as I have said before, I am
> also not convinced of the value of investing time and engineering
> resources in maintaining a custom build system that does not compose
> well with any other project's build system).
>
> David
>
>

Re: plmerge core dumps...

[David Chisnall]

On 13/02/2020 10:35, Richard Frith-Macdonald wrote:

Is it a bug in the linker?


It is, but it is a bug that shows up *only* with ld -r.  LLD did not 
support ld -r for a long time and that made no difference to over 90% of 
the things in the FreeBSD ports collection.  It is a very rarely used 
corner case for the linker and so likely to be very poorly tested (it 
also has quite poorly defined semantics, which is part of the problem).


GNUstep Make uses this for 'subprojects'.  We hit this early because 
-base Additions is a subproject.  These are compiled as .o files and 
then linked to form a single .o file using ld -r.  Most modern build 
systems would typically just pass the list of .o files to the parent 
project to link, but GNUstep Make tries to create a single subproject.o.


It is good that -base tests this corner case of -make, because it would 
otherwise hit external users, but I am not convinced by the value of 
this option in GNUstep Make overall (though, as I have said before, I am 
also not convinced of the value of investing time and engineering 
resources in maintaining a custom build system that does not compose 
well with any other project's build system).


David

Re: plmerge core dumps...

[Andreas Fink]

> On 13 Feb 2020, at 11:35, Richard Frith-Macdonald 
>  wrote:
> 
> 
> 
>> On 13 Feb 2020, at 09:44, Andreas Fink  wrote:
>> 
>> This is not a problem of gnustep but of objectiveC.
>> Yes its a pain that the standard linker doesnt work. But it is what it is. 
>> until someone fixes the standard linker we can't do much.
> 
> I'm not sure where the responsibility lies (maybe nobody knows and that's why 
> it exists):
> Is it a bug in the linker?

yes

> Is it a bug in clang for not invoking the linker correctly?

no 

> Is it a bug in gnustep-make for not passing some extra flag to clang to tell 
> it how to invoke the linker correctly?

no

> 
>> This is mainly given by the fact that modern ObjC support in gcc is still 
>> missing. clang does much better here but there where still some bugs in 
>> clang before version 8 which can get you into trouble.
>> 
>> So golden rule is clang >= 8.0 and linker = ld.gold and it will work.
> 
> Yes, that works for me to build base/gui and quite a lot of other stuff, but 
> not all my code by any means:  I find there are other libraries/tools that 
> fail to link using ld.gold
> Until I can figure out what to do about that, I can't migrate the codebase 
> where I work to use the latest runtime :-(

as far as other libraries  / tools go,  the requirement for ld.gold is only 
there for ObjectiveC code. If you link C code thats not an issue.

Re: plmerge core dumps...

[Richard Frith-Macdonald]

> On 13 Feb 2020, at 09:44, Andreas Fink  wrote:
> 
> This is not a problem of gnustep but of objectiveC.
> Yes its a pain that the standard linker doesnt work. But it is what it is. 
> until someone fixes the standard linker we can't do much.

I'm not sure where the responsibility lies (maybe nobody knows and that's why 
it exists):
Is it a bug in the linker?
Is it a bug in clang for not invoking the linker correctly?
Is it a bug in gnustep-make for not passing some extra flag to clang to tell it 
how to invoke the linker correctly?

> This is mainly given by the fact that modern ObjC support in gcc is still 
> missing. clang does much better here but there where still some bugs in clang 
> before version 8 which can get you into trouble.
> 
> So golden rule is clang >= 8.0 and linker = ld.gold and it will work.

Yes, that works for me to build base/gui and quite a lot of other stuff, but 
not all my code by any means:  I find there are other libraries/tools that fail 
to link using ld.gold
Until I can figure out what to do about that, I can't migrate the codebase 
where I work to use the latest runtime :-(

Re: plmerge core dumps...

[Andreas Fink]

This is not a problem of gnustep but of objectiveC.
Yes its a pain that the standard linker doesnt work. But it is what it is. 
until someone fixes the standard linker we can't do much.
This is mainly given by the fact that modern ObjC support in gcc is still 
missing. clang does much better here but there where still some bugs in clang 
before version 8 which can get you into trouble.

So golden rule is clang >= 8.0 and linker = ld.gold and it will work.



> On 13 Feb 2020, at 10:37, Gregory Casamento  wrote:
> 
> So, what we are, essentially, saying is that GNUstep can't work with the 
> standard Linux linker.   I'm wondering in whose mind this is acceptable.
> 
> GC
> 
> On Wed, Feb 12, 2020 at 9:32 AM Gregory Casamento  > wrote:
> Facepalm this is happening every time I rebuild.  Ugh!  I really can't delete 
> my whole installation every time.
> 
> On Tue, Feb 11, 2020 at 8:56 AM Richard Frith-Macdonald 
> mailto:rich...@frithmacdonald.me.uk>> wrote:
> 
> 
> > On 11 Feb 2020, at 13:47, David Chisnall  > > wrote:
> > 
> > On 11/02/2020 12:30, Richard Frith-Macdonald wrote:
> >> clang -v reported that the normal, system linker was being used
> > 
> > FYI: On most GNU/Linux platforms, BFD is the 'normal, system linker'. For 
> > example:
> > 
> > $ ld -v
> > GNU ld (GNU Binutils for Debian) 2.31.1
> > 
> > This is the BFD linker on a Debian system.
> > 
> > $ ld.gold -v
> > GNU gold (GNU Binutils for Debian 2.31.1) 1.16
> > 
> > This is gold on a Debian system.  It is installed by the binutils package, 
> > but is not installed as the system linker.
> > 
> > $ ld -v
> > LLD 8.0.1 (FreeBSD 366581-128) (compatible with GNU linkers)
> > 
> > This is LLD on a FreeBSD system.  It is the system linker.
> 
> Thanks.
> Mine was a CentOS-7 system with 'alternatives' set to use ld.gold
> I did have three toolchains installed (which may have confused thngs), but 
> all three were set to use ld.gold
> Only when I removed all copies of ld.bfd was I able to build gnustep-base to 
> use the new ABI and have it not crash immediately on startup.
> My best guess at an explanation is that I did something (in the distant past 
> so I don't remember) that confused/broke the 'alternatives' system.
> 
> Something else worth noting wrt the core dump is that it's quite commonplace, 
> when building/installing the core libraries, that if something is horribly 
> broken in base it will show up as soon as you try to build gui and that runs 
> plmerge: a plmerge segfault could have many causes, and I didn't mean to 
> blame it on using the new ABI or the bfd linker.
> 
> 
> -- 
> Gregory Casamento
> GNUstep Lead Developer / OLC, Principal Consultant
> http://www.gnustep.org 
> 
>  - http://heronsperch.blogspot.com 
> 
> http://ind.ie/phoenix/ 
> 
> 
> -- 
> Gregory Casamento
> GNUstep Lead Developer / OLC, Principal Consultant
> http://www.gnustep.org 
> 
>  - http://heronsperch.blogspot.com 
> 
> http://ind.ie/phoenix/ 
> 

Re: plmerge core dumps...

[Gregory Casamento]

So, what we are, essentially, saying is that GNUstep can't work with the
standard Linux linker.   I'm wondering in whose mind this is acceptable.

GC

On Wed, Feb 12, 2020 at 9:32 AM Gregory Casamento 
wrote:

> Facepalm this is happening every time I rebuild.  Ugh!  I really can't
> delete my whole installation every time.
>
> On Tue, Feb 11, 2020 at 8:56 AM Richard Frith-Macdonald <
> rich...@frithmacdonald.me.uk> wrote:
>
>>
>>
>> > On 11 Feb 2020, at 13:47, David Chisnall 
>> wrote:
>> >
>> > On 11/02/2020 12:30, Richard Frith-Macdonald wrote:
>> >> clang -v reported that the normal, system linker was being used
>> >
>> > FYI: On most GNU/Linux platforms, BFD is the 'normal, system linker'.
>> For example:
>> >
>> > $ ld -v
>> > GNU ld (GNU Binutils for Debian) 2.31.1
>> >
>> > This is the BFD linker on a Debian system.
>> >
>> > $ ld.gold -v
>> > GNU gold (GNU Binutils for Debian 2.31.1) 1.16
>> >
>> > This is gold on a Debian system.  It is installed by the binutils
>> package, but is not installed as the system linker.
>> >
>> > $ ld -v
>> > LLD 8.0.1 (FreeBSD 366581-128) (compatible with GNU linkers)
>> >
>> > This is LLD on a FreeBSD system.  It is the system linker.
>>
>> Thanks.
>> Mine was a CentOS-7 system with 'alternatives' set to use ld.gold
>> I did have three toolchains installed (which may have confused thngs),
>> but all three were set to use ld.gold
>> Only when I removed all copies of ld.bfd was I able to build gnustep-base
>> to use the new ABI and have it not crash immediately on startup.
>> My best guess at an explanation is that I did something (in the distant
>> past so I don't remember) that confused/broke the 'alternatives' system.
>>
>> Something else worth noting wrt the core dump is that it's quite
>> commonplace, when building/installing the core libraries, that if something
>> is horribly broken in base it will show up as soon as you try to build gui
>> and that runs plmerge: a plmerge segfault could have many causes, and I
>> didn't mean to blame it on using the new ABI or the bfd linker.
>>
>
>
> --
> Gregory Casamento
> GNUstep Lead Developer / OLC, Principal Consultant
> http://www.gnustep.org
> 
> - http://heronsperch.blogspot.com
> 
> http://ind.ie/phoenix/
> 
>


-- 
Gregory Casamento
GNUstep Lead Developer / OLC, Principal Consultant
http://www.gnustep.org

- http://heronsperch.blogspot.com

http://ind.ie/phoenix/

Re: plmerge core dumps...

[Gregory Casamento]

Facepalm this is happening every time I rebuild.  Ugh!  I really can't
delete my whole installation every time.

On Tue, Feb 11, 2020 at 8:56 AM Richard Frith-Macdonald <
rich...@frithmacdonald.me.uk> wrote:

>
>
> > On 11 Feb 2020, at 13:47, David Chisnall 
> wrote:
> >
> > On 11/02/2020 12:30, Richard Frith-Macdonald wrote:
> >> clang -v reported that the normal, system linker was being used
> >
> > FYI: On most GNU/Linux platforms, BFD is the 'normal, system linker'.
> For example:
> >
> > $ ld -v
> > GNU ld (GNU Binutils for Debian) 2.31.1
> >
> > This is the BFD linker on a Debian system.
> >
> > $ ld.gold -v
> > GNU gold (GNU Binutils for Debian 2.31.1) 1.16
> >
> > This is gold on a Debian system.  It is installed by the binutils
> package, but is not installed as the system linker.
> >
> > $ ld -v
> > LLD 8.0.1 (FreeBSD 366581-128) (compatible with GNU linkers)
> >
> > This is LLD on a FreeBSD system.  It is the system linker.
>
> Thanks.
> Mine was a CentOS-7 system with 'alternatives' set to use ld.gold
> I did have three toolchains installed (which may have confused thngs), but
> all three were set to use ld.gold
> Only when I removed all copies of ld.bfd was I able to build gnustep-base
> to use the new ABI and have it not crash immediately on startup.
> My best guess at an explanation is that I did something (in the distant
> past so I don't remember) that confused/broke the 'alternatives' system.
>
> Something else worth noting wrt the core dump is that it's quite
> commonplace, when building/installing the core libraries, that if something
> is horribly broken in base it will show up as soon as you try to build gui
> and that runs plmerge: a plmerge segfault could have many causes, and I
> didn't mean to blame it on using the new ABI or the bfd linker.
>


-- 
Gregory Casamento
GNUstep Lead Developer / OLC, Principal Consultant
http://www.gnustep.org

- http://heronsperch.blogspot.com

http://ind.ie/phoenix/

Re: plmerge core dumps...

[Richard Frith-Macdonald]

> On 11 Feb 2020, at 13:47, David Chisnall  wrote:
> 
> On 11/02/2020 12:30, Richard Frith-Macdonald wrote:
>> clang -v reported that the normal, system linker was being used
> 
> FYI: On most GNU/Linux platforms, BFD is the 'normal, system linker'. For 
> example:
> 
> $ ld -v
> GNU ld (GNU Binutils for Debian) 2.31.1
> 
> This is the BFD linker on a Debian system.
> 
> $ ld.gold -v
> GNU gold (GNU Binutils for Debian 2.31.1) 1.16
> 
> This is gold on a Debian system.  It is installed by the binutils package, 
> but is not installed as the system linker.
> 
> $ ld -v
> LLD 8.0.1 (FreeBSD 366581-128) (compatible with GNU linkers)
> 
> This is LLD on a FreeBSD system.  It is the system linker.

Thanks.
Mine was a CentOS-7 system with 'alternatives' set to use ld.gold
I did have three toolchains installed (which may have confused thngs), but all 
three were set to use ld.gold
Only when I removed all copies of ld.bfd was I able to build gnustep-base to 
use the new ABI and have it not crash immediately on startup.
My best guess at an explanation is that I did something (in the distant past so 
I don't remember) that confused/broke the 'alternatives' system.

Something else worth noting wrt the core dump is that it's quite commonplace, 
when building/installing the core libraries, that if something is horribly 
broken in base it will show up as soon as you try to build gui and that runs 
plmerge: a plmerge segfault could have many causes, and I didn't mean to blame 
it on using the new ABI or the bfd linker.

Re: plmerge core dumps...

[David Chisnall]

On 11/02/2020 12:30, Richard Frith-Macdonald wrote:

clang -v reported that the normal, system linker was being used


FYI: On most GNU/Linux platforms, BFD is the 'normal, system linker'. 
For example:


$ ld -v
GNU ld (GNU Binutils for Debian) 2.31.1

This is the BFD linker on a Debian system.

$ ld.gold -v
GNU gold (GNU Binutils for Debian 2.31.1) 1.16

This is gold on a Debian system.  It is installed by the binutils 
package, but is not installed as the system linker.


$ ld -v
LLD 8.0.1 (FreeBSD 366581-128) (compatible with GNU linkers)

This is LLD on a FreeBSD system.  It is the system linker.

David

Re: plmerge core dumps...

[Richard Frith-Macdonald]

> On 11 Feb 2020, at 11:46, Gregory Casamento  wrote:
> 
>  Linking service GSspell ...
> 3022 Creating GSspell.service/Resources...
> 3023 Creating GSspell.service/Resources/Info-gnustep.plist...
> 3024Segmentation fault (core dumped)
> 3025/home/travis/staging/share/GNUstep/Makefiles/Instance/service.make:141: 
> recipe for target 'GSspell.service/Resources/Info-gnustep.plist' failed
> 3026make[3]: *** [GSspell.service/Resou
> 
> -- 
> 
> I've been seeing this a lot locally and on CI.  Has there been a change which 
> is causing a segfault in plmerge?

This is where I first noticed a problem when using the new objc-2.0 ABI.
In my case David kindly let me know that the issue was that the bfd linker was 
failing to link the base additions library, causing a recursive exception 
initializing NSObject.
On my system it didn't *l,ook* like this was the case (clang -v reported that 
the normal, system linker was being used),  but rebuklding after removing all 
copies of the bfd linker from my system did solve the problem.