Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-30 Thread Martin Buchholz
I agree with Andrew Hughes. On Wed, Jan 30, 2019 at 9:27 AM Andrew Hughes wrote: > > I'm aware of the benefits of using gold, and also occasional issues with > it, but that's not the debate here. It's already perfectly possible to > build with gold by using --with-ldflags="-fuse-ld=gold", as

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-30 Thread Andrew Hughes
On Thu, 24 Jan 2019 at 13:12, Magnus Ihse Bursie wrote: > > The default binutils linker used by gcc, the bfd linker, is slow. The > new replacement, gold, has been distributed alongside gcc for several > years now, and is a well mature, and much faster, replacement. > > This issue is about

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-29 Thread Volker Simonis
When I last tried gold a few years ago on ppc64 (and ia64, just for the reference :) it didn't work because it didn't support all the required relocations. However, a current experiment with "GNU gold (GNU Binutils for Ubuntu 2.26.1) 1.11" on Ubuntu 16.04/ppc64le succeeded and improved the link

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-26 Thread Martin Buchholz
Another, more productionized, version of my benchmark: processors=12 g++ (Debian 7.3.0-5) 7.3.0 --- -fuse-ld=bfd --- 6.559 user 1.180 system 7.740 total --- -fuse-ld=gold --- 4.575 user 0.600 system 5.176 total --- -fuse-ld=gold -Wl,-threads --- 9.355 user 5.062 system 4.289 total ---

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-26 Thread Andrew Haley
On 1/25/19 3:39 PM, Andrew Haley wrote: > On 1/25/19 2:39 PM, Andrew Haley wrote: >> Massive win for gold here, on my (very old) AArch64 system: > > And on another, faster AArch64 system: > > BFD linker: > > real 0m14.532s > user 0m13.070s > sys 0m1.419s > > gold, single-threaded: > >

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-26 Thread Andrew Haley
On 1/25/19 6:43 PM, Martin Buchholz wrote: > repeating my last experiment with explicit --threads: Weird, so you don't get the same performance improvement with threads that I do. You have fewer cores, maybe? My tests were with 8 and 48, respectively. I'm also seeing more than a factor of 2

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Martin Buchholz
On Fri, Jan 25, 2019 at 10:07 AM Andrew Haley wrote: > > On 1/25/19 5:01 PM, Martin Buchholz wrote: > > I re-ran my linker performance experiment using configure > > --with-native-debug-symbols="internal" > > lld is a big winner here: > > It looks to me like lld and multi-threaded gold would be

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Andrew Haley
On 1/25/19 5:01 PM, Martin Buchholz wrote: > I re-ran my linker performance experiment using configure > --with-native-debug-symbols="internal" > lld is a big winner here: It looks to me like lld and multi-threaded gold would be a near tie. I think that lld uses multi-threading; I wonder why

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Philip Race
Category 3 has to deal with plenty of native code too ... -phil On 1/25/19, 9:15 AM, Martin Buchholz wrote: In our own wrappers around configure, we've introduced the concept of a "developer mode". But this thread suggests there are 3 populations of users invoking configure: 1. release

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Martin Buchholz
In our own wrappers around configure, we've introduced the concept of a "developer mode". But this thread suggests there are 3 populations of users invoking configure: 1. release engineers 2. hotspot developers 3. java library developers Category 1 doesn't care about edit-compile-debug cycle -

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Andrew Haley
On 1/25/19 4:45 PM, Magnus Ihse Bursie wrote: > On 2019-01-25 16:42, Andrew Haley wrote: >> On 1/24/19 2:04 PM, Magnus Ihse Bursie wrote: >>> But I'm leaning more towards just enabling gold on x86_64 -- for other >>> platforms, we might as well keep the good ol' bfd linker. Does that >>> sound

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Martin Buchholz
I re-ran my linker performance experiment using configure --with-native-debug-symbols="internal" lld is a big winner here: --- ld=bfd --- /usr/bin/g++ -fuse-ld=$ld -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,noexecstac 7.30s user 1.26s system 99% cpu 8.559 total /usr/bin/g++ -fuse-ld=$ld

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Magnus Ihse Bursie
On 2019-01-25 16:42, Andrew Haley wrote: On 1/24/19 2:04 PM, Magnus Ihse Bursie wrote: But I'm leaning more towards just enabling gold on x86_64 -- for other platforms, we might as well keep the good ol' bfd linker. Does that sound like a good solution to you? I'd love it to be an option for

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Andrew Haley
On 1/24/19 2:04 PM, Magnus Ihse Bursie wrote: > But I'm leaning more towards just enabling gold on x86_64 -- for other > platforms, we might as well keep the good ol' bfd linker. Does that > sound like a good solution to you? I'd love it to be an option for AArch64. -- Andrew Haley Java

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Andrew Haley
On 1/25/19 2:39 PM, Andrew Haley wrote: > Massive win for gold here, on my (very old) AArch64 system: And on another, faster AArch64 system: BFD linker: real0m14.532s user0m13.070s sys 0m1.419s gold, single-threaded: real0m6.313s user0m6.123s sys 0m0.190s gold,

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-25 Thread Andrew Haley
Massive win for gold here, on my (very old) AArch64 system: BFD linker: $ time /home/aph/gcc-x/aarch64-trunk-install/bin/g++ -Wl,-z,defs -fuse-ld=bfd -Wl,-z,noexecstack ... real0m26.090s user0m24.570s sys 0m1.480s gold, single-threaded: $ time

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Martin Buchholz
On Thu, Jan 24, 2019 at 2:28 PM Erik Joelsson wrote: > > Do the constituent object files have debugging information? Sub-second > > processing times for ~350 MiB of output are rather impressive. > > Ah! That must be it. I just tried with --with-native-debug-symbols=none > and then I get

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Erik Joelsson
On 2019-01-24 14:17, Florian Weimer wrote: * Martin Buchholz: Here's an experiment using the 3 competing open source linkers to link hotspot. This confirms that lld is faster than gold is faster than bfd, but is the one second saving worth the engineering effort? $

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Erik Joelsson
Hello, On 2019-01-24 13:18, Martin Buchholz wrote: I just copied the command out of hotspot/variant-server/libjvm/objs/BUILD_LIBJVM_link.cmdline and lightly edited it. I took the .cmdline verbatim and then just added -fuse-ld=gold, so should be the same. I'm running on Ubuntu 18.04 using a

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Florian Weimer
* Martin Buchholz: > Here's an experiment using the 3 competing open source linkers to link > hotspot. This confirms that lld is faster than gold is faster than > bfd, but is the one second saving worth the engineering effort? > > $ (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Martin Buchholz
I just copied the command out of hotspot/variant-server/libjvm/objs/BUILD_LIBJVM_link.cmdline and lightly edited it. On my old underpowered NUC at home I see slightly worse numbers (but warmed up, files on SSD - are you I/O bound?). (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Erik Joelsson
Are you actually linking libjvm.so in 1.3 seconds? A normal link time for me using bfd is about 23 seconds while gold takes it to 14.2(+-0.2). This is in line with what hotspot developers I have talked to also see (and they have similar hardware). My workstation has a few years on it, but

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Martin Buchholz
Here's an experiment using the 3 competing open source linkers to link hotspot. This confirms that lld is faster than gold is faster than bfd, but is the one second saving worth the engineering effort? $ (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for linker in bfd gold lld; do

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Andrew Haley
On 1/24/19 1:25 PM, Aleksey Shipilev wrote: > That'd be nice. Maybe we should check for the presence of ld.gold, > though, and override LD when it is available? That would also allow > the fallback when ld.gold is misbehaving, but setting LD forcefully. Yes, we absolutely should check. I just

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Magnus Ihse Bursie
On 2019-01-24 18:43, Erik Joelsson wrote: On 2019-01-24 06:04, Magnus Ihse Bursie wrote: On 2019-01-24 14:45, Florian Weimer wrote: * Magnus Ihse Bursie: The default binutils linker used by gcc, the bfd linker, is slow. The new replacement, gold, has been distributed alongside gcc for

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Erik Joelsson
On 2019-01-24 06:04, Magnus Ihse Bursie wrote: On 2019-01-24 14:45, Florian Weimer wrote: * Magnus Ihse Bursie: The default binutils linker used by gcc, the bfd linker, is slow. The new replacement, gold, has been distributed alongside gcc for several years now, and is a well mature, and

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Martin Buchholz
On Thu, Jan 24, 2019 at 7:46 AM Magnus Ihse Bursie wrote: > So, we already tacitly assume a specific linker with the gcc toolchain; we > have just not made that check explicitly. gcc runs on almost every system, but it's harder to replace the compiler than the linker, so gcc ends up being

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Magnus Ihse Bursie
I agree that the relationship between compiler, linker and OS is problematic. In the OpenJDK build system, we have used the concept of "toolchain", by which we mean (amongst other things) a compiler and a linker working together. For some platforms it's a perfect fit (eg Visual Studio), for

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Martin Buchholz
Getting into the business of choosing the linker is big trouble. The system default toolchain may have already chosen a linker. BFD might be configured to have either bfd ld or gold. # Handle --enable-gold, --enable-ld. # --disable-gold [--enable-ld] # Build only ld. Default option. #

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Magnus Ihse Bursie
On 2019-01-24 14:45, Florian Weimer wrote: * Magnus Ihse Bursie: The default binutils linker used by gcc, the bfd linker, is slow. The new replacement, gold, has been distributed alongside gcc for several years now, and is a well mature, and much faster, replacement. The gold linker is an

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Florian Weimer
* Magnus Ihse Bursie: > The default binutils linker used by gcc, the bfd linker, is slow. The > new replacement, gold, has been distributed alongside gcc for several > years now, and is a well mature, and much faster, replacement. The gold linker is an optional component of binutils, not

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Magnus Ihse Bursie
On 2019-01-24 14:20, John Paul Adrian Glaubitz wrote: On 1/24/19 2:11 PM, Magnus Ihse Bursie wrote: The default binutils linker used by gcc, the bfd linker, is slow. The new replacement, gold, has been distributed alongside gcc for several years now, and is a well mature, and much faster,

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Aleksey Shipilev
On 1/24/19 2:15 PM, Magnus Ihse Bursie wrote: > I see now on Wikipedia that gold was added to binutils 2.19.[1] (I thought it > was older still...) > We currently have this check: > TOOLCHAIN_MINIMUM_LD_VERSION_gcc="2.18" > > As part of this fix, I ought to bump it to 2.19, so we know for sure

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread John Paul Adrian Glaubitz
On 1/24/19 2:11 PM, Magnus Ihse Bursie wrote: > The default binutils linker used by gcc, the bfd linker, is slow. > The new replacement, gold, has been distributed alongside gcc for > several years now, and is a well mature, and much faster, replacement. I would recommend testing this change on

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Magnus Ihse Bursie
I see now on Wikipedia that gold was added to binutils 2.19.[1] (I thought it was older still...) We currently have this check: TOOLCHAIN_MINIMUM_LD_VERSION_gcc="2.18" As part of this fix, I ought to bump it to 2.19, so we know for sure that gold is included. Volker, would that be a

RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

2019-01-24 Thread Magnus Ihse Bursie
The default binutils linker used by gcc, the bfd linker, is slow. The new replacement, gold, has been distributed alongside gcc for several years now, and is a well mature, and much faster, replacement. This issue is about replacing ld.bfd with ld.gold for our gcc toolchain. In general, this