Re: Faster rebuild with GNU gold linker

[Ioi Lam]

BTW, for those who want to save an extra second or so in rebuilding hotspot:

    make hotspot VM_VERSION_OBJ=xxx

This doesn't rebuild vm_version.o (which is usually built in a separate 
pass after all other objects have been built).


There are some caveats. See my comments in JDK-8218577.


On 2/20/19 2:09 AM, Andrew Haley wrote:

On 2/20/19 7:54 AM, Ioi Lam wrote:

Ahh, I was using a debug build of lld. After making a release build, the
link time drops to 1.1 seconds! It's definitely better than gold.

Right, exactly. And when you're working on just one file in HotSpot,
this can make a very significant difference to the programmer experience.

Re: Faster rebuild with GNU gold linker

[Andrew Haley]

On 2/20/19 7:54 AM, Ioi Lam wrote:
> Ahh, I was using a debug build of lld. After making a release build, the 
> link time drops to 1.1 seconds! It's definitely better than gold.

Right, exactly. And when you're working on just one file in HotSpot,
this can make a very significant difference to the programmer experience.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: Faster rebuild with GNU gold linker

[Andrew Haley]

On 2/20/19 6:35 AM, Ioi Lam wrote:
> Thanks for the info. I didn't know that this has been discussed so 
> recently. Should have checked befor I posted.
> 
> Anyway, I built the latest lld from GIT, but it's slower than gold on my 
> machine. Just linking libjvm.so is 5.8s vs 3.8s.

This is exactly the opposite of everyone else's results. Cany you please post
the results of using "time" on just the link step? Thanks.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: Faster rebuild with GNU gold linker

[Ioi Lam]

Ahh, I was using a debug build of lld. After making a release build, the 
link time drops to 1.1 seconds! It's definitely better than gold.


On 2/19/19 10:35 PM, Ioi Lam wrote:

Hi Martin,

Thanks for the info. I didn't know that this has been discussed so 
recently. Should have checked befor I posted.


Anyway, I built the latest lld from GIT, but it's slower than gold on 
my machine. Just linking libjvm.so is 5.8s vs 3.8s.


On 2/19/19 5:23 PM, Martin Buchholz wrote:

https://openjdk.markmail.org/thread/q3layufiyjivu42c

On Tue, Feb 19, 2019 at 5:03 PM Ioi Lam > wrote:


    For the impatient engineers 

    I did some measurement between the default GNU linker "ld" vs
    "ld.gold".
    I am trying to get the fastest rebuild time after I modify one cpp
    file.
    With gold, it's down to about 1/3 of the original time.


    slowdebug (~220MB libjvm.so)

      recompile 1 cpp file  |  relink libjvm.so only
    ld:       33 s   25 s
    gold 1 thread:    16 s            9 s
    gold 8 threads:   13 s    6 s


    fastdebug (~360MB libjvm.so)


      recompile 1 cpp file  |  relink libjvm.so only
    ld:           35 s            25 s
    gold 1 thread:    18 s    10 s
    gold 8 threads:   15 s     6 s


    Question: do we want to add built-in support for gold into the JDK
    makefiles?


    Notes:

    To choose gold, run configure with something like:

     --with-extra-ldflags='-fuse-ld=gold
    -Wl,--threads,--thread-count,8'

    I essentially do a "make hotspot" and then move the libjvm.so into
    a JDK
    image, instead of doing a full JDK image build.

    "make hotspot" makes a copy of libjvm.so (from
    support/modules_libs/java.base/server/ to jdk/lib/server/). I
    hacked the
    Makefile to make a hard link instead to avoid the unnecessary I/O.

    libjvm.so is built with --with-native-debug-symbols=internal to 
avoid

    the expensive invocations of objcopy and strip.


    My environment:

    I am using gcc7.3.0 on Ubuntu 16.04.5 on a 5 year old Dell Precision
    T7600 with dual socket Xeon E5-2665 @ 2.40GHz, 64GB RAM, Samsung
    840 PRO
    SSD. I suspect gold can run even faster, but my slow SSD is
    holding it back.

    ld version   = GNU ld (GNU Binutils) 2.30
    gold version = GNU gold (GNU Binutils for Ubuntu 2.26.1) 1.11

    (These are just the versions available to me on my machine, not
    necessarily the best)

Re: Faster rebuild with GNU gold linker

[Ioi Lam]

Hi Martin,

Thanks for the info. I didn't know that this has been discussed so 
recently. Should have checked befor I posted.


Anyway, I built the latest lld from GIT, but it's slower than gold on my 
machine. Just linking libjvm.so is 5.8s vs 3.8s.


On 2/19/19 5:23 PM, Martin Buchholz wrote:

https://openjdk.markmail.org/thread/q3layufiyjivu42c

On Tue, Feb 19, 2019 at 5:03 PM Ioi Lam > wrote:


For the impatient engineers 

I did some measurement between the default GNU linker "ld" vs
"ld.gold".
I am trying to get the fastest rebuild time after I modify one cpp
file.
With gold, it's down to about 1/3 of the original time.


slowdebug (~220MB libjvm.so)

  recompile 1 cpp file  |  relink libjvm.so only
ld:       33 s   25 s
gold 1 thread:    16 s            9 s
gold 8 threads:   13 s    6 s


fastdebug (~360MB libjvm.so)


  recompile 1 cpp file  |  relink libjvm.so only
ld:           35 s            25 s
gold 1 thread:    18 s    10 s
gold 8 threads:   15 s     6 s


Question: do we want to add built-in support for gold into the JDK
makefiles?


Notes:

To choose gold, run configure with something like:

 --with-extra-ldflags='-fuse-ld=gold
-Wl,--threads,--thread-count,8'

I essentially do a "make hotspot" and then move the libjvm.so into
a JDK
image, instead of doing a full JDK image build.

"make hotspot" makes a copy of libjvm.so (from
support/modules_libs/java.base/server/ to jdk/lib/server/). I
hacked the
Makefile to make a hard link instead to avoid the unnecessary I/O.

libjvm.so is built with --with-native-debug-symbols=internal to avoid
the expensive invocations of objcopy and strip.


My environment:

I am using gcc7.3.0 on Ubuntu 16.04.5 on a 5 year old Dell Precision
T7600 with dual socket Xeon E5-2665 @ 2.40GHz, 64GB RAM, Samsung
840 PRO
SSD. I suspect gold can run even faster, but my slow SSD is
holding it back.

ld version   = GNU ld (GNU Binutils) 2.30
gold version = GNU gold (GNU Binutils for Ubuntu 2.26.1) 1.11

(These are just the versions available to me on my machine, not
necessarily the best)

Re: Faster rebuild with GNU gold linker

[Martin Buchholz]

https://openjdk.markmail.org/thread/q3layufiyjivu42c

On Tue, Feb 19, 2019 at 5:03 PM Ioi Lam  wrote:

> For the impatient engineers 
>
> I did some measurement between the default GNU linker "ld" vs "ld.gold".
> I am trying to get the fastest rebuild time after I modify one cpp file.
> With gold, it's down to about 1/3 of the original time.
>
>
> slowdebug (~220MB libjvm.so)
>
>   recompile 1 cpp file  |  relink libjvm.so only
> ld:   33 s   25 s
> gold 1 thread:16 s9 s
> gold 8 threads:   13 s6 s
>
>
> fastdebug (~360MB libjvm.so)
>
>
>   recompile 1 cpp file  |  relink libjvm.so only
> ld:   35 s25 s
> gold 1 thread:18 s10 s
> gold 8 threads:   15 s 6 s
>
>
> Question: do we want to add built-in support for gold into the JDK
> makefiles?
>
>
> Notes:
>
> To choose gold, run configure with something like:
>
>  --with-extra-ldflags='-fuse-ld=gold -Wl,--threads,--thread-count,8'
>
> I essentially do a "make hotspot" and then move the libjvm.so into a JDK
> image, instead of doing a full JDK image build.
>
> "make hotspot" makes a copy of libjvm.so (from
> support/modules_libs/java.base/server/ to jdk/lib/server/). I hacked the
> Makefile to make a hard link instead to avoid the unnecessary I/O.
>
> libjvm.so is built with --with-native-debug-symbols=internal to avoid
> the expensive invocations of objcopy and strip.
>
>
> My environment:
>
> I am using gcc7.3.0 on Ubuntu 16.04.5 on a 5 year old Dell Precision
> T7600 with dual socket Xeon E5-2665 @ 2.40GHz, 64GB RAM, Samsung 840 PRO
> SSD. I suspect gold can run even faster, but my slow SSD is holding it
> back.
>
> ld version   = GNU ld (GNU Binutils) 2.30
> gold version = GNU gold (GNU Binutils for Ubuntu 2.26.1) 1.11
>
> (These are just the versions available to me on my machine, not
> necessarily the best)
>
>

Faster rebuild with GNU gold linker

[Ioi Lam]

For the impatient engineers 

I did some measurement between the default GNU linker "ld" vs "ld.gold". 
I am trying to get the fastest rebuild time after I modify one cpp file. 
With gold, it's down to about 1/3 of the original time.



slowdebug (~220MB libjvm.so)

 recompile 1 cpp file  |  relink libjvm.so only
ld:       33 s   25 s
gold 1 thread:    16 s            9 s
gold 8 threads:   13 s    6 s


fastdebug (~360MB libjvm.so)


 recompile 1 cpp file  |  relink libjvm.so only
ld:           35 s            25 s
gold 1 thread:    18 s    10 s
gold 8 threads:   15 s     6 s


Question: do we want to add built-in support for gold into the JDK 
makefiles?



Notes:

To choose gold, run configure with something like:

    --with-extra-ldflags='-fuse-ld=gold -Wl,--threads,--thread-count,8'

I essentially do a "make hotspot" and then move the libjvm.so into a JDK 
image, instead of doing a full JDK image build.


"make hotspot" makes a copy of libjvm.so (from 
support/modules_libs/java.base/server/ to jdk/lib/server/). I hacked the 
Makefile to make a hard link instead to avoid the unnecessary I/O.


libjvm.so is built with --with-native-debug-symbols=internal to avoid 
the expensive invocations of objcopy and strip.



My environment:

I am using gcc7.3.0 on Ubuntu 16.04.5 on a 5 year old Dell Precision 
T7600 with dual socket Xeon E5-2665 @ 2.40GHz, 64GB RAM, Samsung 840 PRO 
SSD. I suspect gold can run even faster, but my slow SSD is holding it back.


ld version   = GNU ld (GNU Binutils) 2.30
gold version = GNU gold (GNU Binutils for Ubuntu 2.26.1) 1.11

(These are just the versions available to me on my machine, not 
necessarily the best)