Re: A modest proposal: use the gold linker by default

[Henry Robinson]

I'm in favour of the switch, assuming that the details can be worked out.
Reducing the variability in the tools used to build Impala is worthwhile in
and of itself.

On 13 October 2016 at 08:52, Tim Armstrong  wrote:

> Oh interesting. It doesn't look like our toolchain version of binutils
> (2.26) has the fix, but 2.27 does have it.
>
> On Thu, Oct 13, 2016 at 8:48 AM, Todd Lipcon  wrote:
>
> > There's one issue with older versions of gold that for shared objects it
> > doesn't handle weak symbols correctly:
> > https://sourceware.org/bugzilla/show_bug.cgi?id=16979
> >
> > This isn't a problem for static linking, but for dynamic linking causes
> > problems like not seeing tcmalloc's version of malloc.
> >
> > Not a blocker (I use gold almost all the time on my laptop by putting a
> > symlink on my path as 'ld') but something to be aware of.
> >
> > -Todd
> >
> > On Thu, Oct 13, 2016 at 8:28 AM, Jim Apple  wrote:
> >
> > > I think clang doesn't have a CLI option to use gold. Does buildall.sh
> > -asan
> > > interact well with gold?
> > >
> > > On Thu, Oct 13, 2016 at 8:25 AM, Tim Armstrong <
> tarmstr...@cloudera.com>
> > > wrote:
> > >
> > > > The gold linker is much faster for building static binaries so will
> be
> > > much
> > > > friendlier for new developers.
> > > >
> > > > I don't think I've seen a problem attributable to gold since it was
> > > added ~
> > > > 6 months ago. I've been using it since then and we've been using it
> for
> > > > some Impala test jobs as well for at least a couple of months. I feel
> > > > confident that any major problems have been shaken out.
> > > >
> > > > Does anyone have any objections?
> > > >
> > > > - Tim
> > > >
> > >
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>



-- 
Henry Robinson
Software Engineer
Cloudera
415-994-6679

Re: A modest proposal: use the gold linker by default

[Tim Armstrong]

You're right that ASAN+gold in our build system is wonky. It works but
clang actually calls out to ld rather than gold. I think this is purely
because we pass -Wl,-fuse-ld=gold instead of -fuse-ld=gold.

I confirmed that clang can call out to gold correctly if you give it the
right option:

$IMPALA_TOOLCHAIN/llvm-3.8.0-p1/bin/clang -v
-B $IMPALA_TOOLCHAIN/binutils-2.26-p1/bin/ -Wall -fuse-ld=gold
-fsanitize=address helloworld.c -o helloworld && ./helloworld

The last lines of output are:
 "/opt/Impala-Toolchain/binutils-2.26-p1/bin/ld.gold" -z relro
--hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker
/lib64/ld-linux-x86-64.so.2 -o helloworld
/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/crt1.o
/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/5.4.0/crtbegin.o
-L/usr/lib/gcc/x86_64-linux-gnu/5.4.0
-L/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu
-L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu
-L/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../..
-L/opt/Impala-Toolchain/llvm-3.8.0-p1/bin/../lib -L/lib -L/usr/lib
-whole-archive
/opt/Impala-Toolchain/llvm-3.8.0-p1/bin/../lib/clang/3.8.0/lib/linux/libclang_rt.asan-x86_64.a
-no-whole-archive
--dynamic-list=/opt/Impala-Toolchain/llvm-3.8.0-p1/bin/../lib/clang/3.8.0/lib/linux/libclang_rt.asan-x86_64.a.syms
/tmp/helloworld-eb4915.o --no-as-needed -lpthread -lrt -lm -ldl -lgcc
--as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s
--no-as-needed /usr/lib/gcc/x86_64-linux-gnu/5.4.0/crtend.o
/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu/crtn.o
Hello world



On Thu, Oct 13, 2016 at 8:28 AM, Jim Apple  wrote:

> I think clang doesn't have a CLI option to use gold. Does buildall.sh -asan
> interact well with gold?
>
> On Thu, Oct 13, 2016 at 8:25 AM, Tim Armstrong 
> wrote:
>
> > The gold linker is much faster for building static binaries so will be
> much
> > friendlier for new developers.
> >
> > I don't think I've seen a problem attributable to gold since it was
> added ~
> > 6 months ago. I've been using it since then and we've been using it for
> > some Impala test jobs as well for at least a couple of months. I feel
> > confident that any major problems have been shaken out.
> >
> > Does anyone have any objections?
> >
> > - Tim
> >
>

Re: A modest proposal: use the gold linker by default

[Tim Armstrong]

Oh interesting. It doesn't look like our toolchain version of binutils
(2.26) has the fix, but 2.27 does have it.

On Thu, Oct 13, 2016 at 8:48 AM, Todd Lipcon  wrote:

> There's one issue with older versions of gold that for shared objects it
> doesn't handle weak symbols correctly:
> https://sourceware.org/bugzilla/show_bug.cgi?id=16979
>
> This isn't a problem for static linking, but for dynamic linking causes
> problems like not seeing tcmalloc's version of malloc.
>
> Not a blocker (I use gold almost all the time on my laptop by putting a
> symlink on my path as 'ld') but something to be aware of.
>
> -Todd
>
> On Thu, Oct 13, 2016 at 8:28 AM, Jim Apple  wrote:
>
> > I think clang doesn't have a CLI option to use gold. Does buildall.sh
> -asan
> > interact well with gold?
> >
> > On Thu, Oct 13, 2016 at 8:25 AM, Tim Armstrong 
> > wrote:
> >
> > > The gold linker is much faster for building static binaries so will be
> > much
> > > friendlier for new developers.
> > >
> > > I don't think I've seen a problem attributable to gold since it was
> > added ~
> > > 6 months ago. I've been using it since then and we've been using it for
> > > some Impala test jobs as well for at least a couple of months. I feel
> > > confident that any major problems have been shaken out.
> > >
> > > Does anyone have any objections?
> > >
> > > - Tim
> > >
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: A modest proposal: use the gold linker by default

[Todd Lipcon]

There's one issue with older versions of gold that for shared objects it
doesn't handle weak symbols correctly:
https://sourceware.org/bugzilla/show_bug.cgi?id=16979

This isn't a problem for static linking, but for dynamic linking causes
problems like not seeing tcmalloc's version of malloc.

Not a blocker (I use gold almost all the time on my laptop by putting a
symlink on my path as 'ld') but something to be aware of.

-Todd

On Thu, Oct 13, 2016 at 8:28 AM, Jim Apple  wrote:

> I think clang doesn't have a CLI option to use gold. Does buildall.sh -asan
> interact well with gold?
>
> On Thu, Oct 13, 2016 at 8:25 AM, Tim Armstrong 
> wrote:
>
> > The gold linker is much faster for building static binaries so will be
> much
> > friendlier for new developers.
> >
> > I don't think I've seen a problem attributable to gold since it was
> added ~
> > 6 months ago. I've been using it since then and we've been using it for
> > some Impala test jobs as well for at least a couple of months. I feel
> > confident that any major problems have been shaken out.
> >
> > Does anyone have any objections?
> >
> > - Tim
> >
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: A modest proposal: use the gold linker by default

[Jim Apple]

I think clang doesn't have a CLI option to use gold. Does buildall.sh -asan
interact well with gold?

On Thu, Oct 13, 2016 at 8:25 AM, Tim Armstrong 
wrote:

> The gold linker is much faster for building static binaries so will be much
> friendlier for new developers.
>
> I don't think I've seen a problem attributable to gold since it was added ~
> 6 months ago. I've been using it since then and we've been using it for
> some Impala test jobs as well for at least a couple of months. I feel
> confident that any major problems have been shaken out.
>
> Does anyone have any objections?
>
> - Tim
>