Re: ld.so: -fno-builtin?

[Philip Guenther]

On Fri, Dec 23, 2016 at 12:11 PM, Joerg Sonnenberger  wrote:
> On Fri, Dec 23, 2016 at 11:27:15AM -0800, Philip Guenther wrote:
>> This is a form we use inside _libc_ so that calls to those functions
>> generated by gcc will be redirected to aliases with hidden visibility
>> and thus be local calls, without using the PLT.  If that won't work
>> with clang, then we'll want to figure out some other way to get the
>> calls to those functions generated by the compiler to be local calls
>> inside libc.
>
> I'm not sure about all possible platforms, but for all I can think of,
> it is enough if the target is hidden.

Making memcpy hidden is fine for ld.so, but not for libc, thus the
alias and asm rename dance we're currently using.


> In that case ld should be using
> a direct jump/call to the destination without creating a PLT entry.

The linker can eliminate the PLT entry and double-jump, but it
currently can't reoptimize the calling functions when %ebx is no
longer needed for holding the GOT offset for the PLT call.  LTO is
getting better, I guess, but it was nice getting the Right Thing
without requiring it.  


Philip Guenther

Re: ld.so: -fno-builtin?

[Joerg Sonnenberger]

On Fri, Dec 23, 2016 at 11:27:15AM -0800, Philip Guenther wrote:
> This is a form we use inside _libc_ so that calls to those functions
> generated by gcc will be redirected to aliases with hidden visibility
> and thus be local calls, without using the PLT.  If that won't work
> with clang, then we'll want to figure out some other way to get the
> calls to those functions generated by the compiler to be local calls
> inside libc.

I'm not sure about all possible platforms, but for all I can think of,
it is enough if the target is hidden. In that case ld should be using
a direct jump/call to the destination without creating a PLT entry.

Joerg

Re: ld.so: -fno-builtin?

[Joerg Sonnenberger]

On Fri, Dec 23, 2016 at 06:43:42PM +0100, Mark Kettenis wrote:
> > Date: Fri, 23 Dec 2016 18:13:45 +0100
> > From: Joerg Sonnenberger 
> > 
> > On Thu, Dec 22, 2016 at 10:35:25PM -0800, Philip Guenther wrote:
> > > I'm assuming clang handles asm names like gcc, such that declaring
> > >void  *memcpy(void *__restrict, const void *__restrict, __size_t)
> > > __dso_hidden __asm("_dl_memcpy");
> > > 
> > > will make even internally generated calls go to _dl_memcpy instead.
> > 
> > No. The backend normally has no idea about assembler names. What problem
> > are you trying to solve, really?
> 
> Right.  The solution gere is probably to rename _dl_memcpy back to
> memcpy.  One of the reasons why _dl_memcpy exists is that we wanted to
> prevent exporting ld.so's memcpy implementation such that nothing else
> would accidentally pick it up.  But now that we explicitly control
> which symbols we export from ld.so, that risk doesn't exist anymore.

Correct. x86 is more forgiving here than others as it doesn't tend to
pull in anything from libgcc/compiler-rt/whatever, but the same issue
will exist e.g. for the division helper if you care about armv6 etc.

Joerg

Re: ld.so: -fno-builtin?

[Philip Guenther]

On Fri, Dec 23, 2016 at 9:13 AM, Joerg Sonnenberger  wrote:
> On Thu, Dec 22, 2016 at 10:35:25PM -0800, Philip Guenther wrote:
>> I'm assuming clang handles asm names like gcc, such that declaring
>>void  *memcpy(void *__restrict, const void *__restrict, __size_t)
>> __dso_hidden __asm("_dl_memcpy");
>>
>> will make even internally generated calls go to _dl_memcpy instead.
>
> No. The backend normally has no idea about assembler names. What problem
> are you trying to solve, really?

This is a form we use inside _libc_ so that calls to those functions
generated by gcc will be redirected to aliases with hidden visibility
and thus be local calls, without using the PLT.  If that won't work
with clang, then we'll want to figure out some other way to get the
calls to those functions generated by the compiler to be local calls
inside libc.

Reusing for ld.so whatever we make work for libc seems like a good idea.

As kettenis@ notes, the renaming isn't _necessary_ now that we have an
explicit symbol export list for ld.so, but it would still be nice to
be able to tell the compiler to use direct calls, as the generated
code is better than what the linker alone can optimize it to for archs
like i386.


Philip Guenther

Re: ld.so: -fno-builtin?

[Theo de Raadt]

> On Thu, Dec 22, 2016 at 10:35:25PM -0800, Philip Guenther wrote:
> > I'm assuming clang handles asm names like gcc, such that declaring
> >void  *memcpy(void *__restrict, const void *__restrict, __size_t)
> > __dso_hidden __asm("_dl_memcpy");
> > 
> > will make even internally generated calls go to _dl_memcpy instead.
> 
> No. The backend normally has no idea about assembler names. What problem
> are you trying to solve, really?

Joerg, If you don't actually run OpenBSD you won't understand.  Please pipe
down.

Re: ld.so: -fno-builtin?

[Mark Kettenis]

> Date: Fri, 23 Dec 2016 18:13:45 +0100
> From: Joerg Sonnenberger 
> 
> On Thu, Dec 22, 2016 at 10:35:25PM -0800, Philip Guenther wrote:
> > I'm assuming clang handles asm names like gcc, such that declaring
> >void  *memcpy(void *__restrict, const void *__restrict, __size_t)
> > __dso_hidden __asm("_dl_memcpy");
> > 
> > will make even internally generated calls go to _dl_memcpy instead.
> 
> No. The backend normally has no idea about assembler names. What problem
> are you trying to solve, really?

Right.  The solution gere is probably to rename _dl_memcpy back to
memcpy.  One of the reasons why _dl_memcpy exists is that we wanted to
prevent exporting ld.so's memcpy implementation such that nothing else
would accidentally pick it up.  But now that we explicitly control
which symbols we export from ld.so, that risk doesn't exist anymore.

The downside that we will have functions named memcpy in a dynamic
executable still makes debugging a little bit harder, but I'll get
over it.

We probably should not rename all _dl_-prefixed versions of standard C
functions just yet.  Especially those that don't fully implement the
standard C functionality.

Cheers,

Mark

Re: ld.so: -fno-builtin?

[Joerg Sonnenberger]

On Thu, Dec 22, 2016 at 10:35:25PM -0800, Philip Guenther wrote:
> I'm assuming clang handles asm names like gcc, such that declaring
>void  *memcpy(void *__restrict, const void *__restrict, __size_t)
> __dso_hidden __asm("_dl_memcpy");
> 
> will make even internally generated calls go to _dl_memcpy instead.

No. The backend normally has no idea about assembler names. What problem
are you trying to solve, really?

Joerg

Re: ld.so: -fno-builtin?

[Philip Guenther]

On Thu, Dec 22, 2016 at 12:33 PM, Joerg Sonnenberger  wrote:
> On Thu, Dec 22, 2016 at 05:47:05PM +0100, Christian Weisgerber wrote:
>> Building ld.so with clang on amd64 fails with undefined references
>> to memset and memcpy.  That is odd, since neither function appears
>> in the source.  Apparently clang optimizes the _dl_memset and
>> _dl_bcopy functions into calls to memset and memcpy, respectively.
>>
>> I tentatively propose to add -fno-builtin to fix this.
>> (Builds, runs, and passes regress with gcc on amd64 and i386.)
>
> You are lucky if it fixes the problem, infact, -fno-builtin has quite a
> chance to make the problem worse. Clang assumes the same set of basic
> functions to be present even for -ffreestanding environments and can
> lower e.g. initialisation of local variables to memset/memcpy.

Yeah.  I suspect the better answer is to
a) convert _dl_bcopy() to _dl_memcpy()   (it's misnamed anyway, as it
doesn't behave like bcopy), and
b) in a header that's #included everywhere in ld.so, use the gcc asm
names extension to redirect memcpy and memset
   to _dl_memcpy and _dl_memset (and as long as you're doing that,
mark them hidden...)

I'm assuming clang handles asm names like gcc, such that declaring
   void  *memcpy(void *__restrict, const void *__restrict, __size_t)
__dso_hidden __asm("_dl_memcpy");

will make even internally generated calls go to _dl_memcpy instead.


Philip Guenther

Re: ld.so: -fno-builtin?

[Joerg Sonnenberger]

On Thu, Dec 22, 2016 at 05:47:05PM +0100, Christian Weisgerber wrote:
> Building ld.so with clang on amd64 fails with undefined references
> to memset and memcpy.  That is odd, since neither function appears
> in the source.  Apparently clang optimizes the _dl_memset and
> _dl_bcopy functions into calls to memset and memcpy, respectively.
> 
> I tentatively propose to add -fno-builtin to fix this.
> (Builds, runs, and passes regress with gcc on amd64 and i386.)

You are lucky if it fixes the problem, infact, -fno-builtin has quite a
chance to make the problem worse. Clang assumes the same set of basic
functions to be present even for -ffreestanding environments and can
lower e.g. initialisation of local variables to memset/memcpy.

Joerg

ld.so: -fno-builtin?

[Christian Weisgerber]

Building ld.so with clang on amd64 fails with undefined references
to memset and memcpy.  That is odd, since neither function appears
in the source.  Apparently clang optimizes the _dl_memset and
_dl_bcopy functions into calls to memset and memcpy, respectively.

I tentatively propose to add -fno-builtin to fix this.
(Builds, runs, and passes regress with gcc on amd64 and i386.)

Index: Makefile
===
RCS file: /cvs/src/libexec/ld.so/Makefile,v
retrieving revision 1.62
diff -u -p -r1.62 Makefile
--- Makefile4 Jul 2016 21:15:06 -   1.62
+++ Makefile22 Dec 2016 16:27:33 -
@@ -31,6 +31,7 @@ SRCS+=library.c
 .PATH: ${.CURDIR}/${MACHINE_CPU}
 
 DEBUG?=-g
+CFLAGS += -fno-builtin
 CFLAGS += -Wall -Werror
 CFLAGS += -I${.CURDIR} -I${.CURDIR}/${MACHINE_CPU} \
-D'DEF_WEAK(x)=asm("")' -D'DEF_STRONG(x)=asm("")' \
-- 
Christian "naddy" Weisgerber  na...@mips.inka.de