Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Helmut Grohne]

Hi Florian,

On Sat, Dec 04, 2021 at 01:59:14PM +0100, Florian Weimer wrote:
> It is as architecture-independent as ldconfig or getconf.  Perhaps a bit
> more so than getconf.

Both of those are bad examples as both are lies. An amd64 ldconfig
really does not handle arm64 libraries at all. I'd rather not see us
adding more issues.

> Even if /bin/true is an i386 program, assuming that libeatmydata1:i386
> is installed.  Whether ld.so is built for i386 or amd64 will not matter.

That's great.

> With the patch I've posted, you'll get the ELFCLASS64 error.  But I have
> some ideas how to fix that eventually.  Is this sufficient for now?

I'd prefer making it work upfront. People will use it and run into
issues blaming Multi-Arch otherwise.

If you also have ideas on how to make ldconfig work, that would be cool
as well, but it's a different topic and should be moved to d-cross@l.d.o
rather than discussed in this thread.

Helmut

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Florian Weimer]

* Aurelien Jarno:

> Hi,
>
> On 2021-12-02 19:51, Florian Weimer wrote:
>> I'd like to provide an ld.so command as part of glibc.  Today, ld.so can
>> be used to activate preloading, for example.  Compared to LD_PRELOAD,
>> the difference is that it's specific to one process, and won't be
>> inherited by subprocesses—something is that exactly what is needed.
>> There is also some useful diagnostic output in --help,
>> --list-diagnostics.
>
> This sounds a good idea, I guess for instance that in the future the ldd
> feature could be implemented as an option.
>
> However before exposing directly to user, there might be some work need
> to improve error reporting. For instance running the dynamic linker
> against a static library like ldconfig just causes a segmentation fault
> (at least in 2.34, I haven't tried in HEAD). I haven't tried either
> binaries from other libc, but we should make sure that an error is
> correctly reported if users try that. At least a binary from a different
> architecture correctly reports the error.

I filed  for that.

I have a patch that uses execve in those cases.  So ”ld.so
/sbin/ldconfig” will work in the future.  This has some backwards
compatibility impact: In theory there might be objects which do not have
dynamic dependencies, but use a program interpreter, using custom
startup code.  Running those objects would lose dynamic loader
customization.

“ld.so /path/to/lib.so” crashes for completely different reasons: due to
a recently fixed binutils bug, shared objects have a fictitious entry
point set by BFD ld.  Due to the bug, the address is in an loadable,
executable segment (it's the start of the .text section), so we can't
detect this situation in the dynamic loader.  This will only go away
once shared objects are linked with a fixed ld.

>> Having ld.so as a real command makes the name architecture-agnostic.
>> This discourages from hard-coding non-portable paths such as
>> /lib64/ld-linux-x86-64.so.2 or even (the non-ABI-compliant)
>> /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 in scripts that require
>> specific functionality offered by such an explicit loader invocation.
>
> Do you actually have example of usage of the non-ABI-compliant dynamic
> loader in Debian? Independently of the current discussion, this should
> probably be fixed.

There are few suspicious examples in source code:

  

I have not scanned ELF binaries in built packages.

>> I thought that commands with file extensions might be Policy violation.
>> Policy actually talks about file extensions for programs installed in
>> /usr/bin—but only for scripts.  So it's technically okay.  And today,
>> there's already an ld.so manual page, although it's in section 8 and 1.
>> (I think /usr/bin is still appropriate because running ld.so does not
>> require special privileges.)
>> 
>> The initial implementation will be just a symbolic link.  This means
>> that multi-arch support will be missing: the amd64 loader will not be
>> able to redirect execution to the s390x loader.  In principle, it should
>> be possible to find PT_INTERP with a generic ELF parser and redirect to
>> that, but that's vaporware at present.  I don't know yet if it will be
>> possible to implement this without some knowledge of Debian's multi-arch
>> support in the loader.  Upstream doesn't have those features (we only
>> support /usr/lib vs /usr/lib64 and some minor variants of that), so
>> integration might be lacking.
>
> A simple symlink would work as a start, however it means creating a
> new non-multiarch package. If we want that feature to be available to
> all system, we need a way to ensure this package is automatically
> installed.
>
> What is the plan about supporting /usr/lib vs /usr/lib64? If upstream go
> the wrapper way, there might not be a lot of differences with what is
> needed to support multiarch. From what I understand a wrapper would need
> to have a basic understanding of the arguments passed to ld.so to find
> the binary that needs to be loaded. Looking at PT_INTERP is one way to
> go, however we should define what would be the behaviour for non GNU
> libc dynamic loader.

There is not going to be a wrapper.  We will integrate the logic into
ld.so itself, teaching it to execve a different loader.  All ld.so's
will have this capability, so it won't matter which one ends up as
/usr/bin/ld.so, except for performance reasons.

Basically, if we detect an incompatible architecture, we will fetch
PT_INTERP from the executable and execve that, using mostly the original
dynamic linker command line.  At least that's my plan.

>> If someone wants to upstream the multi-arch patches, that would be
>> great.  glibc now accepts submissions under DCO, so copyright assignment
>> should no longer be an obstacle.
>
> I don't think copyright assignment is really the issue there. We have
> two patches left that 

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Aurelien Jarno]

Hi,

On 2021-12-02 19:51, Florian Weimer wrote:
> I'd like to provide an ld.so command as part of glibc.  Today, ld.so can
> be used to activate preloading, for example.  Compared to LD_PRELOAD,
> the difference is that it's specific to one process, and won't be
> inherited by subprocesses—something is that exactly what is needed.
> There is also some useful diagnostic output in --help,
> --list-diagnostics.

This sounds a good idea, I guess for instance that in the future the ldd
feature could be implemented as an option.

However before exposing directly to user, there might be some work need
to improve error reporting. For instance running the dynamic linker
against a static library like ldconfig just causes a segmentation fault
(at least in 2.34, I haven't tried in HEAD). I haven't tried either
binaries from other libc, but we should make sure that an error is
correctly reported if users try that. At least a binary from a different
architecture correctly reports the error.

> Having ld.so as a real command makes the name architecture-agnostic.
> This discourages from hard-coding non-portable paths such as
> /lib64/ld-linux-x86-64.so.2 or even (the non-ABI-compliant)
> /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 in scripts that require
> specific functionality offered by such an explicit loader invocation.

Do you actually have example of usage of the non-ABI-compliant dynamic
loader in Debian? Independently of the current discussion, this should
probably be fixed.

> I thought that commands with file extensions might be Policy violation.
> Policy actually talks about file extensions for programs installed in
> /usr/bin—but only for scripts.  So it's technically okay.  And today,
> there's already an ld.so manual page, although it's in section 8 and 1.
> (I think /usr/bin is still appropriate because running ld.so does not
> require special privileges.)
> 
> The initial implementation will be just a symbolic link.  This means
> that multi-arch support will be missing: the amd64 loader will not be
> able to redirect execution to the s390x loader.  In principle, it should
> be possible to find PT_INTERP with a generic ELF parser and redirect to
> that, but that's vaporware at present.  I don't know yet if it will be
> possible to implement this without some knowledge of Debian's multi-arch
> support in the loader.  Upstream doesn't have those features (we only
> support /usr/lib vs /usr/lib64 and some minor variants of that), so
> integration might be lacking.

A simple symlink would work as a start, however it means creating a
new non-multiarch package. If we want that feature to be available to
all system, we need a way to ensure this package is automatically
installed.

What is the plan about supporting /usr/lib vs /usr/lib64? If upstream go
the wrapper way, there might not be a lot of differences with what is
needed to support multiarch. From what I understand a wrapper would need
to have a basic understanding of the arguments passed to ld.so to find
the binary that needs to be loaded. Looking at PT_INTERP is one way to
go, however we should define what would be the behaviour for non GNU
libc dynamic loader.

> If someone wants to upstream the multi-arch patches, that would be
> great.  glibc now accepts submissions under DCO, so copyright assignment
> should no longer be an obstacle.

I don't think copyright assignment is really the issue there. We have
two patches left that haven't been merged upstream, one from Steve
Langasek and one from me. The problem is more that there is work to add
proper upstream support for multiarch, I guess the idea would be to use
a configure option. On the Debian side multiarch support is basically
done by defining libdir, slibdir and rtlldir in the configparms file,
plus two patches:
- 
https://salsa.debian.org/glibc-team/glibc/-/blob/sid/debian/patches/any/local-ldconfig-multiarch.diff
  This patch basically changes the way ldconfig gets its standard search
  paths. I guess this is something that could be upstream, probably with
  some changes but I haven't recently look at it in details.
- 
https://salsa.debian.org/glibc-team/glibc/-/blob/sid/debian/patches/any/local-ld-multiarch.diff
  This patch does actually two things. The first part makes sure that
  ld.so still looks in the non-multiarch paths. This doesn't look
  upstreamable in the way its done. The second patch just change the way
  the initial '/' is dropped from $(slibdir) so that it works for two
  level paths. This part looks upstreamable.

> Anyway, do you see any problems with providing /usr/bin/ld.so for use by
> skilled end users?

As said above it might require a bit more work to make that possible,
but, but I do not see it as an issue as principle.

Regards,
Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Florian Weimer]

* Helmut Grohne:

> Hi Florian,
>
> On Fri, Dec 03, 2021 at 06:29:33PM +0100, Florian Weimer wrote:
>> We can add a generic ELF parser to that ld.so and use PT_INTERP, as I
>> mentioned below.  I think this is the way to go.  Some care will be
>> needed to avoid endless loops, but that should be it.
>
> Can I ask you to go into a bit more technical detail as to how this is
> supposed to work?

Sure!

> From what was said, I expect that /usr/bin/ld.so is an ELF executable.
> It will likely be part of libc-bin. Do you confirm?

Yes, that's what I expect as well.

> Since libc-bin is Multi-Arch: foreign. The new ld.so really must have an
> architecture-independent API. If it does not, it must not go there.

It is as architecture-independent as ldconfig or getconf.  Perhaps a bit
more so than getconf.

> As far as I understand things, the typical use will be "ld.so
> --preload somelib someprogram". Now consider an i386 ld.so, an amd64
> somelib and an amd64 someprogram. Will that work with the generic ELF
> parser?
>
> At present, it does not seem to work:
>
> $ /lib/ld-linux.so.2 --preload /usr/lib/x86_64-linux-gnu/libeatmydata.so 
> /bin/true
> /bin/true: error while loading shared libraries: /bin/true: wrong ELF class: 
> ELFCLASS64
> $

As has explained elsewhere, you need to use $LIB or just the soname (so
that ld.so searches the right paths).  I expect this to work eventually:

  ld.so --preload libeatmydata.so /bin/true

Even if /bin/true is an i386 program, assuming that libeatmydata1:i386
is installed.  Whether ld.so is built for i386 or amd64 will not matter.

> If that is what you will get from /usr/bin/ld.so, then it must not be
> part of libc-bin or Multi-Arch: foreign must be dropped. The latter
> likely is a non-option due to the amount of resulting breakage.

With the patch I've posted, you'll get the ELFCLASS64 error.  But I have
some ideas how to fix that eventually.  Is this sufficient for now?

Thanks,
Florian

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Helmut Grohne]

Hi Florian,

On Fri, Dec 03, 2021 at 06:29:33PM +0100, Florian Weimer wrote:
> We can add a generic ELF parser to that ld.so and use PT_INTERP, as I
> mentioned below.  I think this is the way to go.  Some care will be
> needed to avoid endless loops, but that should be it.

Can I ask you to go into a bit more technical detail as to how this is
supposed to work?

>From what was said, I expect that /usr/bin/ld.so is an ELF executable.
It will likely be part of libc-bin. Do you confirm?

Since libc-bin is Multi-Arch: foreign. The new ld.so really must have an
architecture-independent API. If it does not, it must not go there.

As far as I understand things, the typical use will be "ld.so
--preload somelib someprogram". Now consider an i386 ld.so, an amd64
somelib and an amd64 someprogram. Will that work with the generic ELF
parser?

At present, it does not seem to work:

$ /lib/ld-linux.so.2 --preload /usr/lib/x86_64-linux-gnu/libeatmydata.so 
/bin/true
/bin/true: error while loading shared libraries: /bin/true: wrong ELF class: 
ELFCLASS64
$

If that is what you will get from /usr/bin/ld.so, then it must not be
part of libc-bin or Multi-Arch: foreign must be dropped. The latter
likely is a non-option due to the amount of resulting breakage.

Helmut

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Theodore Y. Ts'o]

On Fri, Dec 03, 2021 at 05:59:17PM +0100, Florian Weimer wrote:
> * Theodore Y. Ts'o:
> 
> > * How does ld.so --preload *work*?
> 
> The dynamic loader has an array of preloaded sonames, and it processes
> them before loading the dependencies of the main program.  This way,
> definitions in the preloaded objects preempt definitions in the shared
> objects.
> 
> > * Does it modify /bin/ls, so that all users running /bin/ls get the
> > preloaded library?
> 
> No, it's purely a run-time change.

Ah, sorry, I was assuming it was more than just a run-time change.  So
this would effectively be equivalent to something like this, right?

(export LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libeatmydata.so.1 $LD_PRELOAD" ; 
/bin/ls)


By the way, it might be nice if ld.so queried something like 
$HOME/.config/ld.so.preload
as a user-specific version of /etc/ld.so.preload.

I guess there might be some potential security concerns, but if an
attacker has write access to a user's home directory, they probably
can do all sorts of mischief anyway

- Ted

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Simon McVittie]

On Fri, 03 Dec 2021 at 18:29:33 +0100, Florian Weimer wrote:
> > On Thu, 02 Dec 2021 at 19:51:16 +0100, Florian Weimer wrote:
> >> If someone wants to upstream the multi-arch patches, that would be
> >> great.
> >
> > I think multiarch is mostly build-time configuration rather than patches.
> 
> We would have to take the table out of dpkg-architecture and put it into
> upstream glibc (or gcc or binutils), otherwise you can't build a
> multi-arch glibc on a non-Debian system.

Sorry, you asked about patches, so I thought you were under the
impression that Debian was patching glibc to have it use multiarch
library directories. I believe it's mainly done with build-time
configuration rather than by patching, so there isn't necessarily
anything to upstream, because most of what's necessary to enable/allow
that build-time configuration is already upstream - unless you want
glibc to be generically aware of multiarch paths even when built on
non-Debian? Is that your goal here?

As I said, a 99% implementation of multiarch tuples is to take the GNU
tuple that any Autotools-based build system already relies on, discard the
vendor part, and normalize a finite number of special cases (i[3456]86
and arm* are the only ones I'm aware of). I believe the people who did
the early design of multiarch were hoping to standardize it via something
like LSB, but that effort seems approximately as dead as LSB itself.

systemd has an independent implementation of the list of known multiarch
tuples:
https://github.com/systemd/systemd/blob/main/src/basic/architecture.h
https://github.com/systemd/systemd/blob/main/src/basic/architecture.c

Some sort of change to the expansion of $LIB is maybe the only thing
needed in addition to build-time configuration, because the upstream
implementation of $LIB makes the assumption that only the last path
segment of the ${libdir} is desired in $LIB, which is usually the case
but happens to be untrue for multiarch.

> In addition to get the right value for $LIB, it's also desirable to get
> the default search paths right.

This is done by runtime configuration rather than patching, at the moment,
for example this file installed as part of libc6:amd64:

$ cat /etc/ld.so.conf.d/x86_64-linux-gnu.conf
# Multiarch support
/usr/local/lib/x86_64-linux-gnu
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu

If you would prefer this to be hard-coded into ldconfig, I suspect there's
no implementation right now that could be upstreamed.

> And there's also /usr/libexec/getconf to worry about.

At the moment, /usr/bin/getconf is only installed for the "main"
architecture (more precisely, for whichever architecture of libc-bin is
installed). If there's meant to be one getconf per architecture, then it
wouldn't be able to appear in /usr/bin or /usr/libexec with that name.
As with ld.so, this is not unique to multiarch: multilib would have the
same problem.

Is that /usr/libexec/getconf upstream, or is /usr/libexec/getconf
something else?

smcv

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Bastian Blank]

On Fri, Dec 03, 2021 at 11:33:25AM -0500, Theodore Y. Ts'o wrote:
> Some stupid questions that I couldn't answer by reading the man page
> or doing a quick google search
> * How does ld.so --preload *work*?

ld.so is the ELF interpreter.  If you run a normal binary, the kernel
rewrites this request to load and execute /lib64/ld-linux-x86-64.so.2
instead.  This interpreter then loads the real binary, the libraries and
jumps to the real code.

If you run ld.so directly, that kernel redirection is just not done.
The rest of it's tasks however are done just as usual.

--preload is just as if the interpreter would see LD_PRELOAD set.

> * Does it modify /bin/ls, so that all users running /bin/ls get the
> preloaded library?

No, preload does not modify binary files, just the loaded binary in
memory.

> * Does it modify something in the user's home directory?

No.

> * How do you undo the effects ld.so --preload?

You just don't use ld.so and don't set LD_PRELOAD.

Bastian

-- 
He's dead, Jim.
-- McCoy, "The Devil in the Dark", stardate 3196.1

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Florian Weimer]

* Simon McVittie:

> On Thu, 02 Dec 2021 at 19:51:16 +0100, Florian Weimer wrote:
>> Having ld.so as a real command makes the name architecture-agnostic.
>> This discourages from hard-coding non-portable paths such as
>> /lib64/ld-linux-x86-64.so.2 or even (the non-ABI-compliant)
>> /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 in scripts that require
>> specific functionality offered by such an explicit loader invocation.
>
> This works up to a point, but because there is only one /usr/bin/ld.so,
> it can only work for one architecture per machine, so saying it's
> architecture-agnostic is still a bit of a stretch.

We can add a generic ELF parser to that ld.so and use PT_INTERP, as I
mentioned below.  I think this is the way to go.  Some care will be
needed to avoid endless loops, but that should be it.

Things will break if people link with --dynamic-linker=/usr/bin/ld.so,
but that's just broken (like using --dynamic-linker=/lib/dl-2.33.so
today).

>> In principle, it should
>> be possible to find PT_INTERP with a generic ELF parser and redirect to
>> that, but that's vaporware at present.  I don't know yet if it will be
>> possible to implement this without some knowledge of Debian's multi-arch
>> support in the loader.
>
> I believe Debian uses the interoperable (ABI-compliant) ELF interpreter
> as listed on https://sourceware.org/glibc/wiki/ABIList for all
> architectures - it certainly does for all *common* architectures (for
> example our x86_64 executables use /lib64/ld-linux-x86-64.so.2, which is
> a special exception to the rule that we don't usually use lib64).

I'm not aware of any Debian divergence yet, either.

If we can just run any specified PT_INTERP and use something else for
loop detection (e.g., an additional argument), then it should probably
work out of the box.  I was just trying to set expectations because I
had not really thought about it in detail, in particular the loop
avoidance scheme and it whether it must know about all the known
loaders.

Some distributions also want to avoid code execution from ldd.  Another
thing to consider before lifting paths out of PT_INTERP.

> I had naively believed that all distros do the same, but unfortunately
> my work on the Steam Runtime has taught me otherwise: for example, Arch
> Linux has a non-standard ELF interpreter /usr/lib/ld-linux-x86-64.so for
> executables that are built from the glibc source package (but uses the
> interoperable ELF interpreter for everything else),

/usr/lib/ld-linux-x86-64.so could be a botched attempt at completing
UsrMove.  The upstream makefiles are not really set up for that.

> and Exherbo consistently puts their dynamic linkers in
> /usr/x86_64-pc-linux-gnu/lib.

No idea about that one.

> Does glibc automatically set up the interoperable ELF interpreter, or is
> it something that distros' glibc maintainers have to "just know" if they
> are using a non-default ${libdir}?

With ./configure --prefix=/usr, upstream glibc is expected to use the
official path in the file system (and it should no longer be a symbolic
link, either).  The just-built binaries should use that path, too.

But the dynamic linker pathname is not entirely unique, which creates
problems for Debian-style multi-arch.

>> If someone wants to upstream the multi-arch patches, that would be
>> great.  glibc now accepts submissions under DCO, so copyright assignment
>> should no longer be an obstacle.
>
> (Please note that I am not a glibc maintainer and cannot speak for them.)
>
> I think multiarch is mostly build-time configuration rather than patches.
> The main thing needing patching is that we want ${LIB} to expand to
> lib/x86_64-linux-gnu instead of just x86_64-linux-gnu, so that the
> "/usr/${LIB}/libfoo.so.0" idiom works, but glibc would normally only take
> the last component of the ${libdir}:
>
> https://salsa.debian.org/glibc-team/glibc/-/blob/sid/debian/patches/any/local-ld-multiarch.diff

That must get the data from somewhere else.  Looking at

  

it seems to come from DEB_HOST_MULTIARCH, and that's:

| DEB_HOST_MULTIARCH?= $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)

We would have to take the table out of dpkg-architecture and put it into
upstream glibc (or gcc or binutils), otherwise you can't build a
multi-arch glibc on a non-Debian system.  Something like a generic
--with-multiarch-tuple= configure option would sidestep that, but risk
that different distributions end up with different multi-arch tuples.

> The freedesktop.org SDK used for Flatpak also uses Debian-style multiarch
> (but is not otherwise Debian-derived), and addresses that differently, in a
> way that might be more upstream-suitable:
>
> https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/blob/master/patches/glibc/fix-dl-dst-lib.patch

The use of realpath again assumes that the file system already contains
the answer, which is not true if you have a different architecture or

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Florian Weimer]

* Theodore Y. Ts'o:

> * How does ld.so --preload *work*?

The dynamic loader has an array of preloaded sonames, and it processes
them before loading the dependencies of the main program.  This way,
definitions in the preloaded objects preempt definitions in the shared
objects.

> * Does it modify /bin/ls, so that all users running /bin/ls get the
> preloaded library?

No, it's purely a run-time change.

The global setting is in /etc/ld.so.preload.

> * Does it modify something in the user's home directory?

No.  Well, the shell might put that command into .bash_history, or
something like that.

> * How do you undo the effects ld.so --preload?

You run the program without the --preload option.  Or unset LD_PRELOAD.

Thanks,
Florian

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Simon McVittie]

On Thu, 02 Dec 2021 at 19:51:16 +0100, Florian Weimer wrote:
> Having ld.so as a real command makes the name architecture-agnostic.
> This discourages from hard-coding non-portable paths such as
> /lib64/ld-linux-x86-64.so.2 or even (the non-ABI-compliant)
> /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 in scripts that require
> specific functionality offered by such an explicit loader invocation.

This works up to a point, but because there is only one /usr/bin/ld.so,
it can only work for one architecture per machine, so saying it's
architecture-agnostic is still a bit of a stretch.

In multiarch, in principle there is no such thing as "the" primary
architecture (it is valid to combine sed:amd64 and coreutils:i386 on an
amd64 kernel), but in practice it's usually the case that "most"
executables come from the same architecture as dpkg.

So if we only have one ld.so, then on typical Debian x86_64 machines
it will only work for x86_64 executables, and not for i386 executables
(or cross-executables via qemu-aarch64-static or whatever). Similarly,
on Red Hat-style multilib, it will only work for x86_64 and not for
i386. Does that give you the functionality you are expecting?

One way to make it closer to architecture-agnostic would be to name it
${tuple}-ld.so, similar to how gcc (cross-)compilers are named. From
Debian's point of view, ideally the tuple would be a multiarch tuple,
which is a GNU tuple normalized to eliminate differences within an
ABI-compatible family of architectures:

- start from the GNU tuple, e.g. i686-pc-linux-gnu
- discard the vendor part, e.g. i686-linux-gnu
  - this version is the tools prefix used in cross-compilation
- replace i?86 with i386 and arm* with arm, e.g. i386-linux-gnu
  - this version is the Debian multiarch tuple

(Or perhaps better to have symlinks with both the cross-tools prefix
and the multiarch tuple, where they differ - which I believe is only
i386 and 32-bit ARM, because most/all other architectures sensibly change
the GNU tuple if and only if the ABI is different.)

> The initial implementation will be just a symbolic link.  This means
> that multi-arch support will be missing: the amd64 loader will not be
> able to redirect execution to the s390x loader.

... or to the i386 loader, which is probably a concern for more people
(that affects Red Hat-style multilib, which is present in some form on
most distros, and not just Debian-style multiarch, which is only seen in
Debian derivatives and the freedesktop.org SDK).

> In principle, it should
> be possible to find PT_INTERP with a generic ELF parser and redirect to
> that, but that's vaporware at present.  I don't know yet if it will be
> possible to implement this without some knowledge of Debian's multi-arch
> support in the loader.

I believe Debian uses the interoperable (ABI-compliant) ELF interpreter
as listed on https://sourceware.org/glibc/wiki/ABIList for all
architectures - it certainly does for all *common* architectures (for
example our x86_64 executables use /lib64/ld-linux-x86-64.so.2, which is
a special exception to the rule that we don't usually use lib64).

I had naively believed that all distros do the same, but unfortunately
my work on the Steam Runtime has taught me otherwise: for example, Arch
Linux has a non-standard ELF interpreter /usr/lib/ld-linux-x86-64.so for
executables that are built from the glibc source package (but uses the
interoperable ELF interpreter for everything else), and Exherbo
consistently puts their dynamic linkers in /usr/x86_64-pc-linux-gnu/lib.

Does glibc automatically set up the interoperable ELF interpreter, or is
it something that distros' glibc maintainers have to "just know" if they
are using a non-default ${libdir}?

> If someone wants to upstream the multi-arch patches, that would be
> great.  glibc now accepts submissions under DCO, so copyright assignment
> should no longer be an obstacle.

(Please note that I am not a glibc maintainer and cannot speak for them.)

I think multiarch is mostly build-time configuration rather than patches.
The main thing needing patching is that we want ${LIB} to expand to
lib/x86_64-linux-gnu instead of just x86_64-linux-gnu, so that the
"/usr/${LIB}/libfoo.so.0" idiom works, but glibc would normally only take
the last component of the ${libdir}:

https://salsa.debian.org/glibc-team/glibc/-/blob/sid/debian/patches/any/local-ld-multiarch.diff

The freedesktop.org SDK used for Flatpak also uses Debian-style multiarch
(but is not otherwise Debian-derived), and addresses that differently, in a
way that might be more upstream-suitable:

https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/blob/master/patches/glibc/fix-dl-dst-lib.patch

smcv

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Theodore Y. Ts'o]

On Fri, Dec 03, 2021 at 02:46:27PM +0100, Bastian Blank wrote:
> On Fri, Dec 03, 2021 at 01:57:08PM +0100, Florian Weimer wrote:
> > Right, thanks for providing a concrete example.  A (somewhat) portable
> > version would look like this:
> >   ld.so --preload '/usr/$LIB/libeatmydata.so.1.3.0' /bin/sl
> 
> You mean
>   ld.so --preload libeatmydata.so.1.3.0 /bin/ls

Some stupid questions that I couldn't answer by reading the man page
or doing a quick google search

* How does ld.so --preload *work*?

* Does it modify /bin/ls, so that all users running /bin/ls get the
preloaded library?

* Does it modify something in the user's home directory?

* How do you undo the effects ld.so --preload?

"Inquiring minds want to know"  :-)

Thanks,

- Ted

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Florian Weimer]

* Bastian Blank:

> On Fri, Dec 03, 2021 at 01:57:08PM +0100, Florian Weimer wrote:
>> Right, thanks for providing a concrete example.  A (somewhat) portable
>> version would look like this:
>>   ld.so --preload '/usr/$LIB/libeatmydata.so.1.3.0' /bin/sl
>
> You mean
>   ld.so --preload libeatmydata.so.1.3.0 /bin/ls
> ?

Right, that is even better.

No objects to /usr/bin/ld.so then?

Thanks,
Florian

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Bastian Blank]

On Fri, Dec 03, 2021 at 01:57:08PM +0100, Florian Weimer wrote:
> Right, thanks for providing a concrete example.  A (somewhat) portable
> version would look like this:
>   ld.so --preload '/usr/$LIB/libeatmydata.so.1.3.0' /bin/sl

You mean
  ld.so --preload libeatmydata.so.1.3.0 /bin/ls
?

ld.so is able to find the correct path itself.  So only looking up the
correct ld.so implementation would be needed.

Bastian

-- 
Behind every great man, there is a woman -- urging him on.
-- Harry Mudd, "I, Mudd", stardate 4513.3

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Florian Weimer]

* Paul Wise:

> Florian Weimer wrote:
>
>> I'd like to provide an ld.so command as part of glibc.
>
> Will this happen in glibc upstream or just in Debian?

Upstream, and then Debian.  The symbolic link would likely and up in
libc-bin in Debian.

>> Today, ld.so can be used to activate preloading, for example. 
>> Compared to LD_PRELOAD, the difference is that it's specific to one
>> process, and won't be inherited by subprocesses—something is that
>> exactly what is needed.
>
> That appears to be activated like this:
>
> /lib64/ld-linux-x86-64.so.2 --preload 
> /usr/lib/x86_64-linux-gnu/libeatmydata.so.1.3.0 /bin/ls

Right, thanks for providing a concrete example.  A (somewhat) portable
version would look like this:

  ld.so --preload '/usr/$LIB/libeatmydata.so.1.3.0' /bin/sl

This assumes that $LIB expands to the multi-arch subdirectory.
(In upstream, it switches between lib, lib64, libx32 as needed.)

>> Anyway, do you see any problems with providing /usr/bin/ld.so for use
>> by skilled end users?
>
> It means more folks get exposed to ld.so features, which might mean
> more support and feature requests for glibc upstream,. For example the
> set of features provided by environment variables is different to the
> set of features provided by command-line options.

The intent of this change is to expose these loader features to more
users and tools.  This came up for --list-diagnostics, where we'd
otherwise had to teach the sos tool (and others) how to find the loader
path.

Thanks,
Florian

Re: /usr/bin/ld.so as a symbolic link for the dynamic loader

[Paul Wise]

Florian Weimer wrote:

> I'd like to provide an ld.so command as part of glibc.

Will this happen in glibc upstream or just in Debian?

> Today, ld.so can be used to activate preloading, for example. 
> Compared to LD_PRELOAD, the difference is that it's specific to one
> process, and won't be inherited by subprocesses—something is that
> exactly what is needed.

That appears to be activated like this:

/lib64/ld-linux-x86-64.so.2 --preload 
/usr/lib/x86_64-linux-gnu/libeatmydata.so.1.3.0 /bin/ls

> Anyway, do you see any problems with providing /usr/bin/ld.so for use
> by skilled end users?

It means more folks get exposed to ld.so features, which might mean
more support and feature requests for glibc upstream,. For example the
set of features provided by environment variables is different to the
set of features provided by command-line options.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part