Re: sanitizers broken (was RE: libc/libsys split coming soon) [errno in libsys: any analogy to __elf_aux_vector ?]
On Thu, Feb 22, 2024 at 10:16:30AM -0800, Mark Millard wrote: > Brooks Davis wrote on > Date: Thu, 22 Feb 2024 02:03:09 UTC : > > > On Thu, Feb 22, 2024 at 02:57:13AM +0200, Konstantin Belousov wrote: > > > On Wed, Feb 21, 2024 at 08:20:25PM +, Brooks Davis wrote: > > > > That's probably worth a shot. Static linking will work anyway because > > > > libc.a in effect embeds libsys to retain compatability. > > > Please do not add libsys.so to the ABI. Right now it is an implementation > > > detail of libthr and libc, and it is preferable to not change it, at least > > > not yet, and definitely not to solve some minor internal issues. > > > > Indeed, on further reflection I agree. Another option occured to me > > which I intend to persue tomorrow: explicitly link the sanitizer .so > > files with libsys. At least in the base system that should be straight > > foward. > > Does the errno move to libsys have any problems similar to > the __elf_aux_vector move to libsys --that might also lead > to needing -lsys (for things as the are now)? I don't think so. With errno, there is still a copy in libc, it's just not used because the libsys on takes precidence. With __elf_aux_vector we were working around a different issue where the wrong copy was being updated by rtld which I resolved by moving it entierly. It's worth noting that any software that links with the errno symbol is buggy as errno is defined as a macro that calls a function as permitted by POSIX. I'm not convinced we should be exposing it for linkage at all. -- Brooks
Re: sanitizers broken (was RE: libc/libsys split coming soon) [errno in libsys: any analogy to __elf_aux_vector ?]
Brooks Davis wrote on Date: Thu, 22 Feb 2024 02:03:09 UTC : > On Thu, Feb 22, 2024 at 02:57:13AM +0200, Konstantin Belousov wrote: > > On Wed, Feb 21, 2024 at 08:20:25PM +, Brooks Davis wrote: > > > That's probably worth a shot. Static linking will work anyway because > > > libc.a in effect embeds libsys to retain compatability. > > Please do not add libsys.so to the ABI. Right now it is an implementation > > detail of libthr and libc, and it is preferable to not change it, at least > > not yet, and definitely not to solve some minor internal issues. > > Indeed, on further reflection I agree. Another option occured to me > which I intend to persue tomorrow: explicitly link the sanitizer .so > files with libsys. At least in the base system that should be straight > foward. Does the errno move to libsys have any problems similar to the __elf_aux_vector move to libsys --that might also lead to needing -lsys (for things as the are now)? For reference: https://lists.freebsd.org/archives/dev-commits-src-main/2024-February/022025.html === Mark Millard marklmi at yahoo.com
Re: sanitizers broken (was RE: libc/libsys split coming soon)
Am 2024-02-21 10:52, schrieb hartmut.bra...@dlr.de: Hi, I updated yesterday and now event a minimal program with cc -fsanitize=address produces ld: error: undefined symbol: __elf_aux_vector referenced by sanitizer_linux_libcdep.cpp:950 (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a cc: error: linker command failed with exit code 1 (use -v to see invocation) I think this is caused by the libsys split. There are other issues too. Discussed in multiple places. I opened https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277222 this morning, maybe it can be used to centralize the libsys issues (= I don't mind of you add a comment there, but maybe brooks wants to have a separate PR). Bye, Alexander. -- http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.orgnetch...@freebsd.org : PGP 0x8F31830F9F2772BF signature.asc Description: OpenPGP digital signature
Re: sanitizers broken (was RE: libc/libsys split coming soon)
On 21 Feb 2024, at 20:00, Brooks Davis wrote: > > The sanitizers reach somewhat questionably into libc internals that are > exported to allow rtld to update them. I was unable to find an solution > that didn't break this and I felt that fixing things like closefrom() > using non-deprecated syscalls was more important than avoiding changes > to the sanitizer interface. On Darwin, Apple added a special __interpose section that contains pairs of functions to be replaced and replacements. Within the library supplying the interposer, the symbol is resolved to the next version along, but everything that links to the interposing library sees the wrapped version. I wonder if it’s worth teaching rtld to do something equivalent. It’s a fairly lightweight generic mechanism that avoids a lot of the hacks that the sanitisers (and other things, such as instrumented malloc wrappers) do. David
Re: sanitizers broken (was RE: libc/libsys split coming soon)
On Thu, Feb 22, 2024 at 02:57:13AM +0200, Konstantin Belousov wrote: > On Wed, Feb 21, 2024 at 08:20:25PM +, Brooks Davis wrote: > > That's probably worth a shot. Static linking will work anyway because > > libc.a in effect embeds libsys to retain compatability. > Please do not add libsys.so to the ABI. Right now it is an implementation > detail of libthr and libc, and it is preferable to not change it, at least > not yet, and definitely not to solve some minor internal issues. Indeed, on further reflection I agree. Another option occured to me which I intend to persue tomorrow: explicitly link the sanitizer .so files with libsys. At least in the base system that should be straight foward. -- Brooks > > > > > -- Brooks > > > > On Wed, Feb 21, 2024 at 09:12:41PM +0100, Dimitry Andric wrote: > > > Can't we just add libsys.so to the /usr/lib/libc.so linker script? That > > > would work for everything except static linking? > > > > > > -Dimitry > > > > > > > On 21 Feb 2024, at 21:00, Brooks Davis wrote: > > > > > > > > TL;DR: you can work around this by adding -lsys to the link line and I > > > > aim to improve the situation soon. > > > > > > > > The sanitizers reach somewhat questionably into libc internals that are > > > > exported to allow rtld to update them. I was unable to find an solution > > > > that didn't break this and I felt that fixing things like closefrom() > > > > using non-deprecated syscalls was more important than avoiding changes > > > > to the sanitizer interface. > > > > > > > > I'm trying to find a way to better solution to the sanitizer. A few > > > > ideas I'm considering: > > > > - Teach clang to add -lsys when linking with sanitizers on sufficently > > > > new systems (con: doesn't fix gcc). > > > > - Make the symbol weak in the sanitizer and complain when it's not > > > > found or call back to using environ. The latter migth have > > > > limitations around direct exec with rtld. > > > > - Relocate __elf_aux_vector to csu so the symbol is always available. > > > > - Adding a new interface to access __elf_aux_vector directly. > > > > > > > > I'll continue to work on this. > > > > > > > > -- Brooks > > > > > > > > On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote: > > > >> Hi, > > > >> > > > >> I updated yesterday and now event a minimal program with > > > >> > > > >> cc -fsanitize=address > > > >> > > > >> produces > > > >> > > > >> ld: error: undefined symbol: __elf_aux_vector > > > > referenced by sanitizer_linux_libcdep.cpp:950 > > > > (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) > > > > sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in > > > > archive /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a > > > >> cc: error: linker command failed with exit code 1 (use -v to see > > > >> invocation) > > > >> > > > >> I think this is caused by the libsys split. > > > >> > > > >> Cheers, > > > >> Harti > > > >> > > > >> -Original Message- > > > >> From: owner-freebsd-curr...@freebsd.org > > > >> On Behalf Of Brooks Davis > > > >> Sent: Friday, February 2, 2024 11:32 PM > > > >> To: curr...@freebsd.org > > > >> Subject: libc/libsys split coming soon > > > >> > > > >> TL;DR: The implementation of system calls is moving to a seperate > > > >> library (libsys). No changes are required to existing software > > > >> (except to ensure that libsys is present when building custom disk > > > >> images). > > > >> > > > >> Code: https://github.com/freebsd/freebsd-src/pull/908 > > > >> > > > >> After nearly a decade of intermittent work, I'm about to land a series > > > >> of patches which moves system calls, vdso support, and libc's parsing > > > >> of the ELF auxiliary argument vector into a separate library (libsys). > > > >> I plan to do this early next week (February 5th). > > > >> > > > >> This change serves three primary purposes: > > > >> 1. It's easier to completely replace system call implementations for > > > >> tracing or compartmentalization purposes. > > > >> 2. It simplifies the implementation of restrictions on system calls > > > >> such > > > >> as those implemented by OpenBSD's msyscall(2) > > > >> (https://man.openbsd.org/msyscall.2). > > > >> 3. It allows language runtimes to link with libsys for system call > > > >> implementations without requiring libc. > > > >> > > > >> libsys is an auxiliary filter for libc. This means that for any > > > >> symbol defined by both, the libsys version takes precedence at > > > >> runtime. For system call implementations, libc contains empty stubs. > > > >> For others it contains copies of the functions (this could be further > > > >> refined at a later date). The statically linked libc contains the > > > >> full implementations so linking libsys is not required. > > > >> > > > >> Additionally, libthr is now linked with libsys to provide > > >
Re: sanitizers broken (was RE: libc/libsys split coming soon)
On Wed, Feb 21, 2024 at 08:20:25PM +, Brooks Davis wrote: > That's probably worth a shot. Static linking will work anyway because > libc.a in effect embeds libsys to retain compatability. Please do not add libsys.so to the ABI. Right now it is an implementation detail of libthr and libc, and it is preferable to not change it, at least not yet, and definitely not to solve some minor internal issues. > > -- Brooks > > On Wed, Feb 21, 2024 at 09:12:41PM +0100, Dimitry Andric wrote: > > Can't we just add libsys.so to the /usr/lib/libc.so linker script? That > > would work for everything except static linking? > > > > -Dimitry > > > > > On 21 Feb 2024, at 21:00, Brooks Davis wrote: > > > > > > TL;DR: you can work around this by adding -lsys to the link line and I > > > aim to improve the situation soon. > > > > > > The sanitizers reach somewhat questionably into libc internals that are > > > exported to allow rtld to update them. I was unable to find an solution > > > that didn't break this and I felt that fixing things like closefrom() > > > using non-deprecated syscalls was more important than avoiding changes > > > to the sanitizer interface. > > > > > > I'm trying to find a way to better solution to the sanitizer. A few > > > ideas I'm considering: > > > - Teach clang to add -lsys when linking with sanitizers on sufficently > > > new systems (con: doesn't fix gcc). > > > - Make the symbol weak in the sanitizer and complain when it's not > > > found or call back to using environ. The latter migth have > > > limitations around direct exec with rtld. > > > - Relocate __elf_aux_vector to csu so the symbol is always available. > > > - Adding a new interface to access __elf_aux_vector directly. > > > > > > I'll continue to work on this. > > > > > > -- Brooks > > > > > > On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote: > > >> Hi, > > >> > > >> I updated yesterday and now event a minimal program with > > >> > > >> cc -fsanitize=address > > >> > > >> produces > > >> > > >> ld: error: undefined symbol: __elf_aux_vector > > > referenced by sanitizer_linux_libcdep.cpp:950 > > > (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) > > > sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in > > > archive /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a > > >> cc: error: linker command failed with exit code 1 (use -v to see > > >> invocation) > > >> > > >> I think this is caused by the libsys split. > > >> > > >> Cheers, > > >> Harti > > >> > > >> -Original Message- > > >> From: owner-freebsd-curr...@freebsd.org > > >> On Behalf Of Brooks Davis > > >> Sent: Friday, February 2, 2024 11:32 PM > > >> To: curr...@freebsd.org > > >> Subject: libc/libsys split coming soon > > >> > > >> TL;DR: The implementation of system calls is moving to a seperate > > >> library (libsys). No changes are required to existing software (except > > >> to ensure that libsys is present when building custom disk images). > > >> > > >> Code: https://github.com/freebsd/freebsd-src/pull/908 > > >> > > >> After nearly a decade of intermittent work, I'm about to land a series > > >> of patches which moves system calls, vdso support, and libc's parsing of > > >> the ELF auxiliary argument vector into a separate library (libsys). I > > >> plan to do this early next week (February 5th). > > >> > > >> This change serves three primary purposes: > > >> 1. It's easier to completely replace system call implementations for > > >> tracing or compartmentalization purposes. > > >> 2. It simplifies the implementation of restrictions on system calls such > > >> as those implemented by OpenBSD's msyscall(2) > > >> (https://man.openbsd.org/msyscall.2). > > >> 3. It allows language runtimes to link with libsys for system call > > >> implementations without requiring libc. > > >> > > >> libsys is an auxiliary filter for libc. This means that for any symbol > > >> defined by both, the libsys version takes precedence at runtime. For > > >> system call implementations, libc contains empty stubs. For others it > > >> contains copies of the functions (this could be further refined at a > > >> later date). The statically linked libc contains the full > > >> implementations so linking libsys is not required. > > >> > > >> Additionally, libthr is now linked with libsys to provide _umtx_op_err(). > > >> > > >> The overall implementation follows https://reviews.freebsd.org/D14609, > > >> but is redone from scratch as multiple commits to facilitate review and > > >> assist git's rename detection. > > >> > > >> Testing: > > >> - Boot testing on amd64, aarch64, and riscv > > >> - make tinderbox (prior version, final run in progress) > > >> - exp-run: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276391 > > >> - Kyua tests in poudriere amd64 jails: same 359 failures as with the > > >>
Re: sanitizers broken (was RE: libc/libsys split coming soon)
That's probably worth a shot. Static linking will work anyway because libc.a in effect embeds libsys to retain compatability. -- Brooks On Wed, Feb 21, 2024 at 09:12:41PM +0100, Dimitry Andric wrote: > Can't we just add libsys.so to the /usr/lib/libc.so linker script? That would > work for everything except static linking? > > -Dimitry > > > On 21 Feb 2024, at 21:00, Brooks Davis wrote: > > > > TL;DR: you can work around this by adding -lsys to the link line and I > > aim to improve the situation soon. > > > > The sanitizers reach somewhat questionably into libc internals that are > > exported to allow rtld to update them. I was unable to find an solution > > that didn't break this and I felt that fixing things like closefrom() > > using non-deprecated syscalls was more important than avoiding changes > > to the sanitizer interface. > > > > I'm trying to find a way to better solution to the sanitizer. A few > > ideas I'm considering: > > - Teach clang to add -lsys when linking with sanitizers on sufficently > > new systems (con: doesn't fix gcc). > > - Make the symbol weak in the sanitizer and complain when it's not > > found or call back to using environ. The latter migth have > > limitations around direct exec with rtld. > > - Relocate __elf_aux_vector to csu so the symbol is always available. > > - Adding a new interface to access __elf_aux_vector directly. > > > > I'll continue to work on this. > > > > -- Brooks > > > > On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote: > >> Hi, > >> > >> I updated yesterday and now event a minimal program with > >> > >> cc -fsanitize=address > >> > >> produces > >> > >> ld: error: undefined symbol: __elf_aux_vector > > referenced by sanitizer_linux_libcdep.cpp:950 > > (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) > > sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in > > archive /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a > >> cc: error: linker command failed with exit code 1 (use -v to see > >> invocation) > >> > >> I think this is caused by the libsys split. > >> > >> Cheers, > >> Harti > >> > >> -Original Message- > >> From: owner-freebsd-curr...@freebsd.org > >> On Behalf Of Brooks Davis > >> Sent: Friday, February 2, 2024 11:32 PM > >> To: curr...@freebsd.org > >> Subject: libc/libsys split coming soon > >> > >> TL;DR: The implementation of system calls is moving to a seperate library > >> (libsys). No changes are required to existing software (except to ensure > >> that libsys is present when building custom disk images). > >> > >> Code: https://github.com/freebsd/freebsd-src/pull/908 > >> > >> After nearly a decade of intermittent work, I'm about to land a series of > >> patches which moves system calls, vdso support, and libc's parsing of the > >> ELF auxiliary argument vector into a separate library (libsys). I plan to > >> do this early next week (February 5th). > >> > >> This change serves three primary purposes: > >> 1. It's easier to completely replace system call implementations for > >> tracing or compartmentalization purposes. > >> 2. It simplifies the implementation of restrictions on system calls such > >> as those implemented by OpenBSD's msyscall(2) > >> (https://man.openbsd.org/msyscall.2). > >> 3. It allows language runtimes to link with libsys for system call > >> implementations without requiring libc. > >> > >> libsys is an auxiliary filter for libc. This means that for any symbol > >> defined by both, the libsys version takes precedence at runtime. For > >> system call implementations, libc contains empty stubs. For others it > >> contains copies of the functions (this could be further refined at a later > >> date). The statically linked libc contains the full implementations so > >> linking libsys is not required. > >> > >> Additionally, libthr is now linked with libsys to provide _umtx_op_err(). > >> > >> The overall implementation follows https://reviews.freebsd.org/D14609, > >> but is redone from scratch as multiple commits to facilitate review and > >> assist git's rename detection. > >> > >> Testing: > >> - Boot testing on amd64, aarch64, and riscv > >> - make tinderbox (prior version, final run in progress) > >> - exp-run: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276391 > >> - Kyua tests in poudriere amd64 jails: same 359 failures as with the > >>latest freebsdci build > >> > >> Thanks to Ali Mashtizadeh and Tal Garfinkel for D14609 and many apologies > >> for not landing this in a timely manner. Additional thanks to kib@ for > >> many rounds of review, markj@ and kib@ for debugging rtld issues exposed > >> by this patch, and antoine@ for exp-runs. > >> > >> Future work: > >> - Purely functional interfaces to system calls (no errorno). > >>Unfortunately there isn't an obvious way to do this without > >>
Re: sanitizers broken (was RE: libc/libsys split coming soon)
Can't we just add libsys.so to the /usr/lib/libc.so linker script? That would work for everything except static linking? -Dimitry > On 21 Feb 2024, at 21:00, Brooks Davis wrote: > > TL;DR: you can work around this by adding -lsys to the link line and I > aim to improve the situation soon. > > The sanitizers reach somewhat questionably into libc internals that are > exported to allow rtld to update them. I was unable to find an solution > that didn't break this and I felt that fixing things like closefrom() > using non-deprecated syscalls was more important than avoiding changes > to the sanitizer interface. > > I'm trying to find a way to better solution to the sanitizer. A few > ideas I'm considering: > - Teach clang to add -lsys when linking with sanitizers on sufficently > new systems (con: doesn't fix gcc). > - Make the symbol weak in the sanitizer and complain when it's not > found or call back to using environ. The latter migth have > limitations around direct exec with rtld. > - Relocate __elf_aux_vector to csu so the symbol is always available. > - Adding a new interface to access __elf_aux_vector directly. > > I'll continue to work on this. > > -- Brooks > > On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote: >> Hi, >> >> I updated yesterday and now event a minimal program with >> >> cc -fsanitize=address >> >> produces >> >> ld: error: undefined symbol: __elf_aux_vector > referenced by sanitizer_linux_libcdep.cpp:950 > (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) > sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive > /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a >> cc: error: linker command failed with exit code 1 (use -v to see invocation) >> >> I think this is caused by the libsys split. >> >> Cheers, >> Harti >> >> -Original Message- >> From: owner-freebsd-curr...@freebsd.org >> On Behalf Of Brooks Davis >> Sent: Friday, February 2, 2024 11:32 PM >> To: curr...@freebsd.org >> Subject: libc/libsys split coming soon >> >> TL;DR: The implementation of system calls is moving to a seperate library >> (libsys). No changes are required to existing software (except to ensure >> that libsys is present when building custom disk images). >> >> Code: https://github.com/freebsd/freebsd-src/pull/908 >> >> After nearly a decade of intermittent work, I'm about to land a series of >> patches which moves system calls, vdso support, and libc's parsing of the >> ELF auxiliary argument vector into a separate library (libsys). I plan to >> do this early next week (February 5th). >> >> This change serves three primary purposes: >> 1. It's easier to completely replace system call implementations for >> tracing or compartmentalization purposes. >> 2. It simplifies the implementation of restrictions on system calls such >> as those implemented by OpenBSD's msyscall(2) >> (https://man.openbsd.org/msyscall.2). >> 3. It allows language runtimes to link with libsys for system call >> implementations without requiring libc. >> >> libsys is an auxiliary filter for libc. This means that for any symbol >> defined by both, the libsys version takes precedence at runtime. For system >> call implementations, libc contains empty stubs. For others it contains >> copies of the functions (this could be further refined at a later date). >> The statically linked libc contains the full implementations so linking >> libsys is not required. >> >> Additionally, libthr is now linked with libsys to provide _umtx_op_err(). >> >> The overall implementation follows https://reviews.freebsd.org/D14609, >> but is redone from scratch as multiple commits to facilitate review and >> assist git's rename detection. >> >> Testing: >> - Boot testing on amd64, aarch64, and riscv >> - make tinderbox (prior version, final run in progress) >> - exp-run: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276391 >> - Kyua tests in poudriere amd64 jails: same 359 failures as with the >>latest freebsdci build >> >> Thanks to Ali Mashtizadeh and Tal Garfinkel for D14609 and many apologies >> for not landing this in a timely manner. Additional thanks to kib@ for many >> rounds of review, markj@ and kib@ for debugging rtld issues exposed by this >> patch, and antoine@ for exp-runs. >> >> Future work: >> - Purely functional interfaces to system calls (no errorno). >>Unfortunately there isn't an obvious way to do this without >>significant (possibly generated) assembly code. >> - Investigate msyscall(2) and pinsyscalls(2). >> - Reduce the size of stubs in libc. Ive errored on the >>side of not touching the copies that end up in libc to keep diff >>size down. We might want to generate empty stubs instead. >> >> See also: >> - Solaris Linker and Libraries Guide: >>
Re: sanitizers broken (was RE: libc/libsys split coming soon)
TL;DR: you can work around this by adding -lsys to the link line and I aim to improve the situation soon. The sanitizers reach somewhat questionably into libc internals that are exported to allow rtld to update them. I was unable to find an solution that didn't break this and I felt that fixing things like closefrom() using non-deprecated syscalls was more important than avoiding changes to the sanitizer interface. I'm trying to find a way to better solution to the sanitizer. A few ideas I'm considering: - Teach clang to add -lsys when linking with sanitizers on sufficently new systems (con: doesn't fix gcc). - Make the symbol weak in the sanitizer and complain when it's not found or call back to using environ. The latter migth have limitations around direct exec with rtld. - Relocate __elf_aux_vector to csu so the symbol is always available. - Adding a new interface to access __elf_aux_vector directly. I'll continue to work on this. -- Brooks On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote: > Hi, > > I updated yesterday and now event a minimal program with > > cc -fsanitize=address > > produces > > ld: error: undefined symbol: __elf_aux_vector > >>> referenced by sanitizer_linux_libcdep.cpp:950 > >>> (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) > >>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in > >>> archive /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a > cc: error: linker command failed with exit code 1 (use -v to see invocation) > > I think this is caused by the libsys split. > > Cheers, > Harti > > -Original Message- > From: owner-freebsd-curr...@freebsd.org > On Behalf Of Brooks Davis > Sent: Friday, February 2, 2024 11:32 PM > To: curr...@freebsd.org > Subject: libc/libsys split coming soon > > TL;DR: The implementation of system calls is moving to a seperate library > (libsys). No changes are required to existing software (except to ensure > that libsys is present when building custom disk images). > > Code: https://github.com/freebsd/freebsd-src/pull/908 > > After nearly a decade of intermittent work, I'm about to land a series of > patches which moves system calls, vdso support, and libc's parsing of the ELF > auxiliary argument vector into a separate library (libsys). I plan to do > this early next week (February 5th). > > This change serves three primary purposes: > 1. It's easier to completely replace system call implementations for > tracing or compartmentalization purposes. > 2. It simplifies the implementation of restrictions on system calls such > as those implemented by OpenBSD's msyscall(2) > (https://man.openbsd.org/msyscall.2). > 3. It allows language runtimes to link with libsys for system call > implementations without requiring libc. > > libsys is an auxiliary filter for libc. This means that for any symbol > defined by both, the libsys version takes precedence at runtime. For system > call implementations, libc contains empty stubs. For others it contains > copies of the functions (this could be further refined at a later date). The > statically linked libc contains the full implementations so linking libsys is > not required. > > Additionally, libthr is now linked with libsys to provide _umtx_op_err(). > > The overall implementation follows https://reviews.freebsd.org/D14609, > but is redone from scratch as multiple commits to facilitate review and > assist git's rename detection. > > Testing: > - Boot testing on amd64, aarch64, and riscv > - make tinderbox (prior version, final run in progress) > - exp-run: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276391 > - Kyua tests in poudriere amd64 jails: same 359 failures as with the > latest freebsdci build > > Thanks to Ali Mashtizadeh and Tal Garfinkel for D14609 and many apologies for > not landing this in a timely manner. Additional thanks to kib@ for many > rounds of review, markj@ and kib@ for debugging rtld issues exposed by this > patch, and antoine@ for exp-runs. > > Future work: > - Purely functional interfaces to system calls (no errorno). > Unfortunately there isn't an obvious way to do this without > significant (possibly generated) assembly code. > - Investigate msyscall(2) and pinsyscalls(2). > - Reduce the size of stubs in libc. Ive errored on the > side of not touching the copies that end up in libc to keep diff > size down. We might want to generate empty stubs instead. > > See also: > - Solaris Linker and Libraries Guide: > https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter4-4.html > > -- Brooks >
Re: sanitizers broken (was RE: libc/libsys split coming soon)
[Brooks' activity related to commit 99ea67573164637d633e8051eb0a5d52f1f9488e
looks likely for what changed the status: "lib{c,sys}: move auxargs more
firmly into libsys".]
On Feb 21, 2024, at 09:02, Mark Millard wrote:
> On Feb 21, 2024, at 08:38, Mark Millard wrote:
>
>> Mark Johnston wrote on
>> Date: Wed, 21 Feb 2024 13:33:43 UTC :
>>
>>> On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote:
Hi,
I updated yesterday and now event a minimal program with
cc -fsanitize=address
produces
ld: error: undefined symbol: __elf_aux_vector
>>> referenced by sanitizer_linux_libcdep.cpp:950
>>> (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
>>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
>>> /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a
cc: error: linker command failed with exit code 1 (use -v to see
invocation)
I think this is caused by the libsys split.
>>>
>>> I don't see any such problem on a system running 5f7ac491eef4, which
>>> includes the libsys split. Which compiler are you using, and which
>>> revision are you running?
>>
>> Trivial to reproduce via pkgbase install/upgrade:
>>
>> # uname -apKU
>> FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT
>> main-n268410-445d3d227e68 GENERIC-NODEBUG arm64 aarch64 1500014 1500014
>
> Gack: pkgbase does not have the kernel and world at the same place
> relative to git commits of source code. See my note from yesterday:
>
> https://lists.freebsd.org/archives/freebsd-pkgbase/2024-February/000319.html
>
> Looking at /usr/src/tests/sys/kern/sigsys.c in this pkgbse context
> shows the content from about 10 hours after 445d3d227e68's commit:
>
> git: e53b83a849e3 - main - tests/sigsys: initialize parameter passed to
> sysctlbyname() Gleb Smirnoff
> (CommitDate: 2024-02-20 22:37:45 +)
>
> (Note: The next commit's source was not present.)
>
>
>> # more main.c
>> int main(void) { return 0; }
>>
>> # cc -fsanitize=address main.c
>> ld: error: undefined symbol: __elf_aux_vector
> referenced by sanitizer_linux_libcdep.cpp:950
> (/home/bapt/worktrees/main/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
> /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-aarch64.a
> referenced by sanitizer_linux_libcdep.cpp:950
> (/home/bapt/worktrees/main/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
> /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-aarch64.a
>> cc: error: linker command failed with exit code 1 (use -v to see invocation)
>
> I do not see the issue in my (somehwat older) personal builds:
>
> # uname -apKU
> you have mail
> FreeBSD CA72-16Gp-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT #134
> main-n268363-8b67c670a49b-dirty: Sat Feb 17 16:22:31 PST 2024
> root@CA72-16Gp-ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72
> arm64 aarch64 1500014 1500014
>
> (Here kernel and world match git hashes: 8b67c670a49b . But
> my personal builds have patches and other tailoring relative
> to the official git hash they are based on.)
>
My guess is:
Brooks Davis
Date: Mon, 19 Feb 2024 22:44:35 UTC
The branch main has been updated by brooks:
URL:
https://cgit.FreeBSD.org/src/commit/?id=99ea67573164637d633e8051eb0a5d52f1f9488e
commit 99ea67573164637d633e8051eb0a5d52f1f9488e
Author: Brooks Davis
AuthorDate: 2024-02-19 22:44:08 +
Commit: Brooks Davis
CommitDate: 2024-02-19 22:44:08 +
lib{c,sys}: move auxargs more firmly into libsys
Continue to filter the public interface (elf_aux_info()), but entierly
relocate the private interfaces (_elf_aux_info(),
__init_elf_aux_vector(), and __elf_aux_vector) to libsys.
This ensures that rtld updates the correct (only) copy of
__elf_aux_vector. After 968a18975adc9c2a619bb52aa2f009de99fc9e24
updates were confused and __getosreldate was failing, causing
the system to fall back to compat compat12 syscalls in some cases.
Return to explicitly linking libc to libsys and link libthr with libc
and libsys (in that order).
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D43910
. .
===
Mark Millard
marklmi at yahoo.com
Re: sanitizers broken (was RE: libc/libsys split coming soon)
On Feb 21, 2024, at 08:38, Mark Millard wrote:
> Mark Johnston wrote on
> Date: Wed, 21 Feb 2024 13:33:43 UTC :
>
>> On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote:
>>> Hi,
>>>
>>> I updated yesterday and now event a minimal program with
>>>
>>> cc -fsanitize=address
>>>
>>> produces
>>>
>>> ld: error: undefined symbol: __elf_aux_vector
>> referenced by sanitizer_linux_libcdep.cpp:950
>> (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
>> /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a
>>> cc: error: linker command failed with exit code 1 (use -v to see invocation)
>>>
>>> I think this is caused by the libsys split.
>>
>> I don't see any such problem on a system running 5f7ac491eef4, which
>> includes the libsys split. Which compiler are you using, and which
>> revision are you running?
>
> Trivial to reproduce via pkgbase install/upgrade:
>
> # uname -apKU
> FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT
> main-n268410-445d3d227e68 GENERIC-NODEBUG arm64 aarch64 1500014 1500014
Gack: pkgbase does not have the kernel and world at the same place
relative to git commits of source code. See my note from yesterday:
https://lists.freebsd.org/archives/freebsd-pkgbase/2024-February/000319.html
Looking at /usr/src/tests/sys/kern/sigsys.c in this pkgbse context
shows the content from about 10 hours after 445d3d227e68's commit:
git: e53b83a849e3 - main - tests/sigsys: initialize parameter passed to
sysctlbyname() Gleb Smirnoff
(CommitDate: 2024-02-20 22:37:45 +)
(Note: The next commit's source was not present.)
> # more main.c
> int main(void) { return 0; }
>
> # cc -fsanitize=address main.c
> ld: error: undefined symbol: __elf_aux_vector
referenced by sanitizer_linux_libcdep.cpp:950
(/home/bapt/worktrees/main/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
/usr/lib/clang/17/lib/freebsd/libclang_rt.asan-aarch64.a
referenced by sanitizer_linux_libcdep.cpp:950
(/home/bapt/worktrees/main/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
/usr/lib/clang/17/lib/freebsd/libclang_rt.asan-aarch64.a
> cc: error: linker command failed with exit code 1 (use -v to see invocation)
I do not see the issue in my (somehwat older) personal builds:
# uname -apKU
you have mail
FreeBSD CA72-16Gp-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT #134
main-n268363-8b67c670a49b-dirty: Sat Feb 17 16:22:31 PST 2024
root@CA72-16Gp-ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72
arm64 aarch64 1500014 1500014
(Here kernel and world match git hashes: 8b67c670a49b . But
my personal builds have patches and other tailoring relative
to the official git hash they are based on.)
===
Mark Millard
marklmi at yahoo.com
Re: sanitizers broken (was RE: libc/libsys split coming soon)
Mark Johnston wrote on
Date: Wed, 21 Feb 2024 13:33:43 UTC :
> On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote:
> > Hi,
> >
> > I updated yesterday and now event a minimal program with
> >
> > cc -fsanitize=address
> >
> > produces
> >
> > ld: error: undefined symbol: __elf_aux_vector
> > >>> referenced by sanitizer_linux_libcdep.cpp:950
> > >>> (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
> > >>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
> > >>> /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a
> > cc: error: linker command failed with exit code 1 (use -v to see invocation)
> >
> > I think this is caused by the libsys split.
>
> I don't see any such problem on a system running 5f7ac491eef4, which
> includes the libsys split. Which compiler are you using, and which
> revision are you running?
Trivial to reproduce via pkgbase install/upgrade:
# uname -apKU
FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT
main-n268410-445d3d227e68 GENERIC-NODEBUG arm64 aarch64 1500014 1500014
# more main.c
int main(void) { return 0; }
# cc -fsanitize=address main.c
ld: error: undefined symbol: __elf_aux_vector
>>> referenced by sanitizer_linux_libcdep.cpp:950
>>> (/home/bapt/worktrees/main/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
>>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
>>> /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-aarch64.a
>>> referenced by sanitizer_linux_libcdep.cpp:950
>>> (/home/bapt/worktrees/main/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950)
>>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive
>>> /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-aarch64.a
cc: error: linker command failed with exit code 1 (use -v to see invocation)
===
Mark Millard
marklmi at yahoo.com
Re: sanitizers broken (was RE: libc/libsys split coming soon)
On Wed, 21 Feb 2024, Mark Johnston wrote: MJ>On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote: MJ>> Hi, MJ>> MJ>> I updated yesterday and now event a minimal program with MJ>> MJ>> cc -fsanitize=address MJ>> MJ>> produces MJ>> MJ>> ld: error: undefined symbol: __elf_aux_vector MJ>> >>> referenced by sanitizer_linux_libcdep.cpp:950 (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) MJ>> >>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a MJ>> cc: error: linker command failed with exit code 1 (use -v to see invocation) MJ>> MJ>> I think this is caused by the libsys split. MJ> MJ>I don't see any such problem on a system running 5f7ac491eef4, which MJ>includes the libsys split. Which compiler are you using, and which MJ>revision are you running? That is 445d3d227e68. The compiler is the system cc. Harti MJ> MJ>> Cheers, MJ>> Harti MJ>> MJ>> -Original Message- MJ>> From: owner-freebsd-curr...@freebsd.org On Behalf Of Brooks Davis MJ>> Sent: Friday, February 2, 2024 11:32 PM MJ>> To: curr...@freebsd.org MJ>> Subject: libc/libsys split coming soon MJ>> MJ>> TL;DR: The implementation of system calls is moving to a seperate library (libsys). No changes are required to existing software (except to ensure that libsys is present when building custom disk images). MJ>> MJ>> Code: https://github.com/freebsd/freebsd-src/pull/908 MJ>> MJ>> After nearly a decade of intermittent work, I'm about to land a series of patches which moves system calls, vdso support, and libc's parsing of the ELF auxiliary argument vector into a separate library (libsys). I plan to do this early next week (February 5th). MJ>> MJ>> This change serves three primary purposes: MJ>> 1. It's easier to completely replace system call implementations for MJ>> tracing or compartmentalization purposes. MJ>> 2. It simplifies the implementation of restrictions on system calls such MJ>> as those implemented by OpenBSD's msyscall(2) MJ>> (https://man.openbsd.org/msyscall.2). MJ>> 3. It allows language runtimes to link with libsys for system call MJ>> implementations without requiring libc. MJ>> MJ>> libsys is an auxiliary filter for libc. This means that for any symbol defined by both, the libsys version takes precedence at runtime. For system call implementations, libc contains empty stubs. For others it contains copies of the functions (this could be further refined at a later date). The statically linked libc contains the full implementations so linking libsys is not required. MJ>> MJ>> Additionally, libthr is now linked with libsys to provide _umtx_op_err(). MJ>> MJ>> The overall implementation follows https://reviews.freebsd.org/D14609, MJ>> but is redone from scratch as multiple commits to facilitate review and assist git's rename detection. MJ>> MJ>> Testing: MJ>> - Boot testing on amd64, aarch64, and riscv MJ>> - make tinderbox (prior version, final run in progress) MJ>> - exp-run: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276391 MJ>> - Kyua tests in poudriere amd64 jails: same 359 failures as with the MJ>> latest freebsdci build MJ>> MJ>> Thanks to Ali Mashtizadeh and Tal Garfinkel for D14609 and many apologies for not landing this in a timely manner. Additional thanks to kib@ for many rounds of review, markj@ and kib@ for debugging rtld issues exposed by this patch, and antoine@ for exp-runs. MJ>> MJ>> Future work: MJ>> - Purely functional interfaces to system calls (no errorno). MJ>> Unfortunately there isn't an obvious way to do this without MJ>> significant (possibly generated) assembly code. MJ>> - Investigate msyscall(2) and pinsyscalls(2). MJ>> - Reduce the size of stubs in libc. I’ve errored on the MJ>> side of not touching the copies that end up in libc to keep diff MJ>> size down. We might want to generate empty stubs instead. MJ>> MJ>> See also: MJ>> - Solaris Linker and Libraries Guide: MJ>> https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter4-4.html MJ>> MJ>> -- Brooks MJ>> MJ>
Re: sanitizers broken (was RE: libc/libsys split coming soon)
On Wed, Feb 21, 2024 at 09:52:23AM +, hartmut.bra...@dlr.de wrote: > Hi, > > I updated yesterday and now event a minimal program with > > cc -fsanitize=address > > produces > > ld: error: undefined symbol: __elf_aux_vector > >>> referenced by sanitizer_linux_libcdep.cpp:950 > >>> (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) > >>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in > >>> archive /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a > cc: error: linker command failed with exit code 1 (use -v to see invocation) > > I think this is caused by the libsys split. I don't see any such problem on a system running 5f7ac491eef4, which includes the libsys split. Which compiler are you using, and which revision are you running? > Cheers, > Harti > > -Original Message- > From: owner-freebsd-curr...@freebsd.org > On Behalf Of Brooks Davis > Sent: Friday, February 2, 2024 11:32 PM > To: curr...@freebsd.org > Subject: libc/libsys split coming soon > > TL;DR: The implementation of system calls is moving to a seperate library > (libsys). No changes are required to existing software (except to ensure > that libsys is present when building custom disk images). > > Code: https://github.com/freebsd/freebsd-src/pull/908 > > After nearly a decade of intermittent work, I'm about to land a series of > patches which moves system calls, vdso support, and libc's parsing of the ELF > auxiliary argument vector into a separate library (libsys). I plan to do > this early next week (February 5th). > > This change serves three primary purposes: > 1. It's easier to completely replace system call implementations for > tracing or compartmentalization purposes. > 2. It simplifies the implementation of restrictions on system calls such > as those implemented by OpenBSD's msyscall(2) > (https://man.openbsd.org/msyscall.2). > 3. It allows language runtimes to link with libsys for system call > implementations without requiring libc. > > libsys is an auxiliary filter for libc. This means that for any symbol > defined by both, the libsys version takes precedence at runtime. For system > call implementations, libc contains empty stubs. For others it contains > copies of the functions (this could be further refined at a later date). The > statically linked libc contains the full implementations so linking libsys is > not required. > > Additionally, libthr is now linked with libsys to provide _umtx_op_err(). > > The overall implementation follows https://reviews.freebsd.org/D14609, > but is redone from scratch as multiple commits to facilitate review and > assist git's rename detection. > > Testing: > - Boot testing on amd64, aarch64, and riscv > - make tinderbox (prior version, final run in progress) > - exp-run: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276391 > - Kyua tests in poudriere amd64 jails: same 359 failures as with the > latest freebsdci build > > Thanks to Ali Mashtizadeh and Tal Garfinkel for D14609 and many apologies for > not landing this in a timely manner. Additional thanks to kib@ for many > rounds of review, markj@ and kib@ for debugging rtld issues exposed by this > patch, and antoine@ for exp-runs. > > Future work: > - Purely functional interfaces to system calls (no errorno). > Unfortunately there isn't an obvious way to do this without > significant (possibly generated) assembly code. > - Investigate msyscall(2) and pinsyscalls(2). > - Reduce the size of stubs in libc. I’ve errored on the > side of not touching the copies that end up in libc to keep diff > size down. We might want to generate empty stubs instead. > > See also: > - Solaris Linker and Libraries Guide: > https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter4-4.html > > -- Brooks >
sanitizers broken (was RE: libc/libsys split coming soon)
Hi, I updated yesterday and now event a minimal program with cc -fsanitize=address produces ld: error: undefined symbol: __elf_aux_vector >>> referenced by sanitizer_linux_libcdep.cpp:950 >>> (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:950) >>> sanitizer_linux_libcdep.o:(__sanitizer::ReExec()) in archive >>> /usr/lib/clang/17/lib/freebsd/libclang_rt.asan-x86_64.a cc: error: linker command failed with exit code 1 (use -v to see invocation) I think this is caused by the libsys split. Cheers, Harti -Original Message- From: owner-freebsd-curr...@freebsd.org On Behalf Of Brooks Davis Sent: Friday, February 2, 2024 11:32 PM To: curr...@freebsd.org Subject: libc/libsys split coming soon TL;DR: The implementation of system calls is moving to a seperate library (libsys). No changes are required to existing software (except to ensure that libsys is present when building custom disk images). Code: https://github.com/freebsd/freebsd-src/pull/908 After nearly a decade of intermittent work, I'm about to land a series of patches which moves system calls, vdso support, and libc's parsing of the ELF auxiliary argument vector into a separate library (libsys). I plan to do this early next week (February 5th). This change serves three primary purposes: 1. It's easier to completely replace system call implementations for tracing or compartmentalization purposes. 2. It simplifies the implementation of restrictions on system calls such as those implemented by OpenBSD's msyscall(2) (https://man.openbsd.org/msyscall.2). 3. It allows language runtimes to link with libsys for system call implementations without requiring libc. libsys is an auxiliary filter for libc. This means that for any symbol defined by both, the libsys version takes precedence at runtime. For system call implementations, libc contains empty stubs. For others it contains copies of the functions (this could be further refined at a later date). The statically linked libc contains the full implementations so linking libsys is not required. Additionally, libthr is now linked with libsys to provide _umtx_op_err(). The overall implementation follows https://reviews.freebsd.org/D14609, but is redone from scratch as multiple commits to facilitate review and assist git's rename detection. Testing: - Boot testing on amd64, aarch64, and riscv - make tinderbox (prior version, final run in progress) - exp-run: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276391 - Kyua tests in poudriere amd64 jails: same 359 failures as with the latest freebsdci build Thanks to Ali Mashtizadeh and Tal Garfinkel for D14609 and many apologies for not landing this in a timely manner. Additional thanks to kib@ for many rounds of review, markj@ and kib@ for debugging rtld issues exposed by this patch, and antoine@ for exp-runs. Future work: - Purely functional interfaces to system calls (no errorno). Unfortunately there isn't an obvious way to do this without significant (possibly generated) assembly code. - Investigate msyscall(2) and pinsyscalls(2). - Reduce the size of stubs in libc. I’ve errored on the side of not touching the copies that end up in libc to keep diff size down. We might want to generate empty stubs instead. See also: - Solaris Linker and Libraries Guide: https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter4-4.html -- Brooks
