On Wed, Dec 22, 2021 at 12:42 AM Waldek Kozaczuk <[email protected]> wrote:
> The OSV_LIBC_API macro makes sure that during the compilation phase the > posix_madvise symbol is left as GLOBAL DEFAULT in the libc/mman.o where all > other symbols are left hidden. The presence of posix_madvise in those 4 > symbols files is used to feed the version script that is used by the linker > to eliminate all other symbols but the ones listed in the sum of the > symbols files for given architecture - aarch64 or x64. The reason we have > given symbol occur almost always in 4 of those files [(x64 + aarch64) times > 2] is because most symbols are also present in ld-musl-x86_64.so.1 and our > dynamic linker advertises this library as well. And each of the symbol > files was originally generated using scripts/extract_symbols.sh which > identifies symbols to be exported against a version of kernel.elf built > with all symbols exported and given Linux library file from the host. At > the end of the day, it does not matter which file the given symbol is part > of as we collapse them all into a single version script. > > The OSV_LIBC_API used during the compilation phase and the symbol files > used by the linker may be seen as a duplicate mechanism and frankly, the > version script would have been enough to export (keep) symbols we want to. > However OSV_LIBC_API + -fvisibility-hidden "*lets the optimizer produce > better code*" (see https://gcc.gnu.org/wiki/Visibility) and this > mechanism but itself is also not enough because there are still some > symbols left (C++ typeinfo and other symbols, possibly originating from > libstdc++.a) what we need to eliminate using the version script and linker. > Thanks for the explanation. So, my next question would be if we can automatically produce the linker scripts, in one of two ways: 1. Recreate them every time on the user's build machine like you said you build them for the first time, or 2. Create at least parts of it from automatic grepping of OSV_LIBC_API et al. in the source files. The goal is that you won't need to remember to add a symbols in 6 different places for it to work. Keep in mind that we'll note really ave comprehensive tests for all symbols in all cases, so if I forget to add new_function() to libdl, or to aarch64, I might not notice this in my tests. If the answer is no, then my question would be if we could at least not duplicate the linker scripts. Can't x86 and aarch have the same list of exported functions? And if two libraries contain the same symbols, could we somehow automatically copy symbols between them? > On Tuesday, December 21, 2021 at 4:45:42 PM UTC-5 Nadav Har'El wrote: > >> Can you please remind me why we need both OSV_LIBC_API and putting the >> function name in 5 different files? >> Isn't there a big risk that later when all this export stuff is less >> fresh in our mind, we forget one of these 6 places and create a mess? >> >> -- >> Nadav Har'El >> [email protected] >> >> >> On Tue, Dec 21, 2021 at 8:26 PM Waldemar Kozaczuk <[email protected]> >> wrote: >> >>> Some apps/runtimes like dotnet call posix_madvise >>> which we do not implement. This patch adds a simple >>> implementation of it based on madvise with a difference >>> that only POSIX_MADV_DONTNEED is supported. On top of >>> this as required posix_madvise() returns an error without >>> setting errno. >>> >>> Signed-off-by: Waldemar Kozaczuk <[email protected]> >>> --- >>> .../aarch64/osv_ld-musl-aarch64.so.1.symbols | 1 + >>> exported_symbols/aarch64/osv_libc.so.6.symbols | 1 + >>> .../x64/osv_ld-musl-x86_64.so.1.symbols | 1 + >>> exported_symbols/x64/osv_libc.so.6.symbols | 1 + >>> libc/mman.cc | 14 ++++++++++++++ >>> 5 files changed, 18 insertions(+) >>> >>> diff --git a/exported_symbols/aarch64/osv_ld-musl-aarch64.so.1.symbols >>> b/exported_symbols/aarch64/osv_ld-musl-aarch64.so.1.symbols >>> index c463131a..4c284fba 100644 >>> --- a/exported_symbols/aarch64/osv_ld-musl-aarch64.so.1.symbols >>> +++ b/exported_symbols/aarch64/osv_ld-musl-aarch64.so.1.symbols >>> @@ -739,6 +739,7 @@ posix_fadvise >>> posix_fadvise64 >>> posix_fallocate >>> posix_fallocate64 >>> +posix_madvise >>> posix_memalign >>> pow >>> pow10 >>> diff --git a/exported_symbols/aarch64/osv_libc.so.6.symbols >>> b/exported_symbols/aarch64/osv_libc.so.6.symbols >>> index 000191b7..ea0bfcdc 100644 >>> --- a/exported_symbols/aarch64/osv_libc.so.6.symbols >>> +++ b/exported_symbols/aarch64/osv_libc.so.6.symbols >>> @@ -667,6 +667,7 @@ posix_fadvise >>> posix_fadvise64 >>> posix_fallocate >>> posix_fallocate64 >>> +posix_madvise >>> posix_memalign >>> ppoll >>> prctl >>> diff --git a/exported_symbols/x64/osv_ld-musl-x86_64.so.1.symbols >>> b/exported_symbols/x64/osv_ld-musl-x86_64.so.1.symbols >>> index b3f87859..d88e98ed 100644 >>> --- a/exported_symbols/x64/osv_ld-musl-x86_64.so.1.symbols >>> +++ b/exported_symbols/x64/osv_ld-musl-x86_64.so.1.symbols >>> @@ -721,6 +721,7 @@ posix_fadvise >>> posix_fadvise64 >>> posix_fallocate >>> posix_fallocate64 >>> +posix_madvise >>> __posix_getopt >>> posix_memalign >>> pow >>> diff --git a/exported_symbols/x64/osv_libc.so.6.symbols >>> b/exported_symbols/x64/osv_libc.so.6.symbols >>> index 6635cabb..07b5368b 100644 >>> --- a/exported_symbols/x64/osv_libc.so.6.symbols >>> +++ b/exported_symbols/x64/osv_libc.so.6.symbols >>> @@ -596,6 +596,7 @@ posix_fadvise >>> posix_fadvise64 >>> posix_fallocate >>> posix_fallocate64 >>> +posix_madvise >>> __posix_getopt >>> posix_memalign >>> ppoll >>> diff --git a/libc/mman.cc b/libc/mman.cc >>> index 9dd6429a..75a94eb0 100644 >>> --- a/libc/mman.cc >>> +++ b/libc/mman.cc >>> @@ -257,3 +257,17 @@ void *sbrk(intptr_t increment) >>> errno = ENOMEM; >>> return (void *)-1; >>> } >>> + >>> +static unsigned posix_madvise_to_advise(int advice) >>> +{ >>> + if (advice == POSIX_MADV_DONTNEED) { >>> + return mmu::advise_dontneed; >>> + } >>> + return 0; >>> +} >>> + >>> +OSV_LIBC_API >>> +int posix_madvise(void *addr, size_t len, int advice) { >>> + auto err = mmu::advise(addr, len, posix_madvise_to_advise(advice)); >>> + return err.get(); >>> +} >>> -- >>> 2.31.1 >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "OSv Development" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/osv-dev/20211221182614.242226-1-jwkozaczuk%40gmail.com >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "OSv Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/b41a7216-072b-41a0-b44b-f991a10845ddn%40googlegroups.com > <https://groups.google.com/d/msgid/osv-dev/b41a7216-072b-41a0-b44b-f991a10845ddn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CANEVyjvxDoiuGvo8vpkoxnuMp24Kx9ahq1yL1K0Ju5SjH4Cy0A%40mail.gmail.com.
