On Fri, Jul 24, 2020 at 12:02 AM Waldek Kozaczuk <[email protected]> wrote:
> Hi, > > As I have been researching what it would take to add floating-point > support to the AArch64 port > What is the problem with aarch64 floating point, and why is it related to musl? Is it just the esoteric functions like feenableexcept() and fegetenv() you mentioned - or are somehow all the math functions broken? Does the normal floating point - besides those functions - work correctly? Note that OSv uses floating point in the kernel (mostly for calculations in the scheduler...) and in x86 we had a lot of grief coming from this, needing to save the floating point state on context switches and things like that - I wonder if there isn't similar problems in aarch64 which have nothing to do with Musl. > , I started realizing that it might be better (not necessarily easier) to > upgrade musl. > This is definitely a good idea, but won't be trivial. I think I once wrote an outline of how this could be done, but I don't remember where I wrote it :-) The short story is that we already have in libc/ various files copied from an old version of musl which we changed for various reasons, and may or may not need to make similar changes to the newer musl. See commits 5a1d4c2451a94809d1ad580272410d8c51e5171b 51722a9552122cbd834329f4c2f13017f353855a 0b651428b91663255d8da9a4913663d0cd4cc710 as recent examples of things that caused us to change things in Musl. If you have the energy to work on this Musl update, I agree that it's a good idea. Also this time around we should try to make even more of an effort not to modify or fork Musl source files as much as we can. > At first, I hoped I could just add fegetenv support ( > https://linux.die.net/man/3/fegetenv) by adding fenv.S from the latest > musl. But very soon I have discovered we need to support LDBL_MANT_DIG=113 > (please see comments in include/api/aarch64/bits/float.h) which triggers > many missing symbols for LDBL_MANT_DIG == 113 (see also > musl/src/math/__invtrigl.* for example) which I added manually from latest > musl (would really require moving it to libc) but then there are 10 or so > other math-related files like musl/src/math/lgammal.c and vfprintf.c-like > ones that also need updates. And this is just for floating-point support > I'm afraid I don't understand the details here - does floating point *without* these math functions already work? > (no signals, setjmp, and other goodies we also need to support AArch64 > and might get from musl as well). > > This led me to question this approach and realize that it might be better > to simply upgrade musl to the latest version which has the latest AArch64 > support plus all bugs fixed in the last 7 years. > > So I was wondering what you think about my pretty controversial idea? > > Obviously it is not a small effort and somewhat risky as things might > break (what is a chance of that? any concrete examples?). But I think it is > necessary and there is some concrete upside to that. > The risk is that *everything* might break ;-) Every place where we have a file in libc/ instead of musl/, it was because we had a problem with the musl code and had to work around it to fix some bug - and you'll need to watch out not to recreate the same original bug. Hopefully, running the tests will find many of these problems. > > Obviously we cannot simply update musl git subproject to the latest > version as a significant number of files have been changed since and moved > to libc subdirectory (or also somewhere else?) or were manually added from > whatever version of musl at some point. So here is a high-level recipe on > how we could do it: > > For each *.c|*.cc file (and probably headers) under libc/ find > corresponding .c file under musl/src (sometimes the file will be .cc like > libc/misc/getopt.cc where we had to add C++ code): > > - if file not found under musl nothing has to be done (correct?) > - otherwise, find a delta between the two and apply to the copy in new > musl (this would be in most cases manual step, as very often the old file > in musl would have changed since 2013) > > Yes. Note that to complicate things even further, some of these modified musl source files actually did *not come* from the version of Musl we have in musl/, but rather an *older *version. So some of these differences you will see are not deliberate differences we did, but rather actually a few months of Musl evolution that has nothing to do with OSv. You can "git blame" / "git log" those libc files to see what changes we *really* did to them, and *why*. > As I understand many files under libc/ never came from musl - some were > written from scratch (see libc/mman.cc) and some may have come from > different place (see libc/locale/*_l.c which I think delegate to musl). > Right. Some of it we wrote ourselves. In particular, any musl function whose main function is to call some system call (like the mmap() stuff in mman.cc, posix threads stuff, etc.), doesn't make sense in OSv and we wrote a completely different implementation. > > Anything I have missed? > > How do we test all this? Besides unit tests we also have this - > https://github.com/cloudius-systems/osv/wiki/Automated-Testing-Framework - > which allows us to test OSv against 30+ apps. > > Do you have any suggestions for the strategy? Maybe something that would > make future upgrades easier? What about reviewing the changes - how to make > easier to review? Maybe a separate branch? Any step by step approach ideas? > You can start by renaming musl/ to musl-old/ (note that it's a submodule, not a directory!) and create a new musl/ submodule pointing to a recent release of musl, and then start to handle the fallout from this change... You'll probably need to change things to even get these new source files to compile (probably need to change header files, or something) and then start to think about all the files we copied to libc/, whether we can get rid of some of them (e.g., when a big was also fixed in Musl, or maybe you can upstream our changes!), and if not, apply our changes to the new version instead of the ancient version. > > I think that Geraldo Netto started working on this at some point, but I am > not sure where he left it off. > > Waldek > > PS. I am enclosing a file that is an approximation of a diff between libc > and musl/src directories which I achieved more-less in this way: > Please note that as I explained above, some of the files in libc/ were actually taken from an older musl than you see in musl/src, so some of the differences aren't real changes we really need, but just some spurious changes between the two versions in musl. Sometimes it's the combination of the two - we kept a file in libc/ because it had some changes we needed, but it also has some spurious changes vs. the version in musl/src, which we didn't really need. > find libc/ -type f -name \*c > /tmp/libc > > #remove libc prefix > > find_diff (which is:) > > ------ > > #!/bin/bash > > execute_diff() > > { > > local NAME="$1" > > echo "-----------------------------------------" > > echo "DIFF: $NAME" > > diff musl/src$NAME libc$NAME 2>&1 > > } > > export -f execute_diff > > cat /tmp/libc2 | xargs -I{} bash -c "execute_diff {}" > > -- > You received this message because you are subscribed to the Google Groups > "OSv Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/d61435ae-dc51-48a4-938b-e77eb7f2725an%40googlegroups.com > <https://groups.google.com/d/msgid/osv-dev/d61435ae-dc51-48a4-938b-e77eb7f2725an%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CANEVyjtiDb11prKdsB8qn7PxZxS47qJ0X7K2ddjw7%2BW9uRdZ0A%40mail.gmail.com.
