Re: [Toybox] scripts/llvm-buildall.sh?
On 8/4/21 6:10 PM, Patrick Oppenlander wrote: > On Wed, Aug 4, 2021 at 8:53 PM Rob Landley wrote: >> >> On 8/3/21 7:05 PM, Patrick Oppenlander wrote: >> >> What I'd LIKE to do is create a scripts/llvm-buildall.sh that builds all >> >> the >> >> supported musl+llvm targets the way mcm-buildall.sh does for musl+gcc. And >> >> getting llvm-project itself to do that was pretty straightforward (it >> >> builds >> >> them all by default). But clang-rt depends on the target libc headers >> >> being >> >> there first (...why?) and then the invocation is... yeah. >> > >> > I mentioned on the musl list that I've had a go at this too. >> >> Dig dig dig... >> >> https://www.openwall.com/lists/musl/2021/07/26/3 >> >> Ooh, very nice. Thanks. >> >> > Getting the llvm project & musl to play nice together is a pain. In >> > case you missed it, what I came up with is here: >> > >> > https://github.com/apexrtos/musl-cross-make/blob/clang/litecross/Makefile.clang >> >> Let's see... >> >> I made it to the giant span of "foreach" starting on line 49 and had to stop >> for >> health reasons. > > If you want the llvm build system to build multiple builtin or runtime > targets there's a lot of configuration required :( For the _compiler_, it's just: mkdir -p build-llvm && cd build-llvm && cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$PREFIX \ -DLLVM_ENABLE_PROJECTS="clang;lld" "$TOP/llvm-project/llvm" && ninja all install && cd .. && rm -rf build-llvm || exit 1 With $PREFIX and $TOP set appropriately, of course. The resulting clang --print-targets lists 39 options, from aarrcchh6644 to whatever "xcore" is. There's no reason their version of libgcc should be noticeably harder to configure, AND YET. (What do you mean by "runtime targets"?) > There's a bunch of cmake "caches" in the llvm project which set a > bunch of these variables for some canned configurations (e.g. see > llvm-project/clang/cmake/caches/BaremetalARM.cmake). Yeah, I rm -rf the build directory each time because otherwise it ignores what you tell cmake's command line. (Not a fan.) > But yes, I should probably use a single foreach there though. Not > really sure why I did it like that. I'm not a huge fan of trying to scale makefiles beyond trivial usage. Simple things like "if this file exists, set this variable" turn out to be SURPRISINGLY HARD TO DO. (And https://en.wikipedia.org/wiki/COMEFROM was originally a JOKE, yet it's fundamentally how Make works. Mixing imperative and declarative flow control in the same file...) >> The next question is, of course, WHY compiler-rt depends on the libc headers. >> And if it's GOING to depend on a header file: >> >> https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h >> >> Says it's MIT licensed right at the top... > > Nice. They'd need something similar for 15 other OS targets though. Do any of them other than Windows NOT fall under "Posix values complying with LP64"? Even all the RTOSes these days are posix+LP64. Compiling for BARE METAL is "whatever subset of posix we decided to provide for ourselves" plus LP64. What specifically does compiler-rt need out of these headers? (It was cmake's configure plumbing dying when I didn't install the headers before trying to build compiler-rt, not any actual C file trying to #include a header...) Let's see: grep -r 'include[ \t]*<' compiler-rt/lib/builtins #include stdint.h is pilot error (Even WINDOWS standardizes char/short/int size and __PTRDIFF_TYPE__ is another of those builtin #defines in the compiler, yes that includes clang.) compiler-rt/lib/builtins/int_endianness.h doesn't need to #include anything for the same reason: clang -dM -E - < /dev/null | grep ENDIAN #define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__ #define __LITTLE_ENDIAN__ 1 #define __ORDER_BIG_ENDIAN__ 4321 #define __ORDER_LITTLE_ENDIAN__ 1234 #define __ORDER_PDP_ENDIAN__ 3412 unwind-ehabi-helpersh #includes unwind.h but that isn't in libc, it's one of the /usr/lib/gcc/*/*/include headers. Why does compiler-rt/lib/builtins/os_version_check.c exist? ...eprintf? Really? // __eprintf() was used in an old version of . // It can eventually go away, but it is needed when linking // .o files built with the old . clear_cache.c? // The compiler generates calls to __clear_cache() when creating // trampoline functions on the stack for use with nested functions. // It is expected to invalidate the instruction cache for the // specified range. I'm going to stop looking at this now. There should probably be a minimal libgcc replacement project that ISN'T THIS, but gcc also had libgcc and libgcc_eh split into two libraries for a reason. (Not necessarily a GOOD reason given it was the FSF doing it, but still...) > Still, depending on the C library headers seems completely broken. > >> By the way, I tried to subscribe to the llvm-dev mailing list but >> https://lists.llvm.org/mailman/listinfo/llvm-dev is disabled and emailing >> that >>
Re: [Toybox] scripts/llvm-buildall.sh?
On Wed, Aug 4, 2021 at 8:53 PM Rob Landley wrote:
>
> On 8/3/21 7:05 PM, Patrick Oppenlander wrote:
> >> What I'd LIKE to do is create a scripts/llvm-buildall.sh that builds all
> >> the
> >> supported musl+llvm targets the way mcm-buildall.sh does for musl+gcc. And
> >> getting llvm-project itself to do that was pretty straightforward (it
> >> builds
> >> them all by default). But clang-rt depends on the target libc headers being
> >> there first (...why?) and then the invocation is... yeah.
> >
> > I mentioned on the musl list that I've had a go at this too.
>
> Dig dig dig...
>
> https://www.openwall.com/lists/musl/2021/07/26/3
>
> Ooh, very nice. Thanks.
>
> > Getting the llvm project & musl to play nice together is a pain. In
> > case you missed it, what I came up with is here:
> >
> > https://github.com/apexrtos/musl-cross-make/blob/clang/litecross/Makefile.clang
>
> Let's see...
>
> I made it to the giant span of "foreach" starting on line 49 and had to stop
> for
> health reasons.
If you want the llvm build system to build multiple builtin or runtime
targets there's a lot of configuration required :(
There's a bunch of cmake "caches" in the llvm project which set a
bunch of these variables for some canned configurations (e.g. see
llvm-project/clang/cmake/caches/BaremetalARM.cmake).
But yes, I should probably use a single foreach there though. Not
really sure why I did it like that.
> > Glossing over the finer details this basically boils down to:
> > * configure llvm
> > * do an llvm "distribution" build
> > * configure musl
> > * install musl headers
> > * do an llvm "builtins" build
>
> The builtins depending on the libc headers is outright disgusting. (This does,
> however, imply that they don't depend on the LINUX headers, which is... My
> brain
> hurts.)
That's a definite WTF.
> The next question is, of course, WHY compiler-rt depends on the libc headers.
> And if it's GOING to depend on a header file:
>
> https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h
>
> Says it's MIT licensed right at the top...
Nice. They'd need something similar for 15 other OS targets though.
Still, depending on the C library headers seems completely broken.
> By the way, I tried to subscribe to the llvm-dev mailing list but
> https://lists.llvm.org/mailman/listinfo/llvm-dev is disabled and emailing that
> subscribe@ thing multiple times has not gotten a response. That development
> community is approximately as welcoming to outsiders as xfree86 was.
You're telling me.
I've submitted a bunch of patches to phabricator which are in various
states of landed (their word for merged), accepted (waiting for..
something?), and bikeshedding.
I've given up beating my head against it for now and am currently
rebasing this list whenever they release something:
https://github.com/apexrtos/musl-cross-make/tree/clang/patches/llvm-project-12.0.1.src
> > * build musl
> > * install kernel headers
> > * do an llvm "runtimes" build
> >
> > I also create symlinks & config files so that it looks like a normal
> > gcc style toolchain.
>
> The kernel needs to learn that $CROSS-cc is a good compiler name, and
> $CROSS-gcc
> is not. And:
>
> https://github.com/torvalds/linux/blob/v5.13/Makefile#L434
>
> Is just painful. Unfortunately, I'm not sure how you'd say "use the first of
> ${CROSS_COMPILE}{clang,gcc,cc} that exists" in Makefile syntax. The foreach
> stuff expands ALL of them, doesn't seem to have a "break" construct. (And of
> course musl does it in shell script, which is the sane way to do it. :)
It's even worse because llvm doesn't install symlinked binaries, so to
use clang you need to "clang --target=arm-xxx" ala
https://github.com/torvalds/linux/blob/62fb9874f5da54fdb243003b386128037319b219/Makefile#L578
I work around that in my build by constructing symlinks and .cfg files
to make clang look like a traditional triple prefixed toolchain.
> > It would be nice if there was one well maintained place to go for an
> > llvm/musl build recipe.
>
> I dunno about "well maintained" but if I can get it to work I'm happy to
> provide
> a script, and possibly even host binaries although a gigabyte tarball is a bit
> of a stretch for my hosting...
I think my mcm fork should do what you need, but I know you prefer
more minimal solutions to these problems.
Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] scripts/llvm-buildall.sh?
On 8/3/21 7:05 PM, Patrick Oppenlander wrote:
>> What I'd LIKE to do is create a scripts/llvm-buildall.sh that builds all the
>> supported musl+llvm targets the way mcm-buildall.sh does for musl+gcc. And
>> getting llvm-project itself to do that was pretty straightforward (it builds
>> them all by default). But clang-rt depends on the target libc headers being
>> there first (...why?) and then the invocation is... yeah.
>
> I mentioned on the musl list that I've had a go at this too.
Dig dig dig...
https://www.openwall.com/lists/musl/2021/07/26/3
Ooh, very nice. Thanks.
> Getting the llvm project & musl to play nice together is a pain. In
> case you missed it, what I came up with is here:
>
> https://github.com/apexrtos/musl-cross-make/blob/clang/litecross/Makefile.clang
Let's see...
I made it to the giant span of "foreach" starting on line 49 and had to stop for
health reasons.
> Glossing over the finer details this basically boils down to:
> * configure llvm
> * do an llvm "distribution" build
> * configure musl
> * install musl headers
> * do an llvm "builtins" build
The builtins depending on the libc headers is outright disgusting. (This does,
however, imply that they don't depend on the LINUX headers, which is... My brain
hurts.)
The next question is, of course, WHY compiler-rt depends on the libc headers.
And if it's GOING to depend on a header file:
https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h
Says it's MIT licensed right at the top...
By the way, I tried to subscribe to the llvm-dev mailing list but
https://lists.llvm.org/mailman/listinfo/llvm-dev is disabled and emailing that
subscribe@ thing multiple times has not gotten a response. That development
community is approximately as welcoming to outsiders as xfree86 was.
> * build musl
> * install kernel headers
> * do an llvm "runtimes" build
>
> I also create symlinks & config files so that it looks like a normal
> gcc style toolchain.
The kernel needs to learn that $CROSS-cc is a good compiler name, and $CROSS-gcc
is not. And:
https://github.com/torvalds/linux/blob/v5.13/Makefile#L434
Is just painful. Unfortunately, I'm not sure how you'd say "use the first of
${CROSS_COMPILE}{clang,gcc,cc} that exists" in Makefile syntax. The foreach
stuff expands ALL of them, doesn't seem to have a "break" construct. (And of
course musl does it in shell script, which is the sane way to do it. :)
> It would be nice if there was one well maintained place to go for an
> llvm/musl build recipe.
I dunno about "well maintained" but if I can get it to work I'm happy to provide
a script, and possibly even host binaries although a gigabyte tarball is a bit
of a stretch for my hosting...
Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
