Re: [Toybox] [PATCH 1/2] macOS: use getconf rather than nproc for portability.
On Wed, Nov 28, 2018 at 5:33 PM Rob Landley wrote: > > On 11/28/18 7:11 PM, enh wrote: > > yeah, bitrot is my big fear too. i really don't want to have to > > actually get a mac. (even if i'm not paying for it, this current > > macbook pro genuinely is the most awful laptop i've ever used.) > > > > but i suspect i could always just update the mac toybox prebuilt "on > > demand", which i assume would work out to "once every couple of years > > or so". obviously i'll admit that to the build folks up front though > > :-) > > Meanwhile I continue to ponder porting android to Hexagon and such, and would > want it to "just work" as much as possible. (Yeah bionic would need new stuff, > but llvm already has https://twitter.com/landley/status/1062570074752589824 ) > So > "build from source in a genericish way" is worth extra cycles from me when > it's > low hanging fruit... > > I'm told the mac mini is back. Alas their cheapest/crappiest one is $800 so > I'm > not likely to get it. But as an ssh-able test machine, there should be one in > a > lab somewhere? (How do you regression test the mac build now?) we don't have automated testing for macOS. most engineers don't have a mac, and the continuous build infrastructure doesn't build for the mac. (the _NDK_ continuous build does. that actually matters --- most NDK users use Windows, but most of the rest use macOS rather than Linux.) basically we submit changes, and then the next day or so -- depending on how broken the mac build already is -- we might get a mail telling us we've broken the mac build. the mac build is usually broken. so, yeah, you'll get no good answer to this question from me :-) what i'm doing right now is hacking about on a loaner macbook that i'll have to give back soon. if maintaining toybox for mac becomes important enough, i can just give in and get a mac in addition to my big boy linux laptop. > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH 1/2] macOS: use getconf rather than nproc for portability.
On Wed, 28 Nov 2018 17:11:03 -0800 enh via Toybox wrote: > yeah, bitrot is my big fear too. i really don't want to have to > actually get a mac. (even if i'm not paying for it, this current > macbook pro genuinely is the most awful laptop i've ever used.) > > but i suspect i could always just update the mac toybox prebuilt "on > demand", which i assume would work out to "once every couple of years > or so". obviously i'll admit that to the build folks up front though > :-) > On Wed, Nov 28, 2018 at 4:43 PM Rob Landley wrote: > > > > On 11/28/18 5:18 PM, enh wrote: > > > yeah, sed seems to work okay. right now (with a bunch of source > > > hacks that i'll try to clean up and send to you) i have the > > > following subset of the AOSP build tools building: > > > > > > basename cat chmod cmp comm cut dirname dos2unix du echo egrep > > > false fgrep file grep head help hostname id ln md5sum mkdir > > > mktemp od paste patch pwd readlink realpath rm rmdir sed setsid > > > sha1sum sleep sort tee timeout true uname uniq unix2dos wc whoami > > > xargs xxd > > > > > > (note that i say "building". hostname at least doesn't work > > > yet.) > > > > I wonder if anybody has a macosx dev environment I could ssh into > > to test stuff out? Alas I used to. I have a Mac Mini I bought specifically to do cross platform development, mostly of virtual world stuff. A fellow developer used remote desktop to that Mac, to develop and test his bits of the software. Until a few months ago I had fibre to the bedroom, my own custom built router, enough IPv6 for every ant within two blocks, and permission from my ISP to run servers. So I just punched the correct sized holes in the firewall, and it all worked. I had to move, the owners of the building tossed us all out so they could knock it down to build a luxury mansion for them to live in. So now I have ADSL2+ shared to half a dozen units via WiFi that I have no control over, and the new ISP doesn't do IPv6 or allow servers. Luckily my fellow developer bought his own Mac before the move. I could bounce a ssh port through my European server, that I always have my own ssh session open to. A bit clunky, and no bandwidth guarantees if every one else in the building is watching Netflix that evening / weekend. The other problem, we are in the middle of an "extreme heatwave" right now. I turn my Mac off when it gets above 32 C, which it will more often than not for the next week. It got over 34 C today, still 34 inside my home. I just got home, I should open the window. My main Linux desktop I built specifically for these sorts of conditions, I don't have to turn it off when it gets hot. Apple doesn't build for the tropics. > > > > I tried to boot Darwin under qemu rather a lot, but as with BSD: > > > > https://landley.net/notes-2007.html#28-11-2007 > > https://lists.gnu.org/archive/html/qemu-devel/2008-02/msg00443.html > > > > I eventually lost interest... > > > > Rob > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world. pgpSrwRAaaowv.pgp Description: OpenPGP digital signature ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH 1/2] macOS: use getconf rather than nproc for portability.
On 11/28/18 7:11 PM, enh wrote: > yeah, bitrot is my big fear too. i really don't want to have to > actually get a mac. (even if i'm not paying for it, this current > macbook pro genuinely is the most awful laptop i've ever used.) > > but i suspect i could always just update the mac toybox prebuilt "on > demand", which i assume would work out to "once every couple of years > or so". obviously i'll admit that to the build folks up front though > :-) Meanwhile I continue to ponder porting android to Hexagon and such, and would want it to "just work" as much as possible. (Yeah bionic would need new stuff, but llvm already has https://twitter.com/landley/status/1062570074752589824 ) So "build from source in a genericish way" is worth extra cycles from me when it's low hanging fruit... I'm told the mac mini is back. Alas their cheapest/crappiest one is $800 so I'm not likely to get it. But as an ssh-able test machine, there should be one in a lab somewhere? (How do you regression test the mac build now?) Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH 1/2] macOS: use getconf rather than nproc for portability.
yeah, bitrot is my big fear too. i really don't want to have to actually get a mac. (even if i'm not paying for it, this current macbook pro genuinely is the most awful laptop i've ever used.) but i suspect i could always just update the mac toybox prebuilt "on demand", which i assume would work out to "once every couple of years or so". obviously i'll admit that to the build folks up front though :-) On Wed, Nov 28, 2018 at 4:43 PM Rob Landley wrote: > > On 11/28/18 5:18 PM, enh wrote: > > yeah, sed seems to work okay. right now (with a bunch of source hacks > > that i'll try to clean up and send to you) i have the following subset > > of the AOSP build tools building: > > > > basename cat chmod cmp comm cut dirname dos2unix du echo egrep false > > fgrep file grep head help hostname id ln md5sum mkdir mktemp od paste > > patch pwd readlink realpath rm rmdir sed setsid sha1sum sleep sort > > tee timeout true uname uniq unix2dos wc whoami xargs xxd > > > > (note that i say "building". hostname at least doesn't work yet.) > > I wonder if anybody has a macosx dev environment I could ssh into to test > stuff out? > > I tried to boot Darwin under qemu rather a lot, but as with BSD: > > https://landley.net/notes-2007.html#28-11-2007 > https://lists.gnu.org/archive/html/qemu-devel/2008-02/msg00443.html > > I eventually lost interest... > > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH 1/2] macOS: use getconf rather than nproc for portability.
On 11/28/18 5:18 PM, enh wrote: > yeah, sed seems to work okay. right now (with a bunch of source hacks > that i'll try to clean up and send to you) i have the following subset > of the AOSP build tools building: > > basename cat chmod cmp comm cut dirname dos2unix du echo egrep false > fgrep file grep head help hostname id ln md5sum mkdir mktemp od paste > patch pwd readlink realpath rm rmdir sed setsid sha1sum sleep sort > tee timeout true uname uniq unix2dos wc whoami xargs xxd > > (note that i say "building". hostname at least doesn't work yet.) I wonder if anybody has a macosx dev environment I could ssh into to test stuff out? I tried to boot Darwin under qemu rather a lot, but as with BSD: https://landley.net/notes-2007.html#28-11-2007 https://lists.gnu.org/archive/html/qemu-devel/2008-02/msg00443.html I eventually lost interest... Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH 1/2] macOS: use getconf rather than nproc for portability.
On Wed, Nov 28, 2018 at 2:42 PM Rob Landley wrote: > > On 11/28/18 2:16 PM, enh via Toybox wrote: > > Needed to build for macOS. > > --- > > scripts/make.sh | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/scripts/make.sh b/scripts/make.sh > > index 306a7cd..59b1fd2 100755 > > --- a/scripts/make.sh > > +++ b/scripts/make.sh > > @@ -15,7 +15,7 @@ source ./configure > > UNSTRIPPED="generated/unstripped/$(basename "$OUTNAME")" > > > > # Try to keep one more cc invocation going than we have processors > > -[ -z "$CPUS" ] && CPUS=$(($(nproc)+1)) > > +[ -z "$CPUS" ] && CPUS=$(($(getconf _NPROCESSORS_ONLN)+1)) > > Hmmm, I did that because: > > $ taskset 1 nproc > 1 > $ taskset 1 getconf _NPROCESSORS_ONLN > 8 > > Is there another... > > $ taskset 1 getconf -a | grep NPROC > _NPROCESSORS_CONF 8 > _NPROCESSORS_ONLN 8 > > Sigh. > > The theory was if nproc doesn't exist then $(nproc) resolves to an empty > string > and $((+1)) becomes 1 (not an error, I checked), so it should still work, just > single-threaded? And then you could "CPUS=1 make" to set it manually if > necessary... > > Is this an _optimization_ for macos, or is it a build break? (I used to have a > mac laptop but sent it to a college student on twitter who wore out her > computer.) if you're happy with just silently using one core on the mac, you could just 2>/dev/null to avoid the error messages: diff --git a/scripts/make.sh b/scripts/make.sh index 306a7cd..56de2e1 100755 --- a/scripts/make.sh +++ b/scripts/make.sh @@ -15,7 +15,7 @@ source ./configure UNSTRIPPED="generated/unstripped/$(basename "$OUTNAME")" # Try to keep one more cc invocation going than we have processors -[ -z "$CPUS" ] && CPUS=$(($(nproc)+1)) +[ -z "$CPUS" ] && CPUS=$(($(nproc 2>/dev/null)+1)) if [ -z "$SED" ] then > Would this work instead? > > if [ -z "$CPUS" ] > then > # nproc respects taskset, fall back to getconf for macos > [ ! -z "$(type -P nproc)" ] && CPUS=$(($(nproc)+1)) || > CPUS=$(($(getconf _NPROCESSORS_ONLN)) > fi yes, modulo your typo. diff --git a/scripts/make.sh b/scripts/make.sh index 306a7cd..69e649f 100755 --- a/scripts/make.sh +++ b/scripts/make.sh @@ -15,7 +15,12 @@ source ./configure UNSTRIPPED="generated/unstripped/$(basename "$OUTNAME")" # Try to keep one more cc invocation going than we have processors -[ -z "$CPUS" ] && CPUS=$(($(nproc)+1)) +if [ -z "$CPUS" ] +then + # nproc respects taskset, fall back to getconf for macos + [ ! -z "$(type -P nproc)" ] && CPUS=$(($(nproc)+1)) || +CPUS=$(($(getconf _NPROCESSORS_ONLN))) +fi if [ -z "$SED" ] then worked for me. > (I just dug up an aboriginal linux image and confirmed that type -P exists in > bash 2.05b from the dawn of time. Now with extra shellshock! I used to use > which > but Red Hat broke it so their version prints out crap when it _doesn't_ find > stuff. The aristocrats people behind systemd!) > > > if [ -z "$SED" ] > > then > > That was put in for macos previously. I _thought_ this worked... > > Hmmm... Toybox has a bootstrap issue where it provides tools that can build > toybox but sometimes has to run in environments that don't have those tools. I > keep thinking there should be some kind of build script that builds _just_ the > tools needed to build toybox, without requiring a configure step. > Unfortunately, > $CFLAGS and library probes and such remain fiddly enough I haven't done it > yet, > although I should be able to come up with a "no frills" hardwired build that's > portable-ish, for _just_ the build tools? (Yeah, which tools are the build > tools? That's mkroot territoy...) > > And this wouldn't help macos because the resulting nproc (calling > sched_getaffinity()) wouldn't _run_ on macos. (Sed probably would, unless its > libc's regex was crazy...) yeah, sed seems to work okay. right now (with a bunch of source hacks that i'll try to clean up and send to you) i have the following subset of the AOSP build tools building: basename cat chmod cmp comm cut dirname dos2unix du echo egrep false fgrep file grep head help hostname id ln md5sum mkdir mktemp od paste patch pwd readlink realpath rm rmdir sed setsid sha1sum sleep sort tee timeout true uname uniq unix2dos wc whoami xargs xxd (note that i say "building". hostname at least doesn't work yet.) > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH 1/2] macOS: use getconf rather than nproc for portability.
On 11/28/18 2:16 PM, enh via Toybox wrote: > Needed to build for macOS. > --- > scripts/make.sh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/scripts/make.sh b/scripts/make.sh > index 306a7cd..59b1fd2 100755 > --- a/scripts/make.sh > +++ b/scripts/make.sh > @@ -15,7 +15,7 @@ source ./configure > UNSTRIPPED="generated/unstripped/$(basename "$OUTNAME")" > > # Try to keep one more cc invocation going than we have processors > -[ -z "$CPUS" ] && CPUS=$(($(nproc)+1)) > +[ -z "$CPUS" ] && CPUS=$(($(getconf _NPROCESSORS_ONLN)+1)) Hmmm, I did that because: $ taskset 1 nproc 1 $ taskset 1 getconf _NPROCESSORS_ONLN 8 Is there another... $ taskset 1 getconf -a | grep NPROC _NPROCESSORS_CONF 8 _NPROCESSORS_ONLN 8 Sigh. The theory was if nproc doesn't exist then $(nproc) resolves to an empty string and $((+1)) becomes 1 (not an error, I checked), so it should still work, just single-threaded? And then you could "CPUS=1 make" to set it manually if necessary... Is this an _optimization_ for macos, or is it a build break? (I used to have a mac laptop but sent it to a college student on twitter who wore out her computer.) Would this work instead? if [ -z "$CPUS" ] then # nproc respects taskset, fall back to getconf for macos [ ! -z "$(type -P nproc)" ] && CPUS=$(($(nproc)+1)) || CPUS=$(($(getconf _NPROCESSORS_ONLN)) fi (I just dug up an aboriginal linux image and confirmed that type -P exists in bash 2.05b from the dawn of time. Now with extra shellshock! I used to use which but Red Hat broke it so their version prints out crap when it _doesn't_ find stuff. The aristocrats people behind systemd!) > if [ -z "$SED" ] > then That was put in for macos previously. I _thought_ this worked... Hmmm... Toybox has a bootstrap issue where it provides tools that can build toybox but sometimes has to run in environments that don't have those tools. I keep thinking there should be some kind of build script that builds _just_ the tools needed to build toybox, without requiring a configure step. Unfortunately, $CFLAGS and library probes and such remain fiddly enough I haven't done it yet, although I should be able to come up with a "no frills" hardwired build that's portable-ish, for _just_ the build tools? (Yeah, which tools are the build tools? That's mkroot territoy...) And this wouldn't help macos because the resulting nproc (calling sched_getaffinity()) wouldn't _run_ on macos. (Sed probably would, unless its libc's regex was crazy...) Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
