Re: [Toybox] Android binaries?

2020-09-08 Thread enh via Toybox 
On Thu, Sep 3, 2020 at 11:34 PM Rob Landley  wrote:

> On 9/3/20 10:02 AM, enh wrote:
> > Isn't it 6 digits now? 99 processes? (Didn't we have this
> discussion already
> > about ps? In theory top inherits that code...)
> >
> >
> > Yes, but my desktop has 7 digits (now at least --- I don't know whether
> my
> > desktop and laptop always differed like this, or this is from the ubuntu
> ->
> > debian switch).
>
> Are you willing to admit threads were a mistake yet? :)
>

to be clear, i only have 412 different pids on this machine. but the
current largest is 4116808. (the "mistake" is that pids are meant to be a
unique identifier but come from a small pool --- not having something like
pidfd since the 1970s.)


> > We could bump the hard-coded value again, but that seems like a lot of
> wasted
> > empty columns for small systems?
>
> I have confidence that a truly threaded system can hit 8 anyway.
>

well, someone will configure a system to use the full range of an int at
some point, yes.


> Alright, am I writing to the typos array already? That really looks like it
> should be in read-only memory but currently isn't, I forget why. (Vague
> "tried
> it and it didn't work" vibes but what specifically happened? I almost
> certainly
> have a todo item for it somewhere...)
>
> Pushed. Try now?
>

yeah, lgtm on both a phone and a "real computer". thanks!


> Rob
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Android binaries?

2020-09-04 Thread Rob Landley 
On 9/3/20 10:02 AM, enh wrote:
> Isn't it 6 digits now? 99 processes? (Didn't we have this discussion 
> already
> about ps? In theory top inherits that code...)
> 
> 
> Yes, but my desktop has 7 digits (now at least --- I don't know whether my
> desktop and laptop always differed like this, or this is from the ubuntu ->
> debian switch).

Are you willing to admit threads were a mistake yet? :)

> We could bump the hard-coded value again, but that seems like a lot of wasted
> empty columns for small systems?

I have confidence that a truly threaded system can hit 8 anyway.

Alright, am I writing to the typos array already? That really looks like it
should be in read-only memory but currently isn't, I forget why. (Vague "tried
it and it didn't work" vibes but what specifically happened? I almost certainly
have a todo item for it somewhere...)

Pushed. Try now?

Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Android binaries?

2020-09-03 Thread enh via Toybox 
On Thu, Sep 3, 2020, 01:04 Rob Landley  wrote:

> On 9/2/20 12:01 PM, enh wrote:
> > if you choose sdk_arm64 (or sdk_x86_64) off ci.android.com
> > , click on the little "download" icon on most
> recent
> > green build (or whichever build you want), the list of artifacts should
> now
> > contain toybox-static32 and toybox-static64.
>
> Yay! Ish. Hmmm... Ah, I was looking at sdk_x86, not sdk_x86_64. Yay, found
> it.
>
> > so some human effort required (afaik there's no URL that gives you the
> latest
> > green build), but a lot less effort than building AOSP...
>
>
> https://ci.android.com/builds/submitted/6810156/sdk_x86_64-sdk/latest/toybox-static64
> works but https://ci.android.com/builds/submitted/ is 404 which means
> this isn't
> a real web page but a database lookup emitted by CGI parsing the whole
> URL. So
> having a "latest" symlink under submitted isn't a question of a dumb little
> script to ln -sf after a successful build, instead somebody has to find the
> person who understands code that's never shipped to the outside world to
> dig
> through to where a new feature would need to be added in the plumbing.
>
> Sigh. If you hover over the top level cell that says "sdk" the hover text
> says
> "show last known good build", so there IS awareness that this is a feature
> people would want... but of course it's not a URL. It's javascript to
> redisplay
> the table showing just one row. Extra code to be LESS flexible.
>
> > ~/Downloads$ chmod a+x ./toybox-static64
> > ~/Downloads$ ./toybox-static64 date
> > Wed Sep  2 16:44:54 GMT 2020
> >
> > i'm not going to run anything else on this "real computer" or i'll be
> back
> > complaining that we should change top to not use KiB on a machine with
> 64GiB of
> > RAM,
>
> My current laptop (Dell Latitude E6230, I spent almost $200 on this thing
> _before_ upgrading the memory and hard drive) has 16 gigs of ram, same
> number of
> digits.
>
> The human_readable_long() plumbing will adjust the units to show the
> appropriate
> number of digits, the thing is we're telling it "8 digits" and 16229720K
> is 8
> digits (plus a unit).
>
> I think you want commas, which would take up digits and bump it down to
> 16,230M?
>
> > and that we need to examine /proc/sys/kernel/pid_max to know how wide pid
> > fields need to be again :-)
>
> Isn't it 6 digits now? 99 processes? (Didn't we have this discussion
> already
> about ps? In theory top inherits that code...)
>

Yes, but my desktop has 7 digits (now at least --- I don't know whether my
desktop and laptop always differed like this, or this is from the ubuntu ->
debian switch).

We could bump the hard-coded value again, but that seems like a lot of
wasted empty columns for small systems?

>
Rob
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Android binaries?

2020-09-03 Thread Rob Landley 
On 9/2/20 12:01 PM, enh wrote:
> if you choose sdk_arm64 (or sdk_x86_64) off ci.android.com
> , click on the little "download" icon on most recent
> green build (or whichever build you want), the list of artifacts should now
> contain toybox-static32 and toybox-static64.

Yay! Ish. Hmmm... Ah, I was looking at sdk_x86, not sdk_x86_64. Yay, found it.

> so some human effort required (afaik there's no URL that gives you the latest
> green build), but a lot less effort than building AOSP...

https://ci.android.com/builds/submitted/6810156/sdk_x86_64-sdk/latest/toybox-static64
works but https://ci.android.com/builds/submitted/ is 404 which means this isn't
a real web page but a database lookup emitted by CGI parsing the whole URL. So
having a "latest" symlink under submitted isn't a question of a dumb little
script to ln -sf after a successful build, instead somebody has to find the
person who understands code that's never shipped to the outside world to dig
through to where a new feature would need to be added in the plumbing.

Sigh. If you hover over the top level cell that says "sdk" the hover text says
"show last known good build", so there IS awareness that this is a feature
people would want... but of course it's not a URL. It's javascript to redisplay
the table showing just one row. Extra code to be LESS flexible.

> ~/Downloads$ chmod a+x ./toybox-static64 
> ~/Downloads$ ./toybox-static64 date
> Wed Sep  2 16:44:54 GMT 2020
> 
> i'm not going to run anything else on this "real computer" or i'll be back
> complaining that we should change top to not use KiB on a machine with 64GiB 
> of
> RAM,

My current laptop (Dell Latitude E6230, I spent almost $200 on this thing
_before_ upgrading the memory and hard drive) has 16 gigs of ram, same number of
digits.

The human_readable_long() plumbing will adjust the units to show the appropriate
number of digits, the thing is we're telling it "8 digits" and 16229720K is 8
digits (plus a unit).

I think you want commas, which would take up digits and bump it down to 16,230M?

> and that we need to examine /proc/sys/kernel/pid_max to know how wide pid
> fields need to be again :-)

Isn't it 6 digits now? 99 processes? (Didn't we have this discussion already
about ps? In theory top inherits that code...)

Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Android binaries?

2020-09-02 Thread enh via Toybox 
if you choose sdk_arm64 (or sdk_x86_64) off ci.android.com, click on the
little "download" icon on most recent green build (or whichever build you
want), the list of artifacts should now contain toybox-static32 and
toybox-static64.

so some human effort required (afaik there's no URL that gives you the
latest green build), but a lot less effort than building AOSP...

~/Downloads$ chmod a+x ./toybox-static64
~/Downloads$ ./toybox-static64 date
Wed Sep  2 16:44:54 GMT 2020

i'm not going to run anything else on this "real computer" or i'll be back
complaining that we should change top to not use KiB on a machine with
64GiB of RAM, and that we need to examine /proc/sys/kernel/pid_max to know
how wide pid fields need to be again :-)



On Fri, Aug 21, 2020 at 4:36 PM enh  wrote:

>
>
> On Thu, Aug 20, 2020 at 3:57 AM Rob Landley  wrote:
>
>> You said way back when that you were thinking of putting up downloadable
>> current
>> toybox android-built binaries somewhere. Did that ever happen?
>>
>
> the "build a static toybox" part did, but the "add it to the artifacts"
> part didn't. the NDK guy -- who spends 99% of his time with old devices and
> other people also stuck on old devices -- liked the idea but the build guy
> -- who actually gets the bill for storage -- was less excited.
>
>
>> I'm writing a "reporting bugs" FAQ entry because of the recent github
>> thread.
>>
>> I've also had a todo item to salvage todo entries I wrote for busybox
>> forever
>> ago, especially since the busybox devs crapped all over the current
>> versions.
>> For example,
>>
>> https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?h=95718b309169#n361
>> used to be project-agnostic (usable advice no matter what open source
>> project
>> you were talking about), but in current
>> https://busybox.net/FAQ.html#backporting
>> they've inserted a large digression in the middle about configuring
>> busybox
>> source from the command line to make sure it doesn't apply to any other
>> project
>> and can't be generally referenced as advice by other projects.
>>
>> But anyway, the advice was "try to reproduce the bug on a current version
>> before
>> poking the developers because there's a nonzero chance we already fixed
>> it", and
>> for linux toybox I can point them at
>> https://landley.net/toybox/downloads/binaries for current versions (even
>> if they
>> don't want to build it from source for their target)... but those are
>> linked
>> against musl?
>>
>> To check if it's been fixed on _android_, they need a bionic version. (I
>> mean
>> the musl versions will run but all sorts of subtle behavior's different.)
>
>
> yeah, i don't know --- Android's seccomp system call filter is pretty
> narrow and doesn't cover all of the "legacy" system calls
> that musl probably uses. i suspect arm32 musl doesn't work at all, but
> arm64 musl might mostly work?
>
>
>> And
>> even if I build a bionic version with the NDK, that hasn't got all the
>> libraries
>> the AOSP build uses. And the AOSP build here 1) takes FOREVER,  B) is
>> random git
>> snapshot du jour, bundling one with MY releases doesn't sync up with YOUR
>> releases in any useful way and could be broken because of transient fluff
>> du
>> jour, C) there's like 8 api levels for various previous releases still in
>> use
>> that I have no idea how to beat out of current AOSP source anyway, D) me
>> distributing android binaries seems like a layering violation somehow. I
>> do not
>> have the domain expertise to properly support or secure them beyond what
>> I'm
>> already doing.
>>
>
> the long term fix is just to get toybox into a mainline module along with
> libc. but that's not happening this year or next.
>
> i'll see if i can work out how to tell ci.android.com that
> toybox-static32 and toybox-static64 should only be in ndk builds...
>
>
>> Anyway, just wondering...
>>
>> Rob
>>
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Android binaries?

2020-08-24 Thread Rich Felker 
On Mon, Aug 24, 2020 at 10:03:11AM -0700, enh wrote:
> On Mon, Aug 24, 2020 at 9:53 AM Rich Felker  wrote:
> >
> > musl supports running on kernels back to 2.6.0, and historically uses
> > the earliest/simplest syscall that can provide the needed
> > functionality for the function being called. However, often it
> > simplifies code to have older functions implemented in terms of newer
> > more general ones, and in that case, order usually gets switched
> > around so that the newer syscall is tried first and the old one is
> > only used as a fallback if (1) the new one fails with ENOSYS, and (2)
> > arguments are such that the old syscall is suitable. The next release
> > is also moving all the socket functions from the multiplexed
> > socketcall to the newer individual syscalls, with socketcall as a
> > fallback, not for implementation ease reasons but because the separate
> > syscalls are the only ones that are filterable with seccomp.
> >
> 
> fwiw, i don't think that's actually true. afaik Android and ChromeOS both
> do this. looks like there's a CTS test too:
> 
> # socketcall: call=={SYS_CONNECT,SYS_SOCKET,SYS_GETSOCKOPT}
> socketcall: arg0 == 1 || arg0 == 3 || arg0 == 15; return EPERM
> 
> i thought the x86 non-socketcall socket stuff was 3.15 anyway? iirc that's
> why we're still using socketcall --- the only x86 device we shipped
> ourselves (Nexus Player) had a kernel that was too old.

The arguments to socketcall are all in a buffer in memory, which can
be changed asynchronously by another task with access to the memory
(i.e. any other thread in the process). Thus there is no security
property obtained by performing filtering on them; at best you can
prevent casual unintended behavior.

Rich
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Android binaries?

2020-08-24 Thread enh via Toybox 
On Mon, Aug 24, 2020 at 9:53 AM Rich Felker  wrote:

> On Sun, Aug 23, 2020 at 11:54:12PM -0500, Rob Landley wrote:
> > On 8/21/20 6:36 PM, enh wrote:
> > > I'm writing a "reporting bugs" FAQ entry because of the recent
> github thread.
> > >
> > > I've also had a todo item to salvage todo entries I wrote for
> busybox forever
> > > ago, especially since the busybox devs crapped all over the
> current versions.
> > > For example,
> > >
> https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?h=95718b309169#n361
> > > used to be project-agnostic (usable advice no matter what open
> source project
> > > you were talking about), but in current
> https://busybox.net/FAQ.html#backporting
> > > they've inserted a large digression in the middle about
> configuring busybox
> > > source from the command line to make sure it doesn't apply to any
> other project
> > > and can't be generally referenced as advice by other projects.
> > >
> > > But anyway, the advice was "try to reproduce the bug on a current
> version before
> > > poking the developers because there's a nonzero chance we already
> fixed it", and
> > > for linux toybox I can point them at
> > > https://landley.net/toybox/downloads/binaries for current
> versions (even if they
> > > don't want to build it from source for their target)... but those
> are linked
> > > against musl?
> > >
> > > To check if it's been fixed on _android_, they need a bionic
> version. (I mean
> > > the musl versions will run but all sorts of subtle behavior's
> different.)
> > >
> > > yeah, i don't know --- Android's seccomp system call filter is pretty
> narrow and
> > > doesn't cover all of the "legacy" system calls that musl probably
> uses. i
> > > suspect arm32 musl doesn't work at all, but arm64 musl might mostly
> work?
> >
> > Rich is fastidious about only using new syscalls when he can get away
> with it,
> > and keeping the set of used syscalls to a minimum. I'd have to ask him
> what the
> > actual set he's using is though.
> >
> > But mostly I was thinking that when trying to reproduce a bug, swapping
> out the
> > libc probably shouldn't be part of the standard reproduction sequence.
> Even when
> > the bug isn't in libc, what will/won't trigger it can change.
>
> musl supports running on kernels back to 2.6.0, and historically uses
> the earliest/simplest syscall that can provide the needed
> functionality for the function being called. However, often it
> simplifies code to have older functions implemented in terms of newer
> more general ones, and in that case, order usually gets switched
> around so that the newer syscall is tried first and the old one is
> only used as a fallback if (1) the new one fails with ENOSYS, and (2)
> arguments are such that the old syscall is suitable. The next release
> is also moving all the socket functions from the multiplexed
> socketcall to the newer individual syscalls, with socketcall as a
> fallback, not for implementation ease reasons but because the separate
> syscalls are the only ones that are filterable with seccomp.
>

fwiw, i don't think that's actually true. afaik Android and ChromeOS both
do this. looks like there's a CTS test too:

# socketcall: call=={SYS_CONNECT,SYS_SOCKET,SYS_GETSOCKOPT}
socketcall: arg0 == 1 || arg0 == 3 || arg0 == 15; return EPERM

i thought the x86 non-socketcall socket stuff was 3.15 anyway? iirc that's
why we're still using socketcall --- the only x86 device we shipped
ourselves (Nexus Player) had a kernel that was too old.


> Aside from that, the syscalls used by a particular function are an
> implementation detail that can and will change from version to
> version, and sometimes these changes are out of necessity to fix a
> bug. For example, it often comes up that newly realized
> requirements/corner-cases for async-signal-safety, locking corner
> cases, fork-related issues, etc. impose a requirement that a function
> block signals during some critical section. Another case where this
> happens is where it's discovered that there are kernel API bugs or
> corner cases the kernel does not handle that need special treatment in
> userspace, that require additional syscalls.
>
> Rich
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Android binaries?

2020-08-24 Thread Rich Felker 
On Sun, Aug 23, 2020 at 11:54:12PM -0500, Rob Landley wrote:
> On 8/21/20 6:36 PM, enh wrote:
> > I'm writing a "reporting bugs" FAQ entry because of the recent github 
> > thread.
> > 
> > I've also had a todo item to salvage todo entries I wrote for busybox 
> > forever
> > ago, especially since the busybox devs crapped all over the current 
> > versions.
> > For example,
> > 
> > https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?h=95718b309169#n361
> > used to be project-agnostic (usable advice no matter what open source 
> > project
> > you were talking about), but in current 
> > https://busybox.net/FAQ.html#backporting
> > they've inserted a large digression in the middle about configuring 
> > busybox
> > source from the command line to make sure it doesn't apply to any other 
> > project
> > and can't be generally referenced as advice by other projects.
> > 
> > But anyway, the advice was "try to reproduce the bug on a current 
> > version before
> > poking the developers because there's a nonzero chance we already fixed 
> > it", and
> > for linux toybox I can point them at
> > https://landley.net/toybox/downloads/binaries for current versions 
> > (even if they
> > don't want to build it from source for their target)... but those are 
> > linked
> > against musl?
> > 
> > To check if it's been fixed on _android_, they need a bionic version. 
> > (I mean
> > the musl versions will run but all sorts of subtle behavior's 
> > different.) 
> > 
> > yeah, i don't know --- Android's seccomp system call filter is pretty 
> > narrow and
> > doesn't cover all of the "legacy" system calls that musl probably uses. i
> > suspect arm32 musl doesn't work at all, but arm64 musl might mostly work?
> 
> Rich is fastidious about only using new syscalls when he can get away with it,
> and keeping the set of used syscalls to a minimum. I'd have to ask him what 
> the
> actual set he's using is though.
> 
> But mostly I was thinking that when trying to reproduce a bug, swapping out 
> the
> libc probably shouldn't be part of the standard reproduction sequence. Even 
> when
> the bug isn't in libc, what will/won't trigger it can change.

musl supports running on kernels back to 2.6.0, and historically uses
the earliest/simplest syscall that can provide the needed
functionality for the function being called. However, often it
simplifies code to have older functions implemented in terms of newer
more general ones, and in that case, order usually gets switched
around so that the newer syscall is tried first and the old one is
only used as a fallback if (1) the new one fails with ENOSYS, and (2)
arguments are such that the old syscall is suitable. The next release
is also moving all the socket functions from the multiplexed
socketcall to the newer individual syscalls, with socketcall as a
fallback, not for implementation ease reasons but because the separate
syscalls are the only ones that are filterable with seccomp.

Aside from that, the syscalls used by a particular function are an
implementation detail that can and will change from version to
version, and sometimes these changes are out of necessity to fix a
bug. For example, it often comes up that newly realized
requirements/corner-cases for async-signal-safety, locking corner
cases, fork-related issues, etc. impose a requirement that a function
block signals during some critical section. Another case where this
happens is where it's discovered that there are kernel API bugs or
corner cases the kernel does not handle that need special treatment in
userspace, that require additional syscalls.

Rich
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Android binaries?

2020-08-24 Thread enh via Toybox 
On Sun, Aug 23, 2020 at 9:45 PM Rob Landley  wrote:

> On 8/21/20 6:36 PM, enh wrote:
> >
> >
> > On Thu, Aug 20, 2020 at 3:57 AM Rob Landley  > > wrote:
> >
> > You said way back when that you were thinking of putting up
> downloadable current
> > toybox android-built binaries somewhere. Did that ever happen?
> >
> > the "build a static toybox" part did, but the "add it to the artifacts"
> part
> > didn't. the NDK guy -- who spends 99% of his time with old devices and
> other
> > people also stuck on old devices -- liked the idea but the build guy --
> who
> > actually gets the bill for storage -- was less excited.
>
> I've commented before that AOSP shipping dynamically linked prebuilt
> binaries is
> one of those "so close, yet so far" things...
>

for the host binaries? yeah, static would certainly help some people, but
as long as we're using glibc for the host...


> > I'm writing a "reporting bugs" FAQ entry because of the recent
> github thread.
> >
> > I've also had a todo item to salvage todo entries I wrote for
> busybox forever
> > ago, especially since the busybox devs crapped all over the current
> versions.
> > For example,
> >
> https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?h=95718b309169#n361
> > used to be project-agnostic (usable advice no matter what open
> source project
> > you were talking about), but in current
> https://busybox.net/FAQ.html#backporting
> > they've inserted a large digression in the middle about configuring
> busybox
> > source from the command line to make sure it doesn't apply to any
> other project
> > and can't be generally referenced as advice by other projects.
> >
> > But anyway, the advice was "try to reproduce the bug on a current
> version before
> > poking the developers because there's a nonzero chance we already
> fixed it", and
> > for linux toybox I can point them at
> > https://landley.net/toybox/downloads/binaries for current versions
> (even if they
> > don't want to build it from source for their target)... but those
> are linked
> > against musl?
> >
> > To check if it's been fixed on _android_, they need a bionic
> version. (I mean
> > the musl versions will run but all sorts of subtle behavior's
> different.)
> >
> > yeah, i don't know --- Android's seccomp system call filter is pretty
> narrow and
> > doesn't cover all of the "legacy" system calls that musl probably uses. i
> > suspect arm32 musl doesn't work at all, but arm64 musl might mostly work?
>
> Rich is fastidious about only using new syscalls when he can get away with
> it,
> and keeping the set of used syscalls to a minimum. I'd have to ask him
> what the
> actual set he's using is though.
>
> But mostly I was thinking that when trying to reproduce a bug, swapping
> out the
> libc probably shouldn't be part of the standard reproduction sequence.
> Even when
> the bug isn't in libc, what will/won't trigger it can change.
>
> > And
> > even if I build a bionic version with the NDK, that hasn't got all
> the libraries
> > the AOSP build uses. And the AOSP build here 1) takes FOREVER,  B)
> is random git
> > snapshot du jour, bundling one with MY releases doesn't sync up with
> YOUR
> > releases in any useful way and could be broken because of transient
> fluff du
> > jour, C) there's like 8 api levels for various previous releases
> still in use
> > that I have no idea how to beat out of current AOSP source anyway,
> D) me
> > distributing android binaries seems like a layering violation
> somehow. I do not
> > have the domain expertise to properly support or secure them beyond
> what I'm
> > already doing.
> >
> > the long term fix is just to get toybox into a mainline module along
> with libc.
> > but that's not happening this year or next.
> >
> > i'll see if i can work out how to tell ci.android.com <
> http://ci.android.com>
> > that toybox-static32 and toybox-static64 should only be in ndk builds...
>
> I make generally enthusiastic noises, with only the vaguest idea of what
> that
> means in context. Yay, doing the thing.
>
> Rob
>
> P.S. If you're talking about space consumed by AOSP prebuilts I have
> various
> suggestions, but that seems more a political can of worms than a technical
> one?
> I don't know the context/constraints they're working under.
>

not prebuilts, _builds_. every build that the build bots build is kept.
(exactly how long depends on exactly what kind of build. obviously some are
more important than others.)


> For example "why are you shipping 2 megs of
>
> prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9/bin/x86_64-linux-android-ld.bfd
> when ld links to .gold instead" is preceded by


depending on what you're doing, you might be using bfd, gold, or lld. lld
arrived before we'd managed to get everyone over to gold. lld is good
enough for the platform at this point but the NDK still hasn't

Re: [Toybox] Android binaries?

2020-08-23 Thread Rob Landley 
On 8/21/20 6:36 PM, enh wrote:
> 
> 
> On Thu, Aug 20, 2020 at 3:57 AM Rob Landley  > wrote:
> 
> You said way back when that you were thinking of putting up downloadable 
> current
> toybox android-built binaries somewhere. Did that ever happen?
> 
> the "build a static toybox" part did, but the "add it to the artifacts" part
> didn't. the NDK guy -- who spends 99% of his time with old devices and other
> people also stuck on old devices -- liked the idea but the build guy -- who
> actually gets the bill for storage -- was less excited.

I've commented before that AOSP shipping dynamically linked prebuilt binaries is
one of those "so close, yet so far" things...

> I'm writing a "reporting bugs" FAQ entry because of the recent github 
> thread.
> 
> I've also had a todo item to salvage todo entries I wrote for busybox 
> forever
> ago, especially since the busybox devs crapped all over the current 
> versions.
> For example,
> 
> https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?h=95718b309169#n361
> used to be project-agnostic (usable advice no matter what open source 
> project
> you were talking about), but in current 
> https://busybox.net/FAQ.html#backporting
> they've inserted a large digression in the middle about configuring 
> busybox
> source from the command line to make sure it doesn't apply to any other 
> project
> and can't be generally referenced as advice by other projects.
> 
> But anyway, the advice was "try to reproduce the bug on a current version 
> before
> poking the developers because there's a nonzero chance we already fixed 
> it", and
> for linux toybox I can point them at
> https://landley.net/toybox/downloads/binaries for current versions (even 
> if they
> don't want to build it from source for their target)... but those are 
> linked
> against musl?
> 
> To check if it's been fixed on _android_, they need a bionic version. (I 
> mean
> the musl versions will run but all sorts of subtle behavior's different.) 
> 
> yeah, i don't know --- Android's seccomp system call filter is pretty narrow 
> and
> doesn't cover all of the "legacy" system calls that musl probably uses. i
> suspect arm32 musl doesn't work at all, but arm64 musl might mostly work?

Rich is fastidious about only using new syscalls when he can get away with it,
and keeping the set of used syscalls to a minimum. I'd have to ask him what the
actual set he's using is though.

But mostly I was thinking that when trying to reproduce a bug, swapping out the
libc probably shouldn't be part of the standard reproduction sequence. Even when
the bug isn't in libc, what will/won't trigger it can change.

> And
> even if I build a bionic version with the NDK, that hasn't got all the 
> libraries
> the AOSP build uses. And the AOSP build here 1) takes FOREVER,  B) is 
> random git
> snapshot du jour, bundling one with MY releases doesn't sync up with YOUR
> releases in any useful way and could be broken because of transient fluff 
> du
> jour, C) there's like 8 api levels for various previous releases still in 
> use
> that I have no idea how to beat out of current AOSP source anyway, D) me
> distributing android binaries seems like a layering violation somehow. I 
> do not
> have the domain expertise to properly support or secure them beyond what 
> I'm
> already doing.
> 
> the long term fix is just to get toybox into a mainline module along with 
> libc.
> but that's not happening this year or next.
> 
> i'll see if i can work out how to tell ci.android.com 
> that toybox-static32 and toybox-static64 should only be in ndk builds...

I make generally enthusiastic noises, with only the vaguest idea of what that
means in context. Yay, doing the thing.

Rob

P.S. If you're talking about space consumed by AOSP prebuilts I have various
suggestions, but that seems more a political can of worms than a technical one?
I don't know the context/constraints they're working under.

For example "why are you shipping 2 megs of
prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9/bin/x86_64-linux-android-ld.bfd
when ld links to .gold instead" is preceded by "why are you shipping gcc when
llvm deprecated it and it was supposedly removed" which is preceded by "why does
the x86 directory have a mips directory next to it when I thought you end of
lifed that platform"... The dates on the mips files are may 6 2019 and I
downloaded this whole thing fresh in something like february...? I'm sure
there's a reason, but dunno what it is.

If the NDK guys are interested in experiments to save space, I note that
building an x86-64 targeted compiler as static i686 binaries linked against musl
can be SMALLER than dynamically native x86-64 binaries, and sometimes runs
faster (due to lighter cache footprint and less memory bus bandwidth for the
data structures) and does not

Re: [Toybox] Android binaries?

2020-08-21 Thread enh via Toybox 
On Thu, Aug 20, 2020 at 3:57 AM Rob Landley  wrote:

> You said way back when that you were thinking of putting up downloadable
> current
> toybox android-built binaries somewhere. Did that ever happen?
>

the "build a static toybox" part did, but the "add it to the artifacts"
part didn't. the NDK guy -- who spends 99% of his time with old devices and
other people also stuck on old devices -- liked the idea but the build guy
-- who actually gets the bill for storage -- was less excited.


> I'm writing a "reporting bugs" FAQ entry because of the recent github
> thread.
>
> I've also had a todo item to salvage todo entries I wrote for busybox
> forever
> ago, especially since the busybox devs crapped all over the current
> versions.
> For example,
>
> https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?h=95718b309169#n361
> used to be project-agnostic (usable advice no matter what open source
> project
> you were talking about), but in current
> https://busybox.net/FAQ.html#backporting
> they've inserted a large digression in the middle about configuring busybox
> source from the command line to make sure it doesn't apply to any other
> project
> and can't be generally referenced as advice by other projects.
>
> But anyway, the advice was "try to reproduce the bug on a current version
> before
> poking the developers because there's a nonzero chance we already fixed
> it", and
> for linux toybox I can point them at
> https://landley.net/toybox/downloads/binaries for current versions (even
> if they
> don't want to build it from source for their target)... but those are
> linked
> against musl?
>
> To check if it's been fixed on _android_, they need a bionic version. (I
> mean
> the musl versions will run but all sorts of subtle behavior's different.)


yeah, i don't know --- Android's seccomp system call filter is pretty
narrow and doesn't cover all of the "legacy" system calls
that musl probably uses. i suspect arm32 musl doesn't work at all, but
arm64 musl might mostly work?


> And
> even if I build a bionic version with the NDK, that hasn't got all the
> libraries
> the AOSP build uses. And the AOSP build here 1) takes FOREVER,  B) is
> random git
> snapshot du jour, bundling one with MY releases doesn't sync up with YOUR
> releases in any useful way and could be broken because of transient fluff
> du
> jour, C) there's like 8 api levels for various previous releases still in
> use
> that I have no idea how to beat out of current AOSP source anyway, D) me
> distributing android binaries seems like a layering violation somehow. I
> do not
> have the domain expertise to properly support or secure them beyond what
> I'm
> already doing.
>

the long term fix is just to get toybox into a mainline module along with
libc. but that's not happening this year or next.

i'll see if i can work out how to tell ci.android.com that toybox-static32
and toybox-static64 should only be in ndk builds...


> Anyway, just wondering...
>
> Rob
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net