Re: [Toybox] Impact of global struct size

[Patrick Oppenlander]

On Thu, Jan 4, 2024 at 4:30 AM Rob Landley  wrote:
>
> I note that I've written over a hundred lines of rant in response to his
> previous email already. I should dig back through this and turn it into proper
> documentation at some point. (Especially since Elliott knows more of this 
> stuff
> than I do so I'm likely to get corrected a lot here...)
>
> On 1/2/24 20:54, enh wrote:
> >> You can look at /proc/self/maps (and /proc/self/smaps, and
> >> /proc/self/smaps_rollup) to see them for a running process (replace "self" 
> >> with
> >> any running PID, self is a symlink to your current PID). The six sections 
> >> are:
> >>
> >>   text - the executable functions: mmap(MAP_PRIVATE, PROT_READ|PROT_EXEC)
> >>   rodata - const globals, string constants, etc: mmap(MAP_PRIVATE, 
> >> PROT_READ)
> >>   data - writeable data initialized to nonzero: mmap(MAP_PRIVATE, 
> >> PROT_WRITE)
> >>   bss - writeable data initialized to zero: mmap(MAP_ANON, PROT_WRITE)
> >>   stack - function call stack, also contains environment data
> >>   heap - backing store for malloc() and free()
> >
> > (Android and modern linux distros require the relro section too.
>
> I thought that was only needed for dynamic linking? Then again you don't 
> allow a
> lot of static stuff to run on the final system anyway...
>
> (The line between PIE and dynamic linking confuses even me. How does static 
> PIE
> relocate itself? I _think_ I looked it up once, but "it's statically links in 
> a
> dynamic linker in the pile of crt1.o and begin.o files" _can't_ be right...)
>
> > interestingly, there _is_ an elf program header for the stack, to
> > signal that you don't want an executable stack. iirc Android and [very
> > very recently] modern linux distros won't let you start a process with
> > an executable main stack, but afaik the code for the option no-one has
> > wanted/needed for a very long time is still in the kernel.)
>
> Cool.
>
> These days there's also vdso and vvar, which are provided by the kernel at
> runtime. The first is a .text section with magic functions you can call as an
> alternative to syscalls, and the second is a magic .rodata section that 
> provides
> volatile variables the OS updates which you can just reach out and look at.
>
> Between the two of them you can do things like check the current timestamp
> without a system call. What they actually provide varies by OS (and then your
> libc has to be taught to use each new capability out of there instead of 
> making
> the syscalls).
>
> "cat /proc/self/maps" and they're the last two entries if present.
>
> There is a "man 7 vdso" but I dunno how up to date it is. (Which gets us back 
> to
> Michael Kerrisk's retirement and the new guy NOT MAINTAINING A WEB COPY. 
> Grrr.)
>
> Maintaining backwards compatibility means keeping a lot of old stuff. I had a
> talk with Rich Felker last night on IRC about what musl-libc's syscall
> requirements actually _are_, and what it would take to repot it on top of a
> posix-ish RTOS du jour. (Makes the trusting trust cleansing cycle smaller if 
> you
> can cross compile Linux from an RTOS...)

I did the "run linux-musl binaries on an RTOS" part a few years ago
and ended up with this list:

https://github.com/apexrtos/apex/blob/master/sys/kern/syscall_table.c

It's by no means exhaustive, but it was enough to run a useful set of
toybox toys, busybox's ash and enough other stuff to build a
commercial product running on an armv7-m (nommu) chip on top of it. I
had a risc-v port working and was in the middle of getting powerpc
(mmu) stuff running when circumstances changed and I had to move on.

I'm not sure how many more syscalls would be required to be able to
compile Linux, but probably not a whole lot.

Patrick

> We didn't come to a conclusion, but I _did_ get permission from skarnet to use
> his git://git.skarnet.org/mdevd under 0BSD. (POrting that to toybox seems 
> easier
> than bringing my old mdev code up to speed for all the
> https://github.com/slashbeast/mdev-like-a-boss stuff it's grown since I handed
> it off.
>
> >> The first three of those literally exist in the ELF file, as in it mmap()s 
> >> a
> >> block of data out of the file at a starting offset, and the memory is thus
> >> automatically populated with data from the file. The text and rodata ones 
> >> don't
> >> really care if it's MAP_PRIVATE or MAP_SHARED because they can never write
> >> anything back to the file, but the data one cares that it's MAP_PRIVATE: 
> >> any
> >> changes stay local and do NOT get written back to the file. And the bss is 
> >> an
> >> anonymous mapping so starts zeroed, the file doesn't bother wasting space 
> >> on a
> >> run of zeroes when the OS can just provide that on request. (It stands for 
> >> Block
> >> Starting Symbol which I assume meant something useful 40 years ago on DEC 
> >> hardware.)
> >
> > (close, but it was IBM and the name was slightly different:
> > https://en.wikipedia.org/wiki/.bss#Origin)
>
> That says United 

Re: [Toybox] scripts/llvm-buildall.sh?

[Patrick Oppenlander]

On Thu, Aug 5, 2021 at 3:28 PM Rob Landley  wrote:
>
> On 8/4/21 6:10 PM, Patrick Oppenlander wrote:
> > On Wed, Aug 4, 2021 at 8:53 PM Rob Landley  wrote:
> >>
> >> On 8/3/21 7:05 PM, Patrick Oppenlander wrote:
> >> >> What I'd LIKE to do is create a scripts/llvm-buildall.sh that builds 
> >> >> all the
> >> >> supported musl+llvm targets the way mcm-buildall.sh does for musl+gcc. 
> >> >> And
> >> >> getting llvm-project itself to do that was pretty straightforward (it 
> >> >> builds
> >> >> them all by default). But clang-rt depends on the target libc headers 
> >> >> being
> >> >> there first (...why?) and then the invocation is... yeah.
> >> >
> >> > I mentioned on the musl list that I've had a go at this too.
> >>
> >> Dig dig dig...
> >>
> >>   https://www.openwall.com/lists/musl/2021/07/26/3
> >>
> >> Ooh, very nice. Thanks.
> >>
> >> > Getting the llvm project & musl to play nice together is a pain. In
> >> > case you missed it, what I came up with is here:
> >> >
> >> > https://github.com/apexrtos/musl-cross-make/blob/clang/litecross/Makefile.clang
> >>
> >> Let's see...
> >>
> >> I made it to the giant span of "foreach" starting on line 49 and had to 
> >> stop for
> >> health reasons.
> >
> > If you want the llvm build system to build multiple builtin or runtime
> > targets there's a lot of configuration required :(
>
> For the _compiler_, it's just:
>
> mkdir -p build-llvm && cd build-llvm &&
> cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$PREFIX \
> -DLLVM_ENABLE_PROJECTS="clang;lld" "$TOP/llvm-project/llvm" &&
> ninja all install &&
> cd .. && rm -rf build-llvm || exit 1
>
> With $PREFIX and $TOP set appropriately, of course. The resulting clang
> --print-targets lists 39 options, from aarrcchh6644 to whatever "xcore" is.
>
> There's no reason their version of libgcc should be noticeably harder to
> configure, AND YET. (What do you mean by "runtime targets"?)

My understanding, which might be incorrect, is that "builtins" is a
subset of "runtimes".

Runtimes includes the compiler runtime (libclang_rt.builtins.a, ala
libgcc), atomics, unwind, sanitizers, libc, libc++, ...

Builtins just builds libclang_rt.builtins.a

> > There's a bunch of cmake "caches" in the llvm project which set a
> > bunch of these variables for some canned configurations (e.g. see
> > llvm-project/clang/cmake/caches/BaremetalARM.cmake).
>
> Yeah, I rm -rf the build directory each time because otherwise it ignores what
> you tell cmake's command line. (Not a fan.)

cmake is just so broken.

I can't even get it to generate usable makefiles (parallel builds seem
to be permanently broken). And building llvm single threaded takes a
ridiculous amount of time.

> > But yes, I should probably use a single foreach there though. Not
> > really sure why I did it like that.
>
> I'm not a huge fan of trying to scale makefiles beyond trivial usage. Simple
> things like "if this file exists, set this variable" turn out to be 
> SURPRISINGLY
> HARD TO DO. (And https://en.wikipedia.org/wiki/COMEFROM was originally a JOKE,
> yet it's fundamentally how Make works. Mixing imperative and declarative flow
> control in the same file...)
>
> >> The next question is, of course, WHY compiler-rt depends on the libc 
> >> headers.
> >> And if it's GOING to depend on a header file:
> >>
> >> https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h
> >>
> >> Says it's MIT licensed right at the top...
> >
> > Nice. They'd need something similar for 15 other OS targets though.
>
> Do any of them other than Windows NOT fall under "Posix values complying with
> LP64"? Even all the RTOSes these days are posix+LP64. Compiling for BARE METAL
> is "whatever subset of posix we decided to provide for ourselves" plus LP64.
>
> What specifically does compiler-rt need out of these headers? (It was cmake's
> configure plumbing dying when I didn't install the headers before trying to
> build compiler-rt, not any actual C file trying to #include a header...)
>
> Let's see: grep -r 'include[ \t]*<' compiler-rt/lib/builtins
>
> #include stdint.h is pilot error (Even WINDOWS standardizes char/short/int 
> size
> and __PTRDIFF_TYPE__ is another of those builtin #defines in t

Re: [Toybox] scripts/llvm-buildall.sh?

[Patrick Oppenlander]

On Wed, Aug 4, 2021 at 8:53 PM Rob Landley  wrote:
>
> On 8/3/21 7:05 PM, Patrick Oppenlander wrote:
> >> What I'd LIKE to do is create a scripts/llvm-buildall.sh that builds all 
> >> the
> >> supported musl+llvm targets the way mcm-buildall.sh does for musl+gcc. And
> >> getting llvm-project itself to do that was pretty straightforward (it 
> >> builds
> >> them all by default). But clang-rt depends on the target libc headers being
> >> there first (...why?) and then the invocation is... yeah.
> >
> > I mentioned on the musl list that I've had a go at this too.
>
> Dig dig dig...
>
>   https://www.openwall.com/lists/musl/2021/07/26/3
>
> Ooh, very nice. Thanks.
>
> > Getting the llvm project & musl to play nice together is a pain. In
> > case you missed it, what I came up with is here:
> >
> > https://github.com/apexrtos/musl-cross-make/blob/clang/litecross/Makefile.clang
>
> Let's see...
>
> I made it to the giant span of "foreach" starting on line 49 and had to stop 
> for
> health reasons.

If you want the llvm build system to build multiple builtin or runtime
targets there's a lot of configuration required :(

There's a bunch of cmake "caches" in the llvm project which set a
bunch of these variables for some canned configurations (e.g. see
llvm-project/clang/cmake/caches/BaremetalARM.cmake).

But yes, I should probably use a single foreach there though. Not
really sure why I did it like that.

> > Glossing over the finer details this basically boils down to:
> > * configure llvm
> > * do an llvm "distribution" build
> > * configure musl
> > * install musl headers
> > * do an llvm "builtins" build
>
> The builtins depending on the libc headers is outright disgusting. (This does,
> however, imply that they don't depend on the LINUX headers, which is... My 
> brain
> hurts.)

That's a definite WTF.

> The next question is, of course, WHY compiler-rt depends on the libc headers.
> And if it's GOING to depend on a header file:
>
> https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h
>
> Says it's MIT licensed right at the top...

Nice. They'd need something similar for 15 other OS targets though.

Still, depending on the C library headers seems completely broken.

> By the way, I tried to subscribe to the llvm-dev mailing list but
> https://lists.llvm.org/mailman/listinfo/llvm-dev is disabled and emailing that
> subscribe@ thing multiple times has not gotten a response. That development
> community is approximately as welcoming to outsiders as xfree86 was.

You're telling me.

I've submitted a bunch of patches to phabricator which are in various
states of landed (their word for merged), accepted (waiting for..
something?), and bikeshedding.

I've given up beating my head against it for now and am currently
rebasing this list whenever they release something:

https://github.com/apexrtos/musl-cross-make/tree/clang/patches/llvm-project-12.0.1.src

> > * build musl
> > * install kernel headers
> > * do an llvm "runtimes" build
> >
> > I also create symlinks & config files so that it looks like a normal
> > gcc style toolchain.
>
> The kernel needs to learn that $CROSS-cc is a good compiler name, and 
> $CROSS-gcc
> is not. And:
>
>   https://github.com/torvalds/linux/blob/v5.13/Makefile#L434
>
> Is just painful. Unfortunately, I'm not sure how you'd say "use the first of
> ${CROSS_COMPILE}{clang,gcc,cc} that exists" in Makefile syntax. The foreach
> stuff expands ALL of them, doesn't seem to have a "break" construct. (And of
> course musl does it in shell script, which is the sane way to do it. :)

It's even worse because llvm doesn't install symlinked binaries, so to
use clang you need to "clang --target=arm-xxx" ala

https://github.com/torvalds/linux/blob/62fb9874f5da54fdb243003b386128037319b219/Makefile#L578

I work around that in my build by constructing symlinks and .cfg files
to make clang look like a traditional triple prefixed toolchain.

> > It would be nice if there was one well maintained place to go for an
> > llvm/musl build recipe.
>
> I dunno about "well maintained" but if I can get it to work I'm happy to 
> provide
> a script, and possibly even host binaries although a gigabyte tarball is a bit
> of a stretch for my hosting...

I think my mcm fork should do what you need, but I know you prefer
more minimal solutions to these problems.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] scripts/llvm-buildall.sh?

[Patrick Oppenlander]

Hi Rob,

On Tue, Aug 3, 2021 at 7:08 PM Rob Landley  wrote:
>
> In https://landley.net/toybox/faq.html#cross there are three build options 
> listed:
>
> 1) using your linux distro's host toolchain (probably glibc based)
>
> 2) using the Android NDK
>
> 3) using the musl+gcc toolchain I supply a scripts/mcm-buildall.sh to build
>
> (Except for #3 I won't post binaries of because GPLv3 is too toxic for me to
> mess with without a corporate legal department providing cover, which has
> admittedly been a limiting factor.)
>
> Recently I've been poking at adding Qualcomm Hexagon support to the
> https://landley.net/toybox/downloads/binaries/mkroot/latest/ target list, and
> hexagon only has upstream support in llvm, not in vanilla gcc. (Which is
> probably my fault actually; long story, related to
> https://landley.net/notes-2012.html#24-02-2012.)
>
> I started down this path because I found out last month that qemu had an llvm
> build script checked into its test suite, and I went down a longish rathole to
> make that work for me (starting at
> https://lists.nongnu.org/archive/html/qemu-devel/2021-07/msg02992.html and
> https://landley.net/notes-2021.html#12-07-2021 and eventually coming up with a
> working script at https://www.openwall.com/lists/musl/2021/07/25/2).
>
> Then I tried to genericize it to build more llvm targets and the "clang-rt"
> package (their libgcc replacement) is, um... extremely brittle.
> (https://landley.net/notes-2021.html#28-07-2021)
>
> What I'd LIKE to do is create a scripts/llvm-buildall.sh that builds all the
> supported musl+llvm targets the way mcm-buildall.sh does for musl+gcc. And
> getting llvm-project itself to do that was pretty straightforward (it builds
> them all by default). But clang-rt depends on the target libc headers being
> there first (...why?) and then the invocation is... yeah.

I mentioned on the musl list that I've had a go at this too.

Getting the llvm project & musl to play nice together is a pain. In
case you missed it, what I came up with is here:

https://github.com/apexrtos/musl-cross-make/blob/clang/litecross/Makefile.clang

Glossing over the finer details this basically boils down to:
* configure llvm
* do an llvm "distribution" build
* configure musl
* install musl headers
* do an llvm "builtins" build
* build musl
* install kernel headers
* do an llvm "runtimes" build

I also create symlinks & config files so that it looks like a normal
gcc style toolchain.

It would be nice if there was one well maintained place to go for an
llvm/musl build recipe.

> I'm currently reading 
> https://llvm.org/docs/HowToCrossCompileBuiltinsOnArm.html
> but the OTHER llvm toolchain I've been using is the Android NDK, and THAT
> successfully builds clang-rt for multiple architectures already:
>
>   android-ndk-r21d$ find . -name 'libclang_rt.*.so' | wc
>22  222375
>   android-ndk-r21d$ find . -name 'libclang_rt.*.a' | wc
>68  687284
>
> Plus it's building bionic from source, which would be nice to know how to do.
>
> So I looked at:
>
>   https://android.googlesource.com/platform/manifest/+/master-ndk/
>
> And, um... huh. Well, clang-rt is part of llvm-project so...
>
>   https://android.googlesource.com/toolchain/llvm-project/+/refs/heads/master
>
> Last updated in 2019?
>
> Am I looking in the wrong place?
>
> Confused,
>
> Rob

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] mkroot works again, pending release.

[Patrick Oppenlander]

On Mon, Oct 26, 2020 at 12:56 PM Rob Landley  wrote:
>
> On 10/25/20 7:57 PM, Patrick Oppenlander wrote:
> > On Mon, Oct 26, 2020 at 11:46 AM Rob Landley  wrote:
> >>
> >> On 10/25/20 6:39 PM, Patrick Oppenlander wrote:
> >>> On Sat, Oct 24, 2020 at 12:43 PM Rob Landley  wrote:
> >>>>
> >>>> On 10/20/20 11:19 PM, Rob Landley wrote:
> >>>>> On 10/20/20 10:03 PM, Patrick Oppenlander wrote:
> >>>>>> On Wed, Oct 21, 2020 at 1:38 PM Rob Landley  wrote:
> >>>>>>> So yeah, allyesconfig throws #warnings during the build for a reason. 
> >>>>>>> I forgot
> >>>>>>> people test like that.
> >>>>>>
> >>>>>> Well that was a lucky coincidence then.
> >>>>>>
> >>>>>> I initially observed the issue on my ARMv7-M NOMMU target.
> >>>>>> "allyesconfig" was the closest hammer to get a toybox build with sh on
> >>>>>> my host, and just happened to also show the same behaviour.
> >>>>>>
> >>>>>> Thanks for looking into it.
> >>>>>
> >>>>> I do need to fix nommu subshells, yes. Good bug report. But fixing it 
> >>>>> might be
> >>>>> out of scope for this release.
> >>>>
> >>>> It was specifically a -c bug, nommu subshells worked fine in other 
> >>>> contexts but
> >>>> when nommu toysh was relaunching itself it didn't NOT parse -c so it ran 
> >>>> that
> >>>> same command again in the new subshell, which spawned ANOTHER subshell, 
> >>>> which...
> >>>>
> >>>> Try now?
> >>>
> >>> Looks like it still isn't working.
> >>
> >> $ ./sh -c 'echo $(echo hello)'
> >> hello
> >>
> >> It is for me, with nommu support enabled? What's your test and what are 
> >> you seeing?
> >
> > Yep, with nommu support enabled, here's the steps I took:
> >
> > % git describe --tags
> > 0.8.4
> > % make defconfig
> > % make menuconfig
> >
> > switch on pending/sh
> > switch on Toybox global settings/Enable nommu...
> > save
> >
> > % make
> > % ./toybox sh -c 'echo $(echo hello)'
> >
> > gives a blank result.
>
> And does not for me, as I said, so this is insufficient reproduction sequence
> for me to see your issue.

Sorry, didn't realise that was the case.

> Architecture? Toolchain? libc? Attach your .config?

x86_64
Arch Linux
gcc 10.2.0 glibc (distro default)

I can give you a shell account on a machine running that setup if it helps.

.config attached.

Patrick


.config
Description: Binary data
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] mkroot works again, pending release.

[Patrick Oppenlander]

On Mon, Oct 26, 2020 at 11:46 AM Rob Landley  wrote:
>
> On 10/25/20 6:39 PM, Patrick Oppenlander wrote:
> > On Sat, Oct 24, 2020 at 12:43 PM Rob Landley  wrote:
> >>
> >> On 10/20/20 11:19 PM, Rob Landley wrote:
> >>> On 10/20/20 10:03 PM, Patrick Oppenlander wrote:
> >>>> On Wed, Oct 21, 2020 at 1:38 PM Rob Landley  wrote:
> >>>>> So yeah, allyesconfig throws #warnings during the build for a reason. I 
> >>>>> forgot
> >>>>> people test like that.
> >>>>
> >>>> Well that was a lucky coincidence then.
> >>>>
> >>>> I initially observed the issue on my ARMv7-M NOMMU target.
> >>>> "allyesconfig" was the closest hammer to get a toybox build with sh on
> >>>> my host, and just happened to also show the same behaviour.
> >>>>
> >>>> Thanks for looking into it.
> >>>
> >>> I do need to fix nommu subshells, yes. Good bug report. But fixing it 
> >>> might be
> >>> out of scope for this release.
> >>
> >> It was specifically a -c bug, nommu subshells worked fine in other 
> >> contexts but
> >> when nommu toysh was relaunching itself it didn't NOT parse -c so it ran 
> >> that
> >> same command again in the new subshell, which spawned ANOTHER subshell, 
> >> which...
> >>
> >> Try now?
> >
> > Looks like it still isn't working.
>
> $ ./sh -c 'echo $(echo hello)'
> hello
>
> It is for me, with nommu support enabled? What's your test and what are you 
> seeing?

Yep, with nommu support enabled, here's the steps I took:

% git describe --tags
0.8.4
% make defconfig
% make menuconfig

switch on pending/sh
switch on Toybox global settings/Enable nommu...
save

% make
% ./toybox sh -c 'echo $(echo hello)'

gives a blank result.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] mkroot works again, pending release.

[Patrick Oppenlander]

On Sat, Oct 24, 2020 at 12:43 PM Rob Landley  wrote:
>
> On 10/20/20 11:19 PM, Rob Landley wrote:
> > On 10/20/20 10:03 PM, Patrick Oppenlander wrote:
> >> On Wed, Oct 21, 2020 at 1:38 PM Rob Landley  wrote:
> >>> So yeah, allyesconfig throws #warnings during the build for a reason. I 
> >>> forgot
> >>> people test like that.
> >>
> >> Well that was a lucky coincidence then.
> >>
> >> I initially observed the issue on my ARMv7-M NOMMU target.
> >> "allyesconfig" was the closest hammer to get a toybox build with sh on
> >> my host, and just happened to also show the same behaviour.
> >>
> >> Thanks for looking into it.
> >
> > I do need to fix nommu subshells, yes. Good bug report. But fixing it might 
> > be
> > out of scope for this release.
>
> It was specifically a -c bug, nommu subshells worked fine in other contexts 
> but
> when nommu toysh was relaunching itself it didn't NOT parse -c so it ran that
> same command again in the new subshell, which spawned ANOTHER subshell, 
> which...
>
> Try now?

Looks like it still isn't working.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] watchdog: Can't call daemon() on nommu

[Patrick Oppenlander]

On Fri, Oct 23, 2020 at 11:59 AM Rob Landley  wrote:
>
>
>
> On 10/22/20 5:49 PM, Rob Landley wrote:
> > On 10/21/20 4:24 AM, Patrick Oppenlander wrote:
> >> Sure, but..
> >>
> >> $ watchdog -F &
> >>
> >> works on nommu
> >
> > So I'm poking at a vfork() based xvdaemon() with nommu support, and the ONLY
> > caller of daemon() without 0,0 arguments is watchdog.
> >
> > I think not closing stdin/stdout/stderr is irrelevant here (it doesn't use 
> > them
> > afterwards? Do you WANT it to stay attached to the tty in -F mode?), but 
> > the no
> > chroot("/") is my question:
> >
> > Do you want to support relative DEV arguments in the watchdog command?
>
> On the assumption we _don't_, I've checked in the new xvdaemon() function and
> moved watchdog and sntp over to it.
>
> It doesn't work with qemu application emulation:
>
>   $ qemu-sh4eb ./skeleton
>   before 21444 3
>   qemu: no user program specified
>
> Apparently exec /proc/self/exe confuses qemu. :P
>
> But other than that, it SEEMS to work? (Yes, I keep delaying the release for
> tweaks...)

Works for me too!

Thanks!

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] watchdog: Can't call daemon() on nommu

[Patrick Oppenlander]

On Fri, 23 Oct 2020, 08:21 Rob Landley,  wrote:

> On 10/22/20 3:58 PM, Patrick Oppenlander wrote:
> > On Thu, 22 Oct 2020, 17:37 Rob Landley,  > <mailto:r...@landley.net>> wrote:
> >
> > On 10/21/20 4:24 AM, Patrick Oppenlander wrote:
> > > Sure, but..
> > >
> > > $ watchdog -F &
> > >
> > > works on nommu
> >
> > Yes, but fork() and daemon() shouldn't be available on nommu systems
> (they
> > _can't_ succeed) and you can only compile time probe for them if
> it's a build
> > break.
> >
> > I have a todo item to make some variant of xdaemon() plumbing that
> works on
> > nommu by relaunching the command, but I haven't gotten around to it
> yet because
> > I'm back in tokyo at $DAYJOB.
> >
> > Neat. Didn't know that was the plan, thanks.
>
> I'm working on a nommu j-core system at $DAYJOB. It has to work
> eventually...
>
> > > Also, watchdog -F is pretty useful as a quick test for newly
> developed
> > > watchdog drivers on nommu platforms.
> >
> > I agree it should be supported. Eventually nothing should depend on
> TOYBOX_FORK,
> > I need to fix up all of those to work on nommu properly.
> >
> > Great! No hurry, just wanted to point it out. I'm playing "annoying
> user" at the
> > moment :)
>
> No problem, thanks for the feedback.
>
> > Enjoy Japan. Are they open for business at the moment?
>
> For tourism, no. If you have a residence card, there's a somewhat elaborate
> procedure involving an in-person trip to the embassy, specific kinds of
> covid
> testing, and a 2 week quarantine when you get here:
>
>   https://landley.net/notes-2020.html#28-08-2020
>   https://landley.net/notes-2020.html#14-09-2020
>   https://landley.net/notes-2020.html#28-09-2020
>
> and so on...
>

Sounds like a nightmare! I'll have a read later.

> We're almost out of a
> > hard lockdown here in Melbourne, Australia. Travel restrictions have
> been eased
> > from 5km to 25km from home just this week.
>
> Can we just put Jacinda Ardern in charge of everywhere? (Not that she'd
> want to,
> but still.)
>

She's fantastic.

Our news system in Australia is broken. We had this tripe circulating last
week https://twitter.com/Trixie_Boo/status/1317926761292722176

One of our previous prime ministers is petitioning for a Royal Commission
because of this kind of thing
https://www.theguardian.com/media/2020/oct/11/kevin-rudd-petition-royal-commission-news-corp-media-domination-australia
.

Our local premier Dan Andrews and his team have done an absolutely
incredible job of managing the situation here under immense pressure from
the opposition and the media. I'm proud of what we have achieved.

Rob
>

Patrick

>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] watchdog: Can't call daemon() on nommu

[Patrick Oppenlander]

On Thu, 22 Oct 2020, 17:37 Rob Landley,  wrote:

> On 10/21/20 4:24 AM, Patrick Oppenlander wrote:
> > Sure, but..
> >
> > $ watchdog -F &
> >
> > works on nommu
>
> Yes, but fork() and daemon() shouldn't be available on nommu systems (they
> _can't_ succeed) and you can only compile time probe fore them if it's a
> build
> break.
>
> I have a todo item to make some variant of xdaemon() plumbing that works on
> nommu by relaunching the command, but I haven't gotten around to it yet
> because
> I'm back in tokyo at $DAYJOB.
>

Neat. Didn't know that was the plan, thanks.

> Also, watchdog -F is pretty useful as a quick test for newly developed
> > watchdog drivers on nommu platforms.
>
> I agree it should be supported. Eventually nothing should depend on
> TOYBOX_FORK,
> I need to fix up all of those to work on nommu properly.
>

Great! No hurry, just wanted to point it out. I'm playing "annoying user"
at the moment :)


> Rob
>

Enjoy Japan. Are they open for business at the moment? We're almost out of
a hard lockdown here in Melbourne, Australia. Travel restrictions have been
eased from 5km to 25km from home just this week.

Stay safe,

Patrick

>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] watchdog: Can't call daemon() on nommu

[Patrick Oppenlander]

Sure, but..

$ watchdog -F &

works on nommu

Also, watchdog -F is pretty useful as a quick test for newly developed
watchdog drivers on nommu platforms.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] [PATCH] blkdiscard: fix uninitialised offset

[patrick . oppenlander]

From: Patrick Oppenlander 

---
 toys/other/blkdiscard.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/toys/other/blkdiscard.c b/toys/other/blkdiscard.c
index a516676e..862e4fed 100644
--- a/toys/other/blkdiscard.c
+++ b/toys/other/blkdiscard.c
@@ -44,7 +44,7 @@ void blkdiscard_main(void)
   unsigned long long ol[2];
 
   // TODO: if numeric arg was long long array could live in TT.
-  if (FLAG(o)) ol[0] = TT.o;
+  ol[0] = TT.o;
   if (FLAG(l)) ol[1] = TT.l;
   else {
 xioctl(fd, BLKGETSIZE64, ol+1);
-- 
2.28.0

___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] mkroot works again, pending release.

[Patrick Oppenlander]

On Wed, Oct 21, 2020 at 3:09 PM Rob Landley  wrote:
>
> On 10/20/20 10:03 PM, Patrick Oppenlander wrote:
> > On Wed, Oct 21, 2020 at 1:38 PM Rob Landley  wrote:
> >> So yeah, allyesconfig throws #warnings during the build for a reason. I 
> >> forgot
> >> people test like that.
> >
> > Well that was a lucky coincidence then.
> >
> > I initially observed the issue on my ARMv7-M NOMMU target.
> > "allyesconfig" was the closest hammer to get a toybox build with sh on
> > my host, and just happened to also show the same behaviour.
> >
> > Thanks for looking into it.
>
> I do need to fix nommu subshells, yes. Good bug report. But fixing it might be
> out of scope for this release.

No problem, happy to wait as I have a working shell.

But I do get excited whenever you mention toysh. It will allow me to
throw away another external dependency, simplify build systems,
improve build times, reduce maintenance overhead, etc. etc.

So, thanks again for working on it!

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] mkroot works again, pending release.

[Patrick Oppenlander]

On Wed, Oct 21, 2020 at 1:38 PM Rob Landley  wrote:
>
>
>
> On 10/20/20 7:14 PM, Patrick Oppenlander wrote:
> > On Wed, Oct 21, 2020 at 11:02 AM Rob Landley  wrote:
> >>
> >>
> >>
> >> On 10/20/20 5:23 PM, Patrick Oppenlander wrote:
> >>> On Tue, Oct 20, 2020 at 10:58 AM Rob Landley  wrote:
> >>>>
> >>>> I got toysh implementing all the bits that mkroot's init script needs, 
> >>>> so I
> >>>> expect to cut a release in the next couple days. There's still a zillion
> >>>> dangling threads but I should be able to park it for now.
> >>>>
> >>>> If anybody wants to holler about something I should fix now, this would 
> >>>> be a
> >>>> good time. I'm back in tokyo working on $DAYJOB again, and am likely to 
> >>>> down
> >>>> tools on toybox after the release for at least a couple months.
> >>>>
> >>>> I note that except for a couple large holes (lack of $((math)) and 
> >>>> function()
> >>>> support, haven't finished trap/signal/jobs, needs command line editing 
> >>>> and
> >>>> history) toysh is... sort of working now? Several known bugs (test suite
> >>>> failures), but it's actually starting to be actually run real scripts. 
> >>>> Needs
> >>>> auditing for memory leaks, and the largest script I've thrown at it _is_ 
> >>>> the
> >>>> mkroot init, but... it's advanced from "don't bother" to "object of 
> >>>> curiosity".
> >>>>
> >>>> Right now, useful bug reports would come in the form of "thing I could 
> >>>> add to
> >>>> tests/sh.test and get to later", and keep in mind I've got...
> >>>>
> >>>>   $ wc sh.tests
> >>>>   845  3791 20673 sh.tests
> >>>>
> >>>> One or two of those already, not yet properly formatted into runnable 
> >>>> regression
> >>>> tests. (And one of the current sh.test entries HANGS because a
> >>>> subshell-plus-redirection test fails to hand off a filehandle properly 
> >>>> and we
> >>>> wait forever reading from nothing, so I need to teach tests to time 
> >>>> out...)
> >>>>
> >>>> Rob
> >>>
> >>> Hi Rob,
> >>>
> >>> first, thanks for all the hard work!
> >>>
> >>> I had a go at using toysh instead of busybox ash and immediately ran into 
> >>> this:
> >>>
> >>> % ./toybox sh
> >>> $ for i in $(./toybox); do echo $i; done
> >>> $
> >>
> >> It ran for me?
> >>
> >>   $ ./toybox sh -c 'for i in $(./toybox); do echo $i; done' | wc
> >> 211 2111355
> >>
> >>> That seems to start an interactive subshell somehow (takes two ^D's to
> >>> quit) instead of looping.
> >>
> >> Hmmm, how do I reproduce this...
> >>
> >> Rob
> >
> > Something strange is happening.
> >
> > % ./toybox sh -c 'for i in $(./toybox); do echo $i; done' | wc
> >   0   0   0
> >
> > Also, it takes a long time.
> >
> > % time ./toybox sh -c 'for i in $(./toybox); do echo $i; done'
> > ./toybox sh -c 'for i in $(./toybox); do echo $i; done'  0.00s user
> > 0.00s system 0% cpu 1.387 total
> >
> > Here's a what I'm doing on my end:
> >
> > % git describe --tags
> > 0.8.3-197-g9c8b4051
> >
> > % make allyesconfig
>
> Um. I haven't tested that in a couple months. (I should add a
> "CONFIG_BREAK_THE_BUILD" to that so people don't just blindly allyesconfig but
> have to switch something OFF to make it work.) Let's see...
>
> .toys/lsb/md5sum.c:86:25: fatal error: openssl/md5.h: No such file or 
> directory
>
> Doesn't compile for me, of course. I don't have the openssl libraries
> installed... The "ar.c" in my pending is half-finished, so is hd.c, mkdosfs,
> strace, I have a typo in my readelf...
>
> Ok, I have repeated your hang.
>
> Ah, of course, you enabled TOYBOX_FORCE_NOMMU which is using the "vfork, exec
> yourself, and send state data through a pipe" path because nommu systems 
> haven't
> got fork(). I haven't tested that path since June, it's conceptually all there
> but hasn't been regression tested and has like 3 pending TODO items.
>
> The problem here is it's sending the whole -c line and not the contents of the
> $(./toybox) to the child process, so it's  looping forking off another child
> process lots of times. :)
>
> So yeah, allyesconfig throws #warnings during the build for a reason. I forgot
> people test like that.

Well that was a lucky coincidence then.

I initially observed the issue on my ARMv7-M NOMMU target.
"allyesconfig" was the closest hammer to get a toybox build with sh on
my host, and just happened to also show the same behaviour.

Thanks for looking into it.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] mkroot works again, pending release.

[Patrick Oppenlander]

On Wed, Oct 21, 2020 at 11:02 AM Rob Landley  wrote:
>
>
>
> On 10/20/20 5:23 PM, Patrick Oppenlander wrote:
> > On Tue, Oct 20, 2020 at 10:58 AM Rob Landley  wrote:
> >>
> >> I got toysh implementing all the bits that mkroot's init script needs, so I
> >> expect to cut a release in the next couple days. There's still a zillion
> >> dangling threads but I should be able to park it for now.
> >>
> >> If anybody wants to holler about something I should fix now, this would be 
> >> a
> >> good time. I'm back in tokyo working on $DAYJOB again, and am likely to 
> >> down
> >> tools on toybox after the release for at least a couple months.
> >>
> >> I note that except for a couple large holes (lack of $((math)) and 
> >> function()
> >> support, haven't finished trap/signal/jobs, needs command line editing and
> >> history) toysh is... sort of working now? Several known bugs (test suite
> >> failures), but it's actually starting to be actually run real scripts. 
> >> Needs
> >> auditing for memory leaks, and the largest script I've thrown at it _is_ 
> >> the
> >> mkroot init, but... it's advanced from "don't bother" to "object of 
> >> curiosity".
> >>
> >> Right now, useful bug reports would come in the form of "thing I could add 
> >> to
> >> tests/sh.test and get to later", and keep in mind I've got...
> >>
> >>   $ wc sh.tests
> >>   845  3791 20673 sh.tests
> >>
> >> One or two of those already, not yet properly formatted into runnable 
> >> regression
> >> tests. (And one of the current sh.test entries HANGS because a
> >> subshell-plus-redirection test fails to hand off a filehandle properly and 
> >> we
> >> wait forever reading from nothing, so I need to teach tests to time out...)
> >>
> >> Rob
> >
> > Hi Rob,
> >
> > first, thanks for all the hard work!
> >
> > I had a go at using toysh instead of busybox ash and immediately ran into 
> > this:
> >
> > % ./toybox sh
> > $ for i in $(./toybox); do echo $i; done
> > $
>
> It ran for me?
>
>   $ ./toybox sh -c 'for i in $(./toybox); do echo $i; done' | wc
> 211 2111355
>
> > That seems to start an interactive subshell somehow (takes two ^D's to
> > quit) instead of looping.
>
> Hmmm, how do I reproduce this...
>
> Rob

Something strange is happening.

% ./toybox sh -c 'for i in $(./toybox); do echo $i; done' | wc
  0   0   0

Also, it takes a long time.

% time ./toybox sh -c 'for i in $(./toybox); do echo $i; done'
./toybox sh -c 'for i in $(./toybox); do echo $i; done'  0.00s user
0.00s system 0% cpu 1.387 total

Here's a what I'm doing on my end:

% git describe --tags
0.8.3-197-g9c8b4051

% make allyesconfig

% make

% ./toybox sh -c 'for i in $(./toybox); do echo $i; done' | wc
  0   0   0

Maybe an strace will help?

% strace ./toybox sh -c 'for i in $(./toybox); do echo $i; done' | wc
execve("./toybox", ["./toybox", "sh", "-c", "for i in $(./toybox); do
echo $i"...], 0x7ffce2875758 /* 72 vars */) = 0
brk(NULL)   = 0x55ec73215000
arch_prctl(0x3001 /* ARCH_??? */, 0x7fff376bc750) = -1 EINVAL (Invalid argument)
access("/etc/ld.so.preload", R_OK)  = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=282302, ...}) = 0
mmap(NULL, 282302, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f8135308000
close(3)= 0
openat(AT_FDCWD, "/usr/lib/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\23\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=14440, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7f8135306000
mmap(NULL, 16400, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f8135301000
mmap(0x7f8135302000, 4096, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f8135302000
mmap(0x7f8135303000, 4096, PROT_READ,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f8135303000
mmap(0x7f8135304000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f8135304000
close(3)= 0
openat(AT_FDCWD, "/usr/lib/libcrypt.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=186112, ...}) = 0
mmap(NULL, 221736, PROT_REA

Re: [Toybox] mkroot works again, pending release.

[Patrick Oppenlander]

On Tue, Oct 20, 2020 at 10:58 AM Rob Landley  wrote:
>
> I got toysh implementing all the bits that mkroot's init script needs, so I
> expect to cut a release in the next couple days. There's still a zillion
> dangling threads but I should be able to park it for now.
>
> If anybody wants to holler about something I should fix now, this would be a
> good time. I'm back in tokyo working on $DAYJOB again, and am likely to down
> tools on toybox after the release for at least a couple months.
>
> I note that except for a couple large holes (lack of $((math)) and function()
> support, haven't finished trap/signal/jobs, needs command line editing and
> history) toysh is... sort of working now? Several known bugs (test suite
> failures), but it's actually starting to be actually run real scripts. Needs
> auditing for memory leaks, and the largest script I've thrown at it _is_ the
> mkroot init, but... it's advanced from "don't bother" to "object of 
> curiosity".
>
> Right now, useful bug reports would come in the form of "thing I could add to
> tests/sh.test and get to later", and keep in mind I've got...
>
>   $ wc sh.tests
>   845  3791 20673 sh.tests
>
> One or two of those already, not yet properly formatted into runnable 
> regression
> tests. (And one of the current sh.test entries HANGS because a
> subshell-plus-redirection test fails to hand off a filehandle properly and we
> wait forever reading from nothing, so I need to teach tests to time out...)
>
> Rob

Hi Rob,

first, thanks for all the hard work!

I had a go at using toysh instead of busybox ash and immediately ran into this:

% ./toybox sh
$ for i in $(./toybox); do echo $i; done
$

That seems to start an interactive subshell somehow (takes two ^D's to
quit) instead of looping.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] hwclock & rtcwake expect /etc/adjtime to exist

[Patrick Oppenlander]

Hi,

A colleague of mine noticed that after a recent toybox update hwclock
now expects /etc/adjtime to be present. Previously it used to assume
UTC if /etc/adjtime is not present which is consistent with
util-linux.

I've attached patches which address this issue in hwclock and rtcwake.

Regards,

Patrick
From 963385ca53af701c20ec6b63312b96dfba917f18 Mon Sep 17 00:00:00 2001
From: Patrick Oppenlander 
Date: Thu, 15 Oct 2020 12:51:10 +1100
Subject: [PATCH 2/2] rtcwake: default to UTC if /etc/adjtime can't be read

This is consistent with the util-linux implementation.
---
 toys/other/rtcwake.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/toys/other/rtcwake.c b/toys/other/rtcwake.c
index 161bd881..a382ee57 100644
--- a/toys/other/rtcwake.c
+++ b/toys/other/rtcwake.c
@@ -55,7 +55,7 @@ void rtcwake_main(void)
 
   if (FLAG(u)) utc = 1;
   else if (FLAG(l)) utc = 0;
-  else utc = !!strstr(xreadfile("/etc/adjtime", toybuf, 2048), "UTC");
+  else utc = !readfile("/etc/adjtime", toybuf, 2048) || !!strstr(toybuf, "UTC");
   if (FLAG(v)) xprintf("RTC time: %s\n", utc ? "UTC" : "local");
 
   if (!TT.d) TT.d = "/dev/rtc0";
-- 
2.28.0

From 85cf3b20e9706be219f0280bc40aa0da17a860e6 Mon Sep 17 00:00:00 2001
From: Patrick Oppenlander 
Date: Thu, 15 Oct 2020 12:50:40 +1100
Subject: [PATCH 1/2] hwclock: default to UTC if /etc/adjtime can't be read

This is consistent with the util-linux implementation.
---
 toys/other/hwclock.c | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/toys/other/hwclock.c b/toys/other/hwclock.c
index ba28cc3d..a6db923a 100644
--- a/toys/other/hwclock.c
+++ b/toys/other/hwclock.c
@@ -40,10 +40,8 @@ void hwclock_main()
 
   if (FLAG(u)) utc = 1;
   else if (FLAG(l)) utc = 0;
-  else {
-xreadfile("/etc/adjtime", toybuf, sizeof(toybuf));
-utc = !!strstr(toybuf, "UTC");
-  }
+  else utc = !readfile("/etc/adjtime", toybuf, sizeof(toybuf)) ||
+!!strstr(toybuf, "UTC");
 
   if (!FLAG(t)) {
 if (!TT.f) TT.f = "/dev/rtc0";
-- 
2.28.0

___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] 0.8.3 is out.

[Patrick Oppenlander]

On Thu, May 14, 2020 at 9:06 AM Rob Landley  wrote:
>
> On 5/13/20 5:26 PM, Patrick Oppenlander wrote:
> > That works for:
> >
> > % make root LINUX=../linux
> >
> > I alread have a bunch of cross compilers installed & in my path:
> >
> > % ls -1 /opt/cross
> > aarch64-linux-musleabi
> > armv7m-linux-musleabi
> > armv7m-linux-musleabihf
> > arm-cdd-linux-musleabi
> > arm-linux-musleabi
> > arm-none-eabi
> > arm_cortex_m0-eabi-newlib
> > m32c-elf
> > m68k-elf
> > powerpc-eabispe
> > x86_64-linux-musl
> >
> > So I tried
> >
> > % make root CROSS_COMPILE=arm-linux-musleabi- LINUX=../linux
> >
> > Which fails with "Unknown $TARGET"
>
> It's taking everything before the first dash and feeding it into a big if/else
> staircase. In theory arrcchh6644 should work, and m68k and x86_64. but "arm" 
> it
> doesn't known armv5l vs armv7l (different kernel configs)...

Neat. I tried:

% make root CROSS_COMPILE=aarch64-linux-musleabi- LINUX=../linux

which worked perfectly.

> > OK, maybe
> >
> > % make root TARGET=arm CROSS_COMPILE=arm-linux-musleabi- LINUX=../linux
> >
> > That fails with "cp: bad 'toybox': No such file or directory"
>
> I suspect there was at least one error message before that, which you didn't 
> copy?

Sorry, perhaps I overlooked something. Here's the full output:

% make root TARGET=arm CROSS_COMPILE=arm-linux-musleabi- LINUX=../linux
scripts/mkroot.sh  -- LINUX=../linux CROSS_COMPILE=arm-linux-musleabi-
TARGET=arm
Building for arm

=== airlock
cleaned
cc -o kconfig/conf kconfig/conf.c kconfig/zconf.tab.c -DKBUILD_NO_NLS=1 \
-DPROJECT_NAME=\"ToyBox\"
scripts/genconfig.sh
kconfig/conf -D /dev/null Config.in > /dev/null
scripts/make.sh
Generate headers from toys/*/*.c...
generated/newtoys.h Library probe...
Make generated/config.h from /home/patrick/src/ext/toybox/root/.airlock.
generated/flags.h generated/globals.h generated/tags.h generated/help.h
Compile 
toybox-arm
scripts/install.sh --symlink --force --airlock
Compile instlist...
Generate headers from toys/*/*.c...
Library probe...
Make generated/config.h from /home/patrick/src/ext/toybox/root/.airlock.
Install commands...
cp: cannot stat 'toybox': No such file or directory
make: *** [Makefile:49: install_airlock] Error 1
make: *** [Makefile:80: root] Error 1
make root TARGET=arm CROSS_COMPILE=arm-linux-musleabi- LINUX=../linux
21.18s user 3.64s system 552% cpu 4.491 total

> > Maybe I should read the script.. OK, looks like TARGET should be
> > armv7l in this case
> >
> > % make root TARGET=armv7l CROSS_COMPILE=arm-linux-musleabi- LINUX=../linux
> >
> > Nope, same problem.
> >
> > OK I think what's happening is that the TARGET variable is leaking
> > into make.sh which then generates toybox-$TARGET which mkroot.sh
> > doesn't expect?
>
> I've never set it on the mkroot command line before, so it's always been a 
> local
> and never exported from mkroot into child processes. I can add an explicit
> "unexport", or I could rip the suffix plumbing out of make.sh.
>

Is there some other usecase for the suffix plubing in make.sh?

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] 0.8.3 is out.

[Patrick Oppenlander]

On Wed, May 13, 2020 at 12:17 PM Rob Landley  wrote:
>
> Release notes on the website.
>
> Rob
>
> P.S. I wrote up a long elaborate email about it to send here, but unplugged my
> laptop due to thunderstorm and then wandered away from it long enough for the
> battery to die. Alas thunderbird does not save unsent messages the way kmail,
> vi, and chrome do. Oh well.
> ___
> Toybox mailing list
> Toybox@lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net

Great work, thanks!

I did a run of "make root LINUX=../linux" which did lots of things and
then barfed with:

=== root.cpio.gz
cpio: --no-preserve-owner is meaningless with --create
Try 'cpio --help' or 'cpio --usage' for more information.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] blkdiscard man page clarification

[Patrick Oppenlander]

Hi Lukas,

while writing a blkdiscard for toybox I came across a discrepancy
between the man page and the util-linux implementation.

For the offset and length arguments, the man page states that "The
provided value will be aligned to the device sector size", but the
implementation does not perform any alignment and returns an error.

% blkdiscard -o 99 /dev/sdc
blkdiscard: /dev/sdc: offset 99 is not aligned to sector size 512

To me, that behaviour makes sense.

A simple resolution would be to change the text to "The provided value
_must_ be aligned to the device sector size".

Kind regards,

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] [PATCH] blkdiscard (for review)

[Patrick Oppenlander]

On Wed, Apr 15, 2020 at 11:48 AM Rob Landley  wrote:
>
> On 4/14/20 6:36 PM, Patrick Oppenlander wrote:
> >>> Without FDPIC every
> >>> toybox fork/exec requires a another copy of the entire program image
> >>> which kinda sucks.
> >>
> >> Run "make change", then look in the "change" directory.
> >>
> >> (It's "change for a $20". There should probably be be a better name for 
> >> it.)
> >
> > What's the use case for the separate binaries?
>
> Mostly that people complain about not having it, but on a nommu system without
> fdpic or binflt (which in my case means static PIE) you load less in for each
> binary that can't share anything with any other binary.

Of course. That makes more sense now :)

> >>> Then again, FDPIC for ARM should be available in gcc 10 (& musl
> >>> hopefully soon after) which ought to cover the majority of that space.
> >>
> >> Oh hey. Those french guys finally got their stuff pushed upstream? (I 
> >> poked Rich
> >> about that 4 years ago, https://www.openwall.com/lists/musl/2016/12/13/2 
> >> but he
> >> decided to wait for it to wind up in the upstream compiler before caring. 
> >> He has
> >> a contract to maintain the fdpic stuff for sh2 and had the concepts 
> >> explained to
> >> him by Jeff Dionne the original author of uClinux, so the base support is 
> >> there.
> >> But there wasn't much he could do about the compiler...)
> >
> > Yep, looks like it went in September 2019.
> >
> > FDPIC for ARM is also on the musl 1.2.2 roadmap
> > https://wiki.musl-libc.org/roadmap.html aimed for June release.
>
> Cool. For cortex-m scripts/mcm-buildall.sh is doing
> "armv7m:eabi:--with-arch=armv7-m --with-mode=thumb --disable-libatomic
> --enable-default-pie" which provides a compiler I can then "make root
> CROSS=armv7m" which makes static pie binaries that qemu-arm -cpu cortex-m3 can
> sort of run?
>
> $ qemu-arm -cpu cortex-m3 root/armv7m/fs/bin/toybox echo hello world
> hello world
>
> But not reliably:
>
> $ qemu-arm -cpu cortex-m3 root/armv7m/fs/bin/toybox cal
> qemu: uncaught target signal 4 (Illegal instruction) - core dumped
> Illegal instruction
>
> (Without the -cpu it runs fine. Something toolchainish I suspect? I don't 
> THINK
> an unaligned access issue would cause illegal instruction and sh4 should catch
> that anyway.)

│  >0x7ff9b00a  mrc 15, 0, r4, cr13, cr0, {3}

It's because armv7-m is missing a hardware TLS register. I guess
qemu-user doesn't know how to handle that.

The way I handle it is trap and emulate. My employer has a custom RTOS
with a minimal linux compatible syscall interface (forked from
http://prex.sourceforge.net and heavily modified) which I maintain. I
have convinced them to open source it, it's just waiting for me to get
my shit together and sort out licensing & copyright issues.

We will probably end up using your 0BSD license. Currently trying to
figure out the best places to put copyright notices. Do you have any
advice on that topic?

> But the main reason I haven't put more work into that target is I haven't got 
> a
> kernel config and qemu-system-arm -M cortex-m-board pair I can use yet. The
> cortex-m board I (briefly) had to run Linux on was an expensive FPGA thing
> Innoflight loaned me for a bit, but took back again last year. (It had a
> cortex-m bolted to the side of the FPGA. The kernel for that was an 
> out-of-tree
> fork maintained by Emcraft in Moscow, and all the make root tagets are vanilla
> kernel with no patches.)

I use the mps2-an385 (which is cortex-m3) board for my (non linux)
testing. There's a .dts for it in the mainline linux kernel, but I've
never tried to run it.

> > That's another confusing thing with blkdiscard. The official
> > documentation says "The provided value will be aligned to the device
> > sector size." which I interpreted as meaning that the code would align
> > an offset of 99 to something sane. But, do you round down or up? So I
> > tried the one installed on my Arch Linux system and:
> >
> > % blkdiscard -o 99 /dev/sdc
> > blkdiscard: /dev/sdc: offset 99 is not aligned to sector size 512
>
> The man page says Lukas Czerner ⟨lczer...@redhat.com⟩ maintains it, so I guess
> ask him which way to go? (I'd say "pick one" but if this is a security thing 
> you
>  want to err on the side of removing data, and if it isn't you'd want to err 
> on
> the side of KEEPING data, and I have no idea?)
>
> If he doesn't reply, I guess err on the side of removing data? If you were 
> told
> to discard da

Re: [Toybox] [PATCH] blkdiscard (for review)

[Patrick Oppenlander]

On Tue, Apr 14, 2020 at 4:34 PM Rob Landley  wrote:
>
> On 4/13/20 6:23 PM, Patrick Oppenlander wrote:
> > On Fri, Apr 10, 2020 at 5:33 PM Rob Landley  wrote:
> >>
> >> On 4/9/20 12:54 AM, Patrick Oppenlander wrote:
> >>> Hi Rob,
> >>>
> >>> I needed blkdiscard for an embedded job so had a go at putting it 
> >>> together.
> >>
> >> Applied, let's see...
> >>
> >>> Length & offset parsing don't match util-linux where MB, GB are powers
> >>> of 1000 and M, MiB, G, GiB are powers of 1024.
> >>>
> >>> I didn't use xioctl to avoid leaking fd if CFG_TOYBOX_FREE is set. Is
> >>> there a better solution to this?
> >>
> >> CFG_TOYBOX_FREE is... squishy.
> >>
> >> In theory there's no point freeing memory and closing files right before a
> >> program exits, because the process is about to go away and the OS will 
> >> clean all
> >> that up. But the android guys like running stuff under valgrand and such 
> >> to find
> >> resource leaks (even when they don't necessarily matter). There are 
> >> long-running
> >> programs that shouldn't leak (if "sed" leaked memory for each line of 
> >> input it
> >> processed it would run out of memory handling a large enough input file), 
> >> but
> >> something like nice/sleep/nproc/mount/insmod ever freeing memory is kind of
> >> optional.
> >
> > Totally agree. In my opinion there's only one argument for commands
> > like blkdiscard cleaning up: integration with the toybox shell to
> > avoid fork/exec on NOMMU to reduce memory use.
>
> Eh, having access to basic capabilities from a "kernel plus one userspace
> package" base is nice. But a separate command for every ioctl in the world is
> kinda sad. (This is why I like /proc and /sys knobs: echo thingy >
> /some/control/knob avoids the need for a special command to do that.)
>
> > Without FDPIC every
> > toybox fork/exec requires a another copy of the entire program image
> > which kinda sucks.
>
> Run "make change", then look in the "change" directory.
>
> (It's "change for a $20". There should probably be be a better name for it.)

What's the use case for the separate binaries?

I thought the point of a single multi-call was that the total is less
than the sum of the parts?

> > Then again, FDPIC for ARM should be available in gcc 10 (& musl
> > hopefully soon after) which ought to cover the majority of that space.
>
> Oh hey. Those french guys finally got their stuff pushed upstream? (I poked 
> Rich
> about that 4 years ago, https://www.openwall.com/lists/musl/2016/12/13/2 but 
> he
> decided to wait for it to wind up in the upstream compiler before caring. He 
> has
> a contract to maintain the fdpic stuff for sh2 and had the concepts explained 
> to
> him by Jeff Dionne the original author of uClinux, so the base support is 
> there.
> But there wasn't much he could do about the compiler...)

Yep, looks like it went in September 2019.

FDPIC for ARM is also on the musl 1.2.2 roadmap
https://wiki.musl-libc.org/roadmap.html aimed for June release.

> > So arguably simple is best. Would you like me to change it?
>
> I'm already fiddling with it here, but... hmmm.

I sent a simple patch to clean up the error return paths before you
sent this. Feel free to ignore it and do your magic :)

> I have automatic atolx parsing in optstr with # instead of : BUT it stores the
> result in a long which means on 32 bit systems them maximum value is 2 gigs, 
> and
> this combines badly with large file support.
>
> On 64 bit systems, # is obviously the right thing to do. But when supporting 
> 32
> bit systems with large block devices, what you're doing (save the string then
> parse in main) is the right thing to do.

I saw the # support in optstr. That reminds me -- I made a note that
the Toybox M/MB/MiB parsing doesn't match blkdiscard where MB is *1000
and M, MiB are * 1024. I overlooked that you have the 'd' support for
decimal sizes so I guess that's intentional and consistent across all
commands which take size arguments?

Supporting powers of 1000 here is useless anyway. The offset & length
need to be aligned to the underlying device block size, which, for all
devices I know of is a power of 2.

That's another confusing thing with blkdiscard. The official
documentation says "The provided value will be aligned to the device
sector size." which I interpreted as meaning that the code would align
an offset of 99 to something sane. But, do you round down or up? So I
tried the one installed on my Ar

[Toybox] [PATCH] blkdiscard: use xioctl & error_exit

[Patrick Oppenlander]

Hi Rob,

here's a patch to tidy up & simplify the error handling in blkdiscard.

Patrick
From 0534f028a41c64baabb984c562569a34dc9cd571 Mon Sep 17 00:00:00 2001
From: Patrick Oppenlander 
Date: Tue, 14 Apr 2020 15:17:18 +1000
Subject: [PATCH] blkdiscard: use xioctl & error_exit

After list discussion [1] switch to using xioctl & error_exit instead of
cleaning up fd.

For now, leave CFG_TOYBOX_FREE as it is a helpful reminder of what would
need to be cleaned up to support MAYFORK.

1. http://lists.landley.net/pipermail/toybox-landley.net/2020-April/011634.html
---
 toys/pending/blkdiscard.c | 15 ---
 1 file changed, 4 insertions(+), 11 deletions(-)

diff --git a/toys/pending/blkdiscard.c b/toys/pending/blkdiscard.c
index 1f0c09cd..13c48700 100644
--- a/toys/pending/blkdiscard.c
+++ b/toys/pending/blkdiscard.c
@@ -56,23 +56,16 @@ void blkdiscard_main(void)
 
   if (FLAG(l)) len = atolx_range(TT.length, 0, LLONG_MAX);
   else {
-if (ioctl(fd, BLKGETSIZE64, ) < 0) {
-  perror_msg("ioctl %x", (int)BLKGETSIZE64);
-  goto out;
-}
+xioctl(fd, BLKGETSIZE64, );
 len -= off;
   }
 
-  if (FLAG(s) && FLAG(z)) {
-error_msg("secure and zeroout are mutually exclusive");
-goto out;
-  }
+  if (FLAG(s) && FLAG(z))
+error_exit("secure and zeroout are mutually exclusive");
   if (FLAG(s)) req = BLKSECDISCARD;
   if (FLAG(z)) req = BLKZEROOUT;
 
-  if (ioctl(fd, req, (uint64_t[]){off, len}) < 0)
-perror_msg("ioctl %x", req);
+  xioctl(fd, req, (uint64_t[]){off, len});
 
-out:
   if (CFG_TOYBOX_FREE) close(fd);
 }
-- 
2.26.0

___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] [PATCH] blkdiscard (for review)

[Patrick Oppenlander]

On Fri, Apr 10, 2020 at 5:33 PM Rob Landley  wrote:
>
> On 4/9/20 12:54 AM, Patrick Oppenlander wrote:
> > Hi Rob,
> >
> > I needed blkdiscard for an embedded job so had a go at putting it together.
>
> Applied, let's see...
>
> > Length & offset parsing don't match util-linux where MB, GB are powers
> > of 1000 and M, MiB, G, GiB are powers of 1024.
> >
> > I didn't use xioctl to avoid leaking fd if CFG_TOYBOX_FREE is set. Is
> > there a better solution to this?
>
> CFG_TOYBOX_FREE is... squishy.
>
> In theory there's no point freeing memory and closing files right before a
> program exits, because the process is about to go away and the OS will clean 
> all
> that up. But the android guys like running stuff under valgrand and such to 
> find
> resource leaks (even when they don't necessarily matter). There are 
> long-running
> programs that shouldn't leak (if "sed" leaked memory for each line of input it
> processed it would run out of memory handling a large enough input file), but
> something like nice/sleep/nproc/mount/insmod ever freeing memory is kind of
> optional.

Totally agree. In my opinion there's only one argument for commands
like blkdiscard cleaning up: integration with the toybox shell to
avoid fork/exec on NOMMU to reduce memory use. Without FDPIC every
toybox fork/exec requires a another copy of the entire program image
which kinda sucks.

Then again, FDPIC for ARM should be available in gcc 10 (& musl
hopefully soon after) which ought to cover the majority of that space.

So arguably simple is best. Would you like me to change it?

> The use cases I had in mind when I put in CFG_TOYBOX_FREE were:
>
> A) the busybox space I was leaving behind (routers and such)
>
> B) knoppix as described in https://www.youtube.com/watch?v=MkJkyMuBm3g#t=1m18s
>
> C) Countering Trusting Trust
> (https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf)
> not just through diverse double compiling 
> (https://dwheeler.com/trusting-trust/)
> but through a fully auditable system small enough people can examine the
> binaries if they really want to.
>
> All of which can benefit from saving a few bytes here and there, which argues
> for dropping the unnecessary cleanup entirely. But the arguments in _favor_ of
> having it were:
>
> D) Any command that can fully clean up after itself could run MAYFORK from the
> toybox shell, which is faster.
>
> E) If somebody wants to run this stuff on bare metal (or glue it into a
> bootloader like u-boot; some context where you haven't GOT an OS cleaning up
> after you) then each command has to clean up after itself because nothing 
> else will.
>
> Since then Android showed up and ate 80% of the attention, and they like 
> having
> the cleanup because valgrind and their own automated auditing, so they switch
> the option on in their configs. But for a command that's never going to run
> mayfork and never be run from a bare metal context... eh? The leak only ever
> causes a real world problem in the D and E cases, and "blkdiscard" isn't
> something there's any point in micro-optimizing. (Running test/echo/true in 
> the
> shell's process is a signifcant enough timesaver to be worth a nofork audit.)
>
> Rob
>
> P.S. Alright, I'll explain (C):
>
> Long ago the inventor of Unix modified his C compiler to recognize when it was
> compling the login program and insert a backdoor root account, then modified 
> the
> C compiler to recognize when it was compiling _itself_ and insert two 
> backdoors:
> one for the login program, and the new one for itself. Then he deleted the
> backdoor from the compiler source, but gave the BSD guys the hacked binary 
> that
> would trojan both programs in a self-replicating way.
>
> Years later somebody gave him a lifetime achievement award and he explained 
> what
> he'd done in his acceptance speech. Wheeler's PHD thesis was "if you build 
> with
> two different compilers you can clean that sort of trojan out because they'd
> have to recognize _each_other_ and predict you were going to do that".
>
> MY fix is to make a system small and simple enough that a sufficiently trained
> group of professional reverse engineers can make it through "objdump -d" of 
> the
> whole thing in a reasonable amount of time, and then build the system under 
> that
> known good base. (This is what the Norad guys were doing in the video in (B)
> which his why they were using the smallest and simplest implementations they
> could find for high-security systems.) And we're doing the hardware side of 
> that
> over at https://j-core.org.)

Thaks for the writeup, that's a mind bending concept.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] [PATCH] blkdiscard (for review)

[Patrick Oppenlander]

Hi Rob,

I needed blkdiscard for an embedded job so had a go at putting it together.

Length & offset parsing don't match util-linux where MB, GB are powers
of 1000 and M, MiB, G, GiB are powers of 1024.

I didn't use xioctl to avoid leaking fd if CFG_TOYBOX_FREE is set. Is
there a better solution to this?

Hope you find this helpful,

Patrick
From 24e02a645bc2e0fbbd98a1facb6e1a1ed498d3e0 Mon Sep 17 00:00:00 2001
From: Patrick Oppenlander 
Date: Thu, 9 Apr 2020 15:30:12 +1000
Subject: [PATCH] blkdiscard: new command

---
 toys/pending/blkdiscard.c | 78 +++
 1 file changed, 78 insertions(+)
 create mode 100644 toys/pending/blkdiscard.c

diff --git a/toys/pending/blkdiscard.c b/toys/pending/blkdiscard.c
new file mode 100644
index ..1f0c09cd
--- /dev/null
+++ b/toys/pending/blkdiscard.c
@@ -0,0 +1,78 @@
+/* blkdiscard - discard device sectors
+ *
+ * Copyright 2020 Patrick Oppenlander 
+ *
+ * See https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/sys-utils/blkdiscard.8
+ *
+ * These options are not supported:
+ * -v, --verbose
+ * -p, --step
+ * -V, --version
+ *
+ * Size parsing does not match util-linux where MB, GB, TB are multiples of
+ * 1000 and MiB, TiB, GiB are multipes of 1024.
+
+USE_BLKDISCARD(NEWTOY(blkdiscard, "<1>1f(force)l(length):o(offset):s(secure)z(zeroout)", TOYFLAG_BIN))
+
+config BLKDISCARD
+  bool "blkdiscard"
+  default n
+  help
+usage: blkdiscard [options] device
+
+Discard device sectors.
+
+-o, --offset OFF	Byte offset from which to start discarding
+-l, --length LEN	Number of bytes to discard
+-s, --secure		Perform secure discard
+-z, --zeroout		Zero-fill rather than discard
+-f, --force		Disable checking for mounted filesystem
+
+By default OFF is zero and LEN is the device size meaning that the entire
+device will be discarded.
+
+OFF and LEN must be aligned to the device sector size.
+
+WARNING: All discarded data will be permanently lost!
+*/
+
+#define FOR_blkdiscard
+#include "toys.h"
+
+#include 
+
+GLOBALS(
+  char *offset, *length;
+)
+
+void blkdiscard_main(void)
+{
+  int fd, req = BLKDISCARD;
+  uint64_t off = 0, len;
+
+  fd = xopen(toys.optargs[0], O_WRONLY | (FLAG(f) ? 0 : O_EXCL));
+
+  if (FLAG(o)) off = atolx_range(TT.offset, 0, LLONG_MAX);
+
+  if (FLAG(l)) len = atolx_range(TT.length, 0, LLONG_MAX);
+  else {
+if (ioctl(fd, BLKGETSIZE64, ) < 0) {
+  perror_msg("ioctl %x", (int)BLKGETSIZE64);
+  goto out;
+}
+len -= off;
+  }
+
+  if (FLAG(s) && FLAG(z)) {
+error_msg("secure and zeroout are mutually exclusive");
+goto out;
+  }
+  if (FLAG(s)) req = BLKSECDISCARD;
+  if (FLAG(z)) req = BLKZEROOUT;
+
+  if (ioctl(fd, req, (uint64_t[]){off, len}) < 0)
+perror_msg("ioctl %x", req);
+
+out:
+  if (CFG_TOYBOX_FREE) close(fd);
+}
-- 
2.26.0

___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Did I mention why I didn't implement out of tree build?

[Patrick Oppenlander]

On Fri, Nov 15, 2019 at 4:17 AM Rob Landley  wrote:
>
> On 11/13/19 9:25 AM, Jarno Mäkipää wrote:
> > It seems that gmail stopped receiving emails from list again. I got
> > kicked out of list for excessive bounces yesterday... Somehow gmail
> > worked for month or two after I complained about it last time.
> >
> > -Jarno
>
> Oh I still get unsubscribed by gmail weekly. Its spam filter remains 
> profoundly
> broken. (And thanks for the reminder, I should trawl through my spam filter
> again to see what it's false positived on before it's been a month and it
> deletes it for me.)
>
> Just reply to the confirmation email and check the web archive to see what you
> missed while unsubscribed.
>
> (Yeah, I should move to a real mail service, I know...)
>

If you find one that's as convenient as gmail can you let us know please!

I also get unsubscribed from the toybox list at least once a month.
The strange thing is that the binutils, gcc & musl lists don't seem to
cause gmail to barf nearly as much as the toybox list does.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] Minor fixes to makefile

[Patrick Oppenlander]

Hi,

noticed a couple of makefile issues.

First looks like a typo to me.

Second fixes "make -j".

I also think "make silentoldconfig" might be broken in some way as I
ran out of patience after waiting a few minutes for it to complete.

Kind regards,

Patrick
From 1032320464fbcc9f2154715cb28e7198639a5401 Mon Sep 17 00:00:00 2001
From: Patrick Oppenlander 
Date: Fri, 2 Aug 2019 16:35:43 +1000
Subject: [PATCH 2/2] fix parallel make build failure

make: *** No rule to make target 'generated/Config.probed', needed by 'Config.in'.  Stop.
make: *** Waiting for unfinished jobs
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index 1bd3b25a..397e4777 100644
--- a/Makefile
+++ b/Makefile
@@ -31,6 +31,7 @@ $(KCONFIG_CONFIG): $(KCONFIG_TOP)
 	exit 1; fi
 
 $(KCONFIG_TOP): generated/Config.in generated/Config.probed
+generated/Config.probed: generated/Config.in
 generated/Config.in: toys/*/*.c scripts/genconfig.sh
 	scripts/genconfig.sh
 
-- 
2.22.0

From 35ec18da06b02588a1214fa2a4de301c05b5c9e9 Mon Sep 17 00:00:00 2001
From: Patrick Oppenlander 
Date: Fri, 2 Aug 2019 16:35:16 +1000
Subject: [PATCH 1/2] fix typo in Makefile

---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 1c1c4a4c..1bd3b25a 100644
--- a/Makefile
+++ b/Makefile
@@ -26,7 +26,7 @@ include kconfig/Makefile
 -include .singlemake
 
 $(KCONFIG_CONFIG): $(KCONFIG_TOP)
-	@if [ -e "$(KCONFG_CONFIG)" ]; then make silentoldconfig; \
+	@if [ -e "$(KCONFIG_CONFIG)" ]; then make silentoldconfig; \
 	else echo "Not configured (run 'make defconfig' or 'make menuconfig')";\
 	exit 1; fi
 
-- 
2.22.0

___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] [PATCH] xgetrandom: use getrandom result if it succeeds

[Patrick Oppenlander]

Currently xgetrandom always uses the fallback path even if getrandom returns 
success.

Patrick
>From 43d8947b9448a0212198551acd6a0ac6a01a0486 Mon Sep 17 00:00:00 2001
From: Patrick Oppenlander 
Date: Wed, 18 Jul 2018 15:09:13 +1000
Subject: [PATCH] xgetrandom: use getrandom result if it succeeds

---
 lib/portability.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/portability.c b/lib/portability.c
index 1415141..2ba3d29 100644
--- a/lib/portability.c
+++ b/lib/portability.c
@@ -35,8 +35,8 @@ void xgetrandom(void *buf, unsigned buflen, unsigned flags)
   int fd;
 
 #if CFG_TOYBOX_GETRANDOM
-  if (buflen != getrandom(buf, buflen, flags))
-if (!CFG_TOYBOX_ON_ANDROID || errno!=ENOSYS) perror_exit("getrandom");
+  if (buflen == getrandom(buf, buflen, flags)) return;
+  if (!CFG_TOYBOX_ON_ANDROID || errno!=ENOSYS) perror_exit("getrandom");
 #endif
 
   fd = xopen(flags ? "/dev/random" : "/dev/urandom", O_RDONLY);
-- 
2.18.0

___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] [PATCH] sh: fflush(stdout) to make sure that prompt is shown

[Patrick Oppenlander]

Found when testing with musl v1.1.19 on arm.

---
 toys/pending/sh.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/toys/pending/sh.c b/toys/pending/sh.c
index 0bfd32c..efa5c96 100644
--- a/toys/pending/sh.c
+++ b/toys/pending/sh.c
@@ -307,6 +307,7 @@ static void do_prompt(void)
 down:
 putchar(c);
   }
+  fflush(stdout);
 }
 
 void sh_main(void)

--
2.17.0
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Out of tree build support

[Patrick Oppenlander]

On 20/12/17 09:24, Rob Landley wrote:

Really what I need to do is come up with a way to build toybox sed and
toysh...


How far are toybox sed & toysh from being usable to build toybox?

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Out of tree build support

[Patrick Oppenlander]

On 20/12/17 09:24, Rob Landley wrote:


Really what I need to do is come up with a way to build toybox sed and
toysh standalone from a canned script with no config step, and then
build the rest with that.


Makes sense.


Speaking of which, did I ever point you at the cp -s trick?

   $ cp -rs "$PWD"/toybox walrus
   $ cd walrus
   $ make distclean && make defconfig && make

(I taught toybox to work with relative rather than absolute paths. The
gnu/dammit version complains about relative paths, and yes that means
even "cp -rs toybox walrus" won't work unless you're using toybox cp.
It's because they don't bother to calculate the dynamic ../ stack.)


That is a neat trick. First time I've seen it, thanks!


One of my todo items is writing a new kconfig to replace the kernel
plumbing I borrowed way back when. (Which is from 2.6.12.)


Even before that the issue is that there's no way to build kconfig from a 
script, so the *config targets depend on make.


I've pushed the current state of affairs to
https://github.com/pattop/toybox if you're interested in taking a look.


H... I see what you're doing, but you're adding complexity to the
Makefile and I want to strip it out and move it into the scripts.


I think the changes to the top level Makefile aren't too bad. They really just boil 
down to teaching make where to find sources & scripts by sprinkling in a few 
$(SRCDIR)s and don't change any of the logic at all.

kconfig/Makefile probably needs to be replaced by a script so the hackery in 
there is fairly moot. Would you be interested in a script that replaces this?

Are you happy with the direction of the changes to the scripts in general?

It might be worth considering passing $(MAKECMDGOALS) to make.sh (or a higher 
level driver script) at some point. It could then call single.sh or test.sh and 
eliminate the requirement for .singlemake. You could then call make.sh from a 
catch all rule in the top level makefile and really turn the makefile into a 
trivial wrapper.


The symlinks in kconfig are a bit nasty (they really should go in the
build directory to avoid polluting the source dir, but as they're always
the same...). I got this working with vpaths earlier but wasn't too
thrilled with the result.


What's the reason for the symlinks in kconfig anyway? I had a quick look 
through the history and couldn't find anything obvious.


CC'ing you as I don't seem to be getting list emails at the moment.


I seem to be, but I think I've been cc'd on everything the past couple
days anyway?


I'm not receiving emails for this thread. Maybe the mailing list is smart 
enough to know you're CC'ing me?

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Out of tree build support

[Patrick Oppenlander]

On 19/12/17 10:41, Rob Landley wrote:

On 12/18/2017 03:57 PM, Patrick Oppenlander wrote:

Hi Rob,

are you open to accepting patches for building out of tree?

I am working on a project where Toybox will be part of an automated
build system, and this would make life a lot easier.

Patrick


It's one of my todo list items, but there's backstory:

I've tried to make it so scripts/make.sh can be run separately (not via
the Makefile, that should be an essentially cosmetic wrapper around the
script-based build infrastructure). (In theory this means you can build
it on systems that haven't got a functional "make" yet. I have another
todo item to have a scripts/defconfig that populates generated/*.sh with
absolutely minimal dependencies, working even with apple's broken sed...)


That's a nasty chicken-and-egg problem indeed. I just thought you hated make :)


I also have a "configure" file where you can persistently set default
values for overrideable build options. This is in shell syntax rather
than Makefile syntax so only the shell scripts import it, if I import it
from the Makefile it gets confused by the "[ -z "$BLAH" ] && BLAH=" syntax.


I have another project which has the same problem, but it also needs the 
variables available for 'ld' to use in link scripts. The only workable solution 
I know of is to have a config file which is processed by a script to generate 
config.mk, config.ld, config.sh and config.h which can be included as required.

I may be able to hack something together at some point in the future if you're 
at all interested in this idea.


(The build has three conflicting use cases, which affect the
environmental dependencies and expected command line user interface. 1)
Conventional "configure/make/install" using gmake. 2) Portable build not
depending on anything toybox doesn't itself provide (building on BSD and
such). And android's doing ninja, which I should have support for but
ninja is a moving target, the one 14.4 installs doesn't work at all for
android...)


Nasty indeed.

I have done some hacking on the scripts. This currently works:

mkdir /tmp/toybox_build
make -C /tmp/toybox_build -f ~/src/ext/toybox/Makefile defconfig
make -C /tmp/toybox_build -f ~/src/ext/toybox/Makefile

So does this:

mkdir /tmp/toybox_build
make -C /tmp/toybox_build -f ~/src/ext/toybox/Makefile defconfig
cd /tmp/toybox_build && ~/src/ext/toybox/scripts/make.sh

So do other targets (like tests, etc). bloatcheck looks to be broken on trunk 
at the moment (incorrect dep on toybox_old?).

How do you intend to do the defconfig step using scripts?

I've pushed the current state of affairs to https://github.com/pattop/toybox if 
you're interested in taking a look. The symlinks in kconfig are a bit nasty 
(they really should go in the build directory to avoid polluting the source 
dir, but as they're always the same...). I got this working with vpaths earlier 
but wasn't too thrilled with the result.

CC'ing you as I don't seem to be getting list emails at the moment.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] [PATCH 2] Fix segfault in config2help

[Patrick Oppenlander]

On Wed, Sep 27, 2017 at 11:36 AM, Rob Landley  wrote:
> But only on a system with all the package upgrades applied, installing
> straight from DVD ditn't reproduce it. I've een very slowly debugging it
> under a qemu instance (which takes like 15 seconds to pull up "vi" on a
> file, for some reason arch linux's gui is REALLY slow under qemu).

Instead of using qemu you could run it under valgrind or try gcc's
address sanitizer. I just checked and valgrind identifies both of
these as "use after free" problems.

> Either gcc or the libc (or ld, or...) got upgraded recently and started
> Doing A Thing.

I observed the same behaviour -- this started after a system update.

>> I haven't had time to completely track down the reason for this, but it
>> seems that under some conditions sym->help is left dangling after memory
>> is free'd by:
>>
>> // Append trailing lines.
>> while (tfrom) dlist_add(, dlist_zap());
>>
>> Either way, testing sym->enabled appears to fix the problem.
>
> That's a workaround, not a fix. And it means we'd need to rebuild
> generated/help.h every time .config changes (which right now we don't).

True.

> Lemme look at your other patch more closely, that's probably the start
> of the right fix...

There may be a better solution than strdup'ing the name. That was the
easiest fix to get it going again.

Patrick
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] [PATCH 2] Fix segfault in config2help

[Patrick Oppenlander]

For some reason config2help started segfaulting on me.

I haven't had time to completely track down the reason for this, but it seems that 
under some conditions sym->help is left dangling after memory is free'd by:

// Append trailing lines.
while (tfrom) dlist_add(, dlist_zap());

Either way, testing sym->enabled appears to fix the problem.

8<===

commit 6c892548ad623ffe2b026139d7f22efb7763f024 (HEAD -> master)
Author: Patrick Oppenlander <patrick.oppenlan...@gmail.com>
Date:   Thu Sep 21 15:37:42 2017 +1000

fix segfault in config2help

Somehow sym->help can be left non-null after trailing lines are appended.

Fix by not attempting to print help for disabled symbols.


diff --git a/scripts/config2help.c b/scripts/config2help.c
index d238939..cdb250d 100644
--- a/scripts/config2help.c
+++ b/scripts/config2help.c
@@ -397,7 +397,7 @@ int main(int argc, char *argv[])
   while (sym) {
 struct double_list *dd;
 
-if (sym->help) {

+if (sym->help && sym->enabled) {
   int i;
   char *s = xstrdup(sym->name);
 
___

Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net