Re: [PATCH 9/9] loop: Add LOOP_CONFIGURE ioctl

[Lauri Kasanen]

On Fri, 18 Nov 2022 17:11:17 +0800
Xiaoming Ni  wrote:

> >> +#if LINUX_VERSION_CODE >= KERNEL_VERSION(5,8,0)
> >> +static int set_loop_configure(int ffd, int lfd, bb_loop_info *loopinfo)
> >> +#else
> > 
> > This assumes compile-time matches runtime. Ie, if you build on a 5.8
> > system, but run on an earlier kernel, the resulting busybox would be
> > unable to mount loops at all.
> > 
> > Please put it backward runtime compatibility, probably by having the
> > new-kernel code in the same func.
> > 
> 
> + rc = ioctl(lfd, LOOP_CONFIGURE, );
> + if (rc == 0) {
> + return lfd;
> + }
> + if (rc == -1 && errno == EINVAL) /* The system may not support 
> RING_CONFIGURE. */
> + return set_loop_configure_old(ffd, lfd, loopinfo);
> + return -1;
> 
> Is this ok?

Probably, if it works (I don't remember what errno an invalid ioctl
returns). Also wrong comment (RING_CONFIGURE)

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH 9/9] loop: Add LOOP_CONFIGURE ioctl

[Lauri Kasanen]

On Fri, 18 Nov 2022 09:01:58 +0800
Xiaoming Ni  wrote:

> LOOP_CONFIGURE is added to Linux 5.8
...
> diff --git a/libbb/loop.c b/libbb/loop.c
> index 9a7ca666d..d4f4906b0 100644
> --- a/libbb/loop.c
> +++ b/libbb/loop.c
> @@ -126,6 +126,30 @@ static int open_file(const char *file, unsigned flags, 
> int *mode)
>   return ffd;
>  }
>  
> +#if LINUX_VERSION_CODE >= KERNEL_VERSION(5,8,0)
> +static int set_loop_configure(int ffd, int lfd, bb_loop_info *loopinfo)
> +#else

This assumes compile-time matches runtime. Ie, if you build on a 5.8
system, but run on an earlier kernel, the resulting busybox would be
unable to mount loops at all.

Please put it backward runtime compatibility, probably by having the
new-kernel code in the same func.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] volume_id: volume_id_get_buffer with small FSes

[Lauri Kasanen]

On Mon, 16 May 2022 10:03:20 +
Alyssa Ross  wrote:

> On Mon, May 16, 2022 at 08:23:37AM +0300, Lauri Kasanen wrote:
> > On Sun, 15 May 2022 17:18:47 +
> > Alyssa Ross  wrote:
> >
> > > I tested both the cases mentioned in the comments — an empty floppy
> > > drive, and an unused loop device.  Reading from an empty floppy drive
> > > is ENXIO, and reading from an unused loop device returns 0, so it
> > > should be fine to drop the minimum length, and be happy with any read
> > > as long as it returns at least one byte.
> >
> > > -  * accesses. Rationale:
> > > -  * users complained of slow blkid due to empty floppy drives.
> >
> > But is it slow like the comment said? Ie. does this patch cause a 1-2s
> > delay due to the floppy drive spinning up?
>
> How would it?  In the case of a floppy or loop device, the operations
> that are happening are exactly the same.  Both before and after my
> patch, it would try to seek or read, that would fail, and then it would
> set id->error and return.

If you mean that the code comment was wrong, please mention that in
the commit message.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] volume_id: volume_id_get_buffer with small FSes

[Lauri Kasanen]

On Sun, 15 May 2022 17:18:47 +
Alyssa Ross  wrote:

> I tested both the cases mentioned in the comments — an empty floppy
> drive, and an unused loop device.  Reading from an empty floppy drive
> is ENXIO, and reading from an unused loop device returns 0, so it
> should be fine to drop the minimum length, and be happy with any read
> as long as it returns at least one byte.

> -  * accesses. Rationale:
> -  * users complained of slow blkid due to empty floppy drives.

But is it slow like the comment said? Ie. does this patch cause a 1-2s
delay due to the floppy drive spinning up?

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] pw_encrypt: Add option to enable bcrypt support

[Lauri Kasanen]

On Tue, 3 May 2022 13:58:44 +0200
Andreas Helmcke  wrote:


> diff --git a/loginutils/Config.src b/loginutils/Config.src
> index cbb09646b..cdcd7132f 100644
> --- a/loginutils/Config.src
> +++ b/loginutils/Config.src
> @@ -91,6 +91,28 @@ config USE_BB_CRYPT_SHA
>  With this option off, login will fail password check for any
>  user which has password encrypted with these algorithms.
>
> +config USE_BCRYPT
> +   bool "Enable the bcrypt crypt function"
> +   default n
> +   depends on !USE_BB_CRYPT
> +   help
> +   Enable this if you have passwords starting with $2a$, $2y$ or
> +   $2b$ in your /etc/passwd or /etc/shadow files. These passwords
> +   are hashed using the bcrypt algorithm. Requires the use of a C
> +   library that supports bcrypt.

The manpage says 2a is blowfish, which this patch wouldn't enable.

I'd also like a compile-time check for the libc support, something like

#if GLIBC && GLIBC_VERSION < blah
#error Your libc doesn't support bcrypt
#endif

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: httpd ranges do not work over 4gb

[Lauri Kasanen]

On Fri, 8 Oct 2021 15:58:47 +0200
Denys Vlasenko  wrote:

> WRONG. If CONFIG_LFS is off, sizeof(off_t) can be larger than sizeof(int).
> x86_64 is an example.
>
>
> Fixed in git, please test.

Fixed! Thanks.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

httpd ranges do not work over 4gb

[Lauri Kasanen]

Hi,

On current bb master httpd, when the client requests valid Range: bytes
over 4gb, the response is wrong - it sends the entire file instead of
the requested range.

E.g.
Range: bytes=5926192128-5926195199

This is on x86_64, so CONFIG_LFS etc shouldn't matter. I haven't
yet looked deeper if it's just a matter of using 64-bit vars.

- Lauri


bb-config.gz
Description: Binary data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Would a patch implementing the netns feature to iplink.c be interesting?

[Lauri Kasanen]

On Sun, 26 Sep 2021 22:30:43 -0400
blowfist  wrote:

> Hey there,
>
> I've made a few patches to busybox to add a few standard features to
> busybox which aren't yet supported in busybox. I'm using these for my project
> jailTools and I was wondering if they would stand any chance of being merged
> into busybox.

IMO this should be behind a config option, as most systems do not use
any namespaces.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] Add support for zstd decompression

[Lauri Kasanen]

On Sat, 11 Sep 2021 02:51:51 +0200
Denys Vlasenko  wrote:

> Anyone knows whether working
> with zstd people is doable, or it is more of a "Pottering"
> experience?

Cyan is quite responsive, I had an issue with a rare arch and it was
taken care of. Facebook of course has performance as priority 1, so it
may take a bit before he has time, but you know.

> With b[un]zip2, we just waited until the upstream code has
> fossilized enough that backporting pains stopped being a concern.

There's still active development here, both compression and hash, so
this would take a while.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: six patches from tinycore linux

[Lauri Kasanen]

On Thu, 12 Aug 2021 16:59:35 +0200
Emmanuel Deloget  wrote:

> * busybox-1.33.0_skip-loop-control.patch makes get_free_loop() behave
> as if there was no free loop devices (unconditionnaly). It would help
> to have some background info here, as it does not seem to fix a bug.

It fixes a specific case of non-devtmpfs + the loop device returned by
loop-control does not exist in /dev yet. The code after in that case is
racy and buggy, but such a combo is rare, probably just us, so we didn't
submit it.

> * busybox-1.27.1-wget-make-default-timeout-configurable.patch does
> exactly what it says it does - it makes the default wget timeout a
> configurable value.

I sent this a long time ago and it was rejected.

> * busybox-1.33.0_modprobe.patch checks whether the module is an ELF
> bianry before sending it to the kernel. Since the checks that the
> kernel performs are more comprehensive, I don't see much value in
> adding a simple check of the ELF header here -- not to mention that
> the code is duplicated.

This silences very loud kernel complaints when using compressed modules
IIRC. Was discussed on both here and lkml.

> * busybox-1.33.0_tc_depmod.patch : it *seems* that this one solves a
> bug but I cannot wrap my head around it. More particularly, I'm
> wondering if this is a tinycore thing (having modules hidden behind
> symlinks) or if this is a real issue.

Without it modules behind a symlinked dir aren't found; but again, very
likely just affects TC, so didn't send it.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Curious bug in remounting read-only cdrom mount point

[Lauri Kasanen]

Hi,

Your logs show you're running a different binary. correct.txt and
dropbear.txt run the util-linux mount, and only uncorrect.txt runs
busybox mount.

So it's not about SSH or the env, but looks like busybox mount just has
a different (wrong?) return code.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Traceroute bug in 1.33 release (please apply patch from master)

[Lauri Kasanen]

Hi,

Traceroute on the 1.33 release is buggy "traceroute: NO OPT x!". The
patch on master, commit f3a55b306ed3 "traceroute: fix option parsing"
fixes it.

Please apply this patch to the 1.33 branch.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH v2 1/2] modutils: check ELF header before calling finit_module()

[Lauri Kasanen]

On Tue, 5 Jan 2021 20:21:12 +0100
Alex Samorukov  wrote:

> On Tue, 5 Jan 2021 10:47:25 +0800
> > Kang-Che Sung  wrote:
> >
> >> I don't see why the
> >> "Invalid ELF header" message would bother you so much, since you
> >> won't load kernel modules often.
> > As mentioned in my original post, it's a pr_err. It shows up on console!
> I am very likely missing something, but why we are trying to load
> non-elf modules?

We're trying to load compressed ELF modules. Pre-v5.9-rc4 kernels
silently told busybox "I can't open this, send uncompressed". Since
that version, the kernel complains loudly. Hence the issue.

(re-adding busybox list)

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH v2 1/2] modutils: check ELF header before calling finit_module()

[Lauri Kasanen]

On Tue, 5 Jan 2021 10:47:25 +0800
Kang-Che Sung  wrote:

> I don't see why the
> "Invalid ELF header" message would bother you so much, since you
> won't load kernel modules often.

As mentioned in my original post, it's a pr_err. It shows up on console!

So every boot there's 20 scary errors interspersed with boot messages.
No need for the curious to see if dmesg was spammed, it's in their face.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Busybox modprobe hitch with kernel 5.10

[Lauri Kasanen]

On Tue, 29 Dec 2020 22:05:21 +0100
Denys Vlasenko  wrote:

> On Mon, Dec 28, 2020 at 6:15 PM Lauri Kasanen  wrote:
> > Hi,
> >
> > On Busybox 1.31.1, modprobing any module for the 5.10 kernel prints a
> > nasty message:
> > Module has invalid ELF header
> >
> > but the module loads successfully after that. So looks like bb tries
> > two paths, the first fails, second succeeds.
>
> What does strace show?

openat(AT_FDCWD, "kernel/drivers/char/ppdev.ko.gz", 
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
SYS_350(0x3, 0x80b37d5, 0, 0xb7d794b7, 0x3) = -1 ENOEXEC (Exec format error)
close(3)= 0
openat(AT_FDCWD, "kernel/drivers/char/ppdev.ko.gz", O_RDONLY|O_LARGEFILE) = 3
read(3, "\37\213", 2)   = 2
read(3, 
"\10\0\0\0\0\0\2\3\214V\5p\333h\26\226l'\345\332s\353\366\312\350\36S\2713\245\364

strace.log.gz
Description: Binary data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Busybox modprobe hitch with kernel 5.10

[Lauri Kasanen]

Hi,

On Busybox 1.31.1, modprobing any module for the 5.10 kernel prints a
nasty message:
Module has invalid ELF header

but the module loads successfully after that. So looks like bb tries
two paths, the first fails, second succeeds. It's rather unsightly to
get a high priority kernel error though.

The message seems to have been introduced in v5.9 rc4, before that it
was a silent failure. It's a pr_err, so it gets printed to console on
the "quiet" log level too.

CCing the authors and reviewers of the kernel patch, in case it's not
intentional to print such a high error for that path.

- Lauri


busybox-1.31.1_config.nosuid.gz
Description: Binary data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] mount: Surround syslog.h with the config check

[Lauri Kasanen]

This lets bb mount build for limited targets without syslog.h,
as long as the parts using it like NFS are disabled.

Signed-off-by: Lauri Kasanen 
---
 util-linux/mount.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/util-linux/mount.c b/util-linux/mount.c
index 0814a52..831dab9 100644
--- a/util-linux/mount.c
+++ b/util-linux/mount.c
@@ -182,7 +182,9 @@
 //usage:   "Returns 0 for success, number of failed mounts for -a, or 
errno for one mount."

 #include 
+#if ENABLE_FEATURE_SYSLOG
 #include 
+#endif
 #include 
 // Grab more as needed from util-linux's mount/mount_constants.h
 #ifndef MS_DIRSYNC
--
2.6.2

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] shrink last_char_is function even more

[Lauri Kasanen]

On Sun, 19 Jul 2020 22:31:09 +0200
Tito  wrote:

> please don't do this, this will bite us further up the road,
> similarly as all string functions that save a check and break
> havoc afterwards.

The param should be marked with the nonnull attribute, just like the
libc string functions. Then the compiler will warn you if you try to
pass NULL (may need higher optimization, warning levels, or the
analyzer mode in complex cases).

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] unzip: -d should create the dir

[Lauri Kasanen]

On Sat, 18 Apr 2020 00:23:13 +0200 (CEST)
Johannes Schindelin  wrote:

> Doesn't Info-Zip do more like an `mkdir -p` than only a single level?

Not sure, I've never tried to extract in a non-existent subdir tree.
100% of my use of -d is in a single new dir.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] unzip: -d should create the dir

[Lauri Kasanen]

The official Info-Zip unzip creates the dir if it doesn't exist.

Signed-off-by: Lauri Kasanen 
---
 archival/unzip.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/archival/unzip.c b/archival/unzip.c
index 8c4cb98..45e0727 100644
--- a/archival/unzip.c
+++ b/archival/unzip.c
@@ -646,8 +646,10 @@ int unzip_main(int argc, char **argv)
}

/* Change dir if necessary */
-   if (base_dir)
+   if (base_dir) {
+   mkdir(base_dir, 0777); /* Try to create, errors don't matter */
xchdir(base_dir);
+   }

if (quiet <= 1) { /* not -qq */
if (quiet == 0) {
--
2.6.2

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

master build failure (ash, undefined reference to `fill_arglist')

[Lauri Kasanen]

Hi,

Current master fails to build with the attached config.

shell/ash.c: In function 'subevalvar':
shell/ash.c:7287: warning: label 'out1' defined but not used
shell/ash.c: In function 'evalcommand':
shell/ash.c:10173: warning: implicit declaration of function
'fill_arglist' shell/ash.c:10173: warning: assignment makes pointer
from integer without a cast shell/ash.c:10196: warning: implicit
declaration of function 'parse_command_args'

...

shell/lib.a(ash.o): In function `evalcommand':
ash.c:(.text.evalcommand+0xb6): undefined reference to `fill_arglist'
ash.c:(.text.evalcommand+0x198): undefined reference to
`parse_command_args' collect2: ld returned 1 exit status


- Lauri


bb-config.gz
Description: Binary data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Weird signal issues using yocto's warrior busybox (aarch64) when memory is cached

[Lauri Kasanen]

Hi,

Try running your uptime loop script under valgrind?

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

"mount,losetup: use /dev/loop-control is it exists" regression

[Lauri Kasanen]

Hi,

In TC, we noticed a regression from the loop-control commit.
"mount: can't setup loop device: No such file or directory" when trying
to mount more than eight loops, on a kernel with loop-control.

It's possibly a race, could be udev didn't have time to create the
device before bb tried to open() it. Probably bb should mknod() it even
in the loop-control path.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH 8/8] mdev: remember line number in rule

[Lauri Kasanen]

On Mon, 16 Dec 2019 22:56:56 +0100
Jan Klötzke  wrote:

> --- a/util-linux/mdev.c
> +++ b/util-linux/mdev.c
> @@ -299,6 +299,7 @@ struct rule {
>   IF_FEATURE_MDEV_EXEC(char *r_cmd;)
>   regex_t match;
>   struct envmatch *envmatch;
> + int lineno;
>  };

int16_t? Surely mdev.conf cannot be larger than 32767 lines.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Why almost every command is trying to read urandom?

[Lauri Kasanen]

On Tue, 12 Nov 2019 22:43:28 +0100
Mauro Condarelli  wrote:

> Question: is that supposed to be a "harmless warning" or it's the tip of
> an iceberg?

It's harmless as long as the program causing it is not cryptographic.
"The randomness provided may not be high quality".

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] Add svg to built in MIME types list

[Lauri Kasanen]

On Fri, 7 Jun 2019 08:36:42 +0200
Xabier Oneca  --  xOneca  wrote:

> (But there's missing js!)

I second this. That's the only change I had to make recently to
httpd.conf, because FF does not execute JS with the wrong mime type
(other browsers do).

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] Add bc (Last patch)

[Lauri Kasanen]

On Mon, 29 Oct 2018 17:30:09 -0600
Gavin Howard  wrote:

> Okay, as it stands, I see one big showstopper for getting this into
> busybox: code style. I don't know how to fix code style with a script,
> so if it is a showstopper for you (and if it is, I don't blame you), I
> guess I have to respectfully withdraw my patch and apologize for
> wasting your time.
> 
> If, however, it is not, we can talk about what to do about it. And I
> will get started on changing the other things.

Install GNU indent. It has an option for where to put the braces.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Ash glob symlink regression when using libc glob

[Lauri Kasanen]

On Tue, 31 Oct 2017 13:34:48 +0100
Denys Vlasenko  wrote:

> > It fails with glibc 2.24 and 2.7, x86 and x86_64, so probably everything
> > between. Are you on a more recent glibc?
> 
> $ /lib64/libc.so.6
> GNU C Library (GNU libc) development release version 2.26.9000.

This looks relevant:
http://sourceware.org/git/?p=glibc.git;a=commit;h=5554304f0dddf75dc27cc6250fc53355161fd16a

So it was fixed after 2.26 and no released glibc contains the fix. How
about either editing the description to say glibc >= 2.27 only, or even
checking that compile-time?

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Ash glob symlink regression when using libc glob

[Lauri Kasanen]

On Mon, 30 Oct 2017 20:24:08 +0100
Denys Vlasenko <vda.li...@googlemail.com> wrote:

> On Sun, Oct 29, 2017 at 7:43 PM, Lauri Kasanen <c...@gmx.com> wrote:
> > Hi,
> >
> > It was reported the following case fails with bb 1.27.2:
> >
> > ln -s nonexistent mylink
> > mv myli* someother
> >
> > It also fails with master. I bisected it to
> > b3f29b452a660a7293162897424bed205f7f9147, "ash: use glob() from libc".
> > Works with bash.
> 
> Can't reproduce. myli* does expend for me.

It fails with glibc 2.24 and 2.7, x86 and x86_64, so probably everything
between. Are you on a more recent glibc?

My attached config was from the bisect commit, so applying it directly
to master may cause the wrong value for CONFIG_ASH_INTERNAL_GLOB.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Ash glob symlink regression when using libc glob

[Lauri Kasanen]

Hi,

It was reported the following case fails with bb 1.27.2:

ln -s nonexistent mylink
mv myli* someother

It also fails with master. I bisected it to
b3f29b452a660a7293162897424bed205f7f9147, "ash: use glob() from libc".
Works with bash.

- Lauri


bb-config.gz
Description: Binary data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] Silence misguided GCC warning about alignment issues

[Lauri Kasanen]

On Mon, 7 Aug 2017 12:09:45 +0200 (CEST)
Johannes Schindelin  wrote:
> Meaning: GCC gets this all wrong and should not complain. So let's force
> it to be quiet for a couple of lines.

I believe this is both wrong and dangerous. If gcc warns about it, it
will also very likely try to mis-optimize the same case.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: acpid & detection of new input devices

[Lauri Kasanen]

Hi,

Big acpid seems to use inotify on /dev/input. Or perhaps you can set up
an udev rule that sends a signal to acpid.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Problem with cat

[Lauri Kasanen]

On Mon, 13 Mar 2017 19:24:17 + (UTC)
Grant Edwards  wrote:

> That paragraph is wrong (or at least misleading). I've checked the
> source code, and 2.6.33 does not update the output file postition (at
> least not the version I downloaded from kernels.org nor the version I
> got from Atmel). Perhaps the author of that man page had a 2.6.33 with
> some 3.0 stuff backported.

Please report the bug to them. I've found the man-pages project quite
responsive.

- Lauri
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: SSL/TLS client command line API?

[Lauri Kasanen]

On Thu, Jan 19, 2017, at 16:17, Denys Vlasenko wrote:
> Hi folks,
>
> So far bbox is completely SSL/TLS-illiterate. wget can't talk
> HTTPS. sendmail groks neither smtps not STARTTLS command. nc can't
> do TLS either.
...
> Do you know a tool whose command line is suitable for us?

Don't know of any, but this seems to be a reasonably common need. About
a year ago, I needed almost the same thing, "do the SSL magic for me and
let me handle the raw protocol" in a curl-using app, but curl did not
have such a mode, and no plans to add it. Their reply was "do it
manually using openssl", which pushed it down on the interesting-things-
list and so didn't get done.

- Lauri

-- 
http://www.fastmail.com - mmm... Fastmail...

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Ash wildcard usability bug

[Lauri Kasanen]

On Thu, Oct 27, 2016, at 09:14, Kang-Che Sung wrote:
> If you are arguing about usability, you should assume a case that you
> are in an _interactive_ shell. Non-interactive environment is for
> shell scripts, and for that portability and robustness will be more
> important than what you called "usability".

It was implied that I was in an interactive shell; apologies if you got
the impression I was arguing for this in non-interactive shells.

- Lauri

-- 
http://www.fastmail.com - Or how I learned to stop worrying and
  love email again

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Ash wildcard usability bug

[Lauri Kasanen]

On Wed, Oct 26, 2016, at 05:44, Kang-Che Sung wrote:
> I wonder why you can't implement an easy workaround using ls or
> something.
> 
> filename="`ls *zip | head -n 1`"
> nc 10.0.0.1 12345 < "$filename"
> do_something_else < "$filename"
> 
> Yeah. Just avoid the glob on the redirection and instead get the
> filename before that. Any difficulty?

Because typing that takes longer? You're missing the point of an
usability feature.

- Lauri

-- 
http://www.fastmail.com - Choose from over 50 domains or use your own

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Ash wildcard usability bug

[Lauri Kasanen]

> > On Mon, Oct 24, 2016, at 20:09, Denys Vlasenko wrote:
> > > Note that they allow to consistently never do globbing on the
> > > redirect word.
> > >
> > > To me it looks like the best course of action.
> >
> > Surely we all agree that it's terrible usability to not expand it?
>
> It may be terrible, but it's conforming to POSIX. ;-)
>
> But in earnest, I think in most cases it is more convenient to use Tab
> completion rather than *.

Well, I did try to use tab completion: cat < *zip[TAB]

Of course it doesn't work. In my case, I had many files sharing a
prefix, but only one ending in *zip, so typing that was several
times faster than letter- tab, oh more letters, tab, oh, more
letters, tab, more

- Lauri

-- 
http://www.fastmail.com - mmm... Fastmail...

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Ash wildcard usability bug

[Lauri Kasanen]

On Mon, Oct 24, 2016, at 20:09, Denys Vlasenko wrote:
> Note that they allow to consistently never do globbing
> on the redirect word.
> 
> To me it looks like the best course of action.

Surely we all agree that it's terrible usability to not expand it?

- Lauri

-- 
http://www.fastmail.com - IMAP accessible web-mail

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Ash wildcard usability bug

[Lauri Kasanen]

Hi,

The following fails in ash 1.24.1, probably also in git. Works in bash.

nc 10.0.0.1 1234 < *zip

"sh: can't open *zip: no such file"

Also applies to cat, etc. With only one matching file in the directory.

- Lauri

-- 
http://www.fastmail.com - The way an email service should be

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Current git HEAD busybox segfaults on some applets

[Lauri Kasanen]

On Fri, Sep 16, 2016, at 19:41, Rich Felker wrote:
> More to the point, they're unnecessary and unsafe. The *_unlocked
> functions are not well-specified, and except for getc/putc, they're
> not going to yield you any performance benefits anyway.

They were originally added because of my request, and they most
certainly do provide performance benefit (on glibc). The original case
was bb
modprobe being a lot slower than modutils modprobe, googling the
archives for those should give you the numbers.

- Lauri

-- 
http://www.fastmail.com - Or how I learned to stop worrying and
  love email again

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] libbb: More informative "short write" error

[Lauri Kasanen]

On Mon, Sep 5, 2016, at 13:18, Denys Vlasenko wrote:
> How about just replacing bb_error_msg_and_die()
> with bb_perror_msg_and_die()?

$ busybox wget google.com
Connecting to google.com (172.217.20.14:80)
Connecting to www.google.fi (80.239.229.245:80)
wget: short write: No space left on device

Works, thanks.

- Lauri

-- 
http://www.fastmail.com - Send your email first class

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] libbb: More informative "short write" error

[Lauri Kasanen]

Hi,

Over at TinyCore, we receive a huge number of questions of the type "I
got "short write", what does it mean?". Mostly for the rpi port and when
using bb wget. The attached patch makes the message more informative,
and hopefully lets more people know what's up without having to ask.

- Lauri

-- 
http://www.fastmail.com - A no graphics, no pop-ups email service

From 5c9f3c91db1e8f5b7adb7fb8211ece92a5ecc756 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen <cur...@operamail.com>
Date: Sun, 4 Sep 2016 13:08:59 +0300
Subject: [PATCH] libbb: More informative "short write" error

Signed-off-by: Lauri Kasanen <cur...@operamail.com>
---
 libbb/xfuncs_printf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libbb/xfuncs_printf.c b/libbb/xfuncs_printf.c
index e9222f6..101836e 100644
--- a/libbb/xfuncs_printf.c
+++ b/libbb/xfuncs_printf.c
@@ -236,7 +236,7 @@ void FAST_FUNC xwrite(int fd, const void *buf, size_t count)
if (count) {
ssize_t size = full_write(fd, buf, count);
if ((size_t)size != count)
-   bb_error_msg_and_die("short write");
+   bb_error_msg_and_die("short write, out of disk space?");
}
 }
 void FAST_FUNC xwrite_str(int fd, const char *str)
-- 
2.6.2

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Not getting the correct result with SHA3

[Lauri Kasanen]

On Mon, Aug 29, 2016, at 15:10, Denys Vlasenko wrote:
> I guess this is an example why it makes sense to wait for general
> adoption of the tool in common distros before adding it to busybox.
> 
> The bug was that SHA3 uses 0110*1 padding (raw Keccak uses 10*1).
> Easy to fix.
> 
> sha3sum which I found on Fedora is a Perl script (oookay...),
> it defaults to SHA3-224 and has an -a BITS option.
> 
> Fixed both in git. Please try it.

Confirmed working on x86_64. Thanks Denys!

- Lauri

-- 
http://www.fastmail.com - The professional email service

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Not getting the correct result with SHA3

[Lauri Kasanen]

On Fri, Aug 26, 2016, at 21:28, Marlon Smith wrote:
> Hi everyone,
> 
> I'm using Busybox version 1.23.1 and it seems like sha3sum is giving an
> incorrect result.
> 
> For example if I run this command:
> 
> echo "abc" | sha3sum
> 
> I get an incorrect result:
> 
> e0a651904afe783ec83eeb586a2e602a09c43a3c6d795549ed7a58caae661074beaccd16d470ce4eac3ba98feee94bead3916ef4a75c1501
> 1f07df348ce7a9e7
> 
> I'm running BusyBox on an i.MX6 embedded Linux system based on Yocto.
> 
> Does anyone know why this could be the case?

Hi,

We're aware. SHA3 changed after we made the applet, IIRC in the padding
bits added after the data. Patches welcome, I'm really busy.

- Lauri

-- 
http://www.fastmail.com - Access your email from home and the web

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] md5sum: error out on -c empty file

[Lauri Kasanen]

On Sat, Jun 25, 2016, at 20:48, Lauri Kasanen wrote:
> Hi,
> 
> GNU md5sum errors out when passed an empty file via -c. Add a test and
> make bb md5sum do the same.

Ping. We have scripts that need this behavior.

- Lauri

-- 
http://www.fastmail.com - A no graphics, no pop-ups email service

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] getrandom: new applet

[Lauri Kasanen]

On Tue, Jun 28, 2016, at 19:56, Etienne CHAMPETIER wrote:
> first user of this applet will be LEDE (OpenWrt) to save an urandom seed
> using getrandom() (so we are sure /dev/urandom pool is initialized)

In LFS, the boot scripts use dd for this. Is there any need for an
applet?

- Lauri

-- 
http://www.fastmail.com - A fast, anti-spam email service.

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] md5sum: error out on -c empty file

[Lauri Kasanen]

Hi,

GNU md5sum errors out when passed an empty file via -c. Add a test and
make bb md5sum do the same.

- Lauri

-- 
http://www.fastmail.com - Choose from over 50 domains or use your own

From f5a686ea2ac56d91f6cecf16b234b43608a51ec7 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen <cur...@operamail.com>
Date: Sat, 25 Jun 2016 20:41:03 +0300
Subject: [PATCH 1/2] md5sum: Add a test for -c with empty file

Signed-off-by: Lauri Kasanen <cur...@operamail.com>
---
 testsuite/md5sum/md5sum-c-empty | 5 +
 1 file changed, 5 insertions(+)
 create mode 100644 testsuite/md5sum/md5sum-c-empty

diff --git a/testsuite/md5sum/md5sum-c-empty b/testsuite/md5sum/md5sum-c-empty
new file mode 100644
index 000..3a3ed72
--- /dev/null
+++ b/testsuite/md5sum/md5sum-c-empty
@@ -0,0 +1,5 @@
+# An empty file passed to -c must error out.
+
+touch empty
+busybox md5sum -c empty && exit 1
+exit 0
-- 
2.6.2

From 7bc302f6d05290803702037389cce39215b7f199 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen <cur...@operamail.com>
Date: Sat, 25 Jun 2016 20:46:13 +0300
Subject: [PATCH 2/2] md5_sha1_sum: error out on -c empty file

This is needed to match GNU md5sum behavior.

Signed-off-by: Lauri Kasanen <cur...@operamail.com>
---
 coreutils/md5_sha1_sum.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/coreutils/md5_sha1_sum.c b/coreutils/md5_sha1_sum.c
index 1a5342e..89af9a3 100644
--- a/coreutils/md5_sha1_sum.c
+++ b/coreutils/md5_sha1_sum.c
@@ -200,6 +200,7 @@ int md5_sha1_sum_main(int argc UNUSED_PARAM, char **argv)
char *line;
int count_total = 0;
int count_failed = 0;
+   uint8_t found = 0;
 
pre_computed_stream = xfopen_stdin(*argv);
 
@@ -224,6 +225,7 @@ int md5_sha1_sum_main(int argc UNUSED_PARAM, char **argv)
}
*filename_ptr = '\0';
filename_ptr += 2;
+   found = 1;
 
hash_value = hash_file(filename_ptr);
 
@@ -244,6 +246,13 @@ int md5_sha1_sum_main(int argc UNUSED_PARAM, char **argv)
bb_error_msg("WARNING: %d of %d computed 
checksums did NOT match",
count_failed, count_total);
}
+   if (!found) {
+   return_value = EXIT_FAILURE;
+
+   if (!(flags & FLAG_SILENT)) {
+   bb_error_msg("WARNING: no checksum 
lines found");
+   }
+   }
fclose_if_not_stdin(pre_computed_stream);
} else {
uint8_t *hash_value = hash_file(*argv);
-- 
2.6.2

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

New warning from the applet rework

[Lauri Kasanen]

Hi,

Today's git with the attached config, on x86_64 linux, gives this
warning:
libbb/appletlib.c:55: warning: 'run_applet_and_exit' declared 'static'
but never defined

- Lauri

-- 
http://www.fastmail.com - Access all of your messages and folders
  wherever you are



bbconfig.gz
Description: GNU Zip compressed data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Busybox binary ends up only 340 bytes

[Lauri Kasanen]

On Mon, Apr 4, 2016, at 23:52, st...@weaverpad.com wrote:
> I am trying to build Busybox for my ARMv7-M microprocessor based
> board. It seems to compile everything as configured; however, the
> resulting binary is only 340 bytes in size. The map file shows all
> files that were built, but there is no space allocated for them. It's
> as if it has decided to discard all sections as unused. I've tried for
> a while now to figure out why it might be doing this. I am using
> Buildroot to automate the entire process. Can anyone provide any
> guidance on how to go about fixing this? What other information should
> I provide?

I hit similar on m68k a while back - the platform's link scripts were
not prepared for gc-sections, and so everything got pruned. Since
editing your platform may not be an option for you, the easiest
workaround is to remove gc-sections from scripts/trylink. Specifically,
look for the logic that checks for it, and empties the variable - make
it always empty.

- Lauri

-- 
http://www.fastmail.com - Or how I learned to stop worrying and
  love email again

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Testing gcc 5.2 and LTO

[Lauri Kasanen]

Hi,

With my shiny new gcc 5.2 toy, I thought I'd test how its LTO worked for
bb,
and share the numbers. There were a few hundred new warnings, one
parenthesis
in ash, and all the rest about strict aliasing, mainly in minix and top.

All on x86_64, defconfig with sync_fancy and ubi* disabled.

busybox git 68acc0f8353

gcc 4.2.2: 849405 bytes 100%
gcc 5.2.0: 829369 bytes 97.6%
LTO 5.2.0: 828209 bytes 97.5%

So, the current best LTO saved a bit over a kb, that's pretty well done
for bb's manual tuning.

- Lauri

-- 
http://www.fastmail.com - Email service worth paying for. Try it for free

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How to compare two floats?

[Lauri Kasanen]

 On Saturday 23 May 2015 21:41:51 bifferos wrote:
  I'm trying to write a thermostat using busybox shell, however the only way I
  could think to compare two float temperatures (without adding awk) was to
  multiply them up in dc and then compare as integers. In the end I modified
  dc to add a comparison operator, which probably breaks the spirit of dc
  somewhat.  I'm interested if I missed a trick somewhere.  Is there another
  way to do this?  Is this considered a heinous thing to do to dc?

a - b

Then you check if the sign is negative, using the shell's string ops.

- Lauri

-- 
http://www.fastmail.com - Access all of your messages and folders
  wherever you are

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

modinfo broken in both master and 1.23

[Lauri Kasanen]

Hi,

We find that modinfo for gzipped modules is broken in both today's
master and the stable 1.23 branch (1.23.1 release is also broken).

Good output (1.22.1 for example):
$ modinfo ntfs
filename:   kernel/fs/ntfs/ntfs.ko.gz
license:GPL
author: Anton Altaparmakov an...@tuxera.com
description:NTFS 1.2/3.x driver - Copyright (c) 2001-2011 Anton
Altaparmakov and Tuxera Inc.
version:2.1.30
alias:  fs-ntfs
srcversion: 426164F8BB2591455E7748D
depends:
vermagic:   3.16.6-tinycore SMP mod_unload 486

Bad output:
$ modinfo ntfs
filename:   ntfs

Bisect says master broke in 7885452, 1.23 broke in the Apply
post-1.23.0 patches commit. Config attached.

- Lauri

-- 
http://www.fastmail.com - Access all of your messages and folders
  wherever you are



config.gz
Description: GNU Zip compressed data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] Add a gzip fastpath for the xmalloc readers, v3

[Lauri Kasanen]

On Sun, Dec 7, 2014, at 01:51, Denys Vlasenko wrote:
  It misses the plus one zero byte behavior of the non-compressed path
  (xmalloc_read). I bet some callers depend on it, using str* functions on
  the returned buffer.
 
 Fixed, and committed to git. Please try it.

Results are correct and speed is equal to my patch. All good, thanks.

- Lauri

-- 
http://www.fastmail.com - Send your email first class

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] Add a gzip fastpath for the xmalloc readers, v3

[Lauri Kasanen]

On Sat, Dec 6, 2014, at 18:11, Denys Vlasenko wrote:
 Please see attached patch (only compile-tested so far).
 It builds on your work, implementing in-memory compression
 for all unpackers we have.

It misses the plus one zero byte behavior of the non-compressed path
(xmalloc_read). I bet some callers depend on it, using str* functions on
the returned buffer.

- Lauri

-- 
http://www.fastmail.com - Same, same, but different...

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] Add a gzip fastpath for the xmalloc readers, v3

[Lauri Kasanen]

v3: Use the aux struct as requested by Denys
v2: Add missing check on open

The performance and number of processes for a depmod -a with gzipped
modules was abysmal. This patch adds a fast path without fork for well-
behaved gzip files, benefiting all users of xmalloc_open_zipped_read_close.

modinfo radeon.ko.gz, a single-file reader, got 30% faster.
depmod -a, which used to fork over 800 times, got 20% faster. And of course
a whole lot less processes - much saved RAM.

function old new   delta
inflate_unzip_internal  23042521+217
xmalloc_open_zipped_read_close73 201+128
unpack_gz_stream 567 570  +3
--
(add/remove: 0/0 grow/shrink: 3/0 up/down: 348/0) Total: 348 bytes

-- 
http://www.fastmail.com - Does exactly what it says on the tin

From 5ad6804ed4485eae176da45524ea848a00b11929 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Sun, 30 Nov 2014 21:37:10 +0200
Subject: [PATCH] Add a gzip fastpath for the xmalloc readers, v3

v3: Use the aux struct as requested by Denys
v2: Add missing check on open

The performance and number of processes for a depmod -a with gzipped
modules was abysmal. This patch adds a fast path without fork for well-
behaved gzip files, benefiting all users of xmalloc_open_zipped_read_close.

modinfo radeon.ko.gz, a single-file reader, got 30% faster.
depmod -a, which used to fork over 800 times, got 20% faster. And of course
a whole lot less processes - much saved RAM.

function old new   delta
inflate_unzip_internal  23042521+217
xmalloc_open_zipped_read_close73 201+128
unpack_gz_stream 567 570  +3
--
(add/remove: 0/0 grow/shrink: 3/0 up/down: 348/0) Total: 348 bytes

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 archival/libarchive/decompress_gunzip.c | 38 -
 archival/libarchive/open_transformer.c  | 31 ++-
 include/bb_archive.h|  2 ++
 3 files changed, 65 insertions(+), 6 deletions(-)

diff --git a/archival/libarchive/decompress_gunzip.c 
b/archival/libarchive/decompress_gunzip.c
index 7c6f38e..938d21f 100644
--- a/archival/libarchive/decompress_gunzip.c
+++ b/archival/libarchive/decompress_gunzip.c
@@ -971,10 +971,11 @@ static int inflate_get_next_window(STATE_PARAM_ONLY)
 
 /* Called from unpack_gz_stream() and inflate_unzip() */
 static IF_DESKTOP(long long) int
-inflate_unzip_internal(STATE_PARAM int in, int out)
+inflate_unzip_internal(STATE_PARAM transformer_aux_data_t *aux, int in, int 
out)
 {
IF_DESKTOP(long long) int n = 0;
ssize_t nwrote;
+   size_t bufsize = 0;
 
/* Allocate all global buffers (for DYN_ALLOC option) */
gunzip_window = xmalloc(GUNZIP_WSIZE);
@@ -1002,16 +1003,43 @@ inflate_unzip_internal(STATE_PARAM int in, int out)
 
while (1) {
int r = inflate_get_next_window(PASS_STATE_ONLY);
-   nwrote = full_write(out, gunzip_window, gunzip_outbuf_count);
+
+   if (aux-mem_output_size) {
+   nwrote = gunzip_outbuf_count;
+   if (gunzip_outbuf_count + n  bufsize) {
+   // increase by four blocks each time
+   const size_t newsize = bufsize + 4 * 
gunzip_outbuf_count + 1;
+   aux-mem_output_buf = 
xrealloc(aux-mem_output_buf, newsize);
+   bufsize = newsize;
+   }
+
+   if (bufsize  aux-mem_output_size) {
+   free(aux-mem_output_buf);
+   aux-mem_output_buf = NULL;
+   n = -1;
+   goto ret;
+   }
+
+   memcpy(aux-mem_output_buf + n, gunzip_window, 
gunzip_outbuf_count);
+   } else {
+   nwrote = full_write(out, gunzip_window, 
gunzip_outbuf_count);
+   }
if (nwrote != (ssize_t)gunzip_outbuf_count) {
bb_perror_msg(write);
n = -1;
goto ret;
}
-   IF_DESKTOP(n += nwrote;)
+   n += nwrote;
if (r == 0) break;
}
 
+   /* Final realloc, plus zero byte */
+   if (aux-mem_output_size) {
+   aux-mem_output_buf = xrealloc(aux-mem_output_buf, n + 1);
+   aux-mem_output_size = n;
+   aux-mem_output_buf[n] = '\0

[PATCH] Add a gzip fastpath for the xmalloc readers, v2

[Lauri Kasanen]

v2: Add missing check on open

The performance and number of processes for a depmod -a with gzipped
modules was abysmal. This patch adds a fast path without fork for well-
behaved gzip files, benefiting all users of xmalloc_open_zipped_read_close.

modinfo radeon.ko.gz, a single-file reader, got 30% faster.
depmod -a, which used to fork over 800 times, got 20% faster. And of course
a whole lot less processes - much saved RAM.

function old new   delta
inflate_get_next_window-1877   +1877
xmalloc_unpack_gz  - 356+356
check_header_gzip  - 298+298
xmalloc_inflate_unzip_internal - 223+223
xmalloc_open_zipped_read_close73 176+103
inflate_init   -  97 +97
inflate_store_unused   -  35 +35
unpack_gz_stream 567 299-268
inflate_unzip_internal  2304 172   -2132
--
(add/remove: 6/0 grow/shrink: 1/2 up/down: 2989/-2400)Total: 589 bytes

-- 
http://www.fastmail.com - The way an email service should be

From e5a58da5d54b8a8e2eef657ac6be60231683a662 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Thu, 27 Nov 2014 14:48:17 +0200
Subject: [PATCH] Add a gzip fastpath for the xmalloc readers

v2: Add missing check on open

The performance and number of processes for a depmod -a with gzipped
modules was abysmal. This patch adds a fast path without fork for well-
behaved gzip files, benefiting all users of xmalloc_open_zipped_read_close.

modinfo radeon.ko.gz, a single-file reader, got 30% faster.
depmod -a, which used to fork over 800 times, got 20% faster. And of course
a whole lot less processes - much saved RAM.

function old new   delta
inflate_get_next_window-1877   +1877
xmalloc_unpack_gz  - 356+356
check_header_gzip  - 298+298
xmalloc_inflate_unzip_internal - 223+223
xmalloc_open_zipped_read_close73 176+103
inflate_init   -  97 +97
inflate_store_unused   -  35 +35
unpack_gz_stream 567 299-268
inflate_unzip_internal  2304 172   -2132
--
(add/remove: 6/0 grow/shrink: 1/2 up/down: 2989/-2400)Total: 589 bytes

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 archival/libarchive/decompress_gunzip.c | 155 
 archival/libarchive/open_transformer.c  |  20 +
 include/bb_archive.h|   1 +
 3 files changed, 160 insertions(+), 16 deletions(-)

diff --git a/archival/libarchive/decompress_gunzip.c 
b/archival/libarchive/decompress_gunzip.c
index 7c6f38e..1f68ebd 100644
--- a/archival/libarchive/decompress_gunzip.c
+++ b/archival/libarchive/decompress_gunzip.c
@@ -968,19 +968,12 @@ static int inflate_get_next_window(STATE_PARAM_ONLY)
/* Doesnt get here */
 }
 
-
-/* Called from unpack_gz_stream() and inflate_unzip() */
-static IF_DESKTOP(long long) int
-inflate_unzip_internal(STATE_PARAM int in, int out)
+static void inflate_init(STATE_PARAM_ONLY)
 {
-   IF_DESKTOP(long long) int n = 0;
-   ssize_t nwrote;
-
/* Allocate all global buffers (for DYN_ALLOC option) */
gunzip_window = xmalloc(GUNZIP_WSIZE);
gunzip_outbuf_count = 0;
gunzip_bytes_out = 0;
-   gunzip_src_fd = in;
 
/* (re) initialize state */
method = -1;
@@ -994,6 +987,31 @@ inflate_unzip_internal(STATE_PARAM int in, int out)
gunzip_crc = ~0;
 
error_msg = corrupted data;
+}
+
+static void inflate_store_unused(STATE_PARAM_ONLY)
+{
+   /* Store unused bytes in a global buffer so calling applets can access 
it */
+   if (gunzip_bk = 8) {
+   /* Undo too much lookahead. The next read will be byte aligned
+* so we can discard unused bits in the last meaningful byte. */
+   bytebuffer_offset--;
+   bytebuffer[bytebuffer_offset] = gunzip_bb  0xff;
+   gunzip_bb = 8;
+   gunzip_bk -= 8;
+   }
+}
+
+/* Called from unpack_gz_stream() and inflate_unzip() */
+static IF_DESKTOP(long long) int
+inflate_unzip_internal(STATE_PARAM int in, int out)
+{
+   IF_DESKTOP(long long) int n = 0;
+   ssize_t nwrote;
+
+   gunzip_src_fd = in;
+   inflate_init(PASS_STATE_ONLY

[PATCH] Add a gzip fastpath for the xmalloc readers

[Lauri Kasanen]

Hi,

The performance and number of processes for a depmod -a with gzipped
modules was abysmal. This patch adds a fast path without fork for well-
behaved gzip files, benefiting all users of
xmalloc_open_zipped_read_close.

modinfo radeon.ko.gz, a single-file reader, got 30% faster.
depmod -a, which used to fork over 800 times, got 20% faster. And of
course
a whole lot less processes - much saved RAM.

function old new   delta
inflate_get_next_window-1877   +1877
xmalloc_unpack_gz  - 356+356
check_header_gzip  - 298+298
xmalloc_inflate_unzip_internal - 223+223
inflate_init   -  97 +97
xmalloc_open_zipped_read_close73 159 +86
inflate_store_unused   -  35 +35
unpack_gz_stream 567 299-268
inflate_unzip_internal  2304 172   -2132
--
(add/remove: 6/0 grow/shrink: 1/2 up/down: 2972/-2400)Total: 572
bytes

--

It's currently guarded by CONFIG_DESKTOP. If you'd like a new config
option instead, please say so.

- Lauri

-- 
http://www.fastmail.com - Choose from over 50 domains or use your own

From 4eaee06a748ce07b004b40aa10fa1bcc3e5e8928 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Thu, 27 Nov 2014 14:48:17 +0200
Subject: [PATCH] Add a gzip fastpath for the xmalloc readers

The performance and number of processes for a depmod -a with gzipped
modules was abysmal. This patch adds a fast path without fork for well-
behaved gzip files, benefiting all users of xmalloc_open_zipped_read_close.

modinfo radeon.ko.gz, a single-file reader, got 30% faster.
depmod -a, which used to fork over 800 times, got 20% faster. And of course
a whole lot less processes - much saved RAM.

function old new   delta
inflate_get_next_window-1877   +1877
xmalloc_unpack_gz  - 356+356
check_header_gzip  - 298+298
xmalloc_inflate_unzip_internal - 223+223
inflate_init   -  97 +97
xmalloc_open_zipped_read_close73 159 +86
inflate_store_unused   -  35 +35
unpack_gz_stream 567 299-268
inflate_unzip_internal  2304 172   -2132
--
(add/remove: 6/0 grow/shrink: 1/2 up/down: 2972/-2400)Total: 572 bytes

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 archival/libarchive/decompress_gunzip.c | 155 
 archival/libarchive/open_transformer.c  |  16 
 include/bb_archive.h|   1 +
 3 files changed, 156 insertions(+), 16 deletions(-)

diff --git a/archival/libarchive/decompress_gunzip.c 
b/archival/libarchive/decompress_gunzip.c
index 7c6f38e..1f68ebd 100644
--- a/archival/libarchive/decompress_gunzip.c
+++ b/archival/libarchive/decompress_gunzip.c
@@ -968,19 +968,12 @@ static int inflate_get_next_window(STATE_PARAM_ONLY)
/* Doesnt get here */
 }
 
-
-/* Called from unpack_gz_stream() and inflate_unzip() */
-static IF_DESKTOP(long long) int
-inflate_unzip_internal(STATE_PARAM int in, int out)
+static void inflate_init(STATE_PARAM_ONLY)
 {
-   IF_DESKTOP(long long) int n = 0;
-   ssize_t nwrote;
-
/* Allocate all global buffers (for DYN_ALLOC option) */
gunzip_window = xmalloc(GUNZIP_WSIZE);
gunzip_outbuf_count = 0;
gunzip_bytes_out = 0;
-   gunzip_src_fd = in;
 
/* (re) initialize state */
method = -1;
@@ -994,6 +987,31 @@ inflate_unzip_internal(STATE_PARAM int in, int out)
gunzip_crc = ~0;
 
error_msg = corrupted data;
+}
+
+static void inflate_store_unused(STATE_PARAM_ONLY)
+{
+   /* Store unused bytes in a global buffer so calling applets can access 
it */
+   if (gunzip_bk = 8) {
+   /* Undo too much lookahead. The next read will be byte aligned
+* so we can discard unused bits in the last meaningful byte. */
+   bytebuffer_offset--;
+   bytebuffer[bytebuffer_offset] = gunzip_bb  0xff;
+   gunzip_bb = 8;
+   gunzip_bk -= 8;
+   }
+}
+
+/* Called from unpack_gz_stream() and inflate_unzip() */
+static IF_DESKTOP(long long) int
+inflate_unzip_internal(STATE_PARAM int in, int out)
+{
+   IF_DESKTOP(long long) int n = 0;
+   ssize_t nwrote

open_transformer fork, depmod

[Lauri Kasanen]

Hi,

Can anyone fill me in on the reason why open_transformer always forks,
instead of straight decompressing?

I found the original commit from 2003:
http://lists.busybox.net/pipermail/busybox-cvs/2003-November/019040.html
But it doesn't explain why the move was done.

I ask because doing a depmod -a with gzipped modules results in over
800 forks, which is surely not optimal neither from memory nor from
performance points of view.

In the case of gzip, the uncompressed size is available, so there would
be no RAM waste in allocating the maximum specified 64mb first. For
compression formats that do not include the uncompressed size, the
fork+pipe approach does save RAM, but for gzip it seems to be needless
overhead.

- Lauri

-- 
http://www.fastmail.com - Send your email first class

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCHv2] Use sendfile to copy data between file descriptors

[Lauri Kasanen]

On Tue, Oct 21, 2014, at 08:46, Mike Frysinger wrote:
 On 20 Oct 2014 21:00, Lauri Kasanen wrote:
   +config FEATURE_USE_SENDFILE
   +bool Use sendfile system call
   +default y
   +help
   +  When enabled, busybox will use the kernel sendfile() function
   +  instead of read/write loops where applicable.
  
  Please include the kernel version requirement here.
 
 `man 2 sendfile`:
 In Linux kernels before 2.6.33, out_fd must refer to a socket.
 Since Linux 2.6.33 it can be any file.  If it is a regular file,
 then sendfile() changes the file offset appropriately.
 -mike

Yes, I know? The point was to tell the user configuring busybox that.

- Lauri

PS: Sorry, forgot the list the first time.

-- 
http://www.fastmail.fm - Or how I learned to stop worrying and
  love email again

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCHv2] Use sendfile to copy data between file descriptors

[Lauri Kasanen]

 +config FEATURE_USE_SENDFILE
 +bool Use sendfile system call
 +default y
 +help
 +  When enabled, busybox will use the kernel sendfile() function
 +  instead of read/write loops where applicable.

Please include the kernel version requirement here.

 +#if ENABLE_FEATURE_USE_SENDFILE
 +   {
 +   ssize_t sz = size  (size  buffer_size)
 +   ? size : MAXINT(ssize_t) - 0x;
  
 -   rd = safe_read(src_fd, buffer, size  buffer_size ?
 buffer_size : size);
 +   while (1) {

And here. Comment should be enough, as the call will probably error out
on older kernels.

 +   rd = sendfile(dst_fd, src_fd, NULL, sz);
 +   if (rd = 0) {
 +   /* Might be EOF, might be an error,
 +* to make sure fall back to the

- Lauri

-- 
http://www.fastmail.fm - A no graphics, no pop-ups email service

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: dc hitting a compiler bug, or undefined behavior

[Lauri Kasanen]

On Mon, Mar 31, 2014, at 0:37, Ralf Friedl wrote:
  What's even worse is that adding any output to push(), even a puts(hi)
  that does not print the argument or any of the stack vars, fixes it. So
  something magic is going on inside the GCC optimization, I'm afraid this
  is above my pay grade.
  Could you send the file miscutils/dc.o that is created with and without
  this puts(hi) in push()?
  Attached.
 Are you using some special compiler options, especially regarding 
 parameter passing in registers and stack alignment?

None, I'm afraid. CFLAGS etc are all unset. See later on for the gcc
defaults too.

Thanks for the research. I only speak x86 asm on a yep, that's asm
level.

 The instruction at 12 loads the double from address %edi after %edi has 
 been set to point to the parameter area. The instruction at 0 in the 
 failed case is exactly the same, except that %edi has not been setup 
 before. So I would consider this a compiler bug.

Would it be possible for you to submit a gcc bug? I can try to do so
too, but it'll take a (long) while before I have time to build latest
gcc to test.
On bb side, I wonder if this gcc version should be blacklisted, or if
some workaround can be found.

Here's the -S -fverbose-asm output of an empty C file. I see nothing
about alignment, regparms, or otherwise relevant default options.

# GNU C (GCC) version 4.7.2 (i486-pc-linux-gnu)
#   compiled by GNU C version 4.7.2, GMP version 5.1.1, MPFR version
3.1.1, MPC version 1.0.1
# GGC heuristics: --param ggc-min-expand=38 --param
ggc-min-heapsize=15472
# options passed: 
# -iprefix
/tmp/tcloop/gcc/usr/local/bin/../lib/gcc/i486-pc-linux-gnu/4.7.2/
# test.c -mtune=i486 -march=i486 -fverbose-asm
# options enabled:  -fauto-inc-dec -fbranch-count-reg -fcommon
# -fdebug-types-section -fdelete-null-pointer-checks -fdwarf2-cfi-asm
# -fearly-inlining -feliminate-unused-debug-types -ffunction-cse
-fgcse-lm
# -fgnu-runtime -fident -finline-atomics -fira-share-save-slots
# -fira-share-spill-slots -fivopts -fkeep-static-consts
# -fleading-underscore -fmath-errno -fmerge-debug-strings
# -fmove-loop-invariants -fpcc-struct-return -fpeephole
# -fprefetch-loop-arrays -fsched-critical-path-heuristic
# -fsched-dep-count-heuristic -fsched-group-heuristic -fsched-interblock
# -fsched-last-insn-heuristic -fsched-rank-heuristic -fsched-spec
# -fsched-spec-insn-heuristic -fsched-stalled-insns-dep -fshow-column
# -fsigned-zeros -fsplit-ivs-in-unroller -fstrict-volatile-bitfields
# -ftrapping-math -ftree-cselim -ftree-forwprop -ftree-loop-if-convert
# -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize
# -ftree-parallelize-loops= -ftree-phiprop -ftree-pta -ftree-reassoc
# -ftree-scev-cprop -ftree-slp-vectorize -ftree-vect-loop-version
# -funit-at-a-time -fvect-cost-model -fverbose-asm
# -fzero-initialized-in-bss -m32 -m80387 -m96bit-long-double
# -malign-stringops -mfancy-math-387 -mfp-ret-in-387 -mglibc -mieee-fp
# -mno-red-zone -mno-sse4 -mpush-args -msahf -mtls-direct-seg-refs

- Lauri

-- 
http://www.fastmail.fm - Faster than the air-speed velocity of an
  unladen european swallow

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

dc hitting a compiler bug, or undefined behavior

[Lauri Kasanen]

Hi,

I'm seeing busybox dc acting funny when compiled with some versions of
gcc. This is on busybox git. The x86 binary busybox_unstripped and
config are attached.

gcc 4.2.2 - ok
gcc 4.7.2:
nc 10 1 add p
2.738e+93

So either bb is hitting a compiler bug, or undefined behavior somewhere
with new gcc's more aggressive optimizations.

- Lauri

-- 
http://www.fastmail.fm - Email service worth paying for. Try it for free



busybox_unstripped.gz
Description: GNU Zip compressed data


bb-config.gz
Description: GNU Zip compressed data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: dc hitting a compiler bug, or undefined behavior

[Lauri Kasanen]

On Sun, Mar 30, 2014, at 15:05, Ralf Friedl wrote:
  I'm seeing busybox dc acting funny when compiled with some versions of
  gcc. This is on busybox git. The x86 binary busybox_unstripped and
  config are attached.
 
  gcc 4.2.2 - ok
  gcc 4.7.2:
  nc 10 1 add p
  2.738e+93
 
  So either bb is hitting a compiler bug, or undefined behavior somewhere
  with new gcc's more aggressive optimizations.

 I have 4.7.2 on x86 and I get 11 as output. You could add debug output 
 to push and pop to see what happens.

I think it's related to 
http://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
which hit the radeon driver as well
http://lists.freedesktop.org/archives/dri-devel/2013-August/044122.html

but changing the stack[1] to stack[0], stack[], or even stack[10] did
not fix the issue.

What's even worse is that adding any output to push(), even a puts(hi)
that does not print the argument or any of the stack vars, fixes it. So
something magic is going on inside the GCC optimization, I'm afraid this
is above my pay grade.

FWIW printf calls in pop() or any other functions did not affect the
calculation. push() is being given the correct argument after strtod.

- Lauri

-- 
http://www.fastmail.fm - The way an email service should be

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: dc hitting a compiler bug, or undefined behavior

[Lauri Kasanen]

On Sun, Mar 30, 2014, at 18:26, Ralf Friedl wrote:
  What's even worse is that adding any output to push(), even a puts(hi)
  that does not print the argument or any of the stack vars, fixes it. So
  something magic is going on inside the GCC optimization, I'm afraid this
  is above my pay grade.

 Could you send the file miscutils/dc.o that is created with and without 
 this puts(hi) in push()?

Attached.

- Lauri

-- 
http://www.fastmail.fm - Email service worth paying for. Try it for free



fail-dc.o.gz
Description: GNU Zip compressed data


success-dc.o.gz
Description: GNU Zip compressed data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Ntpd config file support

[Lauri Kasanen]

On Sat, Mar 22, 2014, at 1:57, Laurent Bercot wrote:
 On 21/03/2014 23:10, Cathey, Jim wrote:
  The only thing BB would need would be to isolate initialization
  into separate functions that would be grouped together by the
  linker.  (And an associated link control file.)  The usual demand-paged
  kernel will take care of the rest.
 
 Yes, that would definitely be the right approach. However, it conflicts
 with code organization, and thus, maintainability: currently, the code is
 sharded by functionality, which is sane and sound - but the linker would
 need the code to be sharded by type, init or non-init, which is exactly
 orthogonal to functionality. I'm not up to date with latest linkers, but
 unless you can annotate functions inside a single .c, it means that you
 now need to split every single functionality into at least two .c files.

Slightly off tangent, but I remember reading a thesis on link time
optimization for the 2.4 linux kernel.

Among other things they managed to do was automatic recognition of init
code, and moving such to its own section. It was quite interesting tech,
no annotations needed; given the kernel already had some annotations,
the tech merged the newly found init code to the annotated one. It could
even detect parts that could not be annotated, because on some arch they
would be called more than once - if your arch didn't, into the init
section they went.

So this could be done entirely in the linker, no busybox code needed.

- Lauri

-- 
http://www.fastmail.fm - A fast, anti-spam email service.

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] correct_password: Handle NULL from crypt

[Lauri Kasanen]

On Mon, Feb 10, 2014, at 10:35, Rich Felker wrote:
 On Sun, Feb 09, 2014 at 12:52:50PM +0100, Denys Vlasenko wrote:
  The old behavior was to return ?
  (I have no old glibc at hand to check it quickly).
  
  Since we have exactly one crypt() call in the entire busybox,
  the easiest fix would be to just add
  if (NULL) return ;
  there. Are there downsides to that?
 
 Are you sure glibc used to return  on invalid inputs? I thought it
 returned something unmatchable. Returning  is mildly dangerous since
 you could end up storing a blank password in the database and allowing
 logins with no password, but if that's protected against, you should
 be fine.

Yes, it works that way. Tested on 2.7.

- Lauri

-- 
http://www.fastmail.fm - Send your email first class

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] correct_password: Handle NULL from crypt

[Lauri Kasanen]

Hi,

As with many other software, busybox was also broken by the glibc =
2.17 behavior change. Now crypt() returns NULL if either salt or
password is invalid.

This causes busybox 1.21, 1.22, and git su to segfault, when you just
press enter at the password prompt (configured to use system crypt() of
course).

Program terminated with signal 11, Segmentation fault.
#0  0xb760cb84 in strcmp () from /lib/libc.so.6
(gdb) bt full
#0  0xb760cb84 in strcmp () from /lib/libc.so.6
No symbol table info available.
#1  0x080493d3 in ask_and_check_password_extended ()
No symbol table info available.

The attached patch fixes su. You may want to check every other call to
crypt() in busybox.

- Lauri

-- 
http://www.fastmail.fm - A fast, anti-spam email service.

From 586f648074e6f979b79a88e1722188b8e6562bd0 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Mon, 3 Feb 2014 18:47:25 +0200
Subject: [PATCH] correct_password: Handle NULL from crypt

glibc = 2.17 may return NULL from crypt if the password or salt is invalid.

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 libbb/correct_password.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/libbb/correct_password.c b/libbb/correct_password.c
index acadf39..e0dc66b 100644
--- a/libbb/correct_password.c
+++ b/libbb/correct_password.c
@@ -74,6 +74,10 @@ int FAST_FUNC ask_and_check_password_extended(const struct passwd *pw,
 		return -1;
 	}
 	encrypted = pw_encrypt(unencrypted, correct, 1);
+	if (!encrypted) {
+		/* The user-supplied password was invalid, crypt returned NULL. */
+		return 0;
+	}
 	r = (strcmp(encrypted, correct) == 0);
 	free(encrypted);
 	nuke_str(unencrypted);
-- 
1.8.3.1

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH 0/3] less: display ANSI colors with option -R

[Lauri Kasanen]

On Sun, Feb 2, 2014, at 17:56, Rich Felker wrote:
systemd is broken by design.
  
  I did not look too deeply into its design.
  Unless there are serious design flaws I am not aware of,
  the idea per se looks sensible.
 
 The idea is not sensible. If I find time I'll write a short article on
 why for ewontfix, but it basically amounts to:
 
 1. Crash in pid #1 brings down the whole system, so it's not
 acceptable to do anything complex in pid 1.

I fully agree with your mail. FWIW, when I made point 1 in the past,
their response was that since they catch all signals (including segv,
ill), systemd will happily not crash, but go on its merry way. This of
course completely ignores that it cannot fix whatever caused the error,
and may go on to do wildly incorrect things due to it.

- Lauri

-- 
http://www.fastmail.fm - mmm... Fastmail...

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH 0/3] less: display ANSI colors with option -R

[Lauri Kasanen]

On Thu, Jan 23, 2014, at 21:28, Bernhard Reutner-Fischer wrote:
 On 23 January 2014 11:01:05 Denys Vlasenko vda.li...@googlemail.com
  Pottering's attempts to force his crap down our throats have to be
  called out and resisted.
 
 :)
 Fix systemd then.
 But apart from that I'd like to have a less that can display colors.

Another me too. Colors in less are useful with git etc, systemd
doesn't have to have something to do with it.

- Lauri

-- 
http://www.fastmail.fm - Does exactly what it says on the tin

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: set_loop tries to create 1048575 loop devices

[Lauri Kasanen]

On Mon, Jan 20, 2014, at 18:52, Ralf Friedl wrote:
 Hi
 
 I just had mount create a large number of loop devices before I 
 interrupted it.
 The reason is that the loop module is not loaded in the kernel, but it 
 would be better to have an error message instead.
 ...
 
 I don't know whether there are other reasons why open /dev/loop might 
 return ENXIO, but if not, then set_loop should abort when it receives 
 that error.

According to libbb/loop.c, it also returns ENXIO to mean this loop is
free. So can't exactly abort for that.

I recommend you disable ENABLE_FEATURE_MOUNT_LOOP_CREATE if you don't
want the devices created.

- Lauri

-- 
http://www.fastmail.fm - Same, same, but different...

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: set_loop tries to create 1048575 loop devices

[Lauri Kasanen]

  I just had mount create a large number of loop devices before I 
  interrupted it.
  The reason is that the loop module is not loaded in the kernel, but it 
  would be better to have an error message instead.
  ...
  
  I don't know whether there are other reasons why open /dev/loop might 
  return ENXIO, but if not, then set_loop should abort when it receives 
  that error.
 
 According to libbb/loop.c, it also returns ENXIO to mean this loop is
 free. So can't exactly abort for that.
 
 I recommend you disable ENABLE_FEATURE_MOUNT_LOOP_CREATE if you don't
 want the devices created.

My mistake, the ENXIO was from ioctl not open. Checking for it in open
should be ok.

- Lauri

-- 
http://www.fastmail.fm - IMAP accessible web-mail

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: Why '-c DEV' option for switch_root?

[Lauri Kasanen]

On Mon, Dec 16, 2013, at 10:05, Laurent Bercot wrote:
 On 2013-12-16 01:08, Rob Landley wrote:
  The most recent kernel has my initmpfs patches, meaning initramfs
   can now be a tmpfs instead of ramfs.
  [snip blurb]
 
   You're listing reasons why initramfs (or initmpfs, if you prefer) is
 more logical than it was before, more convenient, etc. All this may be
 true, but it does not mean initramfs is actually *useful*.
 
   I have yet to see a case where initramfs is really needed. Every time
 I've seen a system boot on initramfs, the same goals could have been
 achieved via booting on the real root filesystem and doing work during
 initialization, which implies a lot less code, and is more maintainable,
 and safer (if something fails early on).

I disagree, initramfs (and now enhanced upstream with tmpfs) is very
useful for running in RAM without additional steps. There may not be a
real root, maybe we booted via PXE or an unsupported SCSI controller,
etc.

Without initramfs you'd have hacks like initrd - switch to tmpfs -
more code, more complexity. Without initrd either you'd have even worse
hacks with the kernel itself.

- Lauri

-- 
http://www.fastmail.fm - Choose from over 50 domains or use your own

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] wget: User-friendly fallback to http

[Lauri Kasanen]

Hi Denys,

Ping on
http://lists.busybox.net/pipermail/busybox/2013-November/080038.html

If DESKTOP guards or something else is needed, please say.

- Lauri

-- 
http://www.fastmail.fm - The way an email service should be

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] wget: User-friendly fallback to http

[Lauri Kasanen]

Hi,

GNU wget: wget google.com // ok
bb before: wget google.com // wget: not an http or ftp url

This costs 47 bytes here, but improves the usability quite a bit.

- Lauri

-- 
http://www.fastmail.fm - A no graphics, no pop-ups email service

From a89587e950bbfe1b3682a2c38405967f197a34e9 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Mon, 25 Nov 2013 20:53:12 +0200
Subject: [PATCH] wget: User-friendly fallback to http

GNU wget: wget google.com // ok
bb before: wget google.com // wget: not an http or ftp url

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 networking/wget.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/networking/wget.c b/networking/wget.c
index cfbacec..3c5 100644
--- a/networking/wget.c
+++ b/networking/wget.c
@@ -282,6 +282,11 @@ static void parse_url(const char *src_url, struct host_info *h)
 		h-port = bb_lookup_port(ftp, tcp, 21);
 		h-host = url + 6;
 		h-is_ftp = 1;
+	} else if (!strstr(url, //)) {
+		// GNU wget is user-friendly and falls back to http://
+		h-port = bb_lookup_port(http, tcp, 80);
+		h-host = url;
+		h-is_ftp = 0;
 	} else
 		bb_error_msg_and_die(not an http or ftp url: %s, sanitize_string(url));
 
-- 
1.8.3.1

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH 1/2] wget: make default timeout configurable, remove a duplicate line

[Lauri Kasanen]

On Fri, Oct 18, 2013, at 12:52, Tito wrote:
 Hi,
 can the wget scripts not be modified like
 
 [ -r /etc/default/wget.conf ]  . /etc/default/wget.conf
 wget -t $WGET_TIMEOUT

Hi Tito,

The whole point of this patch was to avoid having to modify scripts,
otherwise we'd just have added the lot of -T options.

- Lauri

-- 
http://www.fastmail.fm - Faster than the air-speed velocity of an
  unladen european swallow

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH 1/2] wget: make default timeout configurable, remove a duplicate line

[Lauri Kasanen]

On Wed, Oct 16, 2013, at 22:23, Harald Becker wrote:
  GNU wget does kinda cater to this use: you can specify it in
  the .wgetrc file, but then that's only portable to GNU wget.
 How about adding limited support for .wgetrc then (only for
 timeouts)?
 
 Just a suggestion ...
 
 Support for .wgetrc is much bloat. What about looking for
 environment variable WGET_TIMEOUT? Ok it is not compatible with
 other other wget versions, but BB wget isn't neither. 

Checking for an env var would be more size. Having it as a compile-time
configure option means no size change, and we don't need the default to
change at runtime.

- Lauri

-- 
http://www.fastmail.fm - Or how I learned to stop worrying and
  love email again

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH 1/2] wget: make default timeout configurable, remove a duplicate line

[Lauri Kasanen]

On Mon, Oct 14, 2013, at 19:22, Denys Vlasenko wrote:
wget: make default timeout configurable
   
   Why? Upstream wget uses a fixed default timeout, 900.
  
  We'd like to change it in one place instead of adding -T options to a
  ton of scripts.
 
 But this will make your scripts depend on wget being the busybox's wget.
 This plagued Unix world for some time. Not a good idea.
 
 What upstream wget does to address such a case?

That's acceptable to us. We call it as busybox wget anyway when it
matters, and in other places, it is a mere user experience tweak - it
will not break anything if another wget was used.

GNU wget does kinda cater to this use: you can specify it in the .wgetrc
file, but then that's only portable to GNU wget.

- Lauri

-- 
http://www.fastmail.fm - One of many happy users:
  http://www.fastmail.fm/help/overview_quotes.html

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH 1/2] wget: make default timeout configurable, remove a duplicate line

[Lauri Kasanen]

On Sat, Oct 12, 2013, at 22:48, Denys Vlasenko wrote:
  Small housekeeping for wget.
 
  wget: make default timeout configurable
 
 Why? Upstream wget uses a fixed default timeout, 900.

We'd like to change it in one place instead of adding -T options to a
ton of scripts. Doing it via Kconfig is cleaner than changing the
source.

- Lauri

-- 
http://www.fastmail.fm - The professional email service

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH 1/2] wget: make default timeout configurable, remove a duplicate line

[Lauri Kasanen]

Hi,

Small housekeeping for wget.

- Lauri

-- 
http://www.fastmail.fm - One of many happy users:
  http://www.fastmail.fm/help/overview_quotes.html

From 13755e22b0dc9655d5e5d05c2a0d06873a8ef526 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Tue, 8 Oct 2013 16:40:20 +0300
Subject: [PATCH 1/2] wget: make default timeout configurable, remove a
 duplicate line

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 networking/Config.src | 8 
 networking/wget.c | 3 +--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/networking/Config.src b/networking/Config.src
index e1ae0c9..0a483ae 100644
--- a/networking/Config.src
+++ b/networking/Config.src
@@ -981,6 +981,14 @@ config FEATURE_WGET_TIMEOUT
 	  connection initialization).  When FEATURE_WGET_LONG_OPTIONS is
 	  also enabled, the --timeout option will work in addition to -T.
 
+config WGET_DEFAULT_TIMEOUT
+	int Default wget timeout
+	default 900
+	range 1 2000
+	depends on FEATURE_WGET_TIMEOUT
+	help
+	  The default time, in seconds, to wait before wget gives up.
+
 config ZCIP
 	bool zcip
 	default y
diff --git a/networking/wget.c b/networking/wget.c
index 5dac2b5..f604828 100644
--- a/networking/wget.c
+++ b/networking/wget.c
@@ -87,7 +87,6 @@ struct globals {
 #define G (*ptr_to_globals)
 #define INIT_G() do { \
 	SET_PTR_TO_GLOBALS(xzalloc(sizeof(G))); \
-	IF_FEATURE_WGET_TIMEOUT(G.timeout_seconds = 900;) \
 } while (0)
 
 
@@ -944,7 +943,7 @@ int wget_main(int argc UNUSED_PARAM, char **argv)
 
 	INIT_G();
 
-	IF_FEATURE_WGET_TIMEOUT(G.timeout_seconds = 900;)
+	IF_FEATURE_WGET_TIMEOUT(G.timeout_seconds = CONFIG_WGET_DEFAULT_TIMEOUT;)
 	G.proxy_flag = on;   /* use proxies if env vars are set */
 	G.user_agent = Wget; /* User-Agent header field */
 
-- 
1.8.3.1

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH 2/2] wget: add support for connect timeout

[Lauri Kasanen]

Hi,

Connect timeout support for wget.

Bloatcheck:

function old new   delta
open_socket   38  71 +33
alarm_handler  -  31 +31
wget_main   24322447 +15
ftpcmd   127 130  +3
fgets_and_trim89  92  +3
base64enc 63  66  +3
--
(add/remove: 1/0 grow/shrink: 5/0 up/down: 88/0)   Total: 88
bytes
   textdata bss dec hex filename
  21739 970 104   22813591d busybox_old
  21980 986 104   230705a1e busybox_unstripped


- Lauri

-- 
http://www.fastmail.fm - Or how I learned to stop worrying and
  love email again

From 2a33129f02ce0609ba88e3d4ae98b1bed60654f0 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Tue, 8 Oct 2013 17:11:22 +0300
Subject: [PATCH 2/2] wget: add support for connect timeout

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 networking/Config.src | 16 +---
 networking/wget.c | 18 +-
 2 files changed, 26 insertions(+), 8 deletions(-)

diff --git a/networking/Config.src b/networking/Config.src
index 0a483ae..987ae26 100644
--- a/networking/Config.src
+++ b/networking/Config.src
@@ -970,16 +970,18 @@ config FEATURE_WGET_LONG_OPTIONS
 	  Support long options for the wget applet.
 
 config FEATURE_WGET_TIMEOUT
-	bool Enable read timeout option -T SEC
+	bool Enable timeout option -T SEC
 	default y
 	depends on WGET
 	help
-	  Supports network read timeout for wget, so that wget will give
-	  up and timeout when reading network data, through the -T command
-	  line option.  Currently only network data read timeout is
-	  supported (i.e., timeout is not applied to the DNS nor TCP
-	  connection initialization).  When FEATURE_WGET_LONG_OPTIONS is
-	  also enabled, the --timeout option will work in addition to -T.
+	  Supports network read and connect timeouts for wget,
+	  so that wget will give up and timeout, through the -T
+	  command line option.
+
+	  Currently only connect and network data read timeout are
+	  supported (i.e., timeout is not applied to the DNS query). When 
+	  FEATURE_WGET_LONG_OPTIONS is also enabled, the --timeout option
+	  will work in addition to -T.
 
 config WGET_DEFAULT_TIMEOUT
 	int Default wget timeout
diff --git a/networking/wget.c b/networking/wget.c
index f604828..5e3533a 100644
--- a/networking/wget.c
+++ b/networking/wget.c
@@ -72,6 +72,7 @@ struct globals {
 	const char *user_agent; /* User-Agent header field */
 #if ENABLE_FEATURE_WGET_TIMEOUT
 	unsigned timeout_seconds;
+	bool connecting;
 #endif
 	int output_fd;
 	int o_flags;
@@ -198,12 +199,16 @@ static FILE *open_socket(len_and_sockaddr *lsa)
 {
 	FILE *fp;
 
+	IF_FEATURE_WGET_TIMEOUT(G.connecting = 1; alarm(G.timeout_seconds);)
+
 	/* glibc 2.4 seems to try seeking on it - ??! */
 	/* hopefully it understands what ESPIPE means... */
 	fp = fdopen(xconnect_stream(lsa), r+);
 	if (fp == NULL)
 		bb_perror_msg_and_die(bb_msg_memory_exhausted);
 
+	IF_FEATURE_WGET_TIMEOUT(G.connecting = 0;)
+
 	return fp;
 }
 
@@ -907,6 +912,14 @@ However, in real world it was observed that some web servers
 	free(redirected_path);
 }
 
+#if ENABLE_FEATURE_WGET_TIMEOUT
+static void alarm_handler(int sig UNUSED_PARAM)
+{
+	if (G.connecting)
+		bb_error_msg_and_die(download timed out);
+}
+#endif
+
 int wget_main(int argc, char **argv) MAIN_EXTERNALLY_VISIBLE;
 int wget_main(int argc UNUSED_PARAM, char **argv)
 {
@@ -943,7 +956,10 @@ int wget_main(int argc UNUSED_PARAM, char **argv)
 
 	INIT_G();
 
-	IF_FEATURE_WGET_TIMEOUT(G.timeout_seconds = CONFIG_WGET_DEFAULT_TIMEOUT;)
+#if ENABLE_FEATURE_WGET_TIMEOUT
+	G.timeout_seconds = CONFIG_WGET_DEFAULT_TIMEOUT;
+	signal(SIGALRM, alarm_handler);
+#endif
 	G.proxy_flag = on;   /* use proxies if env vars are set */
 	G.user_agent = Wget; /* User-Agent header field */
 
-- 
1.8.3.1

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Kernel build + ash, trap ERR

[Lauri Kasanen]

Hi

The 3.8 kernel build, in scripts/link-vmlinux.sh, uses #!/bin/sh but
requests a trap on ERR.
This fails on bb ash:
scripts/link-vmlinux.sh: trap: line 105: ERR: invalid signal
specification

Is this something that ash should support, or is it a kernel bug for
using a bash feature in a /bin/sh script?

- Lauri

-- 
http://www.fastmail.fm - Access your email from home and the web

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] sha3sum: New applet, v4

[Lauri Kasanen]

Hi

Running Rob Landley's mips image in qemu, this cut seems to give correct
results on big-endian in all cases.

v4:
- Some cleanups
- Big-endian try 2 - all tests now work in qemu-system-mips

@Baruch - your system had some weird corruption, where the whole area
after being set to all zeros showed an actual value. I don't have any
idea why that happens there.

- Lauri

-- 
http://www.fastmail.fm - Access all of your messages and folders
  wherever you are

From c50a72a34c46e016c9c0962abc64a9682f213e9a Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Thu, 3 Jan 2013 21:10:01 +0200
Subject: [PATCH] sha3sum: New applet, v4

v4:
- Some cleanups
- Big-endian try 2 - all tests now work in qemu-system-mips

v3:
- Align to 64, just in case
- Use the correct type, size_t, in the update function

v2:
- Sorted the enum by alpha (was sorted by algorithm age)
- Use normal rotate here, xor not needed
- Comment cleanup
- Remove *empty* comments
- Unify one be/le path
- Remove #define round
- Move to if(BB_ENDIAN) instead of #if

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 coreutils/Config.src |   12 ++-
 coreutils/Kbuild.src |1 +
 coreutils/md5_sha1_sum.c |   17 
 include/applets.src.h|1 +
 include/libbb.h  |7 ++
 libbb/hash_md5_sha.c |  201 ++
 testsuite/sha3sum.tests  |3 +
 7 files changed, 239 insertions(+), 3 deletions(-)
 create mode 100755 testsuite/sha3sum.tests

diff --git a/coreutils/Config.src b/coreutils/Config.src
index a28449b..0c44c4b 100644
--- a/coreutils/Config.src
+++ b/coreutils/Config.src
@@ -514,6 +514,12 @@ config SHA512SUM
 	help
 	  Compute and check SHA512 message digest
 
+config SHA3SUM
+	bool sha3sum
+	default y
+	help
+	  Compute and check SHA3 (512-bit) message digest
+
 config SLEEP
 	bool sleep
 	default y
@@ -766,13 +772,13 @@ config FEATURE_HUMAN_READABLE
 	help
 	  Allow df, du, and ls to have human readable output.
 
-comment Common options for md5sum, sha1sum, sha256sum, sha512sum
-	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM
+comment Common options for md5sum, sha1sum, sha256sum, sha512sum, sha3sum
+	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM || SHA3SUM
 
 config FEATURE_MD5_SHA1_SUM_CHECK
 	bool Enable -c, -s and -w options
 	default y
-	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM
+	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM || SHA3SUM
 	help
 	  Enabling the -c options allows files to be checked
 	  against pre-calculated hash values.
diff --git a/coreutils/Kbuild.src b/coreutils/Kbuild.src
index d6453f0..b715b9c 100644
--- a/coreutils/Kbuild.src
+++ b/coreutils/Kbuild.src
@@ -62,6 +62,7 @@ lib-$(CONFIG_SEQ)   += seq.o
 lib-$(CONFIG_SHA1SUM)   += md5_sha1_sum.o
 lib-$(CONFIG_SHA256SUM) += md5_sha1_sum.o
 lib-$(CONFIG_SHA512SUM) += md5_sha1_sum.o
+lib-$(CONFIG_SHA3SUM)   += md5_sha1_sum.o
 lib-$(CONFIG_SLEEP) += sleep.o
 lib-$(CONFIG_SPLIT) += split.o
 lib-$(CONFIG_SORT)  += sort.o
diff --git a/coreutils/md5_sha1_sum.c b/coreutils/md5_sha1_sum.c
index 2cb6dd4..b62e7c2 100644
--- a/coreutils/md5_sha1_sum.c
+++ b/coreutils/md5_sha1_sum.c
@@ -55,6 +55,16 @@
 //usage: \n	-s	Don't output anything, status code shows success
 //usage: \n	-w	Warn about improperly formatted checksum lines
 //usage:	)
+//usage:
+//usage:#define sha3sum_trivial_usage
+//usage:	IF_FEATURE_MD5_SHA1_SUM_CHECK([-c[sw]] )[FILE]...
+//usage:#define sha3sum_full_usage \n\n
+//usage:   Print IF_FEATURE_MD5_SHA1_SUM_CHECK( or check)  SHA3-512 checksums
+//usage:	IF_FEATURE_MD5_SHA1_SUM_CHECK( \n
+//usage: \n	-c	Check sums against list in FILEs
+//usage: \n	-s	Don't output anything, status code shows success
+//usage: \n	-w	Warn about improperly formatted checksum lines
+//usage:	)
 
 #include libbb.h
 
@@ -65,6 +75,7 @@ enum {
 	HASH_MD5 = 's', /* md5sum */
 	HASH_SHA1 = '1',
 	HASH_SHA256 = '2',
+	HASH_SHA3 = '3',
 	HASH_SHA512 = '5',
 };
 
@@ -86,6 +97,7 @@ static uint8_t *hash_file(const char *filename)
 {
 	int src_fd, hash_len, count;
 	union _ctx_ {
+		sha3_ctx_t sha3;
 		sha512_ctx_t sha512;
 		sha256_ctx_t sha256;
 		sha1_ctx_t sha1;
@@ -124,6 +136,11 @@ static uint8_t *hash_file(const char *filename)
 		update = (void*)sha512_hash;
 		final = (void*)sha512_end;
 		hash_len = 64;
+	} else if (ENABLE_SHA3SUM  hash_algo == HASH_SHA3) {
+		sha3_begin(context.sha3);
+		update = (void*)sha3_hash;
+		final = (void*)sha3_end;
+		hash_len = 64;
 	} else {
 		xfunc_die(); /* can't reach this */
 	}
diff --git a/include/applets.src.h b/include/applets.src.h
index 597b1c9..29ab167 100644
--- a/include/applets.src.h
+++ b/include/applets.src.h
@@ -328,6 +328,7 @@ IF_SETSEBOOL(APPLET(setsebool, BB_DIR_USR_SBIN, BB_SUID_DROP))
 IF_SETSID(APPLET(setsid, BB_DIR_USR_BIN, BB_SUID_DROP))
 IF_SETUIDGID(APPLET_ODDNAME(setuidgid, chpst, BB_DIR_USR_BIN, BB_SUID_DROP, setuidgid))
 IF_SHA1SUM

Re: [PATCH] sha3sum: New applet, v4

[Lauri Kasanen]

On Thu, Jan 10, 2013, at 13:35, Baruch Siach wrote:
 I just wanted to note the this SHA3 implementation is considerably slower
 than MD5 and SHA1 of Busybox, when run on my system. Sample results:
 
 # ls -l busybox 
 -rwxr-xr-x 1 root root 1296692 Jan 10 11:17 busybox
 
 # time ./busybox sha3sum busybox 
 95756158fff5a75e2c3b4826fba4bf0b88d33c3157c2ae3e209aa0e76354759bc14fef25a017a1b6340aeb4e8df6201534e31b2287364f5227ae40789c3b1e97
  busybox
 
 real0m4.085s
 
 # time ./busybox md5sum busybox 
 892a46b7d153b54f48c53815fd785d74  busybox
 
 real0m0.099s
 
 # time ./busybox sha1sum busybox 
 20ad89fd2476b0e6dbbe637455b7db4dde506ea1  busybox
 
 real0m0.101s
 
 Is this expected?

Yes, from two ways: SHA-3 as an algorithm is 2-3x slower than md5, and
the implementation here is optimized for size.

There exist faster implementations, but bb usually wants the smallest
size.


Another point is that on big-endian it now flips to little-endian and
back to get correct results. This is how the upstream Keccak examples
all do it, and I couldn't get correct results on BE otherwise. The speed
should be better on little-endian systems.

- Lauri

-- 
http://www.fastmail.fm - Does exactly what it says on the tin

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] sha3sum: New applet, v4

[Lauri Kasanen]

 Yes, from two ways: SHA-3 as an algorithm is 2-3x slower than md5, and
 the implementation here is optimized for size.
 
 There exist faster implementations, but bb usually wants the smallest
 size.
 
 
 Another point is that on big-endian it now flips to little-endian and
 back to get correct results. This is how the upstream Keccak examples
 all do it, and I couldn't get correct results on BE otherwise. The speed
 should be better on little-endian systems.

On x86_64 it's about 5x slower than bb sha1sum here, which is within the
target (2-3x plus size vs speed).

- Lauri

-- 
http://www.fastmail.fm - Or how I learned to stop worrying and
  love email again

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] sha3sum: New applet, v2

[Lauri Kasanen]

  # mips64
  $ echo Hello World | ./busybox sha3sum
  bb5596c9990b5e6f821d17150b3c79585180ed05f4570ba82a89816a5680c463d3ce23fa8daff68e0ada7c235e21f619d5fc0e122ee2e5fe959e004742d53ee4
  # x86_64
  $ echo Hello World | ./busybox sha3sum
  9ad2496364575076338331ca40a51a46d6a1d680c4e4d31bc3db6f4e46a5de376e0289692dbbb85fec8d9843f7c0a7efe200e86f2382c07d0674d20862db72d8
   -
 
 Is there an independent, publicly available implementation of SHA-3 that
 would 
 tell us which one is correct?

I used the prints on Wikipedia to test - the SHA-3 page lists the sums
for an empty string, the lazy fox sentence, and the lazy fox sentence
with a dot added. All three match with this implementation (when run on
x86_64).

- Lauri

-- 
http://www.fastmail.fm - The way an email service should be

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] sha3sum: New applet, v2

[Lauri Kasanen]

 A pessimized build give the exact same result. Details below:

Thanks Baruch. It's as I thought - something gives it wrong arguments.
See here:
 
 #1  0x1009dab4 in sha3_hash (state=0xbfca1fc0, data=0x101cafd0 ,
 bytes=21474836485)
 at libbb/hash_md5_sha.c:1055

bytes is 20 Gb (!), yet the memory pointed to by data is a few bytes
long only.

I think I know why, size_t on your platform is not 64-bit, will post a
v3 soon.

- Lauri

-- 
http://www.fastmail.fm - A fast, anti-spam email service.

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] sha3sum: New applet, v3

[Lauri Kasanen]

Hi

This patch should no longer crash @ ppc. But I think it will still have
wrong results on big-endian, and I'm not sure it's something I can fix
without access to such.

v3:
- Align to 64, just in case
- Use the correct type, size_t, in the update function

- Lauri

-- 
http://www.fastmail.fm - The way an email service should be

From 864e27f3a01ebff916ade68d5706ad0088bc5b40 Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Thu, 3 Jan 2013 21:10:01 +0200
Subject: [PATCH] sha3sum: New applet, v3

v3:
- Align to 64, just in case
- Use the correct type, size_t, in the update function

v2:
- Sorted the enum by alpha (was sorted by algorithm age)
- Use normal rotate here, xor not needed
- Comment cleanup
- Remove *empty* comments
- Unify one be/le path
- Remove #define round
- Move to if(BB_ENDIAN) instead of #if

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 coreutils/Config.src |   12 ++-
 coreutils/Kbuild.src |1 +
 coreutils/md5_sha1_sum.c |   17 
 include/applets.src.h|1 +
 include/libbb.h  |7 ++
 libbb/hash_md5_sha.c |  210 ++
 testsuite/sha3sum.tests  |3 +
 7 files changed, 248 insertions(+), 3 deletions(-)
 create mode 100755 testsuite/sha3sum.tests

diff --git a/coreutils/Config.src b/coreutils/Config.src
index a28449b..0c44c4b 100644
--- a/coreutils/Config.src
+++ b/coreutils/Config.src
@@ -514,6 +514,12 @@ config SHA512SUM
 	help
 	  Compute and check SHA512 message digest
 
+config SHA3SUM
+	bool sha3sum
+	default y
+	help
+	  Compute and check SHA3 (512-bit) message digest
+
 config SLEEP
 	bool sleep
 	default y
@@ -766,13 +772,13 @@ config FEATURE_HUMAN_READABLE
 	help
 	  Allow df, du, and ls to have human readable output.
 
-comment Common options for md5sum, sha1sum, sha256sum, sha512sum
-	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM
+comment Common options for md5sum, sha1sum, sha256sum, sha512sum, sha3sum
+	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM || SHA3SUM
 
 config FEATURE_MD5_SHA1_SUM_CHECK
 	bool Enable -c, -s and -w options
 	default y
-	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM
+	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM || SHA3SUM
 	help
 	  Enabling the -c options allows files to be checked
 	  against pre-calculated hash values.
diff --git a/coreutils/Kbuild.src b/coreutils/Kbuild.src
index d6453f0..b715b9c 100644
--- a/coreutils/Kbuild.src
+++ b/coreutils/Kbuild.src
@@ -62,6 +62,7 @@ lib-$(CONFIG_SEQ)   += seq.o
 lib-$(CONFIG_SHA1SUM)   += md5_sha1_sum.o
 lib-$(CONFIG_SHA256SUM) += md5_sha1_sum.o
 lib-$(CONFIG_SHA512SUM) += md5_sha1_sum.o
+lib-$(CONFIG_SHA3SUM)   += md5_sha1_sum.o
 lib-$(CONFIG_SLEEP) += sleep.o
 lib-$(CONFIG_SPLIT) += split.o
 lib-$(CONFIG_SORT)  += sort.o
diff --git a/coreutils/md5_sha1_sum.c b/coreutils/md5_sha1_sum.c
index 2cb6dd4..b62e7c2 100644
--- a/coreutils/md5_sha1_sum.c
+++ b/coreutils/md5_sha1_sum.c
@@ -55,6 +55,16 @@
 //usage: \n	-s	Don't output anything, status code shows success
 //usage: \n	-w	Warn about improperly formatted checksum lines
 //usage:	)
+//usage:
+//usage:#define sha3sum_trivial_usage
+//usage:	IF_FEATURE_MD5_SHA1_SUM_CHECK([-c[sw]] )[FILE]...
+//usage:#define sha3sum_full_usage \n\n
+//usage:   Print IF_FEATURE_MD5_SHA1_SUM_CHECK( or check)  SHA3-512 checksums
+//usage:	IF_FEATURE_MD5_SHA1_SUM_CHECK( \n
+//usage: \n	-c	Check sums against list in FILEs
+//usage: \n	-s	Don't output anything, status code shows success
+//usage: \n	-w	Warn about improperly formatted checksum lines
+//usage:	)
 
 #include libbb.h
 
@@ -65,6 +75,7 @@ enum {
 	HASH_MD5 = 's', /* md5sum */
 	HASH_SHA1 = '1',
 	HASH_SHA256 = '2',
+	HASH_SHA3 = '3',
 	HASH_SHA512 = '5',
 };
 
@@ -86,6 +97,7 @@ static uint8_t *hash_file(const char *filename)
 {
 	int src_fd, hash_len, count;
 	union _ctx_ {
+		sha3_ctx_t sha3;
 		sha512_ctx_t sha512;
 		sha256_ctx_t sha256;
 		sha1_ctx_t sha1;
@@ -124,6 +136,11 @@ static uint8_t *hash_file(const char *filename)
 		update = (void*)sha512_hash;
 		final = (void*)sha512_end;
 		hash_len = 64;
+	} else if (ENABLE_SHA3SUM  hash_algo == HASH_SHA3) {
+		sha3_begin(context.sha3);
+		update = (void*)sha3_hash;
+		final = (void*)sha3_end;
+		hash_len = 64;
 	} else {
 		xfunc_die(); /* can't reach this */
 	}
diff --git a/include/applets.src.h b/include/applets.src.h
index 597b1c9..29ab167 100644
--- a/include/applets.src.h
+++ b/include/applets.src.h
@@ -328,6 +328,7 @@ IF_SETSEBOOL(APPLET(setsebool, BB_DIR_USR_SBIN, BB_SUID_DROP))
 IF_SETSID(APPLET(setsid, BB_DIR_USR_BIN, BB_SUID_DROP))
 IF_SETUIDGID(APPLET_ODDNAME(setuidgid, chpst, BB_DIR_USR_BIN, BB_SUID_DROP, setuidgid))
 IF_SHA1SUM(APPLET_NOEXEC(sha1sum, md5_sha1_sum, BB_DIR_USR_BIN, BB_SUID_DROP, sha1sum))
+IF_SHA3SUM(APPLET_NOEXEC(sha3sum, md5_sha1_sum, BB_DIR_USR_BIN, BB_SUID_DROP, sha3sum))
 IF_SHA256SUM(APPLET_NOEXEC(sha256sum, md5_sha1_sum, BB_DIR_USR_BIN

Re: [PATCH] sha3sum: New applet, v3

[Lauri Kasanen]

 You can send me a debug prints augmented version of your patch to run on
 my PowerPC board if you like.

Thanks. Before we go to that, is the empty string example also wrong on
ppc? echo -n 

- Lauri

-- 
http://www.fastmail.fm - Send your email first class

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] sha3sum: New applet, v2

[Lauri Kasanen]

On Sun, Jan 6, 2013, at 7:18, Baruch Siach wrote:
 Hi Lauri,
 Testing on a big-endian system (PowerPC) I'm getting:
 
 # echo test |./busybox sha3sum
 Segmentation fault (core dumped)
 
 On a PC I get:
 
 $ echo test |./busybox sha3sum
 a5c3e353a185244f82a8de017db6022543d776f8943b457b6368c80854ecdfda1705e5dd514d5087116daccb7dace5ad42ab172bfe79a2c00b7a1e9c76e46b55
  -
 
 Any suggestion?

Well, as mentioned, I don't have such a system. Could you do a debug
build (CONFIG_DEBUG=y) and get valgrind output?

echo test | valgrind ./busybox_unstripped sha3sum

- Lauri

-- 
http://www.fastmail.fm - A fast, anti-spam email service.

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] sha3sum: New applet, v2

[Lauri Kasanen]

   Well, as mentioned, I don't have such a system. Could you do a debug
   build (CONFIG_DEBUG=y) and get valgrind output?
   
   echo test | valgrind ./busybox_unstripped sha3sum
  
  Unfortunately, valgrind does not support this PowerPC variant (e500v2).  
  I'll 
  try gdb and see what I get.
 
 Here it is:
 
 (gdb) target remote 192.168.26.170:1234
 Remote debugging using 192.168.26.170:1234
 0x10001f9c in _start ()
 (gdb) c
 Continuing.
 
 Program received signal SIGSEGV, Segmentation fault.
 KeccakF (state=0xbf8d1f30, in=value optimized out, laneCount=8)
 at libbb/hash_md5_sha.c:977
 977 state[laneCount] ^= SWAP_LE64(in[laneCount]);
 (gdb) bt
 #0  KeccakF (state=0xbf8d1f30, in=value optimized out, laneCount=8)
 at libbb/hash_md5_sha.c:977
 #1  0x10051474 in sha3_hash (state=0xbf8d1f30, data=0x10172fd0 , 
 bytes=value optimized out) at libbb/hash_md5_sha.c:1055
 #2  0x1003b544 in hash_file (filename=value optimized out)
 at coreutils/md5_sha1_sum.c:151
 #3  0x1003b7c4 in md5_sha1_sum_main (argc=value optimized out, 
 argv=value optimized out) at coreutils/md5_sha1_sum.c:247

Both state[8] and in[8] have to be valid at that point, there's no way
from the code for them to have less size than 200 and 72 respectively.

So it looks to me that the swap_le64 itself is crashing, but I'm not
sure about that, it should be a GCC builtin. Alternatively, does your
platform crash from unaligned access? State is 32-bit aligned currently,
not 64.

Can you try changing the alignment at include/libbb.h lines 1639 and
1640?

- Lauri

-- 
http://www.fastmail.fm - Choose from over 50 domains or use your own

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] sha3sum: New applet, v2

[Lauri Kasanen]

 The content of the r28 register is:
 
 (gdb) info registers
 ...
 r280x10173010   269955088
 
 The process' /proc/[pid]/maps file shows:
 
 1014d000-10173000 rwxp  00:00 0  [heap]
 
 which means that we're trying to read just beyond the process allocated
 heap.
 
 From the disassembly code I infer that the crash happens before the call
 to 
 bb_bswap_64 at address 0x10050b80.

Thanks.

Can you also enable CONFIG_PESSIMIZE (disable gcc's optimization) and
post the backtrace? I wonder if somehow it gets called with wrong
arguments (end of string, but wrong size for example), those were
optimized out in the first backtrace.

- Lauri

-- 
http://www.fastmail.fm - Send your email first class

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] sha3sum: New applet, v2

[Lauri Kasanen]

Hi

v2:
- Sorted the enum by alpha (was sorted by algorithm age)
- Use normal rotate here, xor not needed
- Comment cleanup
- Remove *empty* comments
- Unify one be/le path
- Remove #define round
- Move to if(BB_ENDIAN) instead of #if

- Lauri

PS: There were existing #if blocks in the same file, just above in the
md5 code. The requirement felt kinda weird with those there...

-- 
http://www.fastmail.fm - Same, same, but different...

From 84768213e7a004b9e703a1d7091bd43b34822e0c Mon Sep 17 00:00:00 2001
From: Lauri Kasanen cur...@operamail.com
Date: Thu, 3 Jan 2013 21:10:01 +0200
Subject: [PATCH] sha3sum: New applet, v2

- Sorted the enum by alpha (was sorted by algorithm age)
- Use normal rotate here, xor not needed
- Comment cleanup
- Remove *empty* comments
- Unify one be/le path
- Remove #define round
- Move to if(BB_ENDIAN) instead of #if

Signed-off-by: Lauri Kasanen cur...@operamail.com
---
 coreutils/Config.src |   12 ++-
 coreutils/Kbuild.src |1 +
 coreutils/md5_sha1_sum.c |   17 
 include/applets.src.h|1 +
 include/libbb.h  |7 ++
 libbb/hash_md5_sha.c |  212 ++
 testsuite/sha3sum.tests  |3 +
 7 files changed, 250 insertions(+), 3 deletions(-)
 create mode 100755 testsuite/sha3sum.tests

diff --git a/coreutils/Config.src b/coreutils/Config.src
index a28449b..0c44c4b 100644
--- a/coreutils/Config.src
+++ b/coreutils/Config.src
@@ -514,6 +514,12 @@ config SHA512SUM
 	help
 	  Compute and check SHA512 message digest
 
+config SHA3SUM
+	bool sha3sum
+	default y
+	help
+	  Compute and check SHA3 (512-bit) message digest
+
 config SLEEP
 	bool sleep
 	default y
@@ -766,13 +772,13 @@ config FEATURE_HUMAN_READABLE
 	help
 	  Allow df, du, and ls to have human readable output.
 
-comment Common options for md5sum, sha1sum, sha256sum, sha512sum
-	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM
+comment Common options for md5sum, sha1sum, sha256sum, sha512sum, sha3sum
+	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM || SHA3SUM
 
 config FEATURE_MD5_SHA1_SUM_CHECK
 	bool Enable -c, -s and -w options
 	default y
-	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM
+	depends on MD5SUM || SHA1SUM || SHA256SUM || SHA512SUM || SHA3SUM
 	help
 	  Enabling the -c options allows files to be checked
 	  against pre-calculated hash values.
diff --git a/coreutils/Kbuild.src b/coreutils/Kbuild.src
index d6453f0..b715b9c 100644
--- a/coreutils/Kbuild.src
+++ b/coreutils/Kbuild.src
@@ -62,6 +62,7 @@ lib-$(CONFIG_SEQ)   += seq.o
 lib-$(CONFIG_SHA1SUM)   += md5_sha1_sum.o
 lib-$(CONFIG_SHA256SUM) += md5_sha1_sum.o
 lib-$(CONFIG_SHA512SUM) += md5_sha1_sum.o
+lib-$(CONFIG_SHA3SUM)   += md5_sha1_sum.o
 lib-$(CONFIG_SLEEP) += sleep.o
 lib-$(CONFIG_SPLIT) += split.o
 lib-$(CONFIG_SORT)  += sort.o
diff --git a/coreutils/md5_sha1_sum.c b/coreutils/md5_sha1_sum.c
index 2cb6dd4..b62e7c2 100644
--- a/coreutils/md5_sha1_sum.c
+++ b/coreutils/md5_sha1_sum.c
@@ -55,6 +55,16 @@
 //usage: \n	-s	Don't output anything, status code shows success
 //usage: \n	-w	Warn about improperly formatted checksum lines
 //usage:	)
+//usage:
+//usage:#define sha3sum_trivial_usage
+//usage:	IF_FEATURE_MD5_SHA1_SUM_CHECK([-c[sw]] )[FILE]...
+//usage:#define sha3sum_full_usage \n\n
+//usage:   Print IF_FEATURE_MD5_SHA1_SUM_CHECK( or check)  SHA3-512 checksums
+//usage:	IF_FEATURE_MD5_SHA1_SUM_CHECK( \n
+//usage: \n	-c	Check sums against list in FILEs
+//usage: \n	-s	Don't output anything, status code shows success
+//usage: \n	-w	Warn about improperly formatted checksum lines
+//usage:	)
 
 #include libbb.h
 
@@ -65,6 +75,7 @@ enum {
 	HASH_MD5 = 's', /* md5sum */
 	HASH_SHA1 = '1',
 	HASH_SHA256 = '2',
+	HASH_SHA3 = '3',
 	HASH_SHA512 = '5',
 };
 
@@ -86,6 +97,7 @@ static uint8_t *hash_file(const char *filename)
 {
 	int src_fd, hash_len, count;
 	union _ctx_ {
+		sha3_ctx_t sha3;
 		sha512_ctx_t sha512;
 		sha256_ctx_t sha256;
 		sha1_ctx_t sha1;
@@ -124,6 +136,11 @@ static uint8_t *hash_file(const char *filename)
 		update = (void*)sha512_hash;
 		final = (void*)sha512_end;
 		hash_len = 64;
+	} else if (ENABLE_SHA3SUM  hash_algo == HASH_SHA3) {
+		sha3_begin(context.sha3);
+		update = (void*)sha3_hash;
+		final = (void*)sha3_end;
+		hash_len = 64;
 	} else {
 		xfunc_die(); /* can't reach this */
 	}
diff --git a/include/applets.src.h b/include/applets.src.h
index 597b1c9..29ab167 100644
--- a/include/applets.src.h
+++ b/include/applets.src.h
@@ -328,6 +328,7 @@ IF_SETSEBOOL(APPLET(setsebool, BB_DIR_USR_SBIN, BB_SUID_DROP))
 IF_SETSID(APPLET(setsid, BB_DIR_USR_BIN, BB_SUID_DROP))
 IF_SETUIDGID(APPLET_ODDNAME(setuidgid, chpst, BB_DIR_USR_BIN, BB_SUID_DROP, setuidgid))
 IF_SHA1SUM(APPLET_NOEXEC(sha1sum, md5_sha1_sum, BB_DIR_USR_BIN, BB_SUID_DROP, sha1sum))
+IF_SHA3SUM(APPLET_NOEXEC(sha3sum, md5_sha1_sum, BB_DIR_USR_BIN, BB_SUID_DROP, sha3sum))
 IF_SHA256SUM(APPLET_NOEXEC(sha256sum, md5_sha1_sum, BB_DIR_USR_BIN

Re: [PATCH] sha3sum: New applet

[Lauri Kasanen]

Hi

 can't you `git send-email` ?  attaching patches encoded in base64 makes
 it a pain to review.

Sorry, not at the moment, no usable SMTP server to do that with.

 seems like you do an xor in the LE case but not the BE case ?  i'm
 guessing that's a bug.

I don't have anything big-endian to test with. Should I put in a
#warning/#error that BE is untested, or?

  +#defineround laneCount
 
 why ?  why not adjust the code or the local variables instead ?

It was that way in the original code. Will change.


Will fix the other parts. Thanks for the fast review, this must be the
fastest reply I've ever gotten on the list ;)

- Lauri

PS: Resending, because forgot to include the list.

-- 
http://www.fastmail.fm - The way an email service should be

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: su should not chdir

[Lauri Kasanen]

Hi Denys

Seems this patch by Pascal in March was missed:
http://lists.busybox.net/pipermail/busybox/2012-March/077510.html

- Lauri

-- 
http://www.fastmail.fm - A no graphics, no pop-ups email service

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Linking fails when httpd gzip is enabled, but seamless gzip not

[Lauri Kasanen]

Hi

Building bb git master with the attached config, on armv6 gcc 4.4.5,
fails linking with:
/usr/bin/ld: busybox_unstripped: hidden symbol `check_signature16' isn't
defined
/usr/bin/ld: final link failed: Nonrepresentable section on output

After enabling CONFIG_FEATURE_SEAMLESS_GZ, the build succeeds. The same
build didn't need that option on x86_64 though, curiously?

- Lauri

-- 
http://www.fastmail.fm - Email service worth paying for. Try it for free



bb-config
Description: Binary data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: tcpsvd single-applet build fails

[Lauri Kasanen]

 
 Please try attached patch.

Works well, thanks.

- Lauri

-- 
http://www.fastmail.fm - The way an email service should be

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

tcpsvd single-applet build fails

[Lauri Kasanen]

32-bit x86 gcc 4.6.1

libbb/lib.a(appletlib.o): In function `main':
appletlib.c:(.text.main+0x67): undefined reference to `tcpsvd_main'

The config is attached.

- Lauri

-- 
http://www.fastmail.fm - Access all of your messages and folders
  wherever you are



bb-config
Description: Binary data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] tar: support the common file extensions .tlzma and .txz

[Lauri Kasanen]

On Tue, Mar 6, 2012, at 17:02, Denys Vlasenko wrote:
 I consolidated several copies of autodetection code. One in
 get_header_tar.c is gone.
 Now tar recognizes XZ-compressed data using generic setup_unzip_on_fd().
 I added a testsuite entry which verifies that.
 Please try current git.
 Please also review last ~5 commits, maybe I added a few bugs :(

Thanks, it works well now. Didn't catch anything reading the last few
commits.

- Lauri

-- 
http://www.fastmail.fm - Same, same, but different...

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] tar: support the common file extensions .tlzma and .txz

[Lauri Kasanen]

 Thanks, it works well now. Didn't catch anything reading the last few
 commits.

I ran cppcheck on the affected files, and it found one possibly
uninitialized use:
[decompress_bunzip2.c:156]: (error) Uninitialized variable: runCnt

Also, seems the __KERNEL__ ifdefs in the unxz code could be removed?

- Lauri

-- 
http://www.fastmail.fm - Does exactly what it says on the tin

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] tar: support the common file extensions .tlzma and .txz

[Lauri Kasanen]

  I am not convinced .txz and .tlzma extensions are common.
  (In my opinion, existence of .tgz extension was a mistake).
 
 They exist, and GNU tar handles them, so I think bb tar shouldn't puke
 when it sees one. Busybox does handle .tgz and .tbz2, because they have
 the magic detection already written (xz magic is marked todo in
 archival/libarchive/get_header_tar.c).

Replying to myself, but I just remembered all Slackware tarballs are
named .txz (changed in 2009 from tgz).
Perhaps that's worth something for the common argument. :)

- Lauri

-- 
http://www.fastmail.fm - The professional email service

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox