malloc: micro optimizations

2023-10-25 Thread Otto Moerbeek
Hi, a few micro-optimization, including getting rid of some statistics that are not actualy very interesting. Speedup amounts to a few tenths of percents to a few percents, depending on how biased the benchmark is. -Otto Index: stdlib/malloc.c

Re: relayd does not delete control socket on shutdown

2023-10-22 Thread Otto Moerbeek
On Sat, Oct 21, 2023 at 10:40:45PM +0300, Kapetanakis Giannis wrote: > On 21/10/2023 20:39, Florian Obser wrote: > > Which was 8 years ago. I don't understand why you see a change in 7.4. > > > > Anyway, we decided to not clean up control sockets in any of our > > privsep daemons because leaving

Re: Why store pointers for some functions in malloc.c?

2023-10-17 Thread Otto Moerbeek
On Wed, Oct 18, 2023 at 09:23:49AM +0900, Masato Asou wrote: > Hello tech@ and otto, > > Why do only some calling functions store the pinttes in region_info as > below: > > static void * > malloc_bytes(struct dir_info *d, size_t size, void *f) > { > > found: > if (i == 0 && k == 0 &&

malloc: more info in error message for write-after-free with option D

2023-10-10 Thread Otto Moerbeek
Hi, This diff adds better error reporting for write-after-free or the more general write of free memory if malloc option D is active. Knowing the place where allocations were done often helps to find out where the overwrite happened. If option D is active malloc now saves caller info in a

Re: Significance of MALLOC_OPTIONS=G

2023-09-28 Thread Otto Moerbeek
On Fri, Sep 29, 2023 at 12:11:51PM +0900, Masato Asou wrote: > I am investigating what problems can bt detected with MALLOC_OPTIONS. > SEGV occurs even if MALLOC_OPTIONS=G is not specified. Normally, the > areas allocated by malloc() are not contiguous. However, after many > malloc() operations

Re: malloc write after free error checking

2023-09-24 Thread Otto Moerbeek
On Wed, Sep 20, 2023 at 08:08:23AM +0200, Otto Moerbeek wrote: > On Sun, Sep 03, 2023 at 09:21:18AM +0200, Otto Moerbeek wrote: > > > Hello, > > > > I'm seeing some reports of "write after free" reported by malloc by > > peolpe running current. Mall

Re: malloc write after free error checking

2023-09-20 Thread Otto Moerbeek
On Wed, Sep 20, 2023 at 03:02:27PM +0200, Matthieu Herrb wrote: > On Wed, Sep 20, 2023 at 08:08:23AM +0200, Otto Moerbeek wrote: > > > > The other, a write after free that crashed the X server when running > > picard was diagnosed by me. This one was a bit

Re: malloc write after free error checking

2023-09-20 Thread Otto Moerbeek
On Sun, Sep 03, 2023 at 09:21:18AM +0200, Otto Moerbeek wrote: > Hello, > > I'm seeing some reports of "write after free" reported by malloc by > peolpe running current. Malloc has become more strict since begining > of June. Let me explain: > > Small allocatio

Re: malloc: add error message in putleakinfo

2023-09-08 Thread Otto Moerbeek
On Sat, Sep 09, 2023 at 11:27:37AM +0900, Masato Asou wrote: > From: Otto Moerbeek > Date: Fri, 8 Sep 2023 13:39:53 +0200 > > > On Fri, Sep 08, 2023 at 10:08:28AM +0900, Masato Asou wrote: > > > >> From: Masato Asou > >> Dat

Re: malloc: add error message in putleakinfo

2023-09-08 Thread Otto Moerbeek
On Fri, Sep 08, 2023 at 10:08:28AM +0900, Masato Asou wrote: > From: Masato Asou > Date: Fri, 08 Sep 2023 05:45:55 +0900 (JST) > > > There was a mistake in the diff. > > > > From: Masato Asou > > Date: Fri, 08 Sep 2023 05:33:23 +0900 (JST) > > > >> Hi, > >> > >> I have modified diff.

Re: malloc: add error message in putleakinfo

2023-09-07 Thread Otto Moerbeek
On Thu, Sep 07, 2023 at 06:45:55PM +0900, Masato Asou wrote: > Hi, > > I am using MALLOC_OPTIONS=D and kdump report no information when > malloc() exceeds 107350 bytes on my OpenBSD box. I have > investigated and found that mmap() faild in putleakinfo(). I'm ok with the general idea, but

Re: malloc write after free error checking

2023-09-03 Thread Otto Moerbeek
On Sun, Sep 03, 2023 at 09:30:49AM +0200, Otto Moerbeek wrote: > > You can also be set upon a wrong foot: if an out of bounds write on a > > adjacent chunk happens and lands in (another) free chunk, upon > > allocation of that free chunk it will be reported as a "wri

Re: malloc write after free error checking

2023-09-03 Thread Otto Moerbeek
On Sun, Sep 03, 2023 at 09:21:18AM +0200, Otto Moerbeek wrote: > Hello, > > I'm seeing some reports of "write after free" reported by malloc by > peolpe running current. Malloc has become more strict since begining > of June. Let me explain: > > Small allocatio

malloc write after free error checking

2023-09-03 Thread Otto Moerbeek
Hello, I'm seeing some reports of "write after free" reported by malloc by peolpe running current. Malloc has become more strict since begining of June. Let me explain: Small allocations share a page. e.g. a 4k page will hold 8 512 byte allocations. When one such allocation is freed, it will

Re: pax: truncate times to MAX_TIME_T, not INT_MAX

2023-06-26 Thread Otto Moerbeek
On Mon, Jun 26, 2023 at 11:09:10AM -0600, Todd C. Miller wrote: > If the mtime in the file header is larger than MAX_TIME_T, trucate > it to MAX_TIME_T, not INT_MAX. The existing assignment dates from > before we had a MAX_TIME_T definition in pax. How strange the checks use MAX_TIME_T as a

Re: recent malloc changes

2023-06-18 Thread Otto Moerbeek
On Sun, Jun 18, 2023 at 03:51:20PM +0200, Theo Buehler wrote: > > So what's in your malloc options? > > mdnsd -d must have thrown this warning for nearly 7 years. With d, it sets > 'malloc_options = "AFGJPX";'. You removed A and P end of october 2016. Sigh should be "S". I take care to

Re: recent malloc changes

2023-06-18 Thread Otto Moerbeek
On Sun, Jun 18, 2023 at 03:24:17PM +0200, Why 42? The lists account. wrote: > > On Sun, Jun 04, 2023 at 01:03:14PM +0200, Otto Moerbeek wrote: > > Hello, > > > > In the last few weeks a series of mallocs diff have been committed. > > The last one today. That on

recent malloc changes

2023-06-04 Thread Otto Moerbeek
Hello, In the last few weeks a series of mallocs diff have been committed. The last one today. That one allows malloc to detect more cases of write-after-free. While that is a good thing, it might uncover latent bugs in appliations. So if you are running current or snapshots, please keep an eye

Re: malloc: better write-after-free detection for chunks

2023-05-14 Thread Otto Moerbeek
On Sun, May 14, 2023 at 10:42:34AM +0200, Otto Moerbeek wrote: > Hi, > > On free, chunks (the pieces of a pages used for smaller allocations) > are junked and then validated after they leave the delayed free list. > So after free, a chunk always contains junk bytes. This means tha

malloc: better write-after-free detection for chunks

2023-05-14 Thread Otto Moerbeek
Hi, On free, chunks (the pieces of a pages used for smaller allocations) are junked and then validated after they leave the delayed free list. So after free, a chunk always contains junk bytes. This means that if we start with the right contents for a new page of chunks, we can *validate* instead

Re: malloc: less unlock/lock dancing

2023-05-10 Thread Otto Moerbeek
On Wed, May 10, 2023 at 10:08:09AM +0200, Theo Buehler wrote: > > Thanks! It has been committed. I doubt though if the Go runtime uses > > libc malloc. > > I don't know if the pure Go runtime uses libc malloc. However, some > of the test code involves cgo and calls into various C libraries >

Re: malloc: less unlock/lock dancing

2023-05-10 Thread Otto Moerbeek
On Tue, May 09, 2023 at 09:55:32PM +0200, Theo Buehler wrote: > On Thu, May 04, 2023 at 03:40:35PM +0200, Otto Moerbeek wrote: > > On Thu, Apr 27, 2023 at 02:17:10PM +0200, Otto Moerbeek wrote: > > > > > This was introduced to not stall other threads while mmap is

Re: malloc: less unlock/lock dancing

2023-05-04 Thread Otto Moerbeek
On Thu, Apr 27, 2023 at 02:17:10PM +0200, Otto Moerbeek wrote: > This was introduced to not stall other threads while mmap is called by > a thread. But now that mmap is unlocked, I believe it is no longer > useful. > > A full build is slighlty faster with this. But this als

malloc: less unlock/lock dancing

2023-04-27 Thread Otto Moerbeek
This was introduced to not stall other threads while mmap is called by a thread. But now that mmap is unlocked, I believe it is no longer useful. A full build is slighlty faster with this. But this also needs testing with you favorite multithreaded program. -Otto Index: stdlib/malloc.c

malloc leak detection available in -current

2023-04-17 Thread Otto Moerbeek
Hi, OpenBSD current now has built-in malloc leak detection. Make sure you run current and have debug symbols (OpenBSD base libraries have debug symbols, compile your own program with -g). To record the leak report: $ MALLOC_OPTIONS=D ktrace -tu a.out To view the leak report: $ kdump -u malloc

Re: MALLOC_STATS: dump internal state and leak info via utrace(2)

2023-04-15 Thread Otto Moerbeek
On Thu, Apr 13, 2023 at 08:22:45PM +0200, Otto Moerbeek wrote: > On Tue, Apr 11, 2023 at 05:50:43PM +0200, Otto Moerbeek wrote: > > > On Sun, Apr 09, 2023 at 12:17:35PM +0200, Otto Moerbeek wrote: > > > > > On Sun, Apr 09, 2023 at 10:08:25AM +0200, Claudio Jeker w

Re: MALLOC_STATS: dump internal state and leak info via utrace(2)

2023-04-13 Thread Otto Moerbeek
On Tue, Apr 11, 2023 at 05:50:43PM +0200, Otto Moerbeek wrote: > On Sun, Apr 09, 2023 at 12:17:35PM +0200, Otto Moerbeek wrote: > > > On Sun, Apr 09, 2023 at 10:08:25AM +0200, Claudio Jeker wrote: > > > > > On Sun, Apr 09, 2023 at 09:15:12AM +0200, Otto Moerbeek wr

Re: MALLOC_STATS: dump internal state and leak info via utrace(2)

2023-04-11 Thread Otto Moerbeek
On Sun, Apr 09, 2023 at 12:17:35PM +0200, Otto Moerbeek wrote: > On Sun, Apr 09, 2023 at 10:08:25AM +0200, Claudio Jeker wrote: > > > On Sun, Apr 09, 2023 at 09:15:12AM +0200, Otto Moerbeek wrote: > > > On Sun, Apr 09, 2023 at 08:20:43AM +0200, Otto Moerbeek wrote: > >

Re: MALLOC_STATS: dump internal state and leak info via utrace(2)

2023-04-09 Thread Otto Moerbeek
On Sun, Apr 09, 2023 at 10:08:25AM +0200, Claudio Jeker wrote: > On Sun, Apr 09, 2023 at 09:15:12AM +0200, Otto Moerbeek wrote: > > On Sun, Apr 09, 2023 at 08:20:43AM +0200, Otto Moerbeek wrote: > > > > > On Sun, Apr 09, 2023 at 07:53:31AM +0200, Sebastien Marie wrote: &g

Re: MALLOC_STATS: dump internal state and leak info via utrace(2)

2023-04-09 Thread Otto Moerbeek
On Sun, Apr 09, 2023 at 08:20:43AM +0200, Otto Moerbeek wrote: > On Sun, Apr 09, 2023 at 07:53:31AM +0200, Sebastien Marie wrote: > > > On Fri, Apr 07, 2023 at 09:52:52AM +0200, Otto Moerbeek wrote: > > > > Hi, > > > > > > > > This is work in pro

Re: MALLOC_STATS: dump internal state and leak info via utrace(2)

2023-04-09 Thread Otto Moerbeek
On Sun, Apr 09, 2023 at 07:53:31AM +0200, Sebastien Marie wrote: > On Fri, Apr 07, 2023 at 09:52:52AM +0200, Otto Moerbeek wrote: > > > Hi, > > > > > > This is work in progress. I have to think if the flags to kdump I'm > > > introducing should be two

Re: MALLOC_STATS: dump internal state and leak info via utrace(2)

2023-04-07 Thread Otto Moerbeek
On Wed, Apr 05, 2023 at 10:54:19AM +0200, Otto Moerbeek wrote: > Hi, > > This is work in progress. I have to think if the flags to kdump I'm > introducing should be two or a single one. > > Currently, malloc.c can be compiled with MALLOC_STATS defined. If run > with option

MALLOC_STATS: dump internal state and leak info via utrace(2)

2023-04-05 Thread Otto Moerbeek
Hi, This is work in progress. I have to think if the flags to kdump I'm introducing should be two or a single one. Currently, malloc.c can be compiled with MALLOC_STATS defined. If run with option D it dumps its state to a malloc.out file at exit. This state can be used to find leaks amongst

malloc: variation in junk locations

2023-04-01 Thread Otto Moerbeek
Hi, by default an allocation isn't fully written with junk bytes, only at certain spots. This introduces variations in the spot, so we have a bigger chance of catching write-after-frees in specific spots. After a remark from jsing@ that variation would be nice to have for this. -Otto

malloc: write after free check in more cases

2023-03-30 Thread Otto Moerbeek
Hi, this came up after a remark by tb@; it does a more thorough check of the chunks in the delay list if malloc option F (which is included in S). Catches way more use-after-free's as otherwise we'll have to wait until a chunk is actively re-used. With the new bucket code that might take longer

recv.c: consistency wrt NULL for pointers

2023-03-25 Thread Otto Moerbeek
Hi, Last arg also is a pointer, so pass NULL. -Otto Index: recv.c === RCS file: /home/cvs/src/lib/libc/net/recv.c,v retrieving revision 1.6 diff -u -p -r1.6 recv.c --- recv.c 4 Oct 2015 07:17:27 - 1.6

Re: malloc: change chunk sizes to be multiple of 16 instead of power of 2

2023-03-07 Thread Otto Moerbeek
On Tue, Mar 07, 2023 at 09:02:04AM +0100, Theo Buehler wrote: > > So here's the dif with the fix. > > The new diff went through an amd64 bulk without fallout and also works > fine on some dev machines. No noticeable performance impact for my > workloads. > > It also reads fine to me (ok tb). >

Re: malloc: change chunk sizes to be multiple of 16 instead of power of 2

2023-03-01 Thread Otto Moerbeek
On Wed, Mar 01, 2023 at 08:49:56AM +0100, Theo Buehler wrote: > On Wed, Mar 01, 2023 at 08:39:08AM +0100, Otto Moerbeek wrote: > > On Wed, Mar 01, 2023 at 08:31:47AM +0100, Theo Buehler wrote: > > > > > On Tue, Feb 28, 2023 at 05:52:28PM +0100, Otto Moerbeek wrote:

Re: malloc: change chunk sizes to be multiple of 16 instead of power of 2

2023-02-28 Thread Otto Moerbeek
On Wed, Mar 01, 2023 at 08:31:47AM +0100, Theo Buehler wrote: > On Tue, Feb 28, 2023 at 05:52:28PM +0100, Otto Moerbeek wrote: > > Second iteration. > > > > Gain back performance by allocation chunk_info pages in a bundle, and > > use less buckets is !malloc option S.

Re: malloc: change chunk sizes to be multiple of 16 instead of power of 2

2023-02-28 Thread Otto Moerbeek
Second iteration. Gain back performance by allocation chunk_info pages in a bundle, and use less buckets is !malloc option S. The chunk sizes used are 16, 32, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 384, 448, 512, 640, 768, 896, 1024, 1280, 1536, 1792, 2048 (and a few more for sparc84

Re: malloc: handle to be cleaned chunks the same as regular ones

2023-02-23 Thread Otto Moerbeek
On Sat, Feb 18, 2023 at 04:12:08PM +0100, Otto Moerbeek wrote: > Hi, > > these recent sshd double free issue prompted me to look at malloc > again. I have something bigger brewing, but this diff makes sure the > to be cleaned chunks (via freezero() or malloc_conceal) particpate i

malloc: change chunk sizes to be multiple of 16 instead of power of 2

2023-02-23 Thread Otto Moerbeek
Hi, The basic idea is simple: one of the reasons the recent sshd bug is potentially exploitable is that a (erroneously) freed malloc chunk gets re-used in a different role. My malloc has power of two chunk sizes and so one page of chunks holds many different types of allocations. Userland malloc

malloc: handle to be cleaned chunks the same as regular ones

2023-02-18 Thread Otto Moerbeek
Hi, these recent sshd double free issue prompted me to look at malloc again. I have something bigger brewing, but this diff makes sure the to be cleaned chunks (via freezero() or malloc_conceal) particpate in the delayed freeing just as others. -Otto Index: stdlib/malloc.c

Re: ifconfig.c redundancy the second

2023-01-13 Thread Otto Moerbeek
On Fri, Jan 13, 2023 at 08:04:36PM +0100, Mathias Koehler wrote: > I hope the following message is a format more helpful for you > guys. (Thanks to Otto Moerbeek who gave me a hint.) /% ... %./ is not a comment marker in C. -Otto > Again my question is how should that code

Re: ifconfig.c - redundancy

2023-01-07 Thread Otto Moerbeek
On Fri, Jan 06, 2023 at 09:46:13PM +0100, Mathias Koehler wrote: > lines 1911 - 1913 and 1917 - 1919 are code blocks that are looking > identical. I cannot imagine that that is on purpose. What do you > really wanna express here? Just quoting line numbers does not help a lot. You were looking at

Re: malloc and immutable

2022-12-21 Thread Otto Moerbeek
On Sun, Dec 18, 2022 at 08:10:48PM +0100, Otto Moerbeek wrote: > Hi, > > This is the latest verion of a diff I sent around previously, but now > it's time this gets wider testing for real. > > The main purpose is to rearrange malloc init in such a way that a few > pa

malloc and immutable

2022-12-18 Thread Otto Moerbeek
Hi, This is the latest verion of a diff I sent around previously, but now it's time this gets wider testing for real. The main purpose is to rearrange malloc init in such a way that a few pages containing mallic internal data structures can be made immutable. In addtion to that, each pools is

Re: More on mimmutable

2022-11-17 Thread Otto Moerbeek
On Thu, Nov 17, 2022 at 08:10:05PM -0700, Theo de Raadt wrote: > So this static executable is completely immutable, except for the > OPENBSD_MUTABLE region. This annotation is used in one place now, deep > inside libc's malloc(3) code, where a piece of code flips a data structure > between

Re: M1 Macmini lost hw.cpuspeed

2022-10-24 Thread Otto Moerbeek
On Mon, Oct 24, 2022 at 04:15:40PM +0200, Mark Kettenis wrote: > > Date: Mon, 24 Oct 2022 14:52:00 +0200 > > From: Robert Nagy > > > > On 24/10/22 14:49 +0200, Theo Buehler wrote: > > > On Mon, Oct 24, 2022 at 09:24:14AM +0200, Otto Moerbeek wrote: > > >

M1 Macmini lost hw.cpuspeed

2022-10-24 Thread Otto Moerbeek
Hello, after updating my M1 macmini after my vacatiuon to the latest snap it seems to have lost a few sysctl nodes, making apm(8) fail: [otto@macmini:4]$ ktrace apm Battery state: unknown, 0% remaining, unknown life estimate AC adapter state: not known Performance adjustment mode: invalid (0

malloc diff

2022-10-21 Thread Otto Moerbeek
Hi, this diff has been sent out earlier, but now that more or the immutable parts are in, it is time to test it in this new environment. Thanks, -Otto Index: stdlib/malloc.c === RCS file:

Re: malloc: prep for immutable pages

2022-10-05 Thread Otto Moerbeek
On Wed, Oct 05, 2022 at 02:47:19PM +0200, Marc Espie wrote: > On Tue, Oct 04, 2022 at 10:15:51AM -0600, Theo de Raadt wrote: > > A note on why this chance is coming. > > > > malloc.c (as it is today), does mprotects back and forth between RW and > > R, to protect an internal object. This object

malloc: prep for immutable pages

2022-09-29 Thread Otto Moerbeek
Hi, Rearrange things so that we do not have to flip protection of r/o pages back and forth when swicthing from single-threaded to multi-threaded. Also saves work in many cases by not initing pools until they are used: the pool used for MAP_CONCEAL pages is not used by very many processes and if

Re: Request for testing malloc and multi-threaded applications

2022-09-27 Thread Otto Moerbeek
On Tue, Sep 27, 2022 at 03:31:12PM +0200, Renaud Allard wrote: > On 1/16/19 19:09, Otto Moerbeek wrote: > > On Wed, Jan 16, 2019 at 01:25:25PM +, Stuart Henderson wrote: > > > > > On 2019/01/04 08:09, Otto Moerbeek wrote: > > > > On Thu, Dec 27, 2018 at 0

Re: installer: zap fdisk.8.gz and disklabel.8.gz

2022-08-25 Thread Otto Moerbeek
On Thu, Aug 25, 2022 at 07:32:16PM +, Miod Vallat wrote: > > > The ability to be able to read the manual pages from the binaries > > > themselves, when running is interactive mode, is an intentional feature > > > and the reason they embed a gzipped version of the formatted manpage. > > > >

Re: Race in disk_attach_callback?

2022-07-13 Thread Otto Moerbeek
On Wed, Jul 13, 2022 at 02:18:53PM +0200, Otto Moerbeek wrote: > Hi, > > after a prompt from stsp@ and florian@, reporting that newfs_msdos > fails when given an $duid.i argument, I set down to see what could be > going on. My test using an USB stick failed to reprdouce the pr

Race in disk_attach_callback?

2022-07-13 Thread Otto Moerbeek
Hi, after a prompt from stsp@ and florian@, reporting that newfs_msdos fails when given an $duid.i argument, I set down to see what could be going on. My test using an USB stick failed to reprdouce the problem. Then I started using a vnd, and that shows the issue (once in a while). The feeling is

Re: quiz(6): update european countries

2022-07-04 Thread Otto Moerbeek
On Mon, Jul 04, 2022 at 05:44:33PM -0400, Daniel Dickman wrote: > > > On Sun, 3 Jul 2022, Daniel Dickman wrote: > > > On Sat, Jul 2, 2022 at 9:26 PM Ben Fuller wrote: > > > > > > I noticed Montenegro doesn't have an entry. Presumably this file hasn't > > > been updated since before 2006! > >

Re: dig(1): SVCB and HTTPS RR types

2022-07-03 Thread Otto Moerbeek
On Sun, Jul 03, 2022 at 07:47:27AM +0200, Florian Obser wrote: > anyone? Looks good and works for me, ok. -Otto > > On 2022-06-25 13:15 +02, Florian Obser wrote: > > See https://datatracker.ietf.org/doc/draft-ietf-dnsop-svcb-https/ > > > > $ ./obj/dig @8.8.8.8 +norec

Re: a few fixes for cat bugs

2022-07-02 Thread Otto Moerbeek
On Sat, Jul 02, 2022 at 08:38:53AM +0100, Leah Rowe wrote: > > Hi Otto, > > > Your fixes are not ok. See comment inline. > > The other person (Theo) who responded, raised the same concerns as you. > Sorry for wasting your time. I've reverted the patches myself, locally, > knowing now that I

Re: a few fixes for cat bugs

2022-07-02 Thread Otto Moerbeek
On Sat, Jul 02, 2022 at 03:15:21AM +0100, Leah Rowe wrote: > > Hi, > > I've been playing around with a few OpenBSD userland programs. By that, > I mean I've been studying them extensively. I'm working on creating a > busybox-like userland for Linux with musl libc, and I need quality code > so I

Re: Picky, but much more efficient arc4random_uniform!

2022-05-18 Thread Otto Moerbeek
On Wed, May 18, 2022 at 05:08:02PM +1000, Damien Miller wrote: > On Wed, 18 May 2022, Otto Moerbeek wrote: > > > instrumenting the code to count the number of arc4random calls I see thsi: > > > > openbsd; elapsed = 2.835819; calls = 12340949 > > bitmask; elapse

Re: Picky, but much more efficient arc4random_uniform!

2022-05-17 Thread Otto Moerbeek
On Wed, May 18, 2022 at 02:50:28PM +1000, Damien Miller wrote: > On Tue, 17 May 2022, Raimo Niskanen wrote: > > > Why reinvent the wheel? > > > > Here is a pretty good walkthrough of established methods: > > > > https://www.pcg-random.org/posts/bounded-rands.html > > > > It sounds to me

Re: Picky, but much more efficient arc4random_uniform!

2022-05-15 Thread Otto Moerbeek
On Sun, May 15, 2022 at 08:57:09AM -0300, Crystal Kolipe wrote: > On Sun, May 15, 2022 at 11:44:29AM +0200, Otto Moerbeek wrote: > > On Sun, May 15, 2022 at 04:27:30AM -0500, Luke Small wrote: > > > How did someone prove the current implementation was cryptographically > &g

Re: Picky, but much more efficient arc4random_uniform!

2022-05-15 Thread Otto Moerbeek
the verification. -Otto > > On Sun, May 15, 2022 at 3:15 AM Otto Moerbeek wrote: > > > On Sun, May 15, 2022 at 01:12:28AM -0500, Luke Small wrote: > > > > > This is version 1, which I was pretty sure was secure. > > > > > > I revamped

Re: Picky, but much more efficient arc4random_uniform!

2022-05-15 Thread Otto Moerbeek
On Sun, May 15, 2022 at 01:12:28AM -0500, Luke Small wrote: > This is version 1, which I was pretty sure was secure. > > I revamped it with a few features and implanted the binary search for 'bit' > > in most cases, which aren't intentionally worst-case, it's pretty darned > fast! > > This is

Re: Picky, but much more efficient arc4random_uniform!

2022-05-15 Thread Otto Moerbeek
On Sat, May 14, 2022 at 05:03:17PM -0700, Philip Guenther wrote: > On Sun, 15 May 2022, Steffen Nurpmeso wrote: > > Stuart Henderson wrote in > ... > > |what's the perceived problem you're wanting to solve? and does that > > |problem actually exist in the first place? > > > > The problem is

Re: Picky, but much more efficient arc4random_uniform!

2022-05-14 Thread Otto Moerbeek
On Sat, May 14, 2022 at 05:48:10AM -0500, Luke Small wrote: > arc4random_uniform_fast2 that I made, streams in data from arc4random() and > uses the datastream directly and uses it as a bit by bit right "sliding > window" in the last loop. arc4random_uniform() uses a modulus which I is > simple

Re: Picky, but much more efficient arc4random_uniform!

2022-05-14 Thread Otto Moerbeek
In general, wasting some arc4random bits is not a problem at all. I see a lot of code with no demonstration, explanation or proof why it would be correct (i.e. produces uniform results). You only talk about speed. The current code might not be optimal, but at least it is very clear it's correct.

Re: stop using mquery(2) inside realloc(3)

2022-05-14 Thread Otto Moerbeek
On Fri, May 13, 2022 at 08:21:19PM -0700, Philip Guenther wrote: > If you try to grow a 'large' (at least half a page) allocation with > realloc(3), it'll try to extend the memory mapping for it and if that > works it won't need to move it. (on a side note: it does not do anything at all if

Re: pf igmp icmp6 multicast router alert

2022-04-28 Thread Otto Moerbeek
On Thu, Apr 28, 2022 at 12:36:40AM +0200, Alexander Bluhm wrote: > On Wed, Apr 27, 2022 at 11:47:45PM +0200, Alexander Bluhm wrote: > > New diff: > > - make off and end relative to opts array > > - check length of IPv4 options > > - fix call to pf_walk_option > > - add case IP6OPT_PADN > > - add

Re: errata70.html - update the 7.1 link

2022-04-22 Thread Otto Moerbeek
On Fri, Apr 22, 2022 at 07:22:18PM -0400, Horia Racoviceanu wrote: > - change 7.1 link from errata70.html to errata71.html > Index: errata70.html > === > RCS file: /cvs/www/errata70.html,v > retrieving revision 1.25 > diff -u -p

Re: pf igmp icmp6 multicast router alert

2022-04-21 Thread Otto Moerbeek
On Thu, Apr 21, 2022 at 07:08:38PM +0200, Alexander Bluhm wrote: > Hi, > > IGMP and ICMP6 for multicast packets have router alert options. > Per default pf drops all IP packets with options. Usually people > ask what is wrong until someone points out that they have to use a > pf rule with

Re: fix very small ntpd leak

2022-03-24 Thread Otto Moerbeek
On Thu, Mar 24, 2022 at 10:34:52AM +1000, Jonathan Matthew wrote: > On Wed, Mar 23, 2022 at 04:59:06PM +0100, Otto Moerbeek wrote: > > On Wed, Mar 23, 2022 at 09:09:01PM +1000, Jonathan Matthew wrote: > > > > > We noticed that the ntpd engine process was getting a bit

Re: fix very small ntpd leak

2022-03-23 Thread Otto Moerbeek
On Wed, Mar 23, 2022 at 09:09:01PM +1000, Jonathan Matthew wrote: > We noticed that the ntpd engine process was getting a bit big on some boxes > that we'd accidentally cut off from the ntp servers (routing is hard). > Reading through the code, I noticed the 'query' member of struct ntp_peer > is

Re: request for testing: malloc and large allocations

2022-02-25 Thread Otto Moerbeek
On Tue, Feb 01, 2022 at 08:00:36AM +0100, Otto Moerbeek wrote: > On Fri, Jan 28, 2022 at 05:17:48PM +0100, Otto Moerbeek wrote: > > > On Fri, Jan 28, 2022 at 04:33:28PM +0100, Alexander Bluhm wrote: > > > > > On Sun, Jan 09, 2022 at 02:54:43PM +0100, Otto Moerbe

Re: tcpdump core's on the latest snapshot

2022-02-13 Thread Otto Moerbeek
On Sun, Feb 13, 2022 at 12:54:19PM +0100, Otto Moerbeek wrote: > On Sun, Feb 13, 2022 at 01:12:34PM +0300, Mikhail wrote: > > > Running this command on the latest snapshot produces core file for me: > > > > doas tcpdump -i urtwn0 proto ip6 BTW, the proper way to filte

Re: tcpdump core's on the latest snapshot

2022-02-13 Thread Otto Moerbeek
On Sun, Feb 13, 2022 at 01:12:34PM +0300, Mikhail wrote: > Running this command on the latest snapshot produces core file for me: > > doas tcpdump -i urtwn0 proto ip6 > > Core details: > > misha:/home/misha:3959$ doas lldb --core tcpdump.core tcpdump > (lldb) target create "tcpdump" --core

Re: request for testing: malloc and large allocations

2022-02-05 Thread Otto Moerbeek
On Sat, Feb 05, 2022 at 08:07:42PM +0100, Jan Stary wrote: > On Feb 05 17:35:46, o...@drijf.net wrote: > > On Sat, Feb 05, 2022 at 05:22:50PM +0100, Jan Stary wrote: > > > > > On Feb 02 00:04:37, alexander.bl...@gmx.net wrote: > > > > On Tue, Feb 01, 2022 at 0

Re: request for testing: malloc and large allocations

2022-02-05 Thread Otto Moerbeek
On Sat, Feb 05, 2022 at 05:22:50PM +0100, Jan Stary wrote: > On Feb 02 00:04:37, alexander.bl...@gmx.net wrote: > > On Tue, Feb 01, 2022 at 08:00:36AM +0100, Otto Moerbeek wrote: > > > > Are you running with any malloc flags? > > > > > > This bug report ena

Re: request for testing: malloc and large allocations

2022-01-31 Thread Otto Moerbeek
On Fri, Jan 28, 2022 at 05:17:48PM +0100, Otto Moerbeek wrote: > On Fri, Jan 28, 2022 at 04:33:28PM +0100, Alexander Bluhm wrote: > > > On Sun, Jan 09, 2022 at 02:54:43PM +0100, Otto Moerbeek wrote: > > > currently malloc does cache a number of free'ed regions up

Re: UBSan instrumentation vs -fno-wrapv

2022-01-30 Thread Otto Moerbeek
On Sun, Jan 30, 2022 at 04:46:36PM -0800, Greg Steuck wrote: > In case somebody hits this, here's a resolved issue: -fno-wrapv is > matters for UBSan coverage. > > Confusion starts with: > > $ uname -srm; cat a.c && clang -fsanitize=undefined a.c -c -o a.o && nm a.o > OpenBSD 7.0 amd64 > int

Re: request for testing: malloc and large allocations

2022-01-28 Thread Otto Moerbeek
On Fri, Jan 28, 2022 at 04:33:28PM +0100, Alexander Bluhm wrote: > On Sun, Jan 09, 2022 at 02:54:43PM +0100, Otto Moerbeek wrote: > > currently malloc does cache a number of free'ed regions up to 128k in > > size. This cache is indexed by size (in # of pages), so it is very >

Re: request for testing: malloc and large allocations

2022-01-25 Thread Otto Moerbeek
On Sat, Jan 22, 2022 at 09:25:25AM +0100, Otto Moerbeek wrote: > On Mon, Jan 17, 2022 at 08:42:47AM +0100, Otto Moerbeek wrote: > > > On Sun, Jan 09, 2022 at 02:54:43PM +0100, Otto Moerbeek wrote: > > > > > Hi, > > > > > > currently malloc does

Re: request for testing: malloc and large allocations

2022-01-22 Thread Otto Moerbeek
On Mon, Jan 17, 2022 at 08:42:47AM +0100, Otto Moerbeek wrote: > On Sun, Jan 09, 2022 at 02:54:43PM +0100, Otto Moerbeek wrote: > > > Hi, > > > > currently malloc does cache a number of free'ed regions up to 128k in > > size. This cache is indexed by size

Re: request for testing: malloc and large allocations

2022-01-16 Thread Otto Moerbeek
On Sun, Jan 09, 2022 at 02:54:43PM +0100, Otto Moerbeek wrote: > Hi, > > currently malloc does cache a number of free'ed regions up to 128k in > size. This cache is indexed by size (in # of pages), so it is very > quick to check. > > Some programs allocate and dealloca

request for testing: malloc and large allocations

2022-01-09 Thread Otto Moerbeek
Hi, currently malloc does cache a number of free'ed regions up to 128k in size. This cache is indexed by size (in # of pages), so it is very quick to check. Some programs allocate and deallocate larger allocations in a frantic way. Accodomate those programs by also keeping a cache of regions

Re: cat(1): always use a 64K buffer

2021-12-13 Thread Otto Moerbeek
On Mon, Dec 13, 2021 at 02:52:50AM -0600, Scott Cheloha wrote: > > On Dec 13, 2021, at 01:13, Otto Moerbeek wrote: > > > > On Sun, Dec 12, 2021 at 07:15:51PM -0600, Scott Cheloha wrote: > > > >> cat(1) sizes its I/O buffer according to the st_blksize of the

Re: cat(1): always use a 64K buffer

2021-12-12 Thread Otto Moerbeek
On Sun, Dec 12, 2021 at 07:15:51PM -0600, Scott Cheloha wrote: > cat(1) sizes its I/O buffer according to the st_blksize of the first > file it processes. We don't do this very often in the tree. I'm not > sure if we should trust st_blksize. > > It would be simpler to just choose a value that

Re: asr(3): strip AD flag in responses

2021-11-21 Thread Otto Moerbeek
On Sun, Nov 21, 2021 at 04:51:45PM +0100, Florian Obser wrote: > On 2021-11-20 21:16 +01, Otto Moerbeek wrote: > > On Sat, Nov 20, 2021 at 06:44:58PM +0100, Florian Obser wrote: > > > >> On 2021-11-20 18:41 +01, Florian Obser wrote: > >> > On 2021-1

Re: asr(3): strip AD flag in responses

2021-11-20 Thread Otto Moerbeek
On Sat, Nov 20, 2021 at 06:44:58PM +0100, Florian Obser wrote: > On 2021-11-20 18:41 +01, Florian Obser wrote: > > On 2021-11-20 18:19 +01, Florian Obser wrote: > > > >> +/* > >> + * Clear AD flag in the answer. > >> + */ > >> +static void > >> +clear_ad(struct asr_result *ar) > >> +{ > >> +

Re: asr(3): strip AD flag in responses

2021-11-20 Thread Otto Moerbeek
On Sat, Nov 20, 2021 at 02:40:59PM +0100, Otto Moerbeek wrote: > On Sat, Nov 20, 2021 at 12:20:32PM +0100, Florian Obser wrote: > > > The Authentic Data (AD) flag indicates that the nameserver validated > > the response using DNSSEC. For clients to trust this the nameserve

Re: asr(3): strip AD flag in responses

2021-11-20 Thread Otto Moerbeek
On Sat, Nov 20, 2021 at 12:20:32PM +0100, Florian Obser wrote: > The Authentic Data (AD) flag indicates that the nameserver validated > the response using DNSSEC. For clients to trust this the nameserver > and the path to the nameserver must be trusted. In the general case > this is not true. >

Re: [PATCH] Change maximum size of /usr/src to 3G for autoinstall

2021-11-07 Thread Otto Moerbeek
On Sun, Nov 07, 2021 at 07:44:57PM +0300, Mikhail wrote: > On Sat, Oct 30, 2021 at 11:39:54AM +0300, Mikhail wrote: > > On Sun, Oct 24, 2021 at 02:17:25PM +0300, Mikhail wrote: > > > On Sun, Oct 24, 2021 at 11:32:26AM +0100, Stuart Henderson wrote: > > > > The minimum needs to go up too, a cvs

Re: Missing semicolon in snmpd/parse.y

2021-10-20 Thread Otto Moerbeek
On Wed, Oct 20, 2021 at 01:58:03PM +0200, Gerhard Roth wrote: > Hi, > > the rule for 'listen_udptcp' is missing a semicolon at its end. > > I have no idea what yacc does to the following 'port' rule without > that semicolon. Looks like the generated c code is the same; ok otto@ -Otto

Re: Unwind + NSD usage question

2021-09-28 Thread Otto Moerbeek
On Mon, Sep 27, 2021 at 08:50:06PM -0400, abyx...@mnetic.ch wrote: > Hello, trying to set up unwind with nsd on the same machine serving a > internal domain (home.arpa) with all my machines being part of that domain, > eg router.home.arpa. If I point dig at my nsd instance (dig @127.0.0.1 -p >

Re: libedit: stop ignoring SIGINT

2021-08-09 Thread Otto Moerbeek
On Mon, Aug 09, 2021 at 07:20:31AM -0600, Theo de Raadt wrote: > Ingo Schwarze wrote: > > > as mentioned earlier, deraadt@ reported that sftp(1) ignores Ctrl-C. > > Fixing that without longjmp(3) requires making editline(3) better > > behaved. > > If a library interface encourages use

Re: [CAN IGNORE] Proposal for new bc(1) and dc(1)

2021-07-31 Thread Otto Moerbeek
On Fri, Jul 30, 2021 at 09:54:27AM -0600, Gavin Howard wrote: > Whoops; I thought Theo would make the decision, and his last email made > me think he might have. > > I am happy to help as much as I can to make the process easy for you. > > In the meantime, I think I will release 5.0.0 when it's

Re: [CAN IGNORE] Proposal for new bc(1) and dc(1)

2021-07-30 Thread Otto Moerbeek
On Thu, Jul 29, 2021 at 10:31:34PM -0600, Gavin Howard wrote: > Hello, > > At this point, because of the lack of reply, I am going to assume that > my proposal is rejected. While I am sad, I understand. > > Thank you for taking the time to consider my proposal. > > Gavin Howard > I just did

Re: [CAN IGNORE] Proposal for new bc(1) and dc(1)

2021-06-17 Thread Otto Moerbeek
On Thu, Jun 17, 2021 at 10:01:02AM -0600, Gavin Howard wrote: > Otto, > > > I think it is interesting. As for the incompatibilites, my memory says > > I followed the original dc and bc when deciding on those (GNU chose to > > differs in these cases). Bit it has been 18 years since I wrote the >

  1   2   3   4   5   6   7   8   9   >