Re: shrinking and growing reallocs: a theoretical? bad case for performance

2020-09-02 Thread Otto Moerbeek
On Tue, Sep 01, 2020 at 11:56:36AM +0100, Stuart Henderson wrote: > On 2020/08/31 08:39, Otto Moerbeek wrote: > > A question from Theo made me think about realloc and come up with a > > particular bad case for performance. I do not know if it happens in > > practice, but it was easy to create a

Re: shrinking and growing reallocs: a theoretical? bad case for performance

2020-09-01 Thread Stuart Henderson
On 2020/08/31 08:39, Otto Moerbeek wrote: > A question from Theo made me think about realloc and come up with a > particular bad case for performance. I do not know if it happens in > practice, but it was easy to create a test program to hit the case. Not very scientific testing (a single attempt

Re: shrinking and growing reallocs: a theoretical? bad case for performance

2020-08-31 Thread Otto Moerbeek
On Mon, Aug 31, 2020 at 11:25:51AM -0600, Theo de Raadt wrote: > > Taking advantage of the sparse address space is smart and as 64-bit > > is now the norm, that space is even sparser. > > Fundamentally this is moving various forms of pressure to the kernel, > which does not do the best job yet.

Re: shrinking and growing reallocs: a theoretical? bad case for performance

2020-08-31 Thread Theo de Raadt
> Taking advantage of the sparse address space is smart and as 64-bit > is now the norm, that space is even sparser. Fundamentally this is moving various forms of pressure to the kernel, which does not do the best job yet. The pivot code in mmap for new mappings isn't entirely bug-free so we've

Re: shrinking and growing reallocs: a theoretical? bad case for performance

2020-08-31 Thread Todd C . Miller
On Mon, 31 Aug 2020 08:39:56 +0200, Otto Moerbeek wrote: > 1. I do not put high pages of shrinking reallocs into to cache, but > directly unmap. > > 2. For small shrinking reallocs realloc become a no-op. Pro: no > syscalls at all, cons: the actual allocation is larger, so less > overflow

Re: shrinking and growing reallocs: a theoretical? bad case for performance

2020-08-31 Thread Theo de Raadt
> If I am interpreting this correctly, realloc could be used to groom/shape > the heap such that future allocations are less random and more predictable? Traditionally the word "heap" refers to the zone of allocations in a sbrk allocator, meaning things are packed tightly together in a known

Re: shrinking and growing reallocs: a theoretical? bad case for performance

2020-08-31 Thread Otto Moerbeek
On Mon, Aug 31, 2020 at 08:28:25AM -0400, David Higgs wrote: > On Mon, Aug 31, 2020 at 2:41 AM Otto Moerbeek wrote: > > > Hi, > > > > A question from Theo made me think about realloc and come up with a > > particular bad case for performance. I do not know if it happens in > > practice, but it

Re: shrinking and growing reallocs: a theoretical? bad case for performance

2020-08-31 Thread David Higgs
On Mon, Aug 31, 2020 at 2:41 AM Otto Moerbeek wrote: > Hi, > > A question from Theo made me think about realloc and come up with a > particular bad case for performance. I do not know if it happens in > practice, but it was easy to create a test program to hit the case. > > We're talking

shrinking and growing reallocs: a theoretical? bad case for performance

2020-08-31 Thread Otto Moerbeek
Hi, A question from Theo made me think about realloc and come up with a particular bad case for performance. I do not know if it happens in practice, but it was easy to create a test program to hit the case. We're talking allocation >= a page here. Smaller allocation follow different rules. If