Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-05 Thread jb
jb gmail.com> writes: > ... > So far, the options could be as follows: > - realloc_flags(p, s, option) > where option is one or a combination (where appropriate) of: > REALLOCF_ANY- default (move or no-move; regular > realloc()) Actually, the

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-04 Thread John Baldwin
On Friday, November 29, 2013 6:16:01 am David Chisnall wrote: > > On 28 Nov 2013, at 15:13, jb wrote: > > > Luigi Rizzo iet.unipi.it> writes: > > > >> ... > >> But I don't understand why you find ksize()/malloc_usable_size() dangerous. > >> ... > > > > The original crime is commited when *us

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-02 Thread jb
Luigi Rizzo iet.unipi.it> writes: > ... > > So far, the options could be as follows: > > - realloc_flags(p, s, option) > > where option is one or a combination (where appropriate) of: > > REALLOCF_ANY- default (move or no-move; regular > > realloc()) > > REALLOCF_NO_MOVE

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-02 Thread Luigi Rizzo
On Mon, Dec 2, 2013 at 4:36 AM, jb wrote: > gmx.com> writes: > > > > > So new flags could be [1]: > > - realloc_flags(p, s, REALLOCF_NO_MOVE) > > ... > > - realloc_flags(p, s, REALLOCF_NO_MOVE | REALLOCF_ELASTIC) > > ... > > For this, there could be a REALLOCF_FORCE flag > > In case of realloc_

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-02 Thread jb
gmx.com> writes: > > So new flags could be [1]: > - realloc_flags(p, s, REALLOCF_NO_MOVE) > ... > - realloc_flags(p, s, REALLOCF_NO_MOVE | REALLOCF_ELASTIC) > ... > For this, there could be a REALLOCF_FORCE flag In case of realloc_flags() failing the request, to avoid unnecessary followups wi

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-01 Thread jb
Daniel Nebdal gmail.com> writes: > ... > That could alternatively be solved by having an "if I ask for N bytes right > now, how large would the block be" - API that doesn't promise too much. > Call it something like "malloc_suggest_size(size_t minsize) ", and make the > description something like

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-01 Thread Daniel Nebdal
On Sun, Dec 1, 2013 at 4:04 AM, wrote: > John-Mark Gurney wrote, On 12/01/2013 03:20: > > Either it happens rarely, and always doing a realloc won't hurt >> performance, or it happens often, and then you should be using a larger >> buffer in the first place.. >> > > What if a size-elastic implem

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread dt71
John-Mark Gurney wrote, On 12/01/2013 03:20: Either it happens rarely, and always doing a realloc won't hurt performance, or it happens often, and then you should be using a larger buffer in the first place.. What if a size-elastic implementation of a dynamic data structure would be able to ad

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread John-Mark Gurney
Luigi Rizzo wrote this message on Fri, Nov 29, 2013 at 17:11 -0800: > On Fri, Nov 29, 2013 at 4:49 PM, Adrian Chadd wrote: > > > The reason I wouldn't implement this is to avoid having code that > > _relies_ on this behaviour in order to function or perform well. > > > nobody ever said (or coul

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread jb
gmx.com> writes: > ... > So new flags could be [1]: > - realloc_flags(p, s, REALLOCF_NO_MOVE): Resize object p, without moving > it, to size s. With this restriction, when requesting more memory, and > the specified amount isn't available, don't do anything (when requesting > less memory, alwa

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread Konstantin Belousov
On Fri, Nov 29, 2013 at 08:37:28PM -0800, Tim Kientzle wrote: > > On Nov 29, 2013, at 3:44 PM, jb wrote: > > > Luigi Rizzo iet.unipi.it> writes: > > > >> ... > >> There is a difference between applications peeking into > >> implementation details that should be hidden, and providing > >> inst

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread jb
gmx.com> writes: > ... > It appears that it's not possible to make a proper API with > malloc_usable_size() included, at least when > multi-threading is involved (ie., in the modern world). > > However, it is still useful to create an API that supports the following cases: > ... Well, this i

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Tim Kientzle
On Nov 29, 2013, at 3:44 PM, jb wrote: > Luigi Rizzo iet.unipi.it> writes: > >> ... >> There is a difference between applications peeking into >> implementation details that should be hidden, and providing >> instead limited and specific information through a well defined API. >> ... > > Rig

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread dt71
What is the supposed definition of malloc_usable_size(p) in a hypothetical, upcoming C standard? With the rest of the C standard remaining the same, one could try: Definition: The value of malloc_usable_size(p) is the amount of space allocated for object p, plus the amount of space after objec

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Luigi Rizzo
On Fri, Nov 29, 2013 at 5:02 PM, jb wrote: > Luigi Rizzo iet.unipi.it> writes: > > > ... > > > If you want to improve memory management, that is, have the system > (kernel > > > or user space) handle memory reallocation intelligently and > transparently > > > to the user, then aim at a well defi

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Luigi Rizzo
On Fri, Nov 29, 2013 at 4:49 PM, Adrian Chadd wrote: > The reason I wouldn't implement this is to avoid having code that > _relies_ on this behaviour in order to function or perform well. > nobody ever said (or could reasonably expect to do) that. Applications don't know if the allocator overa

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread jb
Luigi Rizzo iet.unipi.it> writes: > ... > > If you want to improve memory management, that is, have the system (kernel > > or user space) handle memory reallocation intelligently and transparently > > to the user, then aim at a well defined API: > > - reallocate "with no copy", which means new sp

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Adrian Chadd
The reason I wouldn't implement this is to avoid having code that _relies_ on this behaviour in order to function or perform well. Heck, it may not even be portable to other operating systems. Except, Linux, I guess. -adrian ___ freebsd-current@freebsd

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Luigi Rizzo
On Fri, Nov 29, 2013 at 3:44 PM, jb wrote: > Luigi Rizzo iet.unipi.it> writes: > > > ... > > There is a difference between applications peeking into > > implementation details that should be hidden, and providing > > instead limited and specific information through a well defined API. > > ... >

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread jb
Luigi Rizzo iet.unipi.it> writes: > ... > There is a difference between applications peeking into > implementation details that should be hidden, and providing > instead limited and specific information through a well defined API. > ... Right. If you want to improve memory management, that is,

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Luigi Rizzo
On Thu, Nov 28, 2013 at 7:13 AM, jb wrote: > Luigi Rizzo iet.unipi.it> writes: > > > ... > > But I don't understand why you find ksize()/malloc_usable_size() > dangerous. > > ... > > The original crime is commited when *usable size* (an implementation > detail) > is exported (leaked) to the call

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Julian Elischer
On 11/29/13, 7:26 PM, Daniel Nebdal wrote: On Fri, Nov 29, 2013 at 11:59 AM, Gleb Smirnoff wrote: On Thu, Nov 28, 2013 at 03:13:53PM +, jb wrote: j> > But I don't understand why you find ksize()/malloc_usable_size() dangerous. j> > ... j> j> The original crime is commited when *usable size

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Daniel Nebdal
On Fri, Nov 29, 2013 at 11:59 AM, Gleb Smirnoff wrote: > On Thu, Nov 28, 2013 at 03:13:53PM +, jb wrote: > j> > But I don't understand why you find ksize()/malloc_usable_size() > dangerous. > j> > ... > j> > j> The original crime is commited when *usable size* (an implementation > detail) > j

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread David Chisnall
On 28 Nov 2013, at 15:13, jb wrote: > Luigi Rizzo iet.unipi.it> writes: > >> ... >> But I don't understand why you find ksize()/malloc_usable_size() dangerous. >> ... > > The original crime is commited when *usable size* (an implementation detail) > is exported (leaked) to the caller. > To b

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Gleb Smirnoff
On Thu, Nov 28, 2013 at 03:13:53PM +, jb wrote: j> > But I don't understand why you find ksize()/malloc_usable_size() dangerous. j> > ... j> j> The original crime is commited when *usable size* (an implementation detail) j> is exported (leaked) to the caller. j> To be blunt, when a caller requ

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-28 Thread jb
Luigi Rizzo iet.unipi.it> writes: > ... > But I don't understand why you find ksize()/malloc_usable_size() dangerous. > ... The original crime is commited when *usable size* (an implementation detail) is exported (leaked) to the caller. To be blunt, when a caller requests memory of certain size

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-28 Thread Luigi Rizzo
On Thu, Nov 28, 2013 at 01:33:41PM +, jb wrote: > Luigi Rizzo iet.unipi.it> writes: > > > > > in porting some linux kernel code to FreeBSD we > > stumbled upon ksize(), which returns the > > actual size of a kmalloc() block. > > > > We could easily implement it as the first part > > of real

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-28 Thread jb
Luigi Rizzo iet.unipi.it> writes: > > in porting some linux kernel code to FreeBSD we > stumbled upon ksize(), which returns the > actual size of a kmalloc() block. > > We could easily implement it as the first part > of realloc(9) -- see kern/kern_malloc.c > > Would it make sense to add this

[RFC] how to get the size of a malloc(9) block ?

2013-11-28 Thread Luigi Rizzo
in porting some linux kernel code to FreeBSD we stumbled upon ksize(), which returns the actual size of a kmalloc() block. We could easily implement it as the first part of realloc(9) -- see kern/kern_malloc.c Would it make sense to add this to the malloc(9) API ? The userspace equivalent seems t