Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-05 Thread jb
jb jb.1234abcd at gmail.com writes: ... So far, the options could be as follows: - realloc_flags(p, s, option) where option is one or a combination (where appropriate) of: REALLOCF_ANY- default (move or no-move; regular realloc())

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-04 Thread John Baldwin
On Friday, November 29, 2013 6:16:01 am David Chisnall wrote: On 28 Nov 2013, at 15:13, jb jb.1234a...@gmail.com wrote: Luigi Rizzo rizzo at iet.unipi.it writes: ... But I don't understand why you find ksize()/malloc_usable_size() dangerous. ... The original crime is commited

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-02 Thread jb
dt71 at gmx.com writes: So new flags could be [1]: - realloc_flags(p, s, REALLOCF_NO_MOVE) ... - realloc_flags(p, s, REALLOCF_NO_MOVE | REALLOCF_ELASTIC) ... For this, there could be a REALLOCF_FORCE flag In case of realloc_flags() failing the request, to avoid unnecessary followups

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-02 Thread Luigi Rizzo
On Mon, Dec 2, 2013 at 4:36 AM, jb jb.1234a...@gmail.com wrote: dt71 at gmx.com writes: So new flags could be [1]: - realloc_flags(p, s, REALLOCF_NO_MOVE) ... - realloc_flags(p, s, REALLOCF_NO_MOVE | REALLOCF_ELASTIC) ... For this, there could be a REALLOCF_FORCE flag In case

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-02 Thread jb
Luigi Rizzo rizzo at iet.unipi.it writes: ... So far, the options could be as follows: - realloc_flags(p, s, option) where option is one or a combination (where appropriate) of: REALLOCF_ANY- default (move or no-move; regular realloc()) REALLOCF_NO_MOVE

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-01 Thread Daniel Nebdal
On Sun, Dec 1, 2013 at 4:04 AM, d...@gmx.com wrote: John-Mark Gurney wrote, On 12/01/2013 03:20: Either it happens rarely, and always doing a realloc won't hurt performance, or it happens often, and then you should be using a larger buffer in the first place.. What if a size-elastic

Re: [RFC] how to get the size of a malloc(9) block ?

2013-12-01 Thread jb
Daniel Nebdal dnebdal at gmail.com writes: ... That could alternatively be solved by having an if I ask for N bytes right now, how large would the block be - API that doesn't promise too much. Call it something like malloc_suggest_size(size_t minsize) , and make the description something

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread jb
dt71 at gmx.com writes: ... It appears that it's not possible to make a proper API with malloc_usable_size() included, at least when multi-threading is involved (ie., in the modern world). However, it is still useful to create an API that supports the following cases: ... Well, this

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread Konstantin Belousov
On Fri, Nov 29, 2013 at 08:37:28PM -0800, Tim Kientzle wrote: On Nov 29, 2013, at 3:44 PM, jb jb.1234a...@gmail.com wrote: Luigi Rizzo rizzo at iet.unipi.it writes: ... There is a difference between applications peeking into implementation details that should be hidden, and

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread jb
dt71 at gmx.com writes: ... So new flags could be [1]: - realloc_flags(p, s, REALLOCF_NO_MOVE): Resize object p, without moving it, to size s. With this restriction, when requesting more memory, and the specified amount isn't available, don't do anything (when requesting less memory,

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread John-Mark Gurney
Luigi Rizzo wrote this message on Fri, Nov 29, 2013 at 17:11 -0800: On Fri, Nov 29, 2013 at 4:49 PM, Adrian Chadd adr...@freebsd.org wrote: The reason I wouldn't implement this is to avoid having code that _relies_ on this behaviour in order to function or perform well. nobody ever

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-30 Thread dt71
John-Mark Gurney wrote, On 12/01/2013 03:20: Either it happens rarely, and always doing a realloc won't hurt performance, or it happens often, and then you should be using a larger buffer in the first place.. What if a size-elastic implementation of a dynamic data structure would be able to

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Gleb Smirnoff
On Thu, Nov 28, 2013 at 03:13:53PM +, jb wrote: j But I don't understand why you find ksize()/malloc_usable_size() dangerous. j ... j j The original crime is commited when *usable size* (an implementation detail) j is exported (leaked) to the caller. j To be blunt, when a caller requests

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread David Chisnall
On 28 Nov 2013, at 15:13, jb jb.1234a...@gmail.com wrote: Luigi Rizzo rizzo at iet.unipi.it writes: ... But I don't understand why you find ksize()/malloc_usable_size() dangerous. ... The original crime is commited when *usable size* (an implementation detail) is exported (leaked) to

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Daniel Nebdal
On Fri, Nov 29, 2013 at 11:59 AM, Gleb Smirnoff gleb...@freebsd.org wrote: On Thu, Nov 28, 2013 at 03:13:53PM +, jb wrote: j But I don't understand why you find ksize()/malloc_usable_size() dangerous. j ... j j The original crime is commited when *usable size* (an implementation

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Julian Elischer
On 11/29/13, 7:26 PM, Daniel Nebdal wrote: On Fri, Nov 29, 2013 at 11:59 AM, Gleb Smirnoff gleb...@freebsd.org wrote: On Thu, Nov 28, 2013 at 03:13:53PM +, jb wrote: j But I don't understand why you find ksize()/malloc_usable_size() dangerous. j ... j j The original crime is commited

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Luigi Rizzo
On Thu, Nov 28, 2013 at 7:13 AM, jb jb.1234a...@gmail.com wrote: Luigi Rizzo rizzo at iet.unipi.it writes: ... But I don't understand why you find ksize()/malloc_usable_size() dangerous. ... The original crime is commited when *usable size* (an implementation detail) is exported

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread jb
Luigi Rizzo rizzo at iet.unipi.it writes: ... There is a difference between applications peeking into implementation details that should be hidden, and providing instead limited and specific information through a well defined API. ... Right. If you want to improve memory management, that

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Luigi Rizzo
On Fri, Nov 29, 2013 at 3:44 PM, jb jb.1234a...@gmail.com wrote: Luigi Rizzo rizzo at iet.unipi.it writes: ... There is a difference between applications peeking into implementation details that should be hidden, and providing instead limited and specific information through a well

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Adrian Chadd
The reason I wouldn't implement this is to avoid having code that _relies_ on this behaviour in order to function or perform well. Heck, it may not even be portable to other operating systems. Except, Linux, I guess. -adrian ___

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread jb
Luigi Rizzo rizzo at iet.unipi.it writes: ... If you want to improve memory management, that is, have the system (kernel or user space) handle memory reallocation intelligently and transparently to the user, then aim at a well defined API: - reallocate with no copy, which means new space

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Luigi Rizzo
On Fri, Nov 29, 2013 at 4:49 PM, Adrian Chadd adr...@freebsd.org wrote: The reason I wouldn't implement this is to avoid having code that _relies_ on this behaviour in order to function or perform well. nobody ever said (or could reasonably expect to do) that. Applications don't know if the

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Luigi Rizzo
On Fri, Nov 29, 2013 at 5:02 PM, jb jb.1234a...@gmail.com wrote: Luigi Rizzo rizzo at iet.unipi.it writes: ... If you want to improve memory management, that is, have the system (kernel or user space) handle memory reallocation intelligently and transparently to the user, then aim

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread dt71
What is the supposed definition of malloc_usable_size(p) in a hypothetical, upcoming C standard? With the rest of the C standard remaining the same, one could try: Definition: The value of malloc_usable_size(p) is the amount of space allocated for object p, plus the amount of space after

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-29 Thread Tim Kientzle
On Nov 29, 2013, at 3:44 PM, jb jb.1234a...@gmail.com wrote: Luigi Rizzo rizzo at iet.unipi.it writes: ... There is a difference between applications peeking into implementation details that should be hidden, and providing instead limited and specific information through a well defined

[RFC] how to get the size of a malloc(9) block ?

2013-11-28 Thread Luigi Rizzo
in porting some linux kernel code to FreeBSD we stumbled upon ksize(), which returns the actual size of a kmalloc() block. We could easily implement it as the first part of realloc(9) -- see kern/kern_malloc.c Would it make sense to add this to the malloc(9) API ? The userspace equivalent seems

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-28 Thread jb
Luigi Rizzo rizzo at iet.unipi.it writes: in porting some linux kernel code to FreeBSD we stumbled upon ksize(), which returns the actual size of a kmalloc() block. We could easily implement it as the first part of realloc(9) -- see kern/kern_malloc.c Would it make sense to add this

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-28 Thread Luigi Rizzo
On Thu, Nov 28, 2013 at 01:33:41PM +, jb wrote: Luigi Rizzo rizzo at iet.unipi.it writes: in porting some linux kernel code to FreeBSD we stumbled upon ksize(), which returns the actual size of a kmalloc() block. We could easily implement it as the first part of realloc(9)

Re: [RFC] how to get the size of a malloc(9) block ?

2013-11-28 Thread jb
Luigi Rizzo rizzo at iet.unipi.it writes: ... But I don't understand why you find ksize()/malloc_usable_size() dangerous. ... The original crime is commited when *usable size* (an implementation detail) is exported (leaked) to the caller. To be blunt, when a caller requests memory of certain