Re: new module 'aligned-malloc'

2020-07-23 Thread Florian Weimer
* Paul Eggert: > On 7/23/20 5:24 AM, Florian Weimer wrote: >> Ah, 4096 is too small. >> >> For large allocations (above the mmap threshold), what happens in the >> example is that the process VMA limit is eventually exhausted, at which >> malloc falls back to sbrk, and then the algorithm

Re: new module 'aligned-malloc'

2020-07-23 Thread Paul Eggert
On 7/23/20 5:24 AM, Florian Weimer wrote: Ah, 4096 is too small. For large allocations (above the mmap threshold), what happens in the example is that the process VMA limit is eventually exhausted, at which malloc falls back to sbrk, and then the algorithm succeeds. I don't think it's a good

Re: new module 'aligned-malloc'

2020-07-23 Thread Florian Weimer
* Paul Eggert: > On 7/22/20 12:13 AM, Florian Weimer wrote: >> I don't think it will work. Try to get an allocation of 4096 bytes with >> 4096 bytes alignment using glibc malloc this way. > > That's just a mental exercise since glibc malloc already has > aligned_alloc, but I took the challenge

Re: new module 'aligned-malloc'

2020-07-22 Thread Paul Eggert
On 7/22/20 3:55 PM, Bruno Haible wrote: Of course this performance issue is mostly just for MS-Windows, as other major current platforms already have aligned_alloc or rough equivalent. No, it's not only Windows. It's - macOS - Minix - native Windows macOS has had posix_memalign

Re: new module 'aligned-malloc'

2020-07-22 Thread Paul Eggert
On 7/22/20 4:04 PM, Bruno Haible wrote: Probably most posix_memalign / memalign implementations will round up the request to a 16-bytes allocation. But if some implementation can give me just 8 bytes, properly aligned, without wasting the next 8 bytes, why should I not make use of it? I

Re: new module 'aligned-malloc'

2020-07-22 Thread Jeffrey Walton
On Wed, Jul 22, 2020 at 6:56 PM Bruno Haible wrote: > > Hi Paul, > > > Of course this performance issue is mostly just for MS-Windows, as other > > major > > current platforms already have aligned_alloc or rough equivalent. > > No, it's not only Windows. It's > - macOS > - Minix > - native

Re: new module 'aligned-malloc'

2020-07-22 Thread Bruno Haible
Paul Eggert wrote: > What uses of posix_memalign have a SIZE that is not a multiple of ALIGNMENT? My current use-case is to generate machine code in memory (like a just-in-time compiler). For x86, you can easily have a basic block of 7 bytes, which ought to be aligned on a 16-bytes boundary.

Re: new module 'aligned-malloc'

2020-07-22 Thread Bruno Haible
Hi Paul, > Of course this performance issue is mostly just for MS-Windows, as other > major > current platforms already have aligned_alloc or rough equivalent. No, it's not only Windows. It's - macOS - Minix - native Windows Here's the complete matrix:

Re: new module 'aligned-malloc'

2020-07-22 Thread Paul Eggert
On 7/21/20 9:21 AM, Bruno Haible wrote: I prefer posix_memalign to aligned_alloc, because aligned_alloc requires additionally that the SIZE is a multiple of the ALIGNMENT. What uses of posix_memalign have a SIZE that is not a multiple of ALIGNMENT? I'm asking partly because every time I've

Re: new module 'aligned-malloc'

2020-07-22 Thread Paul Eggert
On 7/21/20 4:29 PM, Bruno Haible wrote: However, the algorithm above would be _grossly_ inefficient - especially for bigger alignments such as 128 or 512. I don't think performance would be that bad, for many applications at least. I suppose we could measure overall performance if the topic

Re: new module 'aligned-malloc'

2020-07-22 Thread Paul Eggert
On 7/22/20 12:13 AM, Florian Weimer wrote: I don't think it will work. Try to get an allocation of 4096 bytes with 4096 bytes alignment using glibc malloc this way. That's just a mental exercise since glibc malloc already has aligned_alloc, but I took the challenge anyway and found that it's

Re: new module 'aligned-malloc'

2020-07-22 Thread Paul Eggert
On 7/21/20 2:43 PM, Jeffrey Walton wrote: Eventually I hope even Microsoft will figure out how to do aligned allocation, and even if my hope is dashed that's OK, our code will still work. I believe Redmond has _aligned_malloc. Unfortunately it requires that the storage be freed via

Re: new module 'aligned-malloc'

2020-07-22 Thread Florian Weimer
* Paul Eggert: > On 7/21/20 8:51 AM, Florian Weimer wrote: >> The official aligned_alloc produces pointers compatible with free. >> This module cannot do that. > > I don't see why not, at least on platforms of interest to Gnulib. On > systems that provide no native way to do an aligned

Re: new module 'aligned-malloc'

2020-07-21 Thread Bruno Haible
Paul Eggert wrote: > I don't see why not, at least on platforms of interest to Gnulib. On systems > that provide no native way to do an aligned allocation, we merely keep > calling > malloc with suitable arguments until we get a pointer that is suitably > aligned. > We then free all the

Re: new module 'aligned-malloc'

2020-07-21 Thread Jeffrey Walton
On Tue, Jul 21, 2020 at 5:18 PM Paul Eggert wrote: > > On 7/21/20 8:51 AM, Florian Weimer wrote: > > The official aligned_alloc produces pointers compatible with free. > > This module cannot do that. > > I don't see why not, at least on platforms of interest to Gnulib. On systems > that provide

Re: new module 'aligned-malloc'

2020-07-21 Thread Paul Eggert
On 7/21/20 8:51 AM, Florian Weimer wrote: The official aligned_alloc produces pointers compatible with free. This module cannot do that. I don't see why not, at least on platforms of interest to Gnulib. On systems that provide no native way to do an aligned allocation, we merely keep calling

Re: new module 'aligned-malloc'

2020-07-21 Thread Bruno Haible
I wrote: > there is hope that some systems (maybe native Windows?) > may get aligned_alloc sooner than posix_memalign. Oh, I was too naïve. Citing Microsoft [1][2]: "aligned_alloc() will probably never be implemented, as C11 specified it in a way that’s incompatible with our implementation

Re: new module 'aligned-malloc'

2020-07-21 Thread Bruno Haible
Hi Paul, > C11 and C++17 have aligned_alloc, which Emacs uses in preference to > posix_memalign. Just noticed it as well, when looking at the glibc documentation [1]. Patch below. > I suggest preferring aligned_alloc to > posix_memalign since aligned_alloc should be available on more systems

Re: new module 'aligned-malloc'

2020-07-21 Thread Florian Weimer
* Paul Eggert: > Also, how about naming the new module 'aligned-alloc' and having it > implement aligned_alloc? That would make for more-seamless > integration. The official aligned_alloc produces pointers compatible with free. This module cannot do that. Thanks, Florian

Re: new module 'aligned-malloc'

2020-07-21 Thread Paul Eggert
On 7/21/20 1:47 AM, Bruno Haible wrote: The posix_memalign and memalign functions only help on those platforms where they exist. It's not possible to emulate posix_memalign or memalign when they are not present, because when malloc() returned p, we can call free (p) but not free (p+4) or free

Re: new module 'aligned-malloc'

2020-07-21 Thread Bruno Haible
Jeffrey Walton wrote: > > And glibc aligns malloc() > > results to 8 bytes on most 32-bit platforms. > > I believe the runtime is required to return a pointer aligned for the > largest C type on the system. On x86 that's 16 bytes due to the xmm > datatypes. Right. The malloc() alignment on

Re: new module 'aligned-malloc'

2020-07-21 Thread Jeffrey Walton
On Tue, Jul 21, 2020 at 7:24 AM Bruno Haible wrote: > > > Checkout https://forum.kde.org/viewtopic.php?p=66274 for some info on > > detection of posix_memalign. > > It's well-known that you can't use standards symbols like __STDC_VERSION__ > or _XOPEN_SOURCE for hardly anything. We wouldn't have

Re: new module 'aligned-malloc'

2020-07-21 Thread Bruno Haible
Hi Jeffrey, > Checkout https://forum.kde.org/viewtopic.php?p=66274 for some info on > detection of posix_memalign. It's well-known that you can't use standards symbols like __STDC_VERSION__ or _XOPEN_SOURCE for hardly anything. We wouldn't have hundreds of AC_CHECK_FUNC calls in gnulib/m4/ if

Re: new module 'aligned-malloc'

2020-07-21 Thread Jeffrey Walton
On Tue, Jul 21, 2020 at 4:48 AM Bruno Haible wrote: > > In new gnulib code, I need to allocate memory blocks through malloc() but with > a 16-bytes alignment. > > The posix_memalign and memalign functions only help on those platforms where > they exist. It's not possible to emulate posix_memalign

new module 'aligned-malloc'

2020-07-21 Thread Bruno Haible
In new gnulib code, I need to allocate memory blocks through malloc() but with a 16-bytes alignment. The posix_memalign and memalign functions only help on those platforms where they exist. It's not possible to emulate posix_memalign or memalign when they are not present, because when malloc()