Re: Proposal: Restore malloc(9) interface
Hi. On 2023/12/31 7:44, Taylor R Campbell wrote: > I propose to deprecate the kmem(9) interface and go back to the > malloc(9) interface. > > > 1. The main difference is that the malloc(9) interface enables >attribution of memory usage: how many bytes have been used for this >purpose vs that purpose, partitioned by named malloc tags like >M_MBUF or M_ACPI. The conversion to the kmem(9) interface lost all >this valuable diagnostic information. > >I've personally spent probably dozens of hours over the last year >or two puzzling over `vmstat -m' output to guess which subsystem >might be leaking memory based on allocation sizes and which >kmem-N pool looks fishy. This is extremely frustrating and a >huge waste of time to recover information we used to gather and >report systematically. I think adding malloc type again is good thing. > 2. A secondary difference is reduced diffs from FreeBSD and OpenBSD >drivers if we use malloc(9). > > 3. A small difference is that kmem(9) distinguishes legacy allocation >from interrupt context, kmem_intr_alloc/free, from front ends that >forbid that, kmem_alloc/free. > >I'm not sure this has provided much valuable diagnostic >information, but it has provided a lot of frustrating crashes. If >we want the same frustrating crashes we could introduce an M_INTR >flag which is mandatory when calling malloc from interrupt context. Current kmen_(intr_)alloc()'s return address is usually aligned with CACHE_LINE_SIZE. (Note that the smallest number is KMEM_ALIGN(== 8) and the max is PAGE_SIZE.) On kern_malloc(), the memory is first allocated with kmem_intr_alloc() and put the malloc_header to the head. As a result, the return address usually aligned with lower than CACHE_LINE_SIZE (at least on amd64). I'm glad if the new malloc(9) API usually returns pointer aligned with CACHE_LINE_SIZE or more. > Note: I am NOT proposing any substantive changes to the implementation > of the allocator -- I'm just proposing that we go back to the old > _interface_, using the new pool-cache-based _implementation_, and to > add lightweight per-CPU, per-tag usage counting to the malloc and free > paths. > > Nor am I suggesting changing anything about uvm_km(9), pool_cache(9), > or anything else -- just changing kmem_alloc(N, KM_[NO]SLEEP) back to > malloc(N, T, M_NOWAIT/WAITOK) and kmem_free(P, N) back to free(P, T), > or possibly free(P, T, N) like OpenBSD does. > > Thoughts? > > > I asked for rationale for the kmem(9) interface last year, and none of > the answers gave any compelling reason to have changed interfaces in > the first place or to finish the conversion now: > > https://mail-index.netbsd.org/tech-kern/2022/10/29/msg028498.html > > As far as I can tell, we just spent umpteen hundreds of hours on > engineering effort over the last decade to convert various drivers and > subsystems from malloc(9) to kmem(9), in exchange for the loss of > valuable diagnostic information about leaks, for increased cost to > porting drivers, and for crashes when old subsystems newly converted > to kmem(9) still allocate from interrupt context. -- --- SAITOH Masanobu (msai...@execsw.org msai...@netbsd.org)
Re: Proposal: Restore malloc(9) interface
On Sat, Dec 30, 2023 at 10:44:52PM +, Taylor R Campbell wrote: > Note: I am NOT proposing any substantive changes to the implementation > of the allocator -- I'm just proposing that we go back to the old > _interface_, using the new pool-cache-based _implementation_, and to > add lightweight per-CPU, per-tag usage counting to the malloc and free > paths. Can we just add tags to kmem(9)? At this point that seems like a path of lesser resistance, and also it avoids having a standard function name with a nonstandard interface. (Also, let's make the tags be typed as pointers instead of an enum) -- David A. Holland dholl...@netbsd.org
Proposal: Restore malloc(9) interface
I propose to deprecate the kmem(9) interface and go back to the malloc(9) interface. 1. The main difference is that the malloc(9) interface enables attribution of memory usage: how many bytes have been used for this purpose vs that purpose, partitioned by named malloc tags like M_MBUF or M_ACPI. The conversion to the kmem(9) interface lost all this valuable diagnostic information. I've personally spent probably dozens of hours over the last year or two puzzling over `vmstat -m' output to guess which subsystem might be leaking memory based on allocation sizes and which kmem-N pool looks fishy. This is extremely frustrating and a huge waste of time to recover information we used to gather and report systematically. 2. A secondary difference is reduced diffs from FreeBSD and OpenBSD drivers if we use malloc(9). 3. A small difference is that kmem(9) distinguishes legacy allocation from interrupt context, kmem_intr_alloc/free, from front ends that forbid that, kmem_alloc/free. I'm not sure this has provided much valuable diagnostic information, but it has provided a lot of frustrating crashes. If we want the same frustrating crashes we could introduce an M_INTR flag which is mandatory when calling malloc from interrupt context. Note: I am NOT proposing any substantive changes to the implementation of the allocator -- I'm just proposing that we go back to the old _interface_, using the new pool-cache-based _implementation_, and to add lightweight per-CPU, per-tag usage counting to the malloc and free paths. Nor am I suggesting changing anything about uvm_km(9), pool_cache(9), or anything else -- just changing kmem_alloc(N, KM_[NO]SLEEP) back to malloc(N, T, M_NOWAIT/WAITOK) and kmem_free(P, N) back to free(P, T), or possibly free(P, T, N) like OpenBSD does. Thoughts? I asked for rationale for the kmem(9) interface last year, and none of the answers gave any compelling reason to have changed interfaces in the first place or to finish the conversion now: https://mail-index.netbsd.org/tech-kern/2022/10/29/msg028498.html As far as I can tell, we just spent umpteen hundreds of hours on engineering effort over the last decade to convert various drivers and subsystems from malloc(9) to kmem(9), in exchange for the loss of valuable diagnostic information about leaks, for increased cost to porting drivers, and for crashes when old subsystems newly converted to kmem(9) still allocate from interrupt context.
