Re: [PATCH] /proc/kmalloc
On Mon, 21 Feb 2005 02:43:44 +, Baruch Even <[EMAIL PROTECTED]> wrote: > One thing I've seen once that might be worth adding is the ability to > mark generations and then ask "what allocations exist from generation x?". In less general terms, I would like to see which module made the allocation so after doing rmmod I could see if the module leaks memory. Many subsystems are already doing this by intercepting kmalloc() and kfree() calls so it would be nice to get rid of the duplication... Pekka - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] /proc/kmalloc
On Mon, 21 Feb 2005 02:43:44 +, Baruch Even [EMAIL PROTECTED] wrote: One thing I've seen once that might be worth adding is the ability to mark generations and then ask what allocations exist from generation x?. In less general terms, I would like to see which module made the allocation so after doing rmmod I could see if the module leaks memory. Many subsystems are already doing this by intercepting kmalloc() and kfree() calls so it would be nice to get rid of the duplication... Pekka - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] /proc/kmalloc
Matt Mackall wrote: I've been sitting on this for over a year now, kicking it out in the hopes that someone finds it useful. kernel.org was down when I was tidying this up so it's against 2.6.10 which is what I had handy. /proc/kmalloc allocation tracing This quick hack adds accounting for kmalloc/kfree callers. This can aid in tracking down memory leaks and large dynamic memory users. The stock version use ~280k of memory for hash tables and can track 32k active allocations. One thing I've seen once that might be worth adding is the ability to mark generations and then ask "what allocations exist from generation x?". So you do something like: echo 5 > /proc/kmalloc_generation run some tests echo 6 > /proc/kmalloc_generation Print all allocations from generation 5: echo 5 > /proc/kmalloc_print_generations Now you get all buffers that were allocated in generation 5 and not released. Not all of these are leaks, but it's easier to wade through this list to see what is and what isn't a leak. Sometimes it's better to summarize all allocations according to the caller who asked for the allocation, it makes it easier to see if there is an undue increase from certain callers. Just some ideas. Baruch - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] /proc/kmalloc
On Sünndag 20 Februar 2005 21:47, Matt Mackall wrote:
> I've been sitting on this for over a year now, kicking it out in the
> hopes that someone finds it useful. kernel.org was down when I was
> tidying this up so it's against 2.6.10 which is what I had handy.
>
> /proc/kmalloc allocation tracing
Nice. I have done something similar for the buddy allocator but never
got around to sending it.
> This quick hack adds accounting for kmalloc/kfree callers. This can
> aid in tracking down memory leaks and large dynamic memory users. The
> stock version use ~280k of memory for hash tables and can track 32k
> active allocations.
>
> Here's some sample output from my laptop:
>
> total bytes allocated: 47118848
> slack bytes allocated: 8717262
> net bytes allocated:2825920
> number of allocs:132796
> number of frees: 122629
> number of callers: 325
> lost callers: 0
> lost allocs: 0
> unknown frees:0
>
>totalslack net alloc/free caller
>2457600 3/3 copy_thread+0x1ad
The format is not really easy to parse, it probably makes sense to
split the two parts into separate files. I also think that debugfs
would be a more appropriate place to put this in than procfs.
> +void __kmalloc_account(const void *caller, const void *addr, int size, int
> req)
> +{
> + int i, hasha, hashc;
> + unsigned long flags;
> +
> + spin_lock_irqsave(_lock, flags);
> + if(req >= 0) /* kmalloc */
> + {
> + /* find callers slot */
> + hashc = kma_hash(caller, MAX_CALLER_TABLE);
> + for (i = 0; i < MAX_CALLER_TABLE; i++) {
> + if (!kma_caller[hashc].caller ||
> + kma_caller[hashc].caller == caller)
> + break;
> + hashc = (hashc + 1) % MAX_CALLER_TABLE;
> + }
The housekeeping that is needed for the hash implementation is rather
complicated. The code that I wrote did a static allocation from inside
a macro, like
#define kmalloc(_size, _gfp) \
({ \
static struct kma_caller _caller \
__attribute__((section(".kmalloc.data"))) = { \
.func = __FUNCTION__, \
.line = __LINE__, \
}; \
_caller.count++; \
_caller.size += (_size); \
__kmalloc((_size), (_gfp)); \
})
Then I could simply print out all allocations by walking through the
special linker section. OTOH, your implementation has the advantage
that it can directly match kmalloc/kfree pairs and that it does not
rely on special linker magic.
Arnd <><
pgp01ROXTHZ22.pgp
Description: signature
[PATCH] /proc/kmalloc
I've been sitting on this for over a year now, kicking it out in the hopes that someone finds it useful. kernel.org was down when I was tidying this up so it's against 2.6.10 which is what I had handy. /proc/kmalloc allocation tracing This quick hack adds accounting for kmalloc/kfree callers. This can aid in tracking down memory leaks and large dynamic memory users. The stock version use ~280k of memory for hash tables and can track 32k active allocations. Here's some sample output from my laptop: total bytes allocated: 47118848 slack bytes allocated: 8717262 net bytes allocated:2825920 number of allocs:132796 number of frees: 122629 number of callers: 325 lost callers: 0 lost allocs: 0 unknown frees:0 totalslack net alloc/free caller 2457600 3/3 copy_thread+0x1ad 192 56 192 1/0 fbcon_startup+0xc9 32768032768 1/0 fbcon_startup+0x267 18720 9360 32 585/584 bit_cursor+0x1a1 81920 8192 1/0 sys_ioperm+0x6c 81920 8192 1/0 register_framebuffer+0x10b 5120 512 1/0 fb_alloc_cmap+0x42 5120 512 1/0 fb_alloc_cmap+0x55 8192 3192 8192 1/0 framebuffer_alloc+0x36 5120 512 1/0 fb_alloc_cmap+0x68 640 64 1/0 fb_add_videomode+0x52 10334976 45215520 80742/80742 soft_cursor+0x67 320 32 1/0 mtrr_file_add+0x8c 128 36 128 1/0 create_driver+0x20 168427520 212992 4112/4060 dup_task_struct+0x3d 15360012/12radeon_do_probe_i2c_edid+0x5f 672 336 67221/0 acpi_os_create_semaphore+0x17 32 32 32 1/0 acpi_os_create_lock+0xd 448 280 7/7 acpi_os_queue_for_execution+0x2a 2016 252 118463/26__request_region+0x20 768 480 76824/0 register_sysctl_table+0x24 328 32 1/0 radeon_agp_alloc+0x6b 128 52 128 1/0 radeon_agp_init+0x3f 224 56 224 7/0 radeon_addmap+0x2b 224 140 224 7/0 radeon_addmap+0x11f 4096 1792 4096 1/0 radeon_addbufs_agp+0x197 1024 896 102432/0 radeon_addbufs_agp+0x273 2880 720 172815/6 groups_alloc+0x35 32 280 1/1 radeon_ctxbitmap_next+0xfd 40960 4096 1/0 radeon_ctxbitmap_init+0x26 2048 732 2048 1/0 radeon_dma_setup+0x1a 32 16 32 1/0 radeon_addctx+0x77 7680 768 6/0 __create_workqueue+0x3b 32 20 32 1/0 radeon_setup+0x97 32 16 32 1/0 radeon_setup+0xd7 64 24 64 2/0 inter_module_register+0x1f 128 320 1/1 sock_kmalloc+0x3d 64 16 64 1/0 radeon_open_helper+0x51 5347584 2076992 1048576 3520/3264 alloc_skb+0x34 4160 1392 416028/0 param_sysfs_setup+0x4c 514304 2147840 1008/1008 pskb_expand_head+0x4c 32 22 32 1/0 radeon_setunique+0x73 32 15 32 1/0 radeon_setunique+0xca 160 24 160 2/0 radeon_realloc+0x21 576 360 19218/12radeon_vm_open+0x32 1920 192 1/0 radeon_stub_getminor+0xa4 19200 51600 112/112 acpi_ut_initialize_buffer+0x24 269344 111231 195008 5588/1890 acpi_ut_callocate+0x37 24416174610 761/761 acpi_ut_allocate+0x38 512 248 512 1/0 radeon_do_init_cp+0x2b 320 266 32010/0 net_sysctl_strdup+0x2a 64 40 64 2/0 use_module+0x8c ... Index: km/init/Kconfig === --- km.orig/init/Kconfig2004-12-24 13:35:24.0 -0800 +++ km/init/Kconfig 2005-02-20 10:51:49.0 -0800 @@ -287,6 +287,13 @@ config KALLSYMS_EXTRA_PASS reported. KALLSYMS_EXTRA_PASS is only a temporary workaround while you wait for kallsyms to be fixed. +config KMALLOC_ACCOUNTING + default n + bool "Enabled accounting of kmalloc/kfree allocations" if EMBEDDED + help + This option records kmalloc and kfree activity and reports it via + /proc/kmalloc. + config FUTEX bool "Enable futex support" if EMBEDDED default y Index: km/mm/slab.c === --- km.orig/mm/slab.c 2004-12-24 13:35:59.0 -0800 +++ km/mm/slab.c2005-02-20 10:50:15.0 -0800 @@ -2427,6 +2427,7 @@
[PATCH] /proc/kmalloc
I've been sitting on this for over a year now, kicking it out in the hopes that someone finds it useful. kernel.org was down when I was tidying this up so it's against 2.6.10 which is what I had handy. /proc/kmalloc allocation tracing This quick hack adds accounting for kmalloc/kfree callers. This can aid in tracking down memory leaks and large dynamic memory users. The stock version use ~280k of memory for hash tables and can track 32k active allocations. Here's some sample output from my laptop: total bytes allocated: 47118848 slack bytes allocated: 8717262 net bytes allocated:2825920 number of allocs:132796 number of frees: 122629 number of callers: 325 lost callers: 0 lost allocs: 0 unknown frees:0 totalslack net alloc/free caller 2457600 3/3 copy_thread+0x1ad 192 56 192 1/0 fbcon_startup+0xc9 32768032768 1/0 fbcon_startup+0x267 18720 9360 32 585/584 bit_cursor+0x1a1 81920 8192 1/0 sys_ioperm+0x6c 81920 8192 1/0 register_framebuffer+0x10b 5120 512 1/0 fb_alloc_cmap+0x42 5120 512 1/0 fb_alloc_cmap+0x55 8192 3192 8192 1/0 framebuffer_alloc+0x36 5120 512 1/0 fb_alloc_cmap+0x68 640 64 1/0 fb_add_videomode+0x52 10334976 45215520 80742/80742 soft_cursor+0x67 320 32 1/0 mtrr_file_add+0x8c 128 36 128 1/0 create_driver+0x20 168427520 212992 4112/4060 dup_task_struct+0x3d 15360012/12radeon_do_probe_i2c_edid+0x5f 672 336 67221/0 acpi_os_create_semaphore+0x17 32 32 32 1/0 acpi_os_create_lock+0xd 448 280 7/7 acpi_os_queue_for_execution+0x2a 2016 252 118463/26__request_region+0x20 768 480 76824/0 register_sysctl_table+0x24 328 32 1/0 radeon_agp_alloc+0x6b 128 52 128 1/0 radeon_agp_init+0x3f 224 56 224 7/0 radeon_addmap+0x2b 224 140 224 7/0 radeon_addmap+0x11f 4096 1792 4096 1/0 radeon_addbufs_agp+0x197 1024 896 102432/0 radeon_addbufs_agp+0x273 2880 720 172815/6 groups_alloc+0x35 32 280 1/1 radeon_ctxbitmap_next+0xfd 40960 4096 1/0 radeon_ctxbitmap_init+0x26 2048 732 2048 1/0 radeon_dma_setup+0x1a 32 16 32 1/0 radeon_addctx+0x77 7680 768 6/0 __create_workqueue+0x3b 32 20 32 1/0 radeon_setup+0x97 32 16 32 1/0 radeon_setup+0xd7 64 24 64 2/0 inter_module_register+0x1f 128 320 1/1 sock_kmalloc+0x3d 64 16 64 1/0 radeon_open_helper+0x51 5347584 2076992 1048576 3520/3264 alloc_skb+0x34 4160 1392 416028/0 param_sysfs_setup+0x4c 514304 2147840 1008/1008 pskb_expand_head+0x4c 32 22 32 1/0 radeon_setunique+0x73 32 15 32 1/0 radeon_setunique+0xca 160 24 160 2/0 radeon_realloc+0x21 576 360 19218/12radeon_vm_open+0x32 1920 192 1/0 radeon_stub_getminor+0xa4 19200 51600 112/112 acpi_ut_initialize_buffer+0x24 269344 111231 195008 5588/1890 acpi_ut_callocate+0x37 24416174610 761/761 acpi_ut_allocate+0x38 512 248 512 1/0 radeon_do_init_cp+0x2b 320 266 32010/0 net_sysctl_strdup+0x2a 64 40 64 2/0 use_module+0x8c ... Index: km/init/Kconfig === --- km.orig/init/Kconfig2004-12-24 13:35:24.0 -0800 +++ km/init/Kconfig 2005-02-20 10:51:49.0 -0800 @@ -287,6 +287,13 @@ config KALLSYMS_EXTRA_PASS reported. KALLSYMS_EXTRA_PASS is only a temporary workaround while you wait for kallsyms to be fixed. +config KMALLOC_ACCOUNTING + default n + bool Enabled accounting of kmalloc/kfree allocations if EMBEDDED + help + This option records kmalloc and kfree activity and reports it via + /proc/kmalloc. + config FUTEX bool Enable futex support if EMBEDDED default y Index: km/mm/slab.c === --- km.orig/mm/slab.c 2004-12-24 13:35:59.0 -0800 +++ km/mm/slab.c2005-02-20 10:50:15.0 -0800 @@ -2427,6 +2427,7 @@ EXPORT_SYMBOL(kmem_cache_alloc_node);
Re: [PATCH] /proc/kmalloc
On Sünndag 20 Februar 2005 21:47, Matt Mackall wrote:
I've been sitting on this for over a year now, kicking it out in the
hopes that someone finds it useful. kernel.org was down when I was
tidying this up so it's against 2.6.10 which is what I had handy.
/proc/kmalloc allocation tracing
Nice. I have done something similar for the buddy allocator but never
got around to sending it.
This quick hack adds accounting for kmalloc/kfree callers. This can
aid in tracking down memory leaks and large dynamic memory users. The
stock version use ~280k of memory for hash tables and can track 32k
active allocations.
Here's some sample output from my laptop:
total bytes allocated: 47118848
slack bytes allocated: 8717262
net bytes allocated:2825920
number of allocs:132796
number of frees: 122629
number of callers: 325
lost callers: 0
lost allocs: 0
unknown frees:0
totalslack net alloc/free caller
2457600 3/3 copy_thread+0x1ad
The format is not really easy to parse, it probably makes sense to
split the two parts into separate files. I also think that debugfs
would be a more appropriate place to put this in than procfs.
+void __kmalloc_account(const void *caller, const void *addr, int size, int
req)
+{
+ int i, hasha, hashc;
+ unsigned long flags;
+
+ spin_lock_irqsave(kma_lock, flags);
+ if(req = 0) /* kmalloc */
+ {
+ /* find callers slot */
+ hashc = kma_hash(caller, MAX_CALLER_TABLE);
+ for (i = 0; i MAX_CALLER_TABLE; i++) {
+ if (!kma_caller[hashc].caller ||
+ kma_caller[hashc].caller == caller)
+ break;
+ hashc = (hashc + 1) % MAX_CALLER_TABLE;
+ }
The housekeeping that is needed for the hash implementation is rather
complicated. The code that I wrote did a static allocation from inside
a macro, like
#define kmalloc(_size, _gfp) \
({ \
static struct kma_caller _caller \
__attribute__((section(.kmalloc.data))) = { \
.func = __FUNCTION__, \
.line = __LINE__, \
}; \
_caller.count++; \
_caller.size += (_size); \
__kmalloc((_size), (_gfp)); \
})
Then I could simply print out all allocations by walking through the
special linker section. OTOH, your implementation has the advantage
that it can directly match kmalloc/kfree pairs and that it does not
rely on special linker magic.
Arnd
pgp01ROXTHZ22.pgp
Description: signature
Re: [PATCH] /proc/kmalloc
Matt Mackall wrote: I've been sitting on this for over a year now, kicking it out in the hopes that someone finds it useful. kernel.org was down when I was tidying this up so it's against 2.6.10 which is what I had handy. /proc/kmalloc allocation tracing This quick hack adds accounting for kmalloc/kfree callers. This can aid in tracking down memory leaks and large dynamic memory users. The stock version use ~280k of memory for hash tables and can track 32k active allocations. One thing I've seen once that might be worth adding is the ability to mark generations and then ask what allocations exist from generation x?. So you do something like: echo 5 /proc/kmalloc_generation run some tests echo 6 /proc/kmalloc_generation Print all allocations from generation 5: echo 5 /proc/kmalloc_print_generations Now you get all buffers that were allocated in generation 5 and not released. Not all of these are leaks, but it's easier to wade through this list to see what is and what isn't a leak. Sometimes it's better to summarize all allocations according to the caller who asked for the allocation, it makes it easier to see if there is an undue increase from certain callers. Just some ideas. Baruch - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
