Re: [Crash-utility] [PATCH RFC] Add "kmem -r" to display accumulated slab statistics like /proc/slabinfo
Hi Kazu,
Nicely done -- queued for crash-7.2.4:
https://github.com/crash-utility/crash/commit/e541c5ca2766d6d6934f435955d70aa7b3fd52b0
Thanks,
Dave
- Original Message -
> On 8/23/2018 4:31 PM, Dave Anderson wrote:
> >
> >
> > - Original Message -
> >> Nowadays, the "kmem -s" output can become very long vertically too,
> >> due to the memcg kmem caches. It look like the longer a system has
> >> run, the longer it becomes.
> >>
> >> crash> kmem -s | wc -l
> >> 19855
> >>
> >> On the other hand, since /proc/slabinfo accumulates the values of
> >> each slab_root_caches and its children, it's still short relatively.
> >> And I think there are many cases that support folks want to see the
> >> accumulated values like /proc/slabinfo from vmcore, in order to
> >> grasp the overview of slab activity quickly.
> >>
> >> We can use something like the attached script to accumulate them,
> >> but I believe it would be more useful to implement it in crash.
> >>
> >> This patch introduces the "kmem -r" option to imitate /proc/slabinfo,
> >> but it is limited to CONFIG_SLUB for now.
> >
> > And it looks like it's limited to Linux 4.11 and later, correct?
>
> Yes.
> The patch below introduced the slab_root_caches list at Linux 4.11.
>
> commit 510ded33e075c2bd662b1efab0110f4240325fc9
> Author: Tejun Heo
> Date: Wed Feb 22 15:41:24 2017 -0800
>
> slab: implement slab_root_caches list
>
>
> Thanks,
> Kazu
>
> >
> > Thanks,
> > Dave
> >
> >
> >
> >>
> >> I tested this patch with the kmem-s2r.awk script:
> >>
> >> crash> kmem -s | awk -f kmem-s2r.awk > kmem-s2r.txt
> >> crash> kmem -r > kmem-r.txt
> >>
> >> # diff -u kmem-s2r.txt kmem-r.txt
> >>
> >> Supported:
> >> crash> kmem -r
> >> crash> kmem -r list
> >> crash> kmem -r
> >>
> >> Signed-off-by: Kazuhito Hagio
> >> ---
> >> defs.h| 5 ++
> >> help.c| 10 +--
> >> memory.c | 219
> >> --
> >> symbols.c | 9 +++
> >> 4 files changed, 220 insertions(+), 23 deletions(-)
> >>
> >> diff --git a/defs.h b/defs.h
> >> index 6fdb478..8687ff1 100644
> >> --- a/defs.h
> >> +++ b/defs.h
> >> @@ -2032,6 +2032,10 @@ struct offset_table {/* stash
> >> of
> >> commonly-used offsets */
> >>long bpf_prog_aux_user;
> >>long user_struct_uid;
> >>long idr_cur;
> >> + long kmem_cache_memcg_params;
> >> + long memcg_cache_params___root_caches_node;
> >> + long memcg_cache_params_children;
> >> + long memcg_cache_params_children_node;
> >> };
> >>
> >> struct size_table { /* stash of commonly-used sizes */
> >> @@ -2438,6 +2442,7 @@ struct vm_table {/* kernel
> >> VM-related
> >> data */
> >> #define PAGEFLAGS (0x400)
> >> #define SLAB_OVERLOAD_PAGE(0x800)
> >> #define SLAB_CPU_CACHE (0x1000)
> >> +#define SLAB_ROOT_CACHES (0x2000)
> >>
> >> #define IS_FLATMEM() (vt->flags & FLATMEM)
> >> #define IS_DISCONTIGMEM() (vt->flags & DISCONTIGMEM)
> >> diff --git a/help.c b/help.c
> >> index aeeb056..ee8b999 100644
> >> --- a/help.c
> >> +++ b/help.c
> >> @@ -6448,7 +6448,7 @@ char *help_kmem[] = {
> >> "kmem",
> >> "kernel memory",
> >> "[-f|-F|-c|-C|-i|-v|-V|-n|-z|-o|-h] [-p | -m member[,member]]\n"
> >> -" [[-s|-S] [slab] [-I slab[,slab]]] [-g [flags]] [[-P] address]]",
> >> +" [[-s|-S|-r] [slab] [-I slab[,slab]]] [-g [flags]] [[-P]
> >> address]]",
> >> " This command displays information about the use of kernel memory.\n",
> >> "-f displays the contents of the system free memory headers.",
> >> "also verifies that the page count equals nr_free_pages.",
> >> @@ -6490,10 +6490,12 @@ char *help_kmem[] = {
> >> "slab data for each per-cpu slab is displayed, along with
> >> the",
> >> "address of each kmem_cache_node, its count of full and
> >> partial",
> >> "slabs, and a list of all tracked slabs.",
> >> -" slab when used with -s or -S, limits the command to only the slab
> >> cache",
> >> -"of name \"slab\". If the slab argument is \"list\", then",
> >> +"-r displays accumulated kmalloc() slab data of each
> >> slab_root_caches",
> >> +"and its children. Available only if CONFIG_SLUB for now.",
> >> +" slab when used with -s, -S or -r, limits the command to only the
> >> slab",
> >> +"cache of name \"slab\". If the slab argument is \"list\",
> >> then",
> >> "all slab cache names and addresses are listed.",
> >> -" -I slab when used with -s or -S, one or more slab cache names in a",
> >> +" -I slab when used with -s, -S or -r, one or more slab cache names in
> >> a",
> >> "comma-separated list may be specified as slab caches to
> >> ignore.",
> >> "-g displays the enumerator value of all bits in the page
> >> structure's",
> >> "\"flags\"
Re: [Crash-utility] [PATCH RFC] Add "kmem -r" to display accumulated slab statistics like /proc/slabinfo
On 8/23/2018 4:31 PM, Dave Anderson wrote:
>
>
> - Original Message -
>> Nowadays, the "kmem -s" output can become very long vertically too,
>> due to the memcg kmem caches. It look like the longer a system has
>> run, the longer it becomes.
>>
>> crash> kmem -s | wc -l
>> 19855
>>
>> On the other hand, since /proc/slabinfo accumulates the values of
>> each slab_root_caches and its children, it's still short relatively.
>> And I think there are many cases that support folks want to see the
>> accumulated values like /proc/slabinfo from vmcore, in order to
>> grasp the overview of slab activity quickly.
>>
>> We can use something like the attached script to accumulate them,
>> but I believe it would be more useful to implement it in crash.
>>
>> This patch introduces the "kmem -r" option to imitate /proc/slabinfo,
>> but it is limited to CONFIG_SLUB for now.
>
> And it looks like it's limited to Linux 4.11 and later, correct?
Yes.
The patch below introduced the slab_root_caches list at Linux 4.11.
commit 510ded33e075c2bd662b1efab0110f4240325fc9
Author: Tejun Heo
Date: Wed Feb 22 15:41:24 2017 -0800
slab: implement slab_root_caches list
Thanks,
Kazu
>
> Thanks,
> Dave
>
>
>
>>
>> I tested this patch with the kmem-s2r.awk script:
>>
>> crash> kmem -s | awk -f kmem-s2r.awk > kmem-s2r.txt
>> crash> kmem -r > kmem-r.txt
>>
>> # diff -u kmem-s2r.txt kmem-r.txt
>>
>> Supported:
>> crash> kmem -r
>> crash> kmem -r list
>> crash> kmem -r
>>
>> Signed-off-by: Kazuhito Hagio
>> ---
>> defs.h| 5 ++
>> help.c| 10 +--
>> memory.c | 219
>> --
>> symbols.c | 9 +++
>> 4 files changed, 220 insertions(+), 23 deletions(-)
>>
>> diff --git a/defs.h b/defs.h
>> index 6fdb478..8687ff1 100644
>> --- a/defs.h
>> +++ b/defs.h
>> @@ -2032,6 +2032,10 @@ struct offset_table {/* stash of
>> commonly-used offsets */
>> long bpf_prog_aux_user;
>> long user_struct_uid;
>> long idr_cur;
>> +long kmem_cache_memcg_params;
>> +long memcg_cache_params___root_caches_node;
>> +long memcg_cache_params_children;
>> +long memcg_cache_params_children_node;
>> };
>>
>> struct size_table { /* stash of commonly-used sizes */
>> @@ -2438,6 +2442,7 @@ struct vm_table {/* kernel VM-related
>> data */
>> #define PAGEFLAGS (0x400)
>> #define SLAB_OVERLOAD_PAGE(0x800)
>> #define SLAB_CPU_CACHE (0x1000)
>> +#define SLAB_ROOT_CACHES (0x2000)
>>
>> #define IS_FLATMEM()(vt->flags & FLATMEM)
>> #define IS_DISCONTIGMEM() (vt->flags & DISCONTIGMEM)
>> diff --git a/help.c b/help.c
>> index aeeb056..ee8b999 100644
>> --- a/help.c
>> +++ b/help.c
>> @@ -6448,7 +6448,7 @@ char *help_kmem[] = {
>> "kmem",
>> "kernel memory",
>> "[-f|-F|-c|-C|-i|-v|-V|-n|-z|-o|-h] [-p | -m member[,member]]\n"
>> -" [[-s|-S] [slab] [-I slab[,slab]]] [-g [flags]] [[-P] address]]",
>> +" [[-s|-S|-r] [slab] [-I slab[,slab]]] [-g [flags]] [[-P] address]]",
>> " This command displays information about the use of kernel memory.\n",
>> "-f displays the contents of the system free memory headers.",
>> "also verifies that the page count equals nr_free_pages.",
>> @@ -6490,10 +6490,12 @@ char *help_kmem[] = {
>> "slab data for each per-cpu slab is displayed, along with the",
>> "address of each kmem_cache_node, its count of full and
>> partial",
>> "slabs, and a list of all tracked slabs.",
>> -" slab when used with -s or -S, limits the command to only the slab
>> cache",
>> -"of name \"slab\". If the slab argument is \"list\", then",
>> +"-r displays accumulated kmalloc() slab data of each
>> slab_root_caches",
>> +"and its children. Available only if CONFIG_SLUB for now.",
>> +" slab when used with -s, -S or -r, limits the command to only the
>> slab",
>> +"cache of name \"slab\". If the slab argument is \"list\",
>> then",
>> "all slab cache names and addresses are listed.",
>> -" -I slab when used with -s or -S, one or more slab cache names in a",
>> +" -I slab when used with -s, -S or -r, one or more slab cache names in
>> a",
>> "comma-separated list may be specified as slab caches to
>> ignore.",
>> "-g displays the enumerator value of all bits in the page
>> structure's",
>> "\"flags\" field.",
>> diff --git a/memory.c b/memory.c
>> index e02ba68..1501b21 100644
>> --- a/memory.c
>> +++ b/memory.c
>> @@ -167,12 +167,12 @@ static int kmem_cache_downsize(void);
>> static int ignore_cache(struct meminfo *, char *);
>> static char *is_kmem_cache_addr(ulong, char *);
>> static char *is_kmem_cache_addr_common(ulong, char *);
>> -static void kmem_cache_list(void);
>> +static void kmem_cache_list(struct meminfo *);
>> static void
Re: [Crash-utility] [PATCH RFC] Add "kmem -r" to display accumulated slab statistics like /proc/slabinfo
- Original Message -
> Nowadays, the "kmem -s" output can become very long vertically too,
> due to the memcg kmem caches. It look like the longer a system has
> run, the longer it becomes.
>
> crash> kmem -s | wc -l
> 19855
>
> On the other hand, since /proc/slabinfo accumulates the values of
> each slab_root_caches and its children, it's still short relatively.
> And I think there are many cases that support folks want to see the
> accumulated values like /proc/slabinfo from vmcore, in order to
> grasp the overview of slab activity quickly.
>
> We can use something like the attached script to accumulate them,
> but I believe it would be more useful to implement it in crash.
>
> This patch introduces the "kmem -r" option to imitate /proc/slabinfo,
> but it is limited to CONFIG_SLUB for now.
And it looks like it's limited to Linux 4.11 and later, correct?
Thanks,
Dave
>
> I tested this patch with the kmem-s2r.awk script:
>
> crash> kmem -s | awk -f kmem-s2r.awk > kmem-s2r.txt
> crash> kmem -r > kmem-r.txt
>
> # diff -u kmem-s2r.txt kmem-r.txt
>
> Supported:
> crash> kmem -r
> crash> kmem -r list
> crash> kmem -r
>
> Signed-off-by: Kazuhito Hagio
> ---
> defs.h| 5 ++
> help.c| 10 +--
> memory.c | 219
> --
> symbols.c | 9 +++
> 4 files changed, 220 insertions(+), 23 deletions(-)
>
> diff --git a/defs.h b/defs.h
> index 6fdb478..8687ff1 100644
> --- a/defs.h
> +++ b/defs.h
> @@ -2032,6 +2032,10 @@ struct offset_table {/* stash of
> commonly-used offsets */
> long bpf_prog_aux_user;
> long user_struct_uid;
> long idr_cur;
> + long kmem_cache_memcg_params;
> + long memcg_cache_params___root_caches_node;
> + long memcg_cache_params_children;
> + long memcg_cache_params_children_node;
> };
>
> struct size_table { /* stash of commonly-used sizes */
> @@ -2438,6 +2442,7 @@ struct vm_table {/* kernel VM-related
> data */
> #define PAGEFLAGS (0x400)
> #define SLAB_OVERLOAD_PAGE(0x800)
> #define SLAB_CPU_CACHE (0x1000)
> +#define SLAB_ROOT_CACHES (0x2000)
>
> #define IS_FLATMEM() (vt->flags & FLATMEM)
> #define IS_DISCONTIGMEM()(vt->flags & DISCONTIGMEM)
> diff --git a/help.c b/help.c
> index aeeb056..ee8b999 100644
> --- a/help.c
> +++ b/help.c
> @@ -6448,7 +6448,7 @@ char *help_kmem[] = {
> "kmem",
> "kernel memory",
> "[-f|-F|-c|-C|-i|-v|-V|-n|-z|-o|-h] [-p | -m member[,member]]\n"
> -" [[-s|-S] [slab] [-I slab[,slab]]] [-g [flags]] [[-P] address]]",
> +" [[-s|-S|-r] [slab] [-I slab[,slab]]] [-g [flags]] [[-P] address]]",
> " This command displays information about the use of kernel memory.\n",
> "-f displays the contents of the system free memory headers.",
> "also verifies that the page count equals nr_free_pages.",
> @@ -6490,10 +6490,12 @@ char *help_kmem[] = {
> "slab data for each per-cpu slab is displayed, along with the",
> "address of each kmem_cache_node, its count of full and
> partial",
> "slabs, and a list of all tracked slabs.",
> -" slab when used with -s or -S, limits the command to only the slab
> cache",
> -"of name \"slab\". If the slab argument is \"list\", then",
> +"-r displays accumulated kmalloc() slab data of each
> slab_root_caches",
> +"and its children. Available only if CONFIG_SLUB for now.",
> +" slab when used with -s, -S or -r, limits the command to only the
> slab",
> +"cache of name \"slab\". If the slab argument is \"list\",
> then",
> "all slab cache names and addresses are listed.",
> -" -I slab when used with -s or -S, one or more slab cache names in a",
> +" -I slab when used with -s, -S or -r, one or more slab cache names in
> a",
> "comma-separated list may be specified as slab caches to
> ignore.",
> "-g displays the enumerator value of all bits in the page
> structure's",
> "\"flags\" field.",
> diff --git a/memory.c b/memory.c
> index e02ba68..1501b21 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -167,12 +167,12 @@ static int kmem_cache_downsize(void);
> static int ignore_cache(struct meminfo *, char *);
> static char *is_kmem_cache_addr(ulong, char *);
> static char *is_kmem_cache_addr_common(ulong, char *);
> -static void kmem_cache_list(void);
> +static void kmem_cache_list(struct meminfo *);
> static void dump_kmem_cache(struct meminfo *);
> static void dump_kmem_cache_percpu_v1(struct meminfo *);
> static void dump_kmem_cache_percpu_v2(struct meminfo *);
> static void dump_kmem_cache_slub(struct meminfo *);
> -static void kmem_cache_list_common(void);
> +static void kmem_cache_list_common(struct meminfo *);
> static ulong get_cpu_slab_ptr(struct meminfo *, int, ulong *);
> static unsigned int
