The RSS counters use a batch size of 32: each CPU accumulates up to 31 pages in either direction before propagating a carry into the counter tree. A smaller batch improves the accuracy of the approximate RSS, and a larger batch reduces the cost of updates.
Add the rss_stat_batch= boot parameter to compare batch sizes on a given machine, along with the percpu_counter_tree.arity= boot parameter for the tree shape. It accepts a power of 2 between 2 and 128, and is ignored with a warning otherwise. Signed-off-by: Mathieu Desnoyers <[email protected]> --- .../admin-guide/kernel-parameters.txt | 10 ++++++++ kernel/fork.c | 23 ++++++++++++++++--- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index c368affa2115..ba2bdeb1480b 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -6792,6 +6792,16 @@ Kernel parameters debug mode. Defaults to CONFIG_RSEQ_DEBUG_DEFAULT_ENABLE. Format: <bool> + rss_stat_batch= [KNL,SMP] + Format: <integer> + Batch size of the per-mm RSS counters: the number of + pages a CPU accumulates before propagating them into + the counter tree. A power of 2 between 2 and 128. + A smaller batch improves the accuracy of the + approximate RSS, a larger batch reduces the update + cost. Default: 32. + This parameter is meant for performance comparison. + rt_group_sched= [KNL] Enable or disable SCHED_RR/FIFO group scheduling when CONFIG_RT_GROUP_SCHED=y. Defaults to !CONFIG_RT_GROUP_SCHED_DEFAULT_DISABLED. diff --git a/kernel/fork.c b/kernel/fork.c index ac4eb1370e62..d0641700cd9c 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -138,9 +138,26 @@ #define MAX_THREADS FUTEX_TID_MASK /* - * Batch size of rss stat approximation + * Batch size of rss stat approximation, tunable with the rss_stat_batch= + * boot parameter to compare batch sizes on a given machine. */ -#define RSS_STAT_BATCH_SIZE 32 +static unsigned int rss_stat_batch_size __ro_after_init = 32; + +static int __init setup_rss_stat_batch(char *str) +{ + unsigned int batch; + + if (kstrtouint(str, 0, &batch) || batch < 2 || !is_power_of_2(batch) || + batch > PERCPU_COUNTER_TREE_MAX_BATCH) { + pr_warn("rss_stat_batch: expecting a power of 2 between 2 and %u\n", + PERCPU_COUNTER_TREE_MAX_BATCH); + return 1; + } + rss_stat_batch_size = batch; + pr_info("rss_stat_batch: %u\n", batch); + return 1; +} +__setup("rss_stat_batch=", setup_rss_stat_batch); /* * Protected counters by write_lock_irq(&tasklist_lock) @@ -1149,7 +1166,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p) if (mm_alloc_sched(mm)) goto fail_sched; - if (percpu_counter_tree_init_many_lazy(mm->rss_stat, NR_MM_COUNTERS, RSS_STAT_BATCH_SIZE)) + if (percpu_counter_tree_init_many_lazy(mm->rss_stat, NR_MM_COUNTERS, rss_stat_batch_size)) goto fail_pcpu; lru_gen_init_mm(mm); -- 2.47.3
