[Fwd: + maps-pssproportional-set-size-accounting-in-smaps.patch added to -mm tree]

2007-09-24 Thread Bernardo Innocenti
Cool!  Andrew picked up the patch I liked.

Andres, how about adding it to olpc-2.6 too?  Along with the
latest Memphis patch, it would give use us invaluable stats
for those trying to reduce memory usage.


 Original Message 
Subject: + maps-pssproportional-set-size-accounting-in-smaps.patch added to -mm 
tree
Date: Mon, 24 Sep 2007 13:32:10 -0700
From: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
CC: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED],[EMAIL 
PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED],[EMAIL PROTECTED], 
[EMAIL PROTECTED]


The patch titled
 maps: PSS(proportional set size) accounting in smaps
has been added to the -mm tree.  Its filename is
 maps-pssproportional-set-size-accounting-in-smaps.patch

*** Remember to use Documentation/SubmitChecklist when testing your code ***

See http://www.zip.com.au/~akpm/linux/patches/stuff/added-to-mm.txt to find
out what to do about this

--
Subject: maps: PSS(proportional set size) accounting in smaps
From: Fengguang Wu [EMAIL PROTECTED]

The proportional set size (PSS) of a process is the count of pages it has
in memory, where each page is divided by the number of processes sharing
it.  So if a process has 1000 pages all to itself, and 1000 shared with one
other process, its PSS will be 1500.

   - lwn.net: ELC: How much memory are applications really using?

The PSS proposed by Matt Mackall is a very nice metic for measuring an
process's memory footprint.  So collect and export it via
/proc/pid/smaps.

Matt Mackall's pagemap/kpagemap and John Berthels's exmap can also do the
job.  They are comprehensive tools.  But for PSS, let's do it in the simple
way.

Cc: John Berthels [EMAIL PROTECTED]
Cc: Bernardo Innocenti [EMAIL PROTECTED]
Cc: Padraig Brady [EMAIL PROTECTED]
Cc: Denys Vlasenko [EMAIL PROTECTED]
Cc: Balbir Singh [EMAIL PROTECTED]
Acked-by: Matt Mackall [EMAIL PROTECTED]
Signed-off-by: Fengguang Wu [EMAIL PROTECTED]
Cc: Hugh Dickins [EMAIL PROTECTED]
Signed-off-by: Andrew Morton [EMAIL PROTECTED]
---

 fs/proc/task_mmu.c |   29 -
 1 files changed, 28 insertions(+), 1 deletion(-)

diff -puN fs/proc/task_mmu.c~maps-pssproportional-set-size-accounting-in-smaps 
fs/proc/task_mmu.c
--- a/fs/proc/task_mmu.c~maps-pssproportional-set-size-accounting-in-smaps
+++ a/fs/proc/task_mmu.c
@@ -324,6 +324,27 @@ struct mem_size_stats
unsigned long private_clean;
unsigned long private_dirty;
unsigned long referenced;
+
+   /*
+* Proportional Set Size(PSS): my share of RSS.
+*
+* PSS of a process is the count of pages it has in memory, where each
+* page is divided by the number of processes sharing it.  So if a
+* process has 1000 pages all to itself, and 1000 shared with one other
+* process, its PSS will be 1500.   - Matt Mackall, lwn.net
+*/
+   u64   pss;
+   /*
+* To keep (accumulated) division errors low, we adopt 64bit pss and
+* use some low bits for division errors. So (pss  PSS_DIV_BITS)
+* would be the real byte count.
+*
+* A shift of 12 before division means(assuming 4K page size):
+*  - 1M 3-user-pages add up to 8KB errors;
+*  - supports mapcount up to 2^24, or 16M;
+*  - supports PSS up to 2^52 bytes, or 4PB.
+*/
+#define PSS_DIV_BITS   12
 };
 
 struct smaps_arg
@@ -341,6 +362,7 @@ static int smaps_pte_range(pmd_t *pmd, u
pte_t *pte, ptent;
spinlock_t *ptl;
struct page *page;
+   int mapcount;
 
pte = pte_offset_map_lock(vma-vm_mm, pmd, addr, ptl);
for (; addr != end; pte++, addr += PAGE_SIZE) {
@@ -357,16 +379,19 @@ static int smaps_pte_range(pmd_t *pmd, u
/* Accumulate the size in pages that have been accessed. */
if (pte_young(ptent) || PageReferenced(page))
mss-referenced += PAGE_SIZE;
-   if (page_mapcount(page) = 2) {
+   mapcount = page_mapcount(page);
+   if (mapcount = 2) {
if (pte_dirty(ptent))
mss-shared_dirty += PAGE_SIZE;
else
mss-shared_clean += PAGE_SIZE;
+   mss-pss += (PAGE_SIZE  PSS_DIV_BITS) / mapcount;
} else {
if (pte_dirty(ptent))
mss-private_dirty += PAGE_SIZE;
else
mss-private_clean += PAGE_SIZE;
+   mss-pss += (PAGE_SIZE  PSS_DIV_BITS);
}
}
pte_unmap_unlock(pte - 1, ptl);
@@ -395,6 +420,7 @@ static int show_smap(struct seq_file *m,
seq_printf(m,
   Size:   %8lu kB\n
   Rss:%8lu kB\n
+ 

Re: [Fwd: + maps-pssproportional-set-size-accounting-in-smaps.patch added to -mm tree]

2007-09-24 Thread Andres Salomon
On Mon, 24 Sep 2007 16:51:07 -0400
Bernardo Innocenti [EMAIL PROTECTED] wrote:

 Cool!  Andrew picked up the patch I liked.
 
 Andres, how about adding it to olpc-2.6 too?  Along with the
 latest Memphis patch, it would give use us invaluable stats
 for those trying to reduce memory usage.
 

Is it useful for the sugar folks' memory donut?  I'm certainly not opposed
to including the patch (though I'd want to look over the lkml thread before
committing), if people find it useful.


___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel


Re: [Fwd: + maps-pssproportional-set-size-accounting-in-smaps.patch added to -mm tree]

2007-09-24 Thread Dan Winship
Andres Salomon wrote:
 On Mon, 24 Sep 2007 16:51:07 -0400
 Bernardo Innocenti [EMAIL PROTECTED] wrote:
 
 Cool!  Andrew picked up the patch I liked.

 Andres, how about adding it to olpc-2.6 too?  Along with the
 latest Memphis patch, it would give use us invaluable stats
 for those trying to reduce memory usage.

 
 Is it useful for the sugar folks' memory donut?  I'm certainly not opposed
 to including the patch (though I'd want to look over the lkml thread before
 committing), if people find it useful.

PSS is basically the same measure we're currently calculating (a 
little more accurate in some ways, a little less in others). Having the 
kernel calculate PSS would let us draw the ring faster (which would be 
good), but wouldn't affect the accuracy.

The biggest problem the ring has right now is that it doesn't take into 
account the fact that code pages can be swapped back out to disk to 
create more RAM, and so it needs to count inactive pages differently 
from active pages if we want the free-space wedge in the donut to be an 
accurate estimation of the user's ability to launch another activity. 
AFAIK the PSS kernel patch doesn't help with this.

-- Dan
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel


Re: [Fwd: + maps-pssproportional-set-size-accounting-in-smaps.patch added to -mm tree]

2007-09-24 Thread Bernardo Innocenti
On 09/24/2007 05:14 PM, Dan Winship wrote:

 The biggest problem the ring has right now is that it doesn't take into 
 account the fact that code pages can be swapped back out to disk to 
 create more RAM, and so it needs to count inactive pages differently 
 from active pages if we want the free-space wedge in the donut to be an 
 accurate estimation of the user's ability to launch another activity. 
 AFAIK the PSS kernel patch doesn't help with this.

Considering only active pages would be an unfair estimate of the overall
memory pressure caused by a running activity.  An activity in the
background would seem to use less memory than one in the foreground.
And the usage estimate would change over time without the application
ever allocating more.

-- 
 \___/
 |___|  Bernardo Innocenti - http://www.codewiz.org/
  \___\ One Laptop Per Child - http://www.laptop.org/
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel