Re: [PATCH] add extra free kbytes tunable

2013-03-08 Thread Simon Jeons
Hi Hugh, On 03/02/2013 11:08 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is

Re: [PATCH] add extra free kbytes tunable

2013-03-08 Thread Simon Jeons
Hi Hugh, On 03/02/2013 11:08 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 11:08 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Sat, 2 Mar 2013, Simon Jeons wrote: > On 03/02/2013 09:42 AM, Hugh Dickins wrote: > > On Sat, 2 Mar 2013, Simon Jeons wrote: > > > In function __add_to_swap_cache if add to radix tree successfully will > > > result > > > in increase NR_FILE_PAGES, why? This is anonymous page instead of file > >

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous page instead of file backed page. Right, that's hard to understand without

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Sat, 2 Mar 2013, Simon Jeons wrote: > > In function __add_to_swap_cache if add to radix tree successfully will result > in increase NR_FILE_PAGES, why? This is anonymous page instead of file backed > page. Right, that's hard to understand without historical background. I think the quick

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 06:33 AM, Hugh Dickins wrote: On Fri, 1 Mar 2013, Simon Jeons wrote: On 03/01/2013 05:22 PM, Simon Jeons wrote: On 02/23/2013 01:56 AM, Johannes Weiner wrote: Mapped file pages have to get scanned twice before they are reclaimed because we don't have enough usage information

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Fri, 1 Mar 2013, Simon Jeons wrote: > On 03/01/2013 05:22 PM, Simon Jeons wrote: > > On 02/23/2013 01:56 AM, Johannes Weiner wrote: > > > Mapped file pages have to get scanned twice before they are reclaimed > > > because we don't have enough usage information after the first scan. > > > > It

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/01/2013 05:22 PM, Simon Jeons wrote: Hi Johannes, On 02/23/2013 01:56 AM, Johannes Weiner wrote: On Tue, Feb 19, 2013 at 09:19:27PM -0800, dormando wrote: The problem is that adding this tunable will constrain future VM implementations. We will forever need to at least retain the

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
Hi Johannes, On 02/23/2013 01:56 AM, Johannes Weiner wrote: On Tue, Feb 19, 2013 at 09:19:27PM -0800, dormando wrote: The problem is that adding this tunable will constrain future VM implementations. We will forever need to at least retain the pseudo-file. We will also need to make some

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
Hi Johannes, On 02/23/2013 01:56 AM, Johannes Weiner wrote: On Tue, Feb 19, 2013 at 09:19:27PM -0800, dormando wrote: The problem is that adding this tunable will constrain future VM implementations. We will forever need to at least retain the pseudo-file. We will also need to make some

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/01/2013 05:22 PM, Simon Jeons wrote: Hi Johannes, On 02/23/2013 01:56 AM, Johannes Weiner wrote: On Tue, Feb 19, 2013 at 09:19:27PM -0800, dormando wrote: The problem is that adding this tunable will constrain future VM implementations. We will forever need to at least retain the

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Fri, 1 Mar 2013, Simon Jeons wrote: On 03/01/2013 05:22 PM, Simon Jeons wrote: On 02/23/2013 01:56 AM, Johannes Weiner wrote: Mapped file pages have to get scanned twice before they are reclaimed because we don't have enough usage information after the first scan. It seems that

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 06:33 AM, Hugh Dickins wrote: On Fri, 1 Mar 2013, Simon Jeons wrote: On 03/01/2013 05:22 PM, Simon Jeons wrote: On 02/23/2013 01:56 AM, Johannes Weiner wrote: Mapped file pages have to get scanned twice before they are reclaimed because we don't have enough usage information

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous page instead of file backed page. Right, that's hard to understand without historical background. I think the quick answer

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous page instead of file backed page. Right, that's hard to understand without

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Sat, 2 Mar 2013, Simon Jeons wrote: On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous page instead of file backed

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 11:08 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous

Re: [PATCH] add extra free kbytes tunable

2013-02-26 Thread Mel Gorman
On Tue, Feb 26, 2013 at 10:13:15AM -0500, Johannes Weiner wrote: > > > > > > I think we should think about capping kswapd zone reclaim cycles just > > > as we do for direct reclaim. It's a little ridiculous that it can run > > > unbounded and reclaim every page in a zone without ever checking

Re: [PATCH] add extra free kbytes tunable

2013-02-26 Thread Johannes Weiner
On Tue, Feb 26, 2013 at 10:47:31AM +, Mel Gorman wrote: > On Fri, Feb 22, 2013 at 12:56:34PM -0500, Johannes Weiner wrote: > > > > > > > > > > > > : We have a server workload wherein machines with 100G+ of "free" memory > > > > : (used by page cache), scattered but frequent random io reads

Re: [PATCH] add extra free kbytes tunable

2013-02-26 Thread Mel Gorman
On Fri, Feb 22, 2013 at 12:56:34PM -0500, Johannes Weiner wrote: > > > > > > > > > : We have a server workload wherein machines with 100G+ of "free" memory > > > : (used by page cache), scattered but frequent random io reads from 12+ > > > : SSD's, and 5gbps+ of internet traffic, will frequently

Re: [PATCH] add extra free kbytes tunable

2013-02-26 Thread Mel Gorman
On Fri, Feb 22, 2013 at 12:56:34PM -0500, Johannes Weiner wrote: SNIP : We have a server workload wherein machines with 100G+ of free memory : (used by page cache), scattered but frequent random io reads from 12+ : SSD's, and 5gbps+ of internet traffic, will frequently hit direct

Re: [PATCH] add extra free kbytes tunable

2013-02-26 Thread Johannes Weiner
On Tue, Feb 26, 2013 at 10:47:31AM +, Mel Gorman wrote: On Fri, Feb 22, 2013 at 12:56:34PM -0500, Johannes Weiner wrote: SNIP : We have a server workload wherein machines with 100G+ of free memory : (used by page cache), scattered but frequent random io reads from 12+ :

Re: [PATCH] add extra free kbytes tunable

2013-02-26 Thread Mel Gorman
On Tue, Feb 26, 2013 at 10:13:15AM -0500, Johannes Weiner wrote: SNIP I think we should think about capping kswapd zone reclaim cycles just as we do for direct reclaim. It's a little ridiculous that it can run unbounded and reclaim every page in a zone without ever checking back

Re: [PATCH] add extra free kbytes tunable

2013-02-22 Thread Johannes Weiner
On Tue, Feb 19, 2013 at 09:19:27PM -0800, dormando wrote: > > > > The problem is that adding this tunable will constrain future VM > > implementations. We will forever need to at least retain the > > pseudo-file. We will also need to make some effort to retain its > > behaviour. > > > > It would

Re: [PATCH] add extra free kbytes tunable

2013-02-22 Thread Johannes Weiner
On Tue, Feb 19, 2013 at 09:19:27PM -0800, dormando wrote: The problem is that adding this tunable will constrain future VM implementations. We will forever need to at least retain the pseudo-file. We will also need to make some effort to retain its behaviour. It would of course be

Re: [PATCH] add extra free kbytes tunable

2013-02-19 Thread dormando
> > The problem is that adding this tunable will constrain future VM > implementations. We will forever need to at least retain the > pseudo-file. We will also need to make some effort to retain its > behaviour. > > It would of course be better to fix things so you don't need to tweak > VM

Re: [PATCH] add extra free kbytes tunable

2013-02-19 Thread Andrew Morton
On Sun, 17 Feb 2013 15:48:31 -0800 (PST) dormando wrote: > Add a userspace visible knob to tell the VM to keep an extra amount > of memory free, by increasing the gap between each zone's min and > low watermarks. The problem is that adding this tunable will constrain future VM implementations.

Re: [PATCH] add extra free kbytes tunable

2013-02-19 Thread Andrew Morton
On Sun, 17 Feb 2013 15:48:31 -0800 (PST) dormando dorma...@rydia.net wrote: Add a userspace visible knob to tell the VM to keep an extra amount of memory free, by increasing the gap between each zone's min and low watermarks. The problem is that adding this tunable will constrain future VM

Re: [PATCH] add extra free kbytes tunable

2013-02-19 Thread dormando
The problem is that adding this tunable will constrain future VM implementations. We will forever need to at least retain the pseudo-file. We will also need to make some effort to retain its behaviour. It would of course be better to fix things so you don't need to tweak VM internals to

[PATCH] add extra free kbytes tunable

2013-02-17 Thread dormando
From: Rik van Riel Add a userspace visible knob to tell the VM to keep an extra amount of memory free, by increasing the gap between each zone's min and low watermarks. This is useful for realtime applications that call system calls and have a bound on the number of allocations that happen in

[PATCH] add extra free kbytes tunable

2013-02-17 Thread dormando
From: Rik van Riel r...@redhat.com Add a userspace visible knob to tell the VM to keep an extra amount of memory free, by increasing the gap between each zone's min and low watermarks. This is useful for realtime applications that call system calls and have a bound on the number of allocations