[PATCH] Fix GFP flags passed from the virtio balloon driver

2010-04-19 Thread Balbir Singh
sometime anyway. To: kvm kvm@vger.kernel.org Cc: Rusty Russell ru...@rustcorp.com.au Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- drivers/virtio/virtio_balloon.c |3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio

[PATCH][RESEND]Fix GFP flags passed from the virtio balloon driver

2010-04-21 Thread Balbir Singh
Fix GFP flags passed from the virtio balloon driver From: Balbir Singh bal...@linux.vnet.ibm.com The virtio balloon driver can dig into the reservation pools of the OS to satisfy a balloon request. This is not advisable and other balloon drivers (drivers/xen/balloon.c) avoid this as well

Re: KVM and the OOM-Killer

2010-05-14 Thread Balbir Singh
* Athanasius k...@miggy.org [2010-05-14 08:33:34]: On Thu, May 13, 2010 at 01:20:31PM +0100, James Stevens wrote: We have a KVM host with 48Gb of RAM and run about 20 KVM clients on it. After some time - different time depending on the kernel version - the VM host kernel will start

Re: KVM and the OOM-Killer

2010-05-14 Thread Balbir Singh
* James Stevens james.stev...@jrcs.co.uk [2010-05-14 09:10:19]: echo -16 /proc/pid/oom_adj Thanks for that - yes, I know about oom_adj, but it doesn't (totally) work. udevd has a default of -17 and it got killed anyway. Also, the only thing this server runs is VMs so if they can't

Re: KVM and the OOM-Killer

2010-05-14 Thread Balbir Singh
* James Stevens james.stev...@jrcs.co.uk [2010-05-14 09:43:04]: Have you looked at memory cgroups and using that with limits with VMs? The problem was *NOT* that my VMs exhausted all memory. I know that is what normally triggers oom-killer, but you have to understand this mine was a very

[REPOST][PATCH 0/3] Unmapped page cache control (v3)

2011-01-20 Thread Balbir Singh
bss dec hex filename 418908 1883023 140888 2442819 254643 mm/built-in.o --- Balbir Singh (3): Move zone_reclaim() outside of CONFIG_NUMA Refactor zone_reclaim code Provide control over unmapped pages Documentation/kernel-parameters.txt |8 ++ include/linux

[REPOST] [PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA (v3)

2011-01-20 Thread Balbir Singh
This patch moves zone_reclaim and associated helpers outside CONFIG_NUMA. This infrastructure is reused in the patches for page cache control that follow. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- include/linux/mmzone.h |4 ++-- include/linux/swap.h |4 ++-- kernel

[REPOST] [PATCH 2/3] Refactor zone_reclaim code (v3)

2011-01-20 Thread Balbir Singh
Changelog v3 1. Renamed zone_reclaim_unmapped_pages to zone_reclaim_pages Refactor zone_reclaim, move reusable functionality outside of zone_reclaim. Make zone_reclaim_unmapped_pages modular Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- mm/vmscan.c | 35

[REPOST] [PATCH 3/3] Provide control over unmapped pages (v3)

2011-01-20 Thread Balbir Singh
) Provide control using zone_reclaim() and a boot parameter. The code reuses functionality from zone_reclaim() to isolate unmapped pages and reclaim them as a priority, ahead of other mapped pages. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- Documentation/kernel-parameters.txt |8

Re: [REPOST] [PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA (v3)

2011-01-20 Thread Balbir Singh
* Christoph Lameter c...@linux.com [2011-01-20 08:49:27]: On Thu, 20 Jan 2011, Balbir Singh wrote: --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -253,11 +253,11 @@ extern int vm_swappiness; extern int remove_mapping(struct address_space *mapping, struct page *page

Re: [REPOST] [PATCH 2/3] Refactor zone_reclaim code (v3)

2011-01-20 Thread Balbir Singh
* Christoph Lameter c...@linux.com [2011-01-20 08:50:40]: Reviewed-by: Christoph Lameter c...@linux.com Thanks for the review! -- Three Cheers, Balbir -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More

Re: [REPOST] [PATCH 3/3] Provide control over unmapped pages (v3)

2011-01-20 Thread Balbir Singh
* Christoph Lameter c...@linux.com [2011-01-20 09:00:09]: On Thu, 20 Jan 2011, Balbir Singh wrote: + unmapped_page_control + [KNL] Available if CONFIG_UNMAPPED_PAGECACHE_CONTROL + is enabled. It controls the amount of unmapped memory

Re: [REPOST] [PATCH 3/3] Provide control over unmapped pages (v3)

2011-01-23 Thread Balbir Singh
* Christoph Lameter c...@linux.com [2011-01-21 09:55:17]: On Fri, 21 Jan 2011, Balbir Singh wrote: * Christoph Lameter c...@linux.com [2011-01-20 09:00:09]: On Thu, 20 Jan 2011, Balbir Singh wrote: + unmapped_page_control + [KNL] Available

[PATCH 0/3] Unmapped Page Cache Control (v4)

2011-01-24 Thread Balbir Singh
. Data from the previous patchsets can be found at https://lkml.org/lkml/2010/11/30/79 --- Balbir Singh (3): Move zone_reclaim() outside of CONFIG_NUMA Refactor zone_reclaim code Provide control over unmapped pages Documentation/kernel-parameters.txt |8 ++ include/linux

[PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA (v4)

2011-01-24 Thread Balbir Singh
This patch moves zone_reclaim and associated helpers outside CONFIG_NUMA. This infrastructure is reused in the patches for page cache control that follow. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- include/linux/mmzone.h |4 ++-- include/linux/swap.h |4 ++-- kernel

[PATCH 1/2] Refactor zone_reclaim code (v4)

2011-01-24 Thread Balbir Singh
Changelog v3 1. Renamed zone_reclaim_unmapped_pages to zone_reclaim_pages Refactor zone_reclaim, move reusable functionality outside of zone_reclaim. Make zone_reclaim_unmapped_pages modular Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com Reviewed-by: Christoph Lameter c...@linux.com

[PATCH 3/3] Provide control over unmapped pages (v4)

2011-01-24 Thread Balbir Singh
as a priority, ahead of other mapped pages. A new sysctl for max_unmapped_ratio is provided and set to 16, indicating 16% of the total zone pages are unmapped, we start shrinking unmapped page cache. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- Documentation/kernel-parameters.txt |8

Re: [PATCH 1/2] Refactor zone_reclaim code (v4)

2011-01-24 Thread Balbir Singh
* Balbir Singh bal...@linux.vnet.ibm.com [2011-01-25 10:40:09]: Changelog v3 1. Renamed zone_reclaim_unmapped_pages to zone_reclaim_pages Refactor zone_reclaim, move reusable functionality outside of zone_reclaim. Make zone_reclaim_unmapped_pages modular Signed-off-by: Balbir Singh bal

Re: [PATCH 3/3] Provide control over unmapped pages (v4)

2011-01-27 Thread Balbir Singh
On Thu, Jan 27, 2011 at 4:42 AM, Minchan Kim minchan@gmail.com wrote: [snip] index 7b56473..2ac8549 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1660,6 +1660,9 @@ zonelist_scan:                        unsigned long mark;                        int ret; +                      

Re: [PATCH 3/3] Provide control over unmapped pages (v4)

2011-01-27 Thread Balbir Singh
* Christoph Lameter c...@linux.com [2011-01-26 10:57:37]: Reviewed-by: Christoph Lameter c...@linux.com Thanks for the review! -- Three Cheers, Balbir -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More

Re: [PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA (v4)

2011-01-27 Thread Balbir Singh
* Christoph Lameter c...@linux.com [2011-01-26 10:56:56]: Reviewed-by: Christoph Lameter c...@linux.com Thanks for the review! -- Three Cheers, Balbir -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More

Re: [PATCH 3/3] Provide control over unmapped pages (v4)

2011-01-27 Thread Balbir Singh
* MinChan Kim minchan@gmail.com [2011-01-28 14:44:50]: On Fri, Jan 28, 2011 at 11:56 AM, Balbir Singh bal...@linux.vnet.ibm.com wrote: On Thu, Jan 27, 2011 at 4:42 AM, Minchan Kim minchan@gmail.com wrote: [snip] index 7b56473..2ac8549 100644 --- a/mm/page_alloc.c +++ b/mm

Re: [PATCH 3/3] Provide control over unmapped pages (v4)

2011-01-28 Thread Balbir Singh
* KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2011-01-28 16:56:05]: On Fri, 28 Jan 2011 16:24:19 +0900 Minchan Kim minchan@gmail.com wrote: On Fri, Jan 28, 2011 at 3:48 PM, Balbir Singh bal...@linux.vnet.ibm.com wrote: * MinChan Kim minchan@gmail.com [2011-01-28 14:44:50

Re: [PATCH 3/3] Provide control over unmapped pages (v4)

2011-01-28 Thread Balbir Singh
* MinChan Kim minchan@gmail.com [2011-01-28 16:24:19]: But the assumption for LRU order to change happens only if the page cannot be successfully freed, which means it is in some way active.. and needs to be moved no? 1. holded page by someone 2. mapped pages 3. active pages 1

Re: [PATCH 3/3] Provide control over unmapped pages (v4)

2011-01-28 Thread Balbir Singh
* KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2011-01-28 17:17:44]: On Fri, 28 Jan 2011 13:49:28 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: * KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2011-01-28 16:56:05]: BTW, it seems this doesn't work when some apps use huge

Re: [PATCH 3/3] Provide control over unmapped pages (v4)

2011-01-31 Thread Balbir Singh
* KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2011-01-31 08:58:53]: On Fri, 28 Jan 2011 09:20:02 -0600 (CST) Christoph Lameter c...@linux.com wrote: On Fri, 28 Jan 2011, KAMEZAWA Hiroyuki wrote: I see it as a tradeoff of when to check? add_to_page_cache or when we are

[PATCH 0/3][RESEND] Provide unmapped page cache control (v4)

2011-02-01 Thread Balbir Singh
and administrators can further tweak this for desired control. Data from the previous patchsets can be found at https://lkml.org/lkml/2010/11/30/79 --- Balbir Singh (3): Move zone_reclaim() outside of CONFIG_NUMA Refactor zone_reclaim code Provide control over unmapped pages

[PATCH 1/3][RESEND] Move zone_reclaim() outside of CONFIG_NUMA (v4)

2011-02-01 Thread Balbir Singh
This patch moves zone_reclaim and associated helpers outside CONFIG_NUMA. This infrastructure is reused in the patches for page cache control that follow. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com Reviewed-by: Christoph Lameter c...@linux.com --- include/linux/mmzone.h |4

[PATCH 2/3][RESEND] Refactor zone_reclaim code (v4)

2011-02-01 Thread Balbir Singh
Changelog v3 1. Renamed zone_reclaim_unmapped_pages to zone_reclaim_pages Refactor zone_reclaim, move reusable functionality outside of zone_reclaim. Make zone_reclaim_unmapped_pages modular Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com Reviewed-by: Christoph Lameter c...@linux.com

[PATCH 3/3][RESEND] Provide control over unmapped pages (v4)

2011-02-01 Thread Balbir Singh
as a priority, ahead of other mapped pages. A new sysctl for max_unmapped_ratio is provided and set to 16, indicating 16% of the total zone pages are unmapped, we start shrinking unmapped page cache. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com Reviewed-by: Christoph Lameter c...@linux.com

Re: [PATCH 3/3][RESEND] Provide control over unmapped pages (v4)

2011-02-09 Thread Balbir Singh
On 02/09/2011 05:27 AM, Andrew Morton wrote: On Tue, 01 Feb 2011 22:25:45 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: Changelog v4 1. Add max_unmapped_ratio and use that as the upper limit to check when to shrink the unmapped page cache (Christoph Lameter) Changelog v2 1. Use

Re: [PATCH 3/3] Provide control over unmapped pages (v4)

2011-02-14 Thread Balbir Singh
for late response. On Fri, Jan 28, 2011 at 8:18 PM, Balbir Singh bal...@linux.vnet.ibm.com wrote: * MinChan Kim minchan@gmail.com [2011-01-28 16:24:19]: But the assumption for LRU order to change happens only if the page cannot be successfully freed, which means it is in some way

[PATCH 0/3] Unmapped page cache control (v5)

2011-03-29 Thread Balbir Singh
/11/30/79 --- Balbir Singh (3): Move zone_reclaim() outside of CONFIG_NUMA Refactor zone_reclaim code Provide control over unmapped pages Documentation/kernel-parameters.txt |8 ++ Documentation/sysctl/vm.txt | 19 + include/linux/mmzone.h | 11

[PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA (v5)

2011-03-29 Thread Balbir Singh
This patch moves zone_reclaim and associated helpers outside CONFIG_NUMA. This infrastructure is reused in the patches for page cache control that follow. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com Reviewed-by: Christoph Lameter c...@linux.com --- include/linux/mmzone.h |4

[PATCH 2/3] Refactor zone_reclaim code (v5)

2011-03-29 Thread Balbir Singh
Changelog v3 1. Renamed zone_reclaim_unmapped_pages to zone_reclaim_pages Refactor zone_reclaim, move reusable functionality outside of zone_reclaim. Make zone_reclaim_unmapped_pages modular Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com Reviewed-by: Christoph Lameter c...@linux.com

[PATCH 3/3] Provide control over unmapped pages (v5)

2011-03-29 Thread Balbir Singh
of other mapped pages. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com Reviewed-by: Christoph Lameter c...@linux.com --- Documentation/kernel-parameters.txt |8 +++ Documentation/sysctl/vm.txt | 19 +++- include/linux/mmzone.h |7 +++ include/linux/swap.h

Re: [PATCH 0/3] Unmapped page cache control (v5)

2011-03-30 Thread Balbir Singh
* Andrew Morton a...@linux-foundation.org [2011-03-30 16:36:07]: On Wed, 30 Mar 2011 11:00:26 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: Data from the previous patchsets can be found at https://lkml.org/lkml/2010/11/30/79 It would be nice if the data for the current patchset

Re: [PATCH 3/3] Provide control over unmapped pages (v5)

2011-03-30 Thread Balbir Singh
* Andrew Morton a...@linux-foundation.org [2011-03-30 16:35:45]: On Wed, 30 Mar 2011 11:02:38 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: Changelog v4 1. Added documentation for max_unmapped_pages 2. Better #ifdef'ing of max_unmapped_pages and min_unmapped_pages Changelog

Re: [PATCH 0/3] Unmapped page cache control (v5)

2011-03-31 Thread Balbir Singh
* KOSAKI Motohiro kosaki.motoh...@jp.fujitsu.com [2011-03-31 14:40:33]: The following series implements page cache control, this is a split out version of patch 1 of version 3 of the page cache optimization patches posted earlier at Previous posting http://lwn.net/Articles/425851/ and

Re: [PATCH 0/3] Unmapped page cache control (v5)

2011-03-31 Thread Balbir Singh
* Dave Chinner da...@fromorbit.com [2011-04-01 08:40:33]: On Wed, Mar 30, 2011 at 11:00:26AM +0530, Balbir Singh wrote: The following series implements page cache control, this is a split out version of patch 1 of version 3 of the page cache optimization patches posted earlier

Re: [PATCH 0/3] Unmapped page cache control (v5)

2011-04-01 Thread Balbir Singh
* KOSAKI Motohiro kosaki.motoh...@jp.fujitsu.com [2011-04-01 16:56:57]: Hi 1) zone reclaim doesn't work if the system has multiple node and the workload is file cache oriented (eg file server, web server, mail server, et al). because zone recliam make some much free pages

Re: [PATCH 0/3] Unmapped page cache control (v5)

2011-04-01 Thread Balbir Singh
* Andrew Morton a...@linux-foundation.org [2011-03-30 22:32:31]: On Thu, 31 Mar 2011 10:57:03 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: * Andrew Morton a...@linux-foundation.org [2011-03-30 16:36:07]: On Wed, 30 Mar 2011 11:00:26 +0530 Balbir Singh bal

Re: [PATCH 0/3] Unmapped page cache control (v5)

2011-04-01 Thread Balbir Singh
* KOSAKI Motohiro kosaki.motoh...@jp.fujitsu.com [2011-04-01 22:21:26]: * KOSAKI Motohiro kosaki.motoh...@jp.fujitsu.com [2011-04-01 16:56:57]: Hi 1) zone reclaim doesn't work if the system has multiple node and the workload is file cache oriented (eg file server, web

[RFC/T/D][PATCH 0/2] KVM page cache optimization (v2)

2010-06-08 Thread Balbir Singh
This is version 2 of the page cache control patches for KVM. This series has two patches, the first controls the amount of unmapped page cache usage via a boot parameter and sysctl. The second patch controls page and slab cache via the balloon driver. Both the patches make heavy use of the

[RFC][PATCH 1/2] Linux/Guest unmapped page cache control

2010-06-08 Thread Balbir Singh
Selectively control Unmapped Page Cache (nospam version) From: Balbir Singh bal...@linux.vnet.ibm.com This patch implements unmapped page cache control via preferred page cache reclaim. The current patch hooks into kswapd and reclaims page cache if the user has requested for unmapped page

[RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-08 Thread Balbir Singh
Balloon unmapped page cache pages first From: Balbir Singh bal...@linux.vnet.ibm.com This patch builds on the ballooning infrastructure by ballooning unmapped page cache pages first. It looks for low hanging fruit first and tries to reclaim clean unmapped pages first. This patch brings

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-10 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-10 12:43:11]: On 06/08/2010 06:51 PM, Balbir Singh wrote: Balloon unmapped page cache pages first From: Balbir Singhbal...@linux.vnet.ibm.com This patch builds on the ballooning infrastructure by ballooning unmapped page cache pages first. It looks

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-10 Thread Balbir Singh
* KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2010-06-11 10:54:41]: On Thu, 10 Jun 2010 17:07:32 -0700 Dave Hansen d...@linux.vnet.ibm.com wrote: On Thu, 2010-06-10 at 19:55 +0530, Balbir Singh wrote: I'm not sure victimizing unmapped cache pages is a good idea. Shouldn't page

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-10 Thread Balbir Singh
* Dave Hansen d...@linux.vnet.ibm.com [2010-06-10 17:07:32]: On Thu, 2010-06-10 at 19:55 +0530, Balbir Singh wrote: I'm not sure victimizing unmapped cache pages is a good idea. Shouldn't page selection use the LRU for recency information instead of the cost of guest reclaim? Dropping

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-11 Thread Balbir Singh
* KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2010-06-11 14:05:53]: On Fri, 11 Jun 2010 10:16:32 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: * KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2010-06-11 10:54:41]: On Thu, 10 Jun 2010 17:07:32 -0700 Dave Hansen d

Re: [PATCH RFC] KVM: busy-spin detector

2010-06-11 Thread Balbir Singh
* Marcelo Tosatti mtosa...@redhat.com [2010-06-10 23:25:51]: The following patch implements a simple busy-spin detector. It considers a vcpu as busy-spinning if there are two consecutive exits due to external interrupt on the same RIP, and sleeps for 100us in that case. It is very likely

Re: [PATCH RFC] KVM: busy-spin detector

2010-06-11 Thread Balbir Singh
* Huang, Zhiteng zhiteng.hu...@intel.com [2010-06-11 23:03:25]: PLE-like design may be more generic than para-virtish when it comes to Windows guest. Hmm.. sounds reasonable Is this busy-spin actually a Lock Holder Preemption problem? Yep, I was hinting towards solving that problem.

Re: [PATCH RFC] KVM: busy-spin detector

2010-06-11 Thread Balbir Singh
* Marcelo Tosatti mtosa...@redhat.com [2010-06-11 14:46:27]: Interesting approach, is there a reason to tie it in with pause loop exits? Hum, i don't see any. PLE exits provide the same detection, but more accurately. Can't we do something more generic or even para-virtish. This is

Re: [RFC][PATCH 1/2] Linux/Guest unmapped page cache control

2010-06-13 Thread Balbir Singh
* Balbir Singh bal...@linux.vnet.ibm.com [2010-06-08 21:21:46]: Selectively control Unmapped Page Cache (nospam version) From: Balbir Singh bal...@linux.vnet.ibm.com This patch implements unmapped page cache control via preferred page cache reclaim. The current patch hooks into kswapd

Re: [RFC][PATCH 1/2] Linux/Guest unmapped page cache control

2010-06-14 Thread Balbir Singh
* KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2010-06-14 09:28:19]: On Mon, 14 Jun 2010 00:01:45 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: * Balbir Singh bal...@linux.vnet.ibm.com [2010-06-08 21:21:46]: Selectively control Unmapped Page Cache (nospam version

Re: [RFC][PATCH 1/2] Linux/Guest unmapped page cache control

2010-06-14 Thread Balbir Singh
* KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2010-06-14 16:00:21]: On Mon, 14 Jun 2010 12:19:55 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: - Why don't you believe LRU ? And if LRU doesn't work well, should it be fixed by a knob rather than generic approach ? - No side

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-14 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-14 11:09:44]: On 06/11/2010 07:56 AM, Balbir Singh wrote: Just to be clear, let's say we have a mapped page (say of /sbin/init) that's been unreferenced since _just_ after the system booted. We also have an unmapped page cache page of a file often used

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-14 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-14 15:40:28]: On 06/14/2010 11:48 AM, Balbir Singh wrote: In this case the order is as follows 1. First we pick free pages if any 2. If we don't have free pages, we go after unmapped page cache and slab cache 3. If that fails as well, we go after

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-14 Thread Balbir Singh
* Dave Hansen d...@linux.vnet.ibm.com [2010-06-14 08:12:56]: On Mon, 2010-06-14 at 14:18 +0530, Balbir Singh wrote: 1. A slab page will not be freed until the entire page is free (all slabs have been kfree'd so to speak). Normal reclaim will definitely free this page, but a lot

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-14 Thread Balbir Singh
* Dave Hansen d...@linux.vnet.ibm.com [2010-06-14 10:09:31]: On Mon, 2010-06-14 at 22:28 +0530, Balbir Singh wrote: If you've got duplicate pages and you know that they are duplicated and can be retrieved at a lower cost, why wouldn't we go after them first? I agree with this in theory

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-14 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-14 18:34:58]: On 06/14/2010 06:12 PM, Dave Hansen wrote: On Mon, 2010-06-14 at 14:18 +0530, Balbir Singh wrote: 1. A slab page will not be freed until the entire page is free (all slabs have been kfree'd so to speak). Normal reclaim will definitely free

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-14 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-14 19:34:00]: On 06/14/2010 06:55 PM, Dave Hansen wrote: On Mon, 2010-06-14 at 18:44 +0300, Avi Kivity wrote: On 06/14/2010 06:33 PM, Dave Hansen wrote: At the same time, I see what you're trying to do with this. It really can be an alternative to

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-15 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-15 09:58:33]: On 06/14/2010 08:45 PM, Balbir Singh wrote: There are two decisions that need to be made: - how much memory a guest should be given - given some guest memory, what's the best use for it The first question can perhaps be answered

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-15 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-15 10:12:44]: On 06/14/2010 08:16 PM, Balbir Singh wrote: * Dave Hansend...@linux.vnet.ibm.com [2010-06-14 10:09:31]: On Mon, 2010-06-14 at 22:28 +0530, Balbir Singh wrote: If you've got duplicate pages and you know that they are duplicated and can

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-15 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-15 12:44:31]: On 06/15/2010 10:49 AM, Balbir Singh wrote: All we need is to select the right page to drop. Do we need to drop to the granularity of the page to drop? I think figuring out the class of pages and making sure that we don't write our own

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-15 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-15 12:54:31]: On 06/15/2010 10:52 AM, Balbir Singh wrote: That is why the policy (in the next set) will come from the host. As to whether the data is truly duplicated, my experiments show up to 60% of the page cache is duplicated. Isn't that incredibly

Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

2010-06-17 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-06-16 14:39:02]: We're talking about an environment which we're always trying to optimize. Imagine that we're always trying to consolidate guests on to smaller numbers of hosts. We're effectively in a state where we _always_ want new guests. If this

Re: [PATCH] emulate accessed bit for EPT

2010-02-03 Thread Balbir Singh
* Rik van Riel r...@redhat.com [2010-02-03 16:11:03]: Currently KVM pretends that pages with EPT mappings never got accessed. This has some side effects in the VM, like swapping out actively used guest pages and needlessly breaking up actively used hugepages. We can avoid those very

Re: [PATCH] emulate accessed bit for EPT

2010-02-04 Thread Balbir Singh
* Rik van Riel r...@redhat.com [2010-02-04 08:40:43]: On 02/03/2010 11:12 PM, Balbir Singh wrote: * Rik van Rielr...@redhat.com [2010-02-03 16:11:03]: Currently KVM pretends that pages with EPT mappings never got accessed. This has some side effects in the VM, like swapping out actively

[PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

2010-03-15 Thread Balbir Singh
Selectively control Unmapped Page Cache (nospam version) From: Balbir Singh bal...@linux.vnet.ibm.com This patch implements unmapped page cache control via preferred page cache reclaim. The current patch hooks into kswapd and reclaims page cache if the user has requested for unmapped page

Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

2010-03-15 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-03-15 09:48:05]: On 03/15/2010 09:22 AM, Balbir Singh wrote: Selectively control Unmapped Page Cache (nospam version) From: Balbir Singhbal...@linux.vnet.ibm.com This patch implements unmapped page cache control via preferred page cache reclaim

Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

2010-03-15 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-03-15 10:27:45]: On 03/15/2010 10:07 AM, Balbir Singh wrote: * Avi Kivitya...@redhat.com [2010-03-15 09:48:05]: On 03/15/2010 09:22 AM, Balbir Singh wrote: Selectively control Unmapped Page Cache (nospam version) From: Balbir Singhbal

Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

2010-03-15 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-03-15 11:27:56]: The knobs are for 1. Selective enablement 2. Selective control of the % of unmapped pages An alternative path is to enable KSM for page cache. Then we have direct read-only guest access to host page cache, without any guest

Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

2010-03-15 Thread Balbir Singh
* Chris Webb ch...@arachsys.com [2010-03-15 20:23:54]: Avi Kivity a...@redhat.com writes: On 03/15/2010 10:07 AM, Balbir Singh wrote: Yes, it is a virtio call away, but is the cost of paying twice in terms of memory acceptable? Usually, it isn't, which is why I recommend cache=off

Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

2010-03-15 Thread Balbir Singh
* Randy Dunlap randy.dun...@oracle.com [2010-03-15 08:46:31]: On Mon, 15 Mar 2010 12:52:15 +0530 Balbir Singh wrote: Hi, If you go ahead with this, please add the boot parameter its description to Documentation/kernel-parameters.txt. I certainly will, thanks for keeping a watch

Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

2010-03-16 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-03-16 13:08:28]: On 03/16/2010 12:44 PM, Christoph Hellwig wrote: On Tue, Mar 16, 2010 at 12:36:31PM +0200, Avi Kivity wrote: Are you talking about direct volume access or qcow2? Doesn't matter. For direct volume access, I still don't get it. The number

Re: [PATCH][RF C/T/D] Unmapped page cache control - via boot parameter

2010-03-17 Thread Balbir Singh
* Anthony Liguori anth...@codemonkey.ws [2010-03-17 10:55:47]: On 03/17/2010 10:14 AM, Chris Webb wrote: Anthony Liguorianth...@codemonkey.ws writes: This really gets down to your definition of safe behaviour. As it stands, if you suffer a power outage, it may lead to guest corruption.

[PATCH 0/3] Series short description

2010-11-30 Thread Balbir Singh
Time 272.672 (0.453178) User Time 189.7 (0.718157) System Time 296.77 (0.845606) Percent CPU 178 (0) Context Switches 118822 (277.434) Sleeps 37542.8 (545.922) More data on the test results with the earlier patch is at http://www.mail-archive.com/kvm@vger.kernel.org/msg43655.html --- Balbir Singh

[PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA

2010-11-30 Thread Balbir Singh
This patch moves zone_reclaim and associated helpers outside CONFIG_NUMA. This infrastructure is reused in the patches for page cache control that follow. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- include/linux/mmzone.h |4 ++-- mm/vmscan.c|2 -- 2 files

[PATCH 2/3] Refactor zone_reclaim

2010-11-30 Thread Balbir Singh
Refactor zone_reclaim, move reusable functionality outside of zone_reclaim. Make zone_reclaim_unmapped_pages modular Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- mm/vmscan.c | 35 +++ 1 files changed, 23 insertions(+), 12 deletions(-) diff --git

[PATCH 3/3] Provide control over unmapped pages

2010-11-30 Thread Balbir Singh
Provide control using zone_reclaim() and a boot parameter. The code reuses functionality from zone_reclaim() to isolate unmapped pages and reclaim them as a priority, ahead of other mapped pages. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- include/linux/swap.h |5 ++- mm

Re: [PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA

2010-11-30 Thread Balbir Singh
* Balbir Singh bal...@linux.vnet.ibm.com [2010-12-01 10:04:08]: * Andrew Morton a...@linux-foundation.org [2010-11-30 14:23:38]: On Tue, 30 Nov 2010 15:45:12 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: This patch moves zone_reclaim and associated helpers outside

Re: [PATCH 2/3] Refactor zone_reclaim

2010-11-30 Thread Balbir Singh
* Balbir Singh bal...@linux.vnet.ibm.com [2010-12-01 10:16:34]: * KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2010-12-01 10:23:29]: On Tue, 30 Nov 2010 15:45:55 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: Refactor zone_reclaim, move reusable functionality outside

Re: [PATCH 3/3] Provide control over unmapped pages

2010-11-30 Thread Balbir Singh
* Balbir Singh bal...@linux.vnet.ibm.com [2010-12-01 10:24:21]: * Andrew Morton a...@linux-foundation.org [2010-11-30 14:25:09]: On Tue, 30 Nov 2010 15:46:31 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: Provide control using zone_reclaim() and a boot parameter. The code

Re: [PATCH 3/3] Provide control over unmapped pages

2010-11-30 Thread Balbir Singh
* Balbir Singh bal...@linux.vnet.ibm.com [2010-12-01 10:46:32]: * KOSAKI Motohiro kosaki.motoh...@jp.fujitsu.com [2010-12-01 09:14:13]: Provide control using zone_reclaim() and a boot parameter. The code reuses functionality from zone_reclaim() to isolate unmapped pages and reclaim

Re: [PATCH 3/3] Provide control over unmapped pages

2010-11-30 Thread Balbir Singh
* Balbir Singh bal...@linux.vnet.ibm.com [2010-12-01 10:48:16]: * KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2010-12-01 10:32:54]: On Tue, 30 Nov 2010 15:46:31 +0530 Balbir Singh bal...@linux.vnet.ibm.com wrote: Provide control using zone_reclaim() and a boot parameter

Re: [PATCH 3/3] Provide control over unmapped pages

2010-12-04 Thread Balbir Singh
* KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com [2010-12-02 11:50:36]: On Thu, 2 Dec 2010 10:22:16 +0900 (JST) KOSAKI Motohiro kosaki.motoh...@jp.fujitsu.com wrote: On Tue, 30 Nov 2010, Andrew Morton wrote: +#define UNMAPPED_PAGE_RATIO 16 Well. Giving 16 a name didn't

Re: [RFC PATCH 0/3] directed yield for Pause Loop Exiting

2010-12-09 Thread Balbir Singh
* Rik van Riel r...@redhat.com [2010-12-02 14:41:29]: When running SMP virtual machines, it is possible for one VCPU to be spinning on a spinlock, while the VCPU that holds the spinlock is not currently running, because the host scheduler preempted it to run something else. Both Intel and

[PATCH 0/3] Provide unmapped page cache control (v2)

2010-12-10 Thread Balbir Singh
bss dec hex filename 418908 1883023 140888 2442819 254643 mm/built-in.o --- Balbir Singh (3): Move zone_reclaim() outside of CONFIG_NUMA Refactor zone_reclaim, move reusable functionality outside Provide control over unmapped pages Documentation/kernel-parameters.txt

[PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA (v2)

2010-12-10 Thread Balbir Singh
Changelog v2 Moved sysctl for min_unmapped_ratio as well This patch moves zone_reclaim and associated helpers outside CONFIG_NUMA. This infrastructure is reused in the patches for page cache control that follow. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- include/linux/mmzone.h

[PATCH 2/3] Refactor zone_reclaim (v2)

2010-12-10 Thread Balbir Singh
Move reusable functionality outside of zone_reclaim. Make zone_reclaim_unmapped_pages modular Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- mm/vmscan.c | 35 +++ 1 files changed, 23 insertions(+), 12 deletions(-) diff --git a/mm/vmscan.c b/mm

[PATCH 3/3] Provide control over unmapped pages (v2)

2010-12-10 Thread Balbir Singh
) 5. Updated Documentation/kernel-parameters.txt (Andrew Morton) Provide control using zone_reclaim() and a boot parameter. The code reuses functionality from zone_reclaim() to isolate unmapped pages and reclaim them as a priority, ahead of other mapped pages. Signed-off-by: Balbir Singh bal

Re: [RFC PATCH 0/3] directed yield for Pause Loop Exiting

2010-12-13 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-12-11 09:31:24]: On 12/10/2010 07:03 AM, Balbir Singh wrote: Scheduler people, please flame me with anything I may have done wrong, so I can do it right for a next version :) This is a good problem statement, there are other things to consider

Re: [RFC PATCH 0/3] directed yield for Pause Loop Exiting

2010-12-13 Thread Balbir Singh
* Avi Kivity a...@redhat.com [2010-12-13 13:57:37]: On 12/11/2010 03:57 PM, Balbir Singh wrote: * Avi Kivitya...@redhat.com [2010-12-11 09:31:24]: On 12/10/2010 07:03 AM, Balbir Singh wrote: Scheduler people, please flame me with anything I may have done wrong, so I can do

Re: [RFC PATCH 0/3] directed yield for Pause Loop Exiting

2010-12-14 Thread Balbir Singh
* Rik van Riel r...@redhat.com [2010-12-13 12:02:51]: On 12/11/2010 08:57 AM, Balbir Singh wrote: If the vpcu holding the lock runs more and capped, the timeslice transfer is a heuristic that will not help. That indicates you really need the cap to be per guest, and not per VCPU. Yes

Re: [PATCH 2/3] Refactor zone_reclaim (v2)

2010-12-14 Thread Balbir Singh
* MinChan Kim minchan@gmail.com [2010-12-14 19:01:26]: Hi Balbir, On Fri, Dec 10, 2010 at 11:31 PM, Balbir Singh bal...@linux.vnet.ibm.com wrote: Move reusable functionality outside of zone_reclaim. Make zone_reclaim_unmapped_pages modular Signed-off-by: Balbir Singh bal

Re: [PATCH 3/3] Provide control over unmapped pages (v2)

2010-12-23 Thread Balbir Singh
* MinChan Kim minchan@gmail.com [2010-12-14 20:02:45]: +                       if (should_reclaim_unmapped_pages(zone)) +                               wakeup_kswapd(zone, order); I think we can put the logic into zone_watermark_okay. I did some checks and zone_watermark_ok is used

Re: [PATCH 2/3] Refactor zone_reclaim (v2)

2010-12-23 Thread Balbir Singh
* MinChan Kim minchan@gmail.com [2010-12-15 07:38:42]: On Tue, Dec 14, 2010 at 8:45 PM, Balbir Singh bal...@linux.vnet.ibm.com wrote: * MinChan Kim minchan@gmail.com [2010-12-14 19:01:26]: Hi Balbir, On Fri, Dec 10, 2010 at 11:31 PM, Balbir Singh bal...@linux.vnet.ibm.com

[PATCH 0/3] Unmapped Page Control (v3)

2010-12-23 Thread Balbir Singh
-in.o CONFIG_UNMAPPED_PAGECACHE_CONTROL disabled, CONFIG_NUMA enabled # size mm/built-in.o textdata bss dec hex filename 418908 1883023 140888 2442819 254643 mm/built-in.o --- Balbir Singh (3): Move zone_reclaim() outside of CONFIG_NUMA Refactor zone_reclaim code

[PATCH 1/3] Move zone_reclaim() outside of CONFIG_NUMA (v3)

2010-12-23 Thread Balbir Singh
This patch moves zone_reclaim and associated helpers outside CONFIG_NUMA. This infrastructure is reused in the patches for page cache control that follow. Signed-off-by: Balbir Singh bal...@linux.vnet.ibm.com --- include/linux/mmzone.h |4 ++-- include/linux/swap.h |4 ++-- kernel

  1   2   >