[PATCH] fix migration with big mem guests

2010-04-04 Thread Izik Eidus
already have timer that is set, so it like it make the most sense to use it... If anyone have any better idea how to solve this issue, I will be very happy to hear. Thanks. From 2d9c25f1fee61f50cb130769c3779707a6ef90d9 Mon Sep 17 00:00:00 2001 From: Izik Eidus iei...@redhat.com Date: Mon, 5 Apr 2010

[PATCH 0/3] ksm support for kvm

2009-09-10 Thread Izik Eidus
Hi, The following seires add ksm support to the kvm mmu. Thanks. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] add SPTE_HOST_WRITEABLE flag to the shadow ptes

2009-09-10 Thread Izik Eidus
this flag notify that the host physical page we are pointing to from the spte is write protected, and therefore we cant change its access to be write unless we run get_user_pages(write = 1). (this is needed for change_pte support in kvm) Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86

[PATCH 3/3] add support for change_pte mmu notifiers

2009-09-10 Thread Izik Eidus
this is needed for kvm if it want ksm to directly map pages into its shadow page tables. Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/mmu.c | 70 ++ virt/kvm/kvm_main.c | 14

Re: [PATCH 2/3] add SPTE_HOST_WRITEABLE flag to the shadow ptes

2009-09-12 Thread Izik Eidus
Marcelo Tosatti wrote: On Thu, Sep 10, 2009 at 07:38:57PM +0300, Izik Eidus wrote: this flag notify that the host physical page we are pointing to from the spte is write protected, and therefore we cant change its access to be write unless we run get_user_pages(write = 1). (this is needed

Re: [PATCH 3/3] add support for change_pte mmu notifiers

2009-09-12 Thread Izik Eidus
Marcelo Tosatti wrote: On Sat, Sep 12, 2009 at 09:41:10AM +0300, Izik Eidus wrote: Marcelo Tosatti wrote: On Thu, Sep 10, 2009 at 07:38:58PM +0300, Izik Eidus wrote: this is needed for kvm if it want ksm to directly map pages into its shadow page tables. Signed-off-by: Izik

Re: [PATCH 2/3] add SPTE_HOST_WRITEABLE flag to the shadow ptes

2009-09-14 Thread Izik Eidus
Marcelo Tosatti wrote: Why can't you use the writable bit in the spte? So that you can only sync a writeable spte if it was writeable before, in sync_page? I could, but there we will add overhead for read only gptes that become writable in the guest... If you prefer to fault on the syncing

[PATCH 0/3] ksm support for kvm v2

2009-09-23 Thread Izik Eidus
Hope i fixed everything i was asked... please tell me if I forgot anything. Izik Eidus (3): kvm: dont hold pagecount reference for mapped sptes pages add SPTE_HOST_WRITEABLE flag to the shadow ptes add support for change_pte mmu notifiers arch/x86/include/asm/kvm_host.h |1 + arch/x86

[PATCH 1/3] kvm: dont hold pagecount reference for mapped sptes pages

2009-09-23 Thread Izik Eidus
not increase the mapcount when mapping page into shadow page table entry, so when comparing pagecount against mapcount, you have no reliable result.) Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/kvm/mmu.c |7 ++- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86

[PATCH 2/3] add SPTE_HOST_WRITEABLE flag to the shadow ptes

2009-09-23 Thread Izik Eidus
this flag notify that the host physical page we are pointing to from the spte is write protected, and therefore we cant change its access to be write unless we run get_user_pages(write = 1). (this is needed for change_pte support in kvm) Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86

[PATCH 3/3] add support for change_pte mmu notifiers

2009-09-23 Thread Izik Eidus
this is needed for kvm if it want ksm to directly map pages into its shadow page tables. Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/mmu.c | 64 +- virt/kvm/kvm_main.c | 14

Re: [PATCH 3/3] add support for change_pte mmu notifiers

2009-09-23 Thread Izik Eidus
Izik Eidus wrote: this is needed for kvm if it want ksm to directly map pages into its shadow page tables. Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/mmu.c | 64 +- virt/kvm

[PATCH 0/3] kvm ksm support v3

2009-09-23 Thread Izik Eidus
Change from v2 : remove unused if. Thanks. Izik Eidus (3): kvm: dont hold pagecount reference for mapped sptes pages add SPTE_HOST_WRITEABLE flag to the shadow ptes add support for change_pte mmu notifiers arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/mmu.c | 84

[PATCH 1/3] kvm: dont hold pagecount reference for mapped sptes pages

2009-09-23 Thread Izik Eidus
not increase the mapcount when mapping page into shadow page table entry, so when comparing pagecount against mapcount, you have no reliable result.) Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/kvm/mmu.c |7 ++- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86

[PATCH 2/3] add SPTE_HOST_WRITEABLE flag to the shadow ptes

2009-09-23 Thread Izik Eidus
this flag notify that the host physical page we are pointing to from the spte is write protected, and therefore we cant change its access to be write unless we run get_user_pages(write = 1). (this is needed for change_pte support in kvm) Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86

[PATCH 3/3] add support for change_pte mmu notifiers

2009-09-23 Thread Izik Eidus
this is needed for kvm if it want ksm to directly map pages into its shadow page tables. Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/mmu.c | 62 +- virt/kvm/kvm_main.c | 14

PATCH: kvm-userspace: ksm support

2009-10-04 Thread Izik Eidus
From a8ca226de8efb4f0447e4ef87bf034cf18996745 Mon Sep 17 00:00:00 2001 From: Izik Eidus iei...@redhat.com Date: Sun, 4 Oct 2009 14:01:31 +0200 Subject: [PATCH] kvm-userspace: add ksm support Calling to madvise(MADV_MERGEABLE) on the memory allocations. Signed-off-by: Izik Eidus iei...@redhat.com

Re: What happens if I use get_user_pages()

2008-07-08 Thread Izik Eidus
ציטוט Arn: If I use the kernel function get_user_pages() within a guest kernel module, will this ensure that the guests pages are never swapped out by the host ? Will it lock these pages in memory ? (Has anyone tried this ? ) As Izik said, the answer is no. Is this the desirable outcome,

Re: KSM Algorithm

2008-07-10 Thread Izik Eidus
ציטוט Sukanto Ghosh: Can anyone answer these queries regarding KSM ? How does KSM offers its services through the /dev/ksm device ? ioctls Are every guest pages scanned in KVM while using KSM or page-scanning and sharing is triggered on some event (low memory, etc) ? no, the scanning

Re: KSM Algorithm

2008-07-10 Thread Izik Eidus
ציטוט Sukanto Ghosh: Doesn't KSM notifies KVM about the shared pages so that KVM can update its sptes accordingly or is it done by KSM itself ? when using kvm, mmu notifiers is a must for ksm, (mmu notifiers update kvm about the changes in the host page table) What about KSM

Re: KSM Algorithm

2008-07-10 Thread Izik Eidus
ציטוט Sukanto Ghosh: One more query, what if multiple processes call ioctl KSM_CREATE_SCAN ? Will there be multiple scanners ? yes Consider a scenario where two processes A B separately call KSM_CREATE_SCAN and then start registering some memory pages/areas via

[PATCH] unalias rework

2008-09-04 Thread izik eidus
that treated as not aliased. Signed-off-by: Izik Eidus [EMAIL PROTECTED] --- arch/ia64/include/asm/kvm_host.h |1 + arch/ia64/kvm/kvm-ia64.c |5 -- arch/s390/include/asm/kvm_host.h |1 + arch/s390/kvm/kvm-s390.c |5 -- arch/x86/kvm/mmu.c |5 +-- arch/x86

Re: [PATCH] unalias rework

2008-10-02 Thread Izik Eidus
Marcelo Tosatti wrote: Hi Izik, On Thu, Sep 04, 2008 at 05:13:20PM +0300, izik eidus wrote: + struct kvm_memory_slot *alias_slot = kvm-memslots[i]; + + if (alias_slot-base_gfn == slot-base_gfn) + return 1; + } + return 0

Re: [PATCH] unalias rework

2008-10-02 Thread Izik Eidus
Marcelo Tosatti wrote: Hi Izik, On Thu, Sep 04, 2008 at 05:13:20PM +0300, izik eidus wrote: +struct kvm_memory_slot *alias_slot = kvm-memslots[i]; + +if (alias_slot-base_gfn == slot-base_gfn) +return 1; +} +return 0; +} + +static void update_alias_slots

fast fix for mmu aliasing troubles

2008-10-03 Thread Izik Eidus
i have sent patch that remove the aliasing and i will resend it again but untill then this patch should be applied as it fix kernel panic. From 61a13744e2367572f3e27ab5c0cce6e080e94d67 Mon Sep 17 00:00:00 2001 From: Izik Eidus [EMAIL PROTECTED] Date: Fri, 3 Oct 2008 17:40:32 +0300 Subject: [PATCH

Re: [Qemu-devel] [RFC] Disk integrity in QEMU

2008-10-12 Thread Izik Eidus
Avi Kivity wrote: LRU typically makes fairly bad decisions since it throws most of the information it has away. I recommend looking up LRU-K and similar algorithms, just to get a feel for this; it is basically the simplest possible algorithm short of random selection. Note that Linux doesn't

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
Andrew Morton wrote: On Tue, 11 Nov 2008 15:21:37 +0200 Izik Eidus [EMAIL PROTECTED] wrote: KSM is a linux driver that allows dynamicly sharing identical memory pages between one or more processes. unlike tradtional page sharing that is made at the allocation of the memory, ksm do

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
Avi Kivity wrote: Andrew Morton wrote: The whole approach seems wrong to me. The kernel lost track of these pages and then we run around post-facto trying to fix that up again. Please explain (for the changelog) why the kernel cannot get this right via the usual sharing, refcounting and

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
Andrew Morton wrote: On Tue, 11 Nov 2008 20:48:16 +0200 Avi Kivity [EMAIL PROTECTED] wrote: Andrew Morton wrote: The whole approach seems wrong to me. The kernel lost track of these pages and then we run around post-facto trying to fix that up again. Please explain (for the

[PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
From: Izik Eidus [EMAIL PROTECTED] ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. pages that are merged are marked as readonly and are COWed when any application try to change them. ksm is working by walking over

[PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
From: Izik Eidus [EMAIL PROTECTED] this function is needed in cases you want to change the userspace virtual mapping into diffrent physical page, KSM need this for merging the identical pages. this function is working by removing the oldpage from the rmap and calling put_page

[PATCH 4/4] MMU_NOTIFIRES: add set_pte_at_notify()

2008-11-11 Thread Izik Eidus
From: Izik Eidus [EMAIL PROTECTED] this function is optimzation for kvm/users of mmu_notifiers for COW pages, it is useful for kvm when ksm is used beacuse it allow kvm not to have to recive VMEXIT and only then map the shared page into the mmu shadow pages, but instead map it directly

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
Andrew Morton wrote: On Tue, 11 Nov 2008 15:21:39 +0200 Izik Eidus [EMAIL PROTECTED] wrote: From: Izik Eidus [EMAIL PROTECTED] this function is needed in cases you want to change the userspace virtual mapping into diffrent physical page, Not sure that I understand that description

[PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
KSM is a linux driver that allows dynamicly sharing identical memory pages between one or more processes. unlike tradtional page sharing that is made at the allocation of the memory, ksm do it dynamicly after the memory was created. Memory is periodically scanned; identical pages are identified

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
Christoph Lameter wrote: page migration as far as i saw cant migrate anonymous page into kernel page. if you want we can change page_migration to do that, but i thought you will rather have ksm changes separate. What do you mean by kernel page? The kernel can allocate a page and then

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
Christoph Lameter wrote: Currently page migration assumes that the page will continue to be part of the existing file or anon vma. exactly, and ksm really need it to get out of the existing anon vma! What you want sounds like assigning a swap pte to an anonymous page? That way a anon

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
Christoph Lameter wrote: On Tue, 11 Nov 2008, Avi Kivity wrote: Christoph Lameter wrote: page migration requires the page to be on the LRU. That could be changed if you have a different means of isolating a page from its page tables. Isn't rmap the means of isolating a page

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
Jonathan Corbet wrote: I don't claim to begin to really understand the deep VM side of this patch, but I can certainly pick nits as I work through it...sorry for the lack of anything more substantive. +static struct list_head slots; Some of these file-static variable names seem a

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
Jonathan Corbet wrote: On Wed, 12 Nov 2008 00:17:39 +0200 Izik Eidus [EMAIL PROTECTED] wrote: +static int ksm_dev_open(struct inode *inode, struct file *filp) +{ + try_module_get(THIS_MODULE); + return 0; +} + +static int ksm_dev_release(struct inode *inode, struct file *filp

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
Jonathan Corbet wrote: [Let's see if I can get through the rest without premature sends...] On Wed, 12 Nov 2008 00:17:39 +0200 Izik Eidus [EMAIL PROTECTED] wrote: Actually, it occurs to me that there's no sanity checks on any of the values passed in by ioctl(). What happens if the user

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
Jonathan Corbet wrote: What about things like cache effects from scanning all those pages? My guess is that, if you're trying to run dozens of Windows guests, cache usage is not at the top of your list of concerns, but I could be wrong. Usually am... Ok, ksm does make the cache of the

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Izik Eidus
Avi Kivity wrote: KAMEZAWA Hiroyuki wrote: Can I make a question ? (I'm working for memory cgroup.) Now, we do charge to anonymous page when - charge(+1) when it's mapped firstly (mapcount 0-1) - uncharge(-1) it's fully unmapped (mapcount 1-0) vir page_remove_rmap(). My quesion is -

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-13 Thread Izik Eidus
ציטוט KAMEZAWA Hiroyuki: Thank you for answers. On Wed, 12 Nov 2008 13:11:12 +0200 Izik Eidus [EMAIL PROTECTED] wrote: Avi Kivity wrote: KAMEZAWA Hiroyuki wrote: Can I make a question ? (I'm working for memory cgroup.) Now, we do charge to anonymous page when - charge(+1

[PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2008-11-16 Thread Izik Eidus
(From v1 to v2 the main change is much more documentation) KSM is a linux driver that allows dynamicly sharing identical memory pages between one or more processes. Unlike tradtional page sharing that is made at the allocation of the memory, ksm do it dynamicly after the memory was created.

[PATCH 1/4] Rmap: Add page_wrprotect() function.

2008-11-16 Thread Izik Eidus
another thread read and writes to/from the first 512bytes of the page. We can lose O_DIRECT reads, the very moment we mark any pte wrprotected... Signed-off-by: Izik Eidus [EMAIL PROTECTED] --- include/linux/rmap.h | 11 mm/rmap.c| 129

[PATCH 2/4] Add replace_page(): change the page pte is pointing to.

2008-11-16 Thread Izik Eidus
of this issue is that newpage cannot be anything but kernel allocated page that is not swappable. Signed-off-by: Izik Eidus [EMAIL PROTECTED] --- include/linux/mm.h |5 +++ mm/memory.c| 80 2 files changed, 85 insertions(+), 0 deletions

[PATCH 3/4] add ksm kernel shared memory driver.

2008-11-16 Thread Izik Eidus
of scanning (so even if there are still more pages to scan, we stop this iteration) __u32 flags: flags to control ksm scaning (right now just ksm_control_flags_run available) Signed-off-by: Izik Eidus

[PATCH 4/4] MMU_NOTIFIRES: add set_pte_at_notify()

2008-11-16 Thread Izik Eidus
. (users of mmu_notifiers that didnt implement the set_pte_at_notify() call back will just recive the mmu_notifier_invalidate_page callback) Signed-off-by: Izik Eidus [EMAIL PROTECTED] --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/mmu.c | 55

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2008-11-20 Thread Izik Eidus
ציטוט Ryota OZAKI: Hi Izik, I've tried your patch set, but ksm doesn't work in my machine. I compiled linux patched with the four patches and configured with KSM and KVM enabled. After boot with the linux, I run two VMs running linux using QEMU with a patch in your mail and started KSM scanner

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2008-11-20 Thread Izik Eidus
ציטוט Izik Eidus: ציטוט Ryota OZAKI: Hi Izik, I've tried your patch set, but ksm doesn't work in my machine. I compiled linux patched with the four patches and configured with KSM and KVM enabled. After boot with the linux, I run two VMs running linux using QEMU with a patch in your mail

[PATCH 0/2] remove kvm vmap usage

2008-12-28 Thread Izik Eidus
Remove the vmap usage from kvm, this is needed both for ksm and get_user_pages != write. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] KVM: introducing kvm_read_guest_virt, kvm_write_guest_virt.

2008-12-28 Thread Izik Eidus
This commit change the name of emulator_read_std into kvm_read_guest_virt, and add new function name kvm_write_guest_virt that allow writing into a guest virtual address. Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/include/asm/kvm_host.h |4 --- arch/x86/kvm/x86.c

[PATCH 2/2] KVM: remove the vmap usage

2008-12-28 Thread Izik Eidus
Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/kvm/x86.c| 62 +--- include/linux/kvm_types.h |3 +- 2 files changed, 14 insertions(+), 51 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c812209..29df564

Re: [PATCH 0/2] remove kvm vmap usage

2008-12-28 Thread Izik Eidus
Izik Eidus wrote: Remove the vmap usage from kvm, this is needed both for ksm and get_user_pages != write. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH] RFC: alias rework

2010-01-25 Thread Izik Eidus
From f94dcd1ccabbcdb51ed7c37c5f58f00a5c1b7eec Mon Sep 17 00:00:00 2001 From: Izik Eidus iei...@redhat.com Date: Mon, 25 Jan 2010 15:49:41 +0200 Subject: [PATCH] RFC: alias rework This patch remove the old way of aliasing inside kvm and move into using aliasing with the same virtual addresses

Re: [PATCH] RFC: alias rework

2010-01-25 Thread Izik Eidus
On Mon, 25 Jan 2010 17:45:53 -0200 Marcelo Tosatti mtosa...@redhat.com wrote: Izik, On Mon, Jan 25, 2010 at 03:53:44PM +0200, Izik Eidus wrote: From f94dcd1ccabbcdb51ed7c37c5f58f00a5c1b7eec Mon Sep 17 00:00:00 2001 From: Izik Eidus iei...@redhat.com Date: Mon, 25 Jan 2010 15:49:41 +0200

Re: [PATCH] RFC: alias rework

2010-01-25 Thread Izik Eidus
On Mon, 25 Jan 2010 18:49:25 -0200 Marcelo Tosatti mtosa...@redhat.com wrote: On Mon, Jan 25, 2010 at 10:40:32PM +0200, Izik Eidus wrote: On Mon, 25 Jan 2010 18:20:39 -0200 Marcelo Tosatti mtosa...@redhat.com wrote: With current code, if a memslot is deleted, access through any aliases

Re: [PATCH] RFC: alias rework

2010-01-26 Thread Izik Eidus
On Tue, 26 Jan 2010 16:14:47 +0200 Avi Kivity a...@redhat.com wrote: On 01/25/2010 10:40 PM, Izik Eidus wrote: Or is this a feature you need? I dont need it (I asked Avi to do something), So he said he want to nuke the aliasing from kvm and keep supporting the old userspace

Re: problems getting KSM running on newer 2.6.32.x kernels

2010-02-09 Thread Izik Eidus
On Tue, 09 Feb 2010 12:51:47 +0200 Avi Kivity a...@redhat.com wrote: On 02/01/2010 12:37 PM, Nikola Ciprich wrote: Hello, it seems to me that after upgrading to some 2.6.32.x release, KSM stopped working for me. I'm not exactly sure which update did this, but enabling KSM doesn't seem

[PATCH 0/1] pci hole remaping

2009-01-10 Thread Izik Eidus
kind of simple, i would send one to qemu later (need to check something first Spice need this, it allow more memory cache (badly needed when runing with multiple screens) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More

[PATCH] kvm-userspace: set pci mem to start at 0xc100000 and vesa to 0xc000000

2009-01-10 Thread Izik Eidus
the maximum memory that would be avaible now is 3giga. Signed-off-by: Izik Eidus iei...@redhat.com --- bios/acpi-dsdt.dsl |2 +- bios/rombios.c |2 +- bios/rombios32.c| 10 +- qemu/hw/pc.c|6 +++--- qemu/hw/vga_int.h

Re: [PATCH] kvm-userspace: set pci mem to start at 0xc100000 and vesa to 0xc000000

2009-01-17 Thread Izik Eidus
Chris Wright wrote: * Izik Eidus (iei...@redhat.com) wrote: This patch make the pci mem region larger (1 giga now). this is needed for pci devices that require large amount of memory such as video cards. for pea guests this patch is not an issue beacuse the guest OS will map the rest

Re: Porting KVM to Mac OS?

2009-01-25 Thread Izik Eidus
Alexander Graf wrote: Hi, On 25.01.2009, at 09:16, Neo Jia wrote: hi, I am thinking if it is possible to port KVM to Mac OS (leopard). Is there anybody doing this already? I've considered doing it, but haven't gotten around to it, due to lack of inspiration. The biggest problem IMHO is

Re: Live memory allocation?

2009-03-26 Thread Izik Eidus
Tomasz Chmielewski wrote: Evert schrieb: Hi all, According to the Wikipedia ( http://en.wikipedia.org/wiki/Comparison_of_platform_virtual_machines ) both VirtualBox VMware server support something called 'Live memory allocation'. Does KVM support this as well? What does this term mean

[PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-03-30 Thread Izik Eidus
of patchs for kvm kernel and kvm-userspace that would allow users of kvm to test ksm with it. The kvm patchs would apply to Avi git tree. Izik Eidus (4): MMU_NOTIFIERS: add set_pte_at_notify() add page_wrprotect(): write protecting page. add replace_page(): change the page pte is pointing

[PATCH 1/4] MMU_NOTIFIERS: add set_pte_at_notify()

2009-03-30 Thread Izik Eidus
. (users of mmu_notifiers that didnt implement the set_pte_at_notify() call back will just recive the mmu_notifier_invalidate_page callback) Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/mmu_notifier.h | 34 ++ mm/memory.c | 10

[PATCH 2/4] add page_wrprotect(): write protecting page.

2009-03-30 Thread Izik Eidus
another thread read and writes to/from the first 512bytes of the page. We can lose O_DIRECT reads, the very moment we mark any pte wrprotected... Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/rmap.h | 11 mm/rmap.c| 139

[PATCH 3/4] add replace_page(): change the page pte is pointing to.

2009-03-30 Thread Izik Eidus
of this issue is that newpage cannot be anything but kernel allocated page that is not swappable. Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/mm.h |5 +++ mm/memory.c| 80 2 files changed, 85 insertions(+), 0 deletions

[PATCH 0/3] kvm support for ksm

2009-03-30 Thread Izik Eidus
apply it against Avi git tree. Izik Eidus (3): kvm: dont hold pagecount reference for mapped sptes pages. kvm: add SPTE_HOST_WRITEABLE flag to the shadow ptes. kvm: add support for change_pte mmu notifiers arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/mmu.c | 89

[PATCH 1/3] kvm: dont hold pagecount reference for mapped sptes pages.

2009-03-30 Thread Izik Eidus
not increase the mapcount when mapping page into shadow page table entry, so when comparing pagecount against mapcount, you have no reliable result.) Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/kvm/mmu.c |7 ++- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86

[PATCH 2/3] kvm: add SPTE_HOST_WRITEABLE flag to the shadow ptes.

2009-03-30 Thread Izik Eidus
this flag notify that the host physical page we are pointing to from the spte is write protected, and therefore we cant change its access to be write unless we run get_user_pages(write = 1). (this is needed for change_pte support in kvm) Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86

[PATCH 3/3] kvm: add support for change_pte mmu notifiers

2009-03-30 Thread Izik Eidus
this is needed for kvm if it want ksm to directly map pages into its shadow page tables. Signed-off-by: Izik Eidus iei...@redhat.com --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/mmu.c | 68 +++ virt/kvm/kvm_main.c

[PATCH 0/2] kvm-userspace ksm support

2009-03-30 Thread Izik Eidus
Apply it against Avi kvm-userspace git tree. Izik Eidus (2): qemu: add ksm support qemu: add ksmctl. qemu/ksm.h | 70 qemu/vl.c | 34 + user/Makefile |6 +++- user/config

[PATCH 1/2] qemu: add ksm support

2009-03-30 Thread Izik Eidus
Signed-off-by: Izik Eidus iei...@redhat.com --- qemu/ksm.h | 70 qemu/vl.c | 34 + 2 files changed, 104 insertions(+), 0 deletions(-) create mode 100644 qemu/ksm.h diff --git a/qemu/ksm.h b/qemu/ksm.h

[PATCH 2/2] qemu: add ksmctl.

2009-03-30 Thread Izik Eidus
userspace tool to control the ksm kernel thread Signed-off-by: Izik Eidus iei...@redhat.com --- user/Makefile |6 +++- user/config-x86-common.mak |2 +- user/ksmctl.c | 69 3 files changed, 75 insertions(+), 2

[PATCH 4/4] add ksm kernel shared memory driver.

2009-03-30 Thread Izik Eidus
: __u32 npages; number of pages to share inside this memory region. __u32 pad; __u64 addr: the begining of the virtual address of this region. KSM_REMOVE_MEMORY_REGION: Remove memory region from ksm. Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/ksm.h| 69

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-03-31 Thread Izik Eidus
KAMEZAWA Hiroyuki wrote: On Tue, 31 Mar 2009 02:59:20 +0300 Izik Eidus iei...@redhat.com wrote: Ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. Pages that are merged are marked as readonly and are COWed when

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-03-31 Thread Izik Eidus
Anthony Liguori wrote: Izik Eidus wrote: Ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. Pages that are merged are marked as readonly and are COWed when any application try to change them. Ksm is used for cases

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-03-31 Thread Izik Eidus
Anthony Liguori wrote: Izik Eidus wrote: I am sending another seires of patchs for kvm kernel and kvm-userspace that would allow users of kvm to test ksm with it. The kvm patchs would apply to Avi git tree. Any reason to not take these through upstream QEMU instead of kvm-userspace

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-04-01 Thread Izik Eidus
KAMEZAWA Hiroyuki wrote: On Tue, 31 Mar 2009 15:21:53 +0300 Izik Eidus iei...@redhat.com wrote: kpage is actually what going to be KsmPage - the shared page... Right now this pages are not swappable..., after ksm will be merged we will make this pages swappable as well

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-04-01 Thread Izik Eidus
Anthony Liguori wrote: Andrea Arcangeli wrote: On Tue, Mar 31, 2009 at 10:54:57AM -0500, Anthony Liguori wrote: You can still disable ksm and simply return ENOSYS for the MADV_ flag. You Anthony, the biggest problem about madvice() is that it is a real system call api, i wouldnt

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-04-02 Thread Izik Eidus
Anthony Liguori wrote: Chris Wright wrote: * Anthony Liguori (anth...@codemonkey.ws) wrote: The ioctl() interface is quite bad for what you're doing. You're telling the kernel extra information about a VA range in userspace. That's what madvise is for. You're tweaking simple

Re: [PATCH 5/4] update ksm userspace interfaces

2009-04-02 Thread Izik Eidus
Chris Wright wrote: * Anthony Liguori (anth...@codemonkey.ws) wrote: Using an interface like madvise() would force the issue to be dealt with properly from the start :-) Yeah, I'm not at all opposed to it. This updates to madvise for register and sysfs for control. madvise issues:

Re: [PATCH 5/4] update ksm userspace interfaces

2009-04-02 Thread Izik Eidus
Chris Wright wrote: * Izik Eidus (iei...@redhat.com) wrote: Is this what we want? How about baby steps... admit that ioctl to control plane is better done via sysfs? Yes -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-04-02 Thread Izik Eidus
Jesper Juhl wrote: Hi, On Tue, 31 Mar 2009, Izik Eidus wrote: KSM is a linux driver that allows dynamicly sharing identical memory pages between one or more processes. Unlike tradtional page sharing that is made at the allocation of the memory, ksm do it dynamicly after the memory

Re: [PATCH 5/4] update ksm userspace interfaces

2009-04-03 Thread Izik Eidus
Gerd Hoffmann wrote: Izik Eidus wrote: The main problem that ksm will face when removing the fd interface is: right now when you register memory into ksm, you open fd, and then ksm do get_task_mm(), we will do mmput when the file will be closed Did you test whenever it really cleans

[PATCH 3/4] add replace_page(): change the page pte is pointing to.

2009-04-04 Thread Izik Eidus
of this issue is that newpage cannot be anything but kernel allocated page that is not swappable. Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/mm.h |5 +++ mm/memory.c| 80 2 files changed, 85 insertions(+), 0 deletions

[PATCH 2/4] add page_wrprotect(): write protecting page.

2009-04-04 Thread Izik Eidus
another thread read and writes to/from the first 512bytes of the page. We can lose O_DIRECT reads, the very moment we mark any pte wrprotected... Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/rmap.h | 11 mm/rmap.c| 139

[PATCH 1/4] MMU_NOTIFIERS: add set_pte_at_notify()

2009-04-04 Thread Izik Eidus
. (users of mmu_notifiers that didnt implement the set_pte_at_notify() call back will just recive the mmu_notifier_invalidate_page callback) Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/mmu_notifier.h | 34 ++ mm/memory.c | 10

[PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-04 Thread Izik Eidus
be done automatically by the system so it sounded extremely attractive when Andrea presented it. I am sending another seires of patchs for kvm kernel and kvm-userspace that would allow users of kvm to test ksm with it. The kvm patchs would apply to Avi git tree. Izik Eidus (4): MMU_NOTIFIERS: add

[PATCH 4/4] add ksm kernel shared memory driver.

2009-04-04 Thread Izik Eidus
of pages to share inside this memory region. __u32 pad; __u64 addr: the begining of the virtual address of this region. __u64 reserved_bits; reserved bits for future usage. KSM_REMOVE_MEMORY_REGION: Remove memory region from ksm. Signed-off-by: Izik Eidus iei...@redhat.com

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-04-06 Thread Izik Eidus
Andrey Panin wrote: On 094, 04 04, 2009 at 05:35:22PM +0300, Izik Eidus wrote: SNIP +static inline u32 calc_checksum(struct page *page) +{ + u32 checksum; + void *addr = kmap_atomic(page, KM_USER0); + checksum = jhash(addr, PAGE_SIZE, 17); Why jhash2() is not used

[PATCH 3/4] add replace_page(): change the page pte is pointing to.

2009-04-08 Thread Izik Eidus
of this issue is that newpage cannot be anything but kernel allocated page that is not swappable. Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/mm.h |5 +++ mm/memory.c| 80 2 files changed, 85 insertions(+), 0 deletions

[PATCH 1/4] MMU_NOTIFIERS: add set_pte_at_notify()

2009-04-08 Thread Izik Eidus
. (users of mmu_notifiers that didnt implement the set_pte_at_notify() call back will just recive the mmu_notifier_invalidate_page callback) Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/mmu_notifier.h | 34 ++ mm/memory.c | 10

[PATCH 0/4] ksm - dynamic page sharing driver for linux v3

2009-04-08 Thread Izik Eidus
kernel and kvm-userspace that would allow users of kvm to test ksm with it. The kvm patchs would apply to Avi git tree. Izik Eidus (4): MMU_NOTIFIERS: add set_pte_at_notify() add page_wrprotect(): write protecting page. add replace_page(): change the page pte is pointing to. add ksm

[PATCH 2/4] add page_wrprotect(): write protecting page.

2009-04-08 Thread Izik Eidus
another thread read and writes to/from the first 512bytes of the page. We can lose O_DIRECT reads, the very moment we mark any pte wrprotected... Signed-off-by: Izik Eidus iei...@redhat.com --- include/linux/rmap.h | 11 mm/rmap.c| 139

[PATCH 4/4] add ksm kernel shared memory driver.

2009-04-08 Thread Izik Eidus
of pages to share inside this memory region. __u32 pad; __u64 addr: the begining of the virtual address of this region. __u64 reserved_bits; reserved bits for future usage. KSM_REMOVE_MEMORY_REGION: Remove memory region from ksm. Signed-off-by: Izik Eidus iei...@redhat.com Signed-off

Re: [PATCH 1/3] kvm: dont hold pagecount reference for mapped sptes pages.

2009-04-09 Thread Izik Eidus
Marcelo Tosatti wrote: On Tue, Mar 31, 2009 at 03:00:02AM +0300, Izik Eidus wrote: When using mmu notifiers, we are allowed to remove the page count reference tooken by get_user_pages to a specific page that is mapped inside the shadow page tables. This is needed so we can balance

Re: [PATCH 1/3] kvm: dont hold pagecount reference for mapped sptes pages.

2009-04-12 Thread Izik Eidus
Izik Eidus wrote: Marcelo Tosatti wrote: On Tue, Mar 31, 2009 at 03:00:02AM +0300, Izik Eidus wrote: When using mmu notifiers, we are allowed to remove the page count reference tooken by get_user_pages to a specific page that is mapped inside the shadow page tables. This is needed so we can

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-04-15 Thread Izik Eidus
Andrew Morton wrote: On Thu, 9 Apr 2009 06:58:41 +0300 Izik Eidus iei...@redhat.com wrote: Confused. In the covering email you indicated that v2 of the patchset had abandoned ioctls and had moved the interface to sysfs. We have abandoned the ioctls that control the ksm behavior (how

  1   2   >