Re: [PATCH 5/4] update ksm userspace interfaces

2009-04-03 Thread Izik Eidus
Gerd Hoffmann wrote: Izik Eidus wrote: The main problem that ksm will face when removing the fd interface is: right now when you register memory into ksm, you open fd, and then ksm do get_task_mm(), we will do mmput when the file will be closed Did you test whenever it really cleans

[PATCH 3/4] add replace_page(): change the page pte is pointing to.

2009-04-04 Thread Izik Eidus
of this issue is that newpage cannot be anything but kernel allocated page that is not swappable. Signed-off-by: Izik Eidus --- include/linux/mm.h |5 +++ mm/memory.c| 80 2 files changed, 85 insertions(+), 0 deletions(-) diff --

[PATCH 2/4] add page_wrprotect(): write protecting page.

2009-04-04 Thread Izik Eidus
while another thread read and writes to/from the first 512bytes of the page. We can lose O_DIRECT reads, the very moment we mark any pte wrprotected..." Signed-off-by: Izik Eidus --- include/linux/rmap.h | 11 mm/rmap.c| 139 +

[PATCH 1/4] MMU_NOTIFIERS: add set_pte_at_notify()

2009-04-04 Thread Izik Eidus
. (users of mmu_notifiers that didnt implement the set_pte_at_notify() call back will just recive the mmu_notifier_invalidate_page callback) Signed-off-by: Izik Eidus --- include/linux/mmu_notifier.h | 34 ++ mm/memory.c | 10 -- mm

[PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-04 Thread Izik Eidus
en fork it N times, to have the data shared. With KSM this would be done automatically by the system so it sounded extremely attractive when Andrea presented it." I am sending another seires of patchs for kvm kernel and kvm-userspace that would allow users of kvm to test ksm with

[PATCH 4/4] add ksm kernel shared memory driver.

2009-04-04 Thread Izik Eidus
pages to share inside this memory region. __u32 pad; __u64 addr: the begining of the virtual address of this region. __u64 reserved_bits; reserved bits for future usage. KSM_REMOVE_MEMORY_REGION: Remove memory region from ksm. Signed-off-by: Izik Eidus --- include/linux/ksm.h

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-04-06 Thread Izik Eidus
Andrey Panin wrote: On 094, 04 04, 2009 at 05:35:22PM +0300, Izik Eidus wrote: +static inline u32 calc_checksum(struct page *page) +{ + u32 checksum; + void *addr = kmap_atomic(page, KM_USER0); + checksum = jhash(addr, PAGE_SIZE, 17); Why jhash2() is not used here

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-06 Thread Izik Eidus
e number of pages that have been shared, you can run top as well) On Sat, Apr 04, 2009 at 05:35:18PM +0300, Izik Eidus wrote: From v1 to v2: 1)Fixed security issue found by Chris Wright: Ksm was checking if page is a shared page by running !PageAnon. Beacuse that Ksm scan only anony

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-06 Thread Izik Eidus
Nick Piggin wrote: On Sunday 05 April 2009 01:35:18 Izik Eidus wrote: This driver is very useful for KVM as in cases of runing multiple guests operation system of the same type. (For desktop work loads we have achived more than x2 memory overcommit (more like x3)) Interesting that it

[PATCH 3/4] add replace_page(): change the page pte is pointing to.

2009-04-08 Thread Izik Eidus
of this issue is that newpage cannot be anything but kernel allocated page that is not swappable. Signed-off-by: Izik Eidus --- include/linux/mm.h |5 +++ mm/memory.c| 80 2 files changed, 85 insertions(+), 0 deletions(-) diff --

[PATCH 1/4] MMU_NOTIFIERS: add set_pte_at_notify()

2009-04-08 Thread Izik Eidus
. (users of mmu_notifiers that didnt implement the set_pte_at_notify() call back will just recive the mmu_notifier_invalidate_page callback) Signed-off-by: Izik Eidus --- include/linux/mmu_notifier.h | 34 ++ mm/memory.c | 10 -- mm

[PATCH 0/4] ksm - dynamic page sharing driver for linux v3

2009-04-08 Thread Izik Eidus
tractive when Andrea presented it." I am sending another seires of patchs for kvm kernel and kvm-userspace that would allow users of kvm to test ksm with it. The kvm patchs would apply to Avi git tree. Izik Eidus (4): MMU_NOTIFIERS: add set_pte_at_notify() add page_wrprotect(): write p

[PATCH 2/4] add page_wrprotect(): write protecting page.

2009-04-08 Thread Izik Eidus
while another thread read and writes to/from the first 512bytes of the page. We can lose O_DIRECT reads, the very moment we mark any pte wrprotected..." Signed-off-by: Izik Eidus --- include/linux/rmap.h | 11 mm/rmap.c| 139 +

[PATCH 4/4] add ksm kernel shared memory driver.

2009-04-08 Thread Izik Eidus
pages to share inside this memory region. __u32 pad; __u64 addr: the begining of the virtual address of this region. __u64 reserved_bits; reserved bits for future usage. KSM_REMOVE_MEMORY_REGION: Remove memory region from ksm. Signed-off-by: Izik Eidus Signed-off-by: Chris Wright

Re: [PATCH 1/3] kvm: dont hold pagecount reference for mapped sptes pages.

2009-04-09 Thread Izik Eidus
Marcelo Tosatti wrote: On Tue, Mar 31, 2009 at 03:00:02AM +0300, Izik Eidus wrote: When using mmu notifiers, we are allowed to remove the page count reference tooken by get_user_pages to a specific page that is mapped inside the shadow page tables. This is needed so we can balance the

Re: [PATCH 2/3] kvm: add SPTE_HOST_WRITEABLE flag to the shadow ptes.

2009-04-09 Thread Izik Eidus
Marcelo Tosatti wrote: On Tue, Mar 31, 2009 at 03:00:03AM +0300, Izik Eidus wrote: this flag notify that the host physical page we are pointing to from the spte is write protected, and therefore we cant change its access to be write unless we run get_user_pages(write = 1). (this is needed

Re: [PATCH 1/3] kvm: dont hold pagecount reference for mapped sptes pages.

2009-04-12 Thread Izik Eidus
Izik Eidus wrote: Marcelo Tosatti wrote: On Tue, Mar 31, 2009 at 03:00:02AM +0300, Izik Eidus wrote: When using mmu notifiers, we are allowed to remove the page count reference tooken by get_user_pages to a specific page that is mapped inside the shadow page tables. This is needed so we can

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-04-15 Thread Izik Eidus
Andrew Morton wrote: On Thu, 9 Apr 2009 06:58:41 +0300 Izik Eidus wrote: Confused. In the covering email you indicated that v2 of the patchset had abandoned ioctls and had moved the interface to sysfs. We have abandoned the ioctls that control the ksm behavior (how much cpu it take

Re: [PATCH 3/4] add replace_page(): change the page pte is pointing to.

2009-04-15 Thread Izik Eidus
Andrea Arcangeli wrote: On Tue, Apr 14, 2009 at 03:09:25PM -0700, Andrew Morton wrote: On Thu, 9 Apr 2009 06:58:40 +0300 Izik Eidus wrote: replace_page() allow changing the mapping of pte from one physical page into diffrent physical page. At a high level, this is very

Re: [PATCH 4/4] add ksm kernel shared memory driver.

2009-04-15 Thread Izik Eidus
Jeremy Fitzhardinge wrote: Andrew Morton wrote: +static pte_t *get_pte(struct mm_struct *mm, unsigned long addr) +{ +pgd_t *pgd; +pud_t *pud; +pmd_t *pmd; +pte_t *ptep = NULL; + +pgd = pgd_offset(mm, addr); +if (!pgd_present(*pgd)) +goto out; + +pud = pud_offs

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v3

2009-04-16 Thread Izik Eidus
Nick Piggin wrote: On Wednesday 15 April 2009 08:09:03 Andrew Morton wrote: On Thu, 9 Apr 2009 06:58:37 +0300 Izik Eidus wrote: KSM is a linux driver that allows dynamicly sharing identical memory pages between one or more processes. Generally looks OK to me. But that

[PATCH 0/5] ksm - dynamic page sharing driver for linux v4

2009-04-19 Thread Izik Eidus
gt; /sys/kernel/mm/ksm/run (Or any other numbers...) Ok, you are ready :-) (Just remember, memory that is swapped, isnt scanned by ksm until it come back to memory, so dont try to raise alot of VMS togather) Thanks. Izik Eidus (5): MMU_NOTIFIERS: add set_pte_at_notify() add get_pte(): hel

[PATCH 1/5] MMU_NOTIFIERS: add set_pte_at_notify()

2009-04-19 Thread Izik Eidus
. (users of mmu_notifiers that didnt implement the set_pte_at_notify() call back will just recive the mmu_notifier_invalidate_page callback) Signed-off-by: Izik Eidus --- include/linux/mmu_notifier.h | 34 ++ mm/memory.c | 10 -- mm

[PATCH 2/5] add get_pte(): helper function: fetching pte for va

2009-04-19 Thread Izik Eidus
get_pte() receive mm_struct of a task, and a virtual address and return the pte corresponding to it. this function return NULL in case it couldnt fetch the pte. Signed-off-by: Izik Eidus --- include/linux/mm.h | 24 1 files changed, 24 insertions(+), 0 deletions

[PATCH 4/5] add replace_page(): change the page pte is pointing to.

2009-04-19 Thread Izik Eidus
of this issue is that newpage cannot be anything but kernel allocated page that is not swappable. Signed-off-by: Izik Eidus --- include/linux/mm.h |5 +++ mm/memory.c| 80 2 files changed, 85 insertions(+), 0 deletions(-) diff --

[PATCH 5/5] add ksm kernel shared memory driver.

2009-04-19 Thread Izik Eidus
and will fork them when write access will happen to them. Ksm scan just memory areas that were registred to be scanned by it. Signed-off-by: Izik Eidus Signed-off-by: Chris Wright Signed-off-by: Andrea Arcangeli --- include/linux/ksm.h| 48 ++ include/linux/miscdevice.h |1 + mm

[PATCH 3/5] add page_wrprotect(): write protecting page.

2009-04-19 Thread Izik Eidus
while another thread read and writes to/from the first 512bytes of the page. We can lose O_DIRECT reads, the very moment we mark any pte wrprotected..." Signed-off-by: Izik Eidus --- include/linux/rmap.h | 11 mm/rmap.c| 139 +

Re: [PATCH 5/5] add ksm kernel shared memory driver.

2009-04-20 Thread Izik Eidus
Avi Kivity wrote: Alan Cox wrote: The minor number you are using already belongs to another project. 10,234 is free but it would be good to know what device naming is proposed. I imagine other folks would like to know why you aren't using sysfs or similar or extending /dev/kvm ? ksm was de

Re: [PATCH 5/5] add ksm kernel shared memory driver.

2009-04-27 Thread Izik Eidus
Andrew Morton wrote: On Mon, 20 Apr 2009 04:36:06 +0300 Izik Eidus wrote: Ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. Pages that are merged are marked as readonly and are COWed when any application try

Re: [PATCH 5/5] add ksm kernel shared memory driver.

2009-04-30 Thread Izik Eidus
On Tue, 28 Apr 2009 02:12:00 +0300 Izik Eidus wrote: > Andrew Morton wrote: > > Breaks sparc64 and probably lots of other architectures: > > > > mm/ksm.c: In function `try_to_merge_two_pages_alloc': > > mm/ksm.c:697: error: `_PAGE_RW' undeclared (first us

Re: [PATCH 5/5] add ksm kernel shared memory driver.

2009-05-13 Thread Izik Eidus
Anthony Liguori wrote: Chris Wright wrote: * Andrew Morton (a...@linux-foundation.org) wrote: Breaks ppc64 allmodcofnig because that architecture doesn't export its copy_user_page() to modules. Things like this and updating to use madvise() I think all point towards s/tristate/bool/.

Re: [PATCH 5/5] add ksm kernel shared memory driver.

2009-05-13 Thread Izik Eidus
Andrew Morton wrote: On Mon, 20 Apr 2009 04:36:06 +0300 Izik Eidus wrote: Ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. Pages that are merged are marked as readonly and are COWed when any application try

Re: problems getting KSM running on newer 2.6.32.x kernels

2010-02-09 Thread Izik Eidus
On Tue, 09 Feb 2010 12:51:47 +0200 Avi Kivity wrote: > On 02/01/2010 12:37 PM, Nikola Ciprich wrote: > > Hello, > > it seems to me that after upgrading to some 2.6.32.x release, KSM stopped > > working > > for me. I'm not exactly sure which update did this, but enabling KSM doesn't > > seem to d

[PATCH] fix migration with big mem guests

2010-04-04 Thread Izik Eidus
ode already have timer that is set, so it like it make the most sense to use it... If anyone have any better idea how to solve this issue, I will be very happy to hear. Thanks. >From 2d9c25f1fee61f50cb130769c3779707a6ef90d9 Mon Sep 17 00:00:00 2001 From: Izik Eidus Date: Mon, 5 Apr 2010 02:05:09

[PATCH] RFC: alias rework

2010-01-25 Thread Izik Eidus
>From f94dcd1ccabbcdb51ed7c37c5f58f00a5c1b7eec Mon Sep 17 00:00:00 2001 From: Izik Eidus Date: Mon, 25 Jan 2010 15:49:41 +0200 Subject: [PATCH] RFC: alias rework This patch remove the old way of aliasing inside kvm and move into using aliasing with the same virtual addresses This patch

Re: [PATCH] RFC: alias rework

2010-01-25 Thread Izik Eidus
On Mon, 25 Jan 2010 17:45:53 -0200 Marcelo Tosatti wrote: > Izik, > > On Mon, Jan 25, 2010 at 03:53:44PM +0200, Izik Eidus wrote: > > >From f94dcd1ccabbcdb51ed7c37c5f58f00a5c1b7eec Mon Sep 17 00:00:00 2001 > > From: Izik Eidus > > Date: Mon, 25 Jan 2010 15:49:41

Re: [PATCH] RFC: alias rework

2010-01-25 Thread Izik Eidus
On Mon, 25 Jan 2010 18:20:39 -0200 Marcelo Tosatti wrote: > With current code, if a memslot is deleted, access through any aliases > that use it will fail (BTW it looks this is not properly handled, but > thats a separate problem). Yea I had some still open concerns about this code (this why I

Re: [PATCH] RFC: alias rework

2010-01-25 Thread Izik Eidus
On Mon, 25 Jan 2010 18:49:25 -0200 Marcelo Tosatti wrote: > On Mon, Jan 25, 2010 at 10:40:32PM +0200, Izik Eidus wrote: > > On Mon, 25 Jan 2010 18:20:39 -0200 > > Marcelo Tosatti wrote: > > > > > With current code, if a memslot is deleted, access through any alias

Re: [PATCH] RFC: alias rework

2010-01-26 Thread Izik Eidus
On Tue, 26 Jan 2010 16:14:47 +0200 Avi Kivity wrote: > On 01/25/2010 10:40 PM, Izik Eidus wrote: > > > >> Or is this a feature you need? > >> > > > > I dont need it (I asked Avi to do something), So he said he want to nuke > > the alia

[PATCH 0/2] remove kvm vmap usage

2008-12-28 Thread Izik Eidus
Remove the vmap usage from kvm, this is needed both for ksm and get_user_pages != write. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] KVM: introducing kvm_read_guest_virt, kvm_write_guest_virt.

2008-12-28 Thread Izik Eidus
This commit change the name of emulator_read_std into kvm_read_guest_virt, and add new function name kvm_write_guest_virt that allow writing into a guest virtual address. Signed-off-by: Izik Eidus --- arch/x86/include/asm/kvm_host.h |4 --- arch/x86/kvm/x86.c | 56

[PATCH 2/2] KVM: remove the vmap usage

2008-12-28 Thread Izik Eidus
Signed-off-by: Izik Eidus --- arch/x86/kvm/x86.c| 62 +--- include/linux/kvm_types.h |3 +- 2 files changed, 14 insertions(+), 51 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c812209..29df564 100644 --- a/arch/x86

Re: [PATCH 0/2] remove kvm vmap usage

2008-12-28 Thread Izik Eidus
Izik Eidus wrote: Remove the vmap usage from kvm, this is needed both for ksm and get_user_pages != write. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo

[PATCH 0/1] pci hole remaping

2009-01-10 Thread Izik Eidus
kind of simple, i would send one to qemu later (need to check something first Spice need this, it allow more memory cache (badly needed when runing with multiple screens) -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More m

[PATCH] kvm-userspace: set pci mem to start at 0xc100000 and vesa to 0xc000000

2009-01-10 Thread Izik Eidus
maximum memory that would be avaible now is 3giga. Signed-off-by: Izik Eidus --- bios/acpi-dsdt.dsl |2 +- bios/rombios.c |2 +- bios/rombios32.c| 10 +- qemu/hw/pc.c|6 +++--- qemu/hw/vga_int.h |2 +- qemu/hw

Re: [PATCH] kvm-userspace: set pci mem to start at 0xc100000 and vesa to 0xc000000

2009-01-17 Thread Izik Eidus
Chris Wright wrote: * Izik Eidus (iei...@redhat.com) wrote: This patch make the pci mem region larger (1 giga now). this is needed for pci devices that require large amount of memory such as video cards. for pea guests this patch is not an issue beacuse the guest OS will map the rest of the

Re: Porting KVM to Mac OS?

2009-01-25 Thread Izik Eidus
Alexander Graf wrote: Hi, On 25.01.2009, at 09:16, Neo Jia wrote: hi, I am thinking if it is possible to port KVM to Mac OS (leopard). Is there anybody doing this already? I've considered doing it, but haven't gotten around to it, due to lack of inspiration. The biggest problem IMHO is the

Re: missing kvm smp tlb flush in invlpg

2009-03-15 Thread Izik Eidus
Marcelo Tosatti wrote: On Thu, Mar 12, 2009 at 06:18:43PM +0100, Andrea Arcangeli wrote: From: Andrea Arcangeli While looking at invlpg out of sync code with Izik I think I noticed a missing smp tlb flush here. Without this the other cpu can still write to a freed host physical page. tlb sm

Re: What happens if I use get_user_pages()

2008-07-08 Thread Izik Eidus
ציטוט Arn: Hi, If I use the kernel function "get_user_pages()" within a guest kernel module, will this ensure that the guests pages are never swapped out by the host ? Will it lock these pages in memory ? (Has anyone tried this ? ) Thanks Arn -- To unsubscribe from this list: send the line "un

Re: What happens if I use get_user_pages()

2008-07-08 Thread Izik Eidus
ציטוט Arn: If I use the kernel function "get_user_pages()" within a guest kernel module, will this ensure that the guests pages are never swapped out by the host ? Will it lock these pages in memory ? (Has anyone tried this ? ) As Izik said, the answer is no. Is this the desirable outcome

Re: KSM Algorithm

2008-07-10 Thread Izik Eidus
ציטוט Sukanto Ghosh: Can anyone answer these queries regarding KSM ? How does KSM offers its services through the /dev/ksm device ? ioctls Are every guest pages scanned in KVM while using KSM or page-scanning and sharing is triggered on some event (low memory, etc) ? no, the scanning r

Re: KSM Algorithm

2008-07-10 Thread Izik Eidus
ציטוט Sukanto Ghosh: Are every guest pages scanned in KVM while using KSM or page-scanning and sharing is triggered on some event (low memory, etc) ? no, the scanning run all the time (with number pages to scan and sleep parameters...) Does the scanner(s) run as separate kernel th

Re: KSM Algorithm

2008-07-10 Thread Izik Eidus
ציטוט Sukanto Ghosh: Doesn't KSM notifies KVM about the shared pages so that KVM can update its sptes accordingly or is it done by KSM itself ? when using kvm, mmu notifiers is a must for ksm, (mmu notifiers update kvm about the changes in the host page table) What about KSM being

Re: KSM Algorithm

2008-07-10 Thread Izik Eidus
ציטוט Sukanto Ghosh: One more query, what if multiple processes call ioctl KSM_CREATE_SCAN ? Will there be multiple scanners ? yes Consider a scenario where two processes A & B separately call KSM_CREATE_SCAN and then start registering some memory pages/areas via KSM_CREATE_SHARED_MEMORY_AR

<    1   2