Re: [PATCH 5/5] add ksm kernel shared memory driver.
On Mon, 20 Apr 2009 04:36:06 +0300 Izik Eidus iei...@redhat.com wrote: Ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. Pages that are merged are marked as readonly and are COWed when any application try to change them. Ksm is used for cases where using fork() is not suitable, one of this cases is where the pages of the application keep changing dynamicly and the application cannot know in advance what pages are going to be identical. Ksm works by walking over the memory pages of the applications it scan in order to find identical pages. It uses a two sorted data strctures called stable and unstable trees to find in effective way the identical pages. When ksm finds two identical pages, it marks them as readonly and merges them into single one page, after the pages are marked as readonly and merged into one page, linux will treat this pages as normal copy_on_write pages and will fork them when write access will happen to them. Ksm scan just memory areas that were registred to be scanned by it. ... + copy_user_highpage(kpage, page1, addr1, vma); ... Breaks ppc64 allmodcofnig because that architecture doesn't export its copy_user_page() to modules. Architectures are inconsistent about this. x86 _does_ export it, because it bounces it to the exported copy_page(). So can I ask that you sit down and work out upon which architectures it really makes sense to offer KSM? Disallow the others in Kconfig and arrange for copy_user_highpage() to be available on the allowed architectures? Thanks. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
* Andrew Morton (a...@linux-foundation.org) wrote: Breaks ppc64 allmodcofnig because that architecture doesn't export its copy_user_page() to modules. Things like this and updating to use madvise() I think all point towards s/tristate/bool/. I don't think CONFIG_KSM=M has huge benefit. thanks, -chris -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
Anthony Liguori wrote: Chris Wright wrote: * Andrew Morton (a...@linux-foundation.org) wrote: Breaks ppc64 allmodcofnig because that architecture doesn't export its copy_user_page() to modules. Things like this and updating to use madvise() I think all point towards s/tristate/bool/. I don't think CONFIG_KSM=M has huge benefit. I agree. I am sending in one sec, the madvise patch that will kick it away from being module anyway... Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
Andrew Morton wrote: On Mon, 20 Apr 2009 04:36:06 +0300 Izik Eidus iei...@redhat.com wrote: Ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. Pages that are merged are marked as readonly and are COWed when any application try to change them. Ksm is used for cases where using fork() is not suitable, one of this cases is where the pages of the application keep changing dynamicly and the application cannot know in advance what pages are going to be identical. Ksm works by walking over the memory pages of the applications it scan in order to find identical pages. It uses a two sorted data strctures called stable and unstable trees to find in effective way the identical pages. When ksm finds two identical pages, it marks them as readonly and merges them into single one page, after the pages are marked as readonly and merged into one page, linux will treat this pages as normal copy_on_write pages and will fork them when write access will happen to them. Ksm scan just memory areas that were registred to be scanned by it. ... + copy_user_highpage(kpage, page1, addr1, vma); ... Breaks ppc64 allmodcofnig because that architecture doesn't export its copy_user_page() to modules. Architectures are inconsistent about this. x86 _does_ export it, because it bounces it to the exported copy_page(). So can I ask that you sit down and work out upon which architectures it really makes sense to offer KSM? Disallow the others in Kconfig and arrange for copy_user_highpage() to be available on the allowed architectures? Hi There is some way (script) that i can run that will allow compile this code for every possible arch? (I dont mind to allow it just for archs that support virtualization - x86, ia64, powerpc, s390, but is it the right thing to do ?) Thanks. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
On Thu, May 14, 2009 at 03:15:05AM +0300, Izik Eidus wrote: Hi There is some way (script) that i can run that will allow compile this code for every possible arch? Segher Boessenkool has a tool for builing cross toolchains and the kernel at git://git.infradead.org/users/segher/buildall.git You can save yourself some time (and pain) and use the built toolchains at: http://bakeyournoodle.com/cross If there is any interest I can get these toolchains hosted on a faster machine (say kernel.org) Yours Tony -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
On Tue, 28 Apr 2009 02:12:00 +0300
Izik Eidus iei...@redhat.com wrote:
Andrew Morton wrote:
Breaks sparc64 and probably lots of other architectures:
mm/ksm.c: In function `try_to_merge_two_pages_alloc':
mm/ksm.c:697: error: `_PAGE_RW' undeclared (first use in this
function)
there should be an official arch-independent way of manipulating
vma-vm_page_prot, but I'm not immediately finding it.
Hi,
vm_get_page_prot() will probably do the work.
I will send you patch that fix it,
but first i am waiting for Andrea and Chris to say they are happy
with small changes that i made to the api after conversation i had
with them (about checking if this api is robust enough so we wont
have to change it later)
When i will get their acks, i will send you patch against this
togather with the api (until then it is ok to just leave it only for
x86)
changes are:
1) limiting the number of memory regions registered per file
descriptor
- so while (1){ (ioctl(KSM_REGISTER_MEMORY_REGION()) ) wont omm the
host
2) checking if memory is overlap in registration (more effective to
ignore such cases)
3) allow removing specific memoy regions inside fd.
Thanks.
Hi,
Following patchs change the api to be more robust, the result change of
the api came after conversation i had with Andrea and Chris about how
to make the api as stable as we can,
In addition i hope this patchset fix the cross compilation problems, i
compiled it on itanium (doesnt have _PAGE_RW) and it seems to work.
Thanks.
From 108b720636d1e679e8d5378469fa1220ce1e6963 Mon Sep 17 00:00:00 2001
From: Izik Eidus iei...@redhat.com
Date: Thu, 30 Apr 2009 20:36:57 +0300
Subject: [PATCH 09/13] ksm: limiting the num of mem regions user can register
per fd.
Right now user can open /dev/ksm fd and register unlimited number of
regions, such behavior may allocate unlimited amount of kernel memory
and get the whole host into out of memory situation.
Signed-off-by: Izik Eidus iei...@redhat.com
---
mm/ksm.c | 15 +++
1 files changed, 15 insertions(+), 0 deletions(-)
diff --git a/mm/ksm.c b/mm/ksm.c
index 6165276..d58db6b 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -48,6 +48,9 @@ static int rmap_hash_size;
module_param(rmap_hash_size, int, 0);
MODULE_PARM_DESC(rmap_hash_size, Hash table size for the reverse mapping);
+static int regions_per_fd;
+module_param(regions_per_fd, int, 0);
+
/*
* ksm_mem_slot - hold information for an userspace scanning range
* (the scanning for this region will be from addr untill addr +
@@ -67,6 +70,7 @@ struct ksm_mem_slot {
*/
struct ksm_sma {
struct list_head sma_slots;
+ int nregions;
};
/**
@@ -453,6 +457,11 @@ static int ksm_sma_ioctl_register_memory_region(struct
ksm_sma *ksm_sma,
struct ksm_mem_slot *slot;
int ret = -EPERM;
+ if ((ksm_sma-nregions + 1) regions_per_fd) {
+ ret = -EBUSY;
+ goto out;
+ }
+
slot = kzalloc(sizeof(struct ksm_mem_slot), GFP_KERNEL);
if (!slot) {
ret = -ENOMEM;
@@ -473,6 +482,7 @@ static int ksm_sma_ioctl_register_memory_region(struct
ksm_sma *ksm_sma,
list_add_tail(slot-link, slots);
list_add_tail(slot-sma_link, ksm_sma-sma_slots);
+ ksm_sma-nregions++;
up_write(slots_lock);
return 0;
@@ -511,6 +521,7 @@ static int ksm_sma_ioctl_remove_memory_region(struct
ksm_sma *ksm_sma)
mmput(slot-mm);
list_del(slot-sma_link);
kfree(slot);
+ ksm_sma-nregions--;
}
up_write(slots_lock);
return 0;
@@ -1389,6 +1400,7 @@ static int ksm_dev_ioctl_create_shared_memory_area(void)
}
INIT_LIST_HEAD(ksm_sma-sma_slots);
+ ksm_sma-nregions = 0;
fd = anon_inode_getfd(ksm-sma, ksm_sma_fops, ksm_sma, 0);
if (fd 0)
@@ -1631,6 +1643,9 @@ static int __init ksm_init(void)
if (r)
goto out_free1;
+ if (!regions_per_fd)
+ regions_per_fd = 1024;
+
ksm_thread = kthread_run(ksm_scan_thread, NULL, kksmd);
if (IS_ERR(ksm_thread)) {
printk(KERN_ERR ksm: creating kthread failed\n);
--
1.5.6.5
From f24a9aa8c049c951a33613909951d115be5f84cd Mon Sep 17 00:00:00 2001
From: Izik Eidus iei...@redhat.com
Date: Thu, 30 Apr 2009 20:37:17 +0300
Subject: [PATCH 10/13] ksm: dont allow overlap memory addresses registrations.
subjects say it all.
Signed-off-by: Izik Eidus iei...@redhat.com
---
mm/ksm.c | 58 ++
1 files changed, 54 insertions(+), 4 deletions(-)
diff --git a/mm/ksm.c b/mm/ksm.c
index d58db6b..982dfff 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -451,21 +451,71 @@ static void remove_page_from_tree(struct mm_struct *mm,
remove_rmap_item_from_tree(rmap_item);
}
+static inline int is_intersecting_address(unsigned long addr,
+
Re: [PATCH 5/5] add ksm kernel shared memory driver.
On Thu, 30 Apr 2009 20:46:24 +0300 Izik Eidus iei...@redhat.com wrote: Following patchs change the api to be more robust, the result change of the api came after conversation i had with Andrea and Chris about how to make the api as stable as we can, In addition i hope this patchset fix the cross compilation problems, i compiled it on itanium (doesnt have _PAGE_RW) and it seems to work. eek, please don't send multiple patches per email - it's surprisingly disruptive to everything. What is the relationship between these patches and the ones I merged the other day? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
On Mon, 20 Apr 2009 04:36:06 +0300 Izik Eidus iei...@redhat.com wrote: Ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. Pages that are merged are marked as readonly and are COWed when any application try to change them. Breaks sparc64 and probably lots of other architectures: mm/ksm.c: In function `try_to_merge_two_pages_alloc': mm/ksm.c:697: error: `_PAGE_RW' undeclared (first use in this function) there should be an official arch-independent way of manipulating vma-vm_page_prot, but I'm not immediately finding it. An alternative (and quite inferior) fix would be to disable ksm on architectures which don't implement _PAGE_RW. That's most of them. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
Andrew Morton wrote:
On Mon, 20 Apr 2009 04:36:06 +0300
Izik Eidus iei...@redhat.com wrote:
Ksm is driver that allow merging identical pages between one or more
applications in way unvisible to the application that use it.
Pages that are merged are marked as readonly and are COWed when any
application try to change them.
Breaks sparc64 and probably lots of other architectures:
mm/ksm.c: In function `try_to_merge_two_pages_alloc':
mm/ksm.c:697: error: `_PAGE_RW' undeclared (first use in this function)
there should be an official arch-independent way of manipulating
vma-vm_page_prot, but I'm not immediately finding it.
Hi,
vm_get_page_prot() will probably do the work.
I will send you patch that fix it,
but first i am waiting for Andrea and Chris to say they are happy with
small changes that i made to the api after conversation i had with them
(about checking if this api is robust enough so we wont have to change
it later)
When i will get their acks, i will send you patch against this togather
with the api (until then it is ok to just leave it only for x86)
changes are:
1) limiting the number of memory regions registered per file descriptor
- so while (1){ (ioctl(KSM_REGISTER_MEMORY_REGION()) ) wont omm the host
2) checking if memory is overlap in registration (more effective to
ignore such cases)
3) allow removing specific memoy regions inside fd.
Thanks.
An alternative (and quite inferior) fix would be to disable ksm on
architectures which don't implement _PAGE_RW. That's most of them.
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
The minor number you are using already belongs to another project. 10,234 is free but it would be good to know what device naming is proposed. I imagine other folks would like to know why you aren't using sysfs or similar or extending /dev/kvm ? Alan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
Alan Cox wrote: The minor number you are using already belongs to another project. 10,234 is free but it would be good to know what device naming is proposed. I imagine other folks would like to know why you aren't using sysfs or similar or extending /dev/kvm ? ksm was deliberately made independent of kvm. While there may or may not be uses of ksm without kvm (you could run ordinary qemu, but no one would do this in a production deployment), keeping them separate helps avoid unnecessary interdependencies. For example all tlb flushes are mediated through mmu notifiers instead of ksm hooking directly into kvm. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 5/5] add ksm kernel shared memory driver.
Avi Kivity wrote: Alan Cox wrote: The minor number you are using already belongs to another project. 10,234 is free but it would be good to know what device naming is proposed. I imagine other folks would like to know why you aren't using sysfs or similar or extending /dev/kvm ? ksm was deliberately made independent of kvm. While there may or may not be uses of ksm without kvm (you could run ordinary qemu, but no one would do this in a production deployment), keeping them separate helps avoid unnecessary interdependencies. For example all tlb flushes are mediated through mmu notifiers instead of ksm hooking directly into kvm. Yes, beside, I do use sysfs for controlling the ksm behavior, Ioctls are provided as easier way for application to register its memory. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 5/5] add ksm kernel shared memory driver.
Ksm is driver that allow merging identical pages between one or more
applications in way unvisible to the application that use it.
Pages that are merged are marked as readonly and are COWed when any
application try to change them.
Ksm is used for cases where using fork() is not suitable,
one of this cases is where the pages of the application keep changing
dynamicly and the application cannot know in advance what pages are
going to be identical.
Ksm works by walking over the memory pages of the applications it
scan in order to find identical pages.
It uses a two sorted data strctures called stable and unstable trees
to find in effective way the identical pages.
When ksm finds two identical pages, it marks them as readonly and merges
them into single one page,
after the pages are marked as readonly and merged into one page, linux
will treat this pages as normal copy_on_write pages and will fork them
when write access will happen to them.
Ksm scan just memory areas that were registred to be scanned by it.
Signed-off-by: Izik Eidus iei...@redhat.com
Signed-off-by: Chris Wright chr...@redhat.com
Signed-off-by: Andrea Arcangeli aarca...@redhat.com
---
include/linux/ksm.h| 48 ++
include/linux/miscdevice.h |1 +
mm/Kconfig |6 +
mm/Makefile|1 +
mm/ksm.c | 1675
5 files changed, 1731 insertions(+), 0 deletions(-)
create mode 100644 include/linux/ksm.h
create mode 100644 mm/ksm.c
diff --git a/include/linux/ksm.h b/include/linux/ksm.h
new file mode 100644
index 000..2c11e9a
--- /dev/null
+++ b/include/linux/ksm.h
@@ -0,0 +1,48 @@
+#ifndef __LINUX_KSM_H
+#define __LINUX_KSM_H
+
+/*
+ * Userspace interface for /dev/ksm - kvm shared memory
+ */
+
+#include linux/types.h
+#include linux/ioctl.h
+
+#include asm/types.h
+
+#define KSM_API_VERSION 1
+
+#define ksm_control_flags_run 1
+
+/* for KSM_REGISTER_MEMORY_REGION */
+struct ksm_memory_region {
+ __u32 npages; /* number of pages to share */
+ __u32 pad;
+ __u64 addr; /* the begining of the virtual address */
+__u64 reserved_bits;
+};
+
+#define KSMIO 0xAB
+
+/* ioctls for /dev/ksm */
+
+#define KSM_GET_API_VERSION _IO(KSMIO, 0x00)
+/*
+ * KSM_CREATE_SHARED_MEMORY_AREA - create the shared memory reagion fd
+ */
+#define KSM_CREATE_SHARED_MEMORY_AREA_IO(KSMIO, 0x01) /* return SMA fd */
+
+/* ioctls for SMA fds */
+
+/*
+ * KSM_REGISTER_MEMORY_REGION - register virtual address memory area to be
+ * scanned by kvm.
+ */
+#define KSM_REGISTER_MEMORY_REGION _IOW(KSMIO, 0x20,\
+ struct ksm_memory_region)
+/*
+ * KSM_REMOVE_MEMORY_REGION - remove virtual address memory area from ksm.
+ */
+#define KSM_REMOVE_MEMORY_REGION _IO(KSMIO, 0x21)
+
+#endif
diff --git a/include/linux/miscdevice.h b/include/linux/miscdevice.h
index beb6ec9..297c0bb 100644
--- a/include/linux/miscdevice.h
+++ b/include/linux/miscdevice.h
@@ -30,6 +30,7 @@
#define HPET_MINOR 228
#define FUSE_MINOR 229
#define KVM_MINOR 232
+#define KSM_MINOR 233
#define MISC_DYNAMIC_MINOR 255
struct device;
diff --git a/mm/Kconfig b/mm/Kconfig
index 57971d2..fb8ac63 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -225,3 +225,9 @@ config HAVE_MLOCKED_PAGE_BIT
config MMU_NOTIFIER
bool
+
+config KSM
+ tristate Enable KSM for page sharing
+ help
+ Enable the KSM kernel module to allow page sharing of equal pages
+ among different tasks.
diff --git a/mm/Makefile b/mm/Makefile
index ec73c68..b885513 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -24,6 +24,7 @@ obj-$(CONFIG_SPARSEMEM_VMEMMAP) += sparse-vmemmap.o
obj-$(CONFIG_TMPFS_POSIX_ACL) += shmem_acl.o
obj-$(CONFIG_SLOB) += slob.o
obj-$(CONFIG_MMU_NOTIFIER) += mmu_notifier.o
+obj-$(CONFIG_KSM) += ksm.o
obj-$(CONFIG_PAGE_POISONING) += debug-pagealloc.o
obj-$(CONFIG_SLAB) += slab.o
obj-$(CONFIG_SLUB) += slub.o
diff --git a/mm/ksm.c b/mm/ksm.c
new file mode 100644
index 000..7fd4158
--- /dev/null
+++ b/mm/ksm.c
@@ -0,0 +1,1675 @@
+/*
+ * Memory merging driver for Linux
+ *
+ * This module enables dynamic sharing of identical pages found in different
+ * memory areas, even if they are not shared by fork()
+ *
+ * Copyright (C) 2008 Red Hat, Inc.
+ * Authors:
+ * Izik Eidus
+ * Andrea Arcangeli
+ * Chris Wright
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.
+ */
+
+#include linux/module.h
+#include linux/errno.h
+#include linux/mm.h
+#include linux/fs.h
+#include linux/miscdevice.h
+#include linux/vmalloc.h
+#include linux/file.h
+#include linux/mman.h
+#include linux/sched.h
+#include linux/rwsem.h
+#include linux/pagemap.h
+#include linux/sched.h
+#include linux/rmap.h
+#include linux/spinlock.h
+#include linux/jhash.h
+#include linux/delay.h
+#include linux/kthread.h
+#include linux/wait.h
