Re: [PATCH 3/4] add ksm kernel shared memory driver.

2008-12-04 Thread Alan Cox
Taken off list Hmmm, list would like to know :-). That would be my choice too but unfortunately I can't do that -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/4] add ksm kernel shared memory driver.

2008-12-03 Thread Pavel Machek
On Tue 2008-12-02 22:10:29, Alan Cox wrote: On Tue, 2 Dec 2008 13:24:11 -0800 Chris Wright [EMAIL PROTECTED] wrote: * Alan Cox ([EMAIL PROTECTED]) wrote: On Tue, 2 Dec 2008 10:07:24 -0800 Chris Wright [EMAIL PROTECTED] wrote: * Alan Cox ([EMAIL PROTECTED]) wrote: + r =

Re: [PATCH 3/4] add ksm kernel shared memory driver.

2008-12-02 Thread Chris Wright
* Alan Cox ([EMAIL PROTECTED]) wrote: + r = !memcmp(old_digest, sha1_item-sha1val, SHA1_DIGEST_SIZE); + mutex_unlock(sha1_lock); + if (r) { + char *old_addr, *new_addr; + old_addr = kmap_atomic(oldpage, KM_USER0); + new_addr = kmap_atomic(newpage,

Re: [PATCH 3/4] add ksm kernel shared memory driver.

2008-12-02 Thread Alan Cox
On Tue, 2 Dec 2008 10:07:24 -0800 Chris Wright [EMAIL PROTECTED] wrote: * Alan Cox ([EMAIL PROTECTED]) wrote: + r = !memcmp(old_digest, sha1_item-sha1val, SHA1_DIGEST_SIZE); + mutex_unlock(sha1_lock); + if (r) { + char *old_addr, *new_addr; + old_addr =

Re: [PATCH 3/4] add ksm kernel shared memory driver.

2008-12-02 Thread Chris Wright
* Alan Cox ([EMAIL PROTECTED]) wrote: On Tue, 2 Dec 2008 10:07:24 -0800 Chris Wright [EMAIL PROTECTED] wrote: * Alan Cox ([EMAIL PROTECTED]) wrote: + r = !memcmp(old_digest, sha1_item-sha1val, SHA1_DIGEST_SIZE); + mutex_unlock(sha1_lock); + if (r) { +

Re: [PATCH 3/4] add ksm kernel shared memory driver.

2008-12-02 Thread Jonathan Corbet
On Tue, 2 Dec 2008 13:24:11 -0800 Chris Wright [EMAIL PROTECTED] wrote: Using current known techniques. A random collision is just as bad news. And, just to clarify, your concern would extend to any digest based comparison? Or are you specifically concerned about sha1? Wouldn't this

Re: [PATCH 3/4] add ksm kernel shared memory driver.

2008-12-02 Thread Alan Cox
On Tue, 2 Dec 2008 13:24:11 -0800 Chris Wright [EMAIL PROTECTED] wrote: * Alan Cox ([EMAIL PROTECTED]) wrote: On Tue, 2 Dec 2008 10:07:24 -0800 Chris Wright [EMAIL PROTECTED] wrote: * Alan Cox ([EMAIL PROTECTED]) wrote: + r = !memcmp(old_digest, sha1_item-sha1val,

Re: [PATCH 3/4] add ksm kernel shared memory driver.

2008-11-28 Thread Alan Cox
+ r = !memcmp(old_digest, sha1_item-sha1val, SHA1_DIGEST_SIZE); + mutex_unlock(sha1_lock); + if (r) { + char *old_addr, *new_addr; + old_addr = kmap_atomic(oldpage, KM_USER0); + new_addr = kmap_atomic(newpage, KM_USER1); + r =

[PATCH 3/4] add ksm kernel shared memory driver.

2008-11-16 Thread Izik Eidus
Ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. Pages that are merged are marked as readonly and are COWed when any application try to change them. Ksm is used for cases where using fork() is not suitable, one of

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-12 Thread Eric Rannaud
On Tue, 2008-11-11 at 17:40 -0500, [EMAIL PROTECTED] wrote: On Tue, 11 Nov 2008 15:03:45 MST, Jonathan Corbet said: Seems reasonably sane to me - only doing the first 128 bytes rather than a full 4K page is some 32 times faster. Yes, you'll have the *occasional* case where two pages were

[PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
From: Izik Eidus [EMAIL PROTECTED] ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. pages that are merged are marked as readonly and are COWed when any application try to change them. ksm is working by walking over

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Andrew Morton
On Tue, 11 Nov 2008 15:21:40 +0200 Izik Eidus [EMAIL PROTECTED] wrote: From: Izik Eidus [EMAIL PROTECTED] ksm is driver that allow merging identical pages between one or more applications in way unvisible to the application that use it. pages that are merged are marked as readonly and are

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Jonathan Corbet
I don't claim to begin to really understand the deep VM side of this patch, but I can certainly pick nits as I work through it...sorry for the lack of anything more substantive. +static struct list_head slots; Some of these file-static variable names seem a little..terse... +#define

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Andrea Arcangeli
On Tue, Nov 11, 2008 at 12:38:51PM -0800, Andrew Morton wrote: Please fully document that interface in the changelog so that we can review your decisions here. This is by far the most important consideration - we can change all the code, but interfaces are for ever. Yes, this is the most

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
Jonathan Corbet wrote: I don't claim to begin to really understand the deep VM side of this patch, but I can certainly pick nits as I work through it...sorry for the lack of anything more substantive. +static struct list_head slots; Some of these file-static variable names seem a

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Jonathan Corbet
On Wed, 12 Nov 2008 00:17:39 +0200 Izik Eidus [EMAIL PROTECTED] wrote: +static int ksm_dev_open(struct inode *inode, struct file *filp) +{ + try_module_get(THIS_MODULE); + return 0; +} + +static int ksm_dev_release(struct inode *inode, struct file *filp) +{ +

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
Jonathan Corbet wrote: On Wed, 12 Nov 2008 00:17:39 +0200 Izik Eidus [EMAIL PROTECTED] wrote: +static int ksm_dev_open(struct inode *inode, struct file *filp) +{ + try_module_get(THIS_MODULE); + return 0; +} + +static int ksm_dev_release(struct inode *inode, struct file *filp)

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
Jonathan Corbet wrote: [Let's see if I can get through the rest without premature sends...] On Wed, 12 Nov 2008 00:17:39 +0200 Izik Eidus [EMAIL PROTECTED] wrote: Actually, it occurs to me that there's no sanity checks on any of the values passed in by ioctl(). What happens if the user

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Valdis . Kletnieks
On Tue, 11 Nov 2008 15:03:45 MST, Jonathan Corbet said: +#define PAGECMP_OFFSET 128 +#define PAGEHASH_SIZE (PAGECMP_OFFSET ? PAGECMP_OFFSET : PAGE_SIZE) +/* hash the page */ +static void page_hash(struct page *page, unsigned char *digest) So is this really saying that you only hash the

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Avi Kivity
Jonathan Corbet wrote: +static struct list_head slots; Some of these file-static variable names seem a little..terse... While ksm was written to be independent of a certain TLA-named kernel subsystem developed two rooms away, they share some naming... this refers to kvm 'memory

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Avi Kivity
Izik Eidus wrote: Any benchmarks on the runtime cost of having KSM running? This one is problematic, ksm can take anything from 0% to 100% cpu its all depend on how fast you run it. it have 3 parameters: number of pages to scan before it go to sleep maximum number of pages to merge while we

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Andrea Arcangeli
Hi Jonathan, On Tue, Nov 11, 2008 at 03:30:28PM -0700, Jonathan Corbet wrote: But it will fail in a totally silent and mysterious way. Doesn't it seem better to verify the values when you can return a meaningful error code to the caller? I think you're right, but just because find_extend_vma

Re: [PATCH 3/4] add ksm kernel shared memory driver

2008-11-11 Thread Izik Eidus
Jonathan Corbet wrote: What about things like cache effects from scanning all those pages? My guess is that, if you're trying to run dozens of Windows guests, cache usage is not at the top of your list of concerns, but I could be wrong. Usually am... Ok, ksm does make the cache of the