RE: [RFC/PATCH] zcache/ramster rewrite and promotion
> From: Pekka Enberg [mailto:penb...@kernel.org] > Subject: Re: [RFC/PATCH] zcache/ramster rewrite and promotion > > On Mon, Aug 6, 2012 at 7:10 PM, Dan Magenheimer > wrote: > > Hmmm.. there's also zbud.c and tmem.c which are critical components > > of both zcache and ramster. And there are header files as well which > > will need to either be in mm/ or somewhere in include/linux/ > > > > Is there a reason or rule that mm/ can't have subdirectories? > > > > Since zcache has at least three .c files plus ramster.c, and > > since mm/frontswap.c and mm/cleancache.c are the foundation on > > which all of these are built, I was thinking grouping all six > > (plus headers) in the same mm/tmem/ subdirectory was a good > > way to keep mm/ from continuing to get more cluttered... not counting > > new zcache and ramster files, there are now 74 .c files in mm/! > > (Personally, I think a directory has too many files in it if > > "ls" doesn't fit in a 25x80 window.) > > > > Thoughts? > > There's no reason we can't have subdirectories. That said, I really > don't see the point of having a separate directory called 'tmem'. It > might make sense to have mm/zcache and/or mm/ramster but I suspect > you can just fold the core code in mm/zcache.c and mm/ramster.c by > slimming down the weird Solaris-like 'tmem' abstractions. I'm not sure I understand... what is Solaris-like about tmem? And what would you slim down? While I agree one can often glom three separate 1000-line .c files into a single 3000-line .c file, I recently spent some time moving the other direction to, I thought, improve readability. Do kernel developers have a preference for huge .c files rather than smaller logically-separated moderate-sized files in a subdirectory? Thanks, Dan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [RFC/PATCH] zcache/ramster rewrite and promotion
From: Pekka Enberg [mailto:penb...@kernel.org] Subject: Re: [RFC/PATCH] zcache/ramster rewrite and promotion On Mon, Aug 6, 2012 at 7:10 PM, Dan Magenheimer dan.magenhei...@oracle.com wrote: Hmmm.. there's also zbud.c and tmem.c which are critical components of both zcache and ramster. And there are header files as well which will need to either be in mm/ or somewhere in include/linux/ Is there a reason or rule that mm/ can't have subdirectories? Since zcache has at least three .c files plus ramster.c, and since mm/frontswap.c and mm/cleancache.c are the foundation on which all of these are built, I was thinking grouping all six (plus headers) in the same mm/tmem/ subdirectory was a good way to keep mm/ from continuing to get more cluttered... not counting new zcache and ramster files, there are now 74 .c files in mm/! (Personally, I think a directory has too many files in it if ls doesn't fit in a 25x80 window.) Thoughts? There's no reason we can't have subdirectories. That said, I really don't see the point of having a separate directory called 'tmem'. It might make sense to have mm/zcache and/or mm/ramster but I suspect you can just fold the core code in mm/zcache.c and mm/ramster.c by slimming down the weird Solaris-like 'tmem' abstractions. I'm not sure I understand... what is Solaris-like about tmem? And what would you slim down? While I agree one can often glom three separate 1000-line .c files into a single 3000-line .c file, I recently spent some time moving the other direction to, I thought, improve readability. Do kernel developers have a preference for huge .c files rather than smaller logically-separated moderate-sized files in a subdirectory? Thanks, Dan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
On Mon, Aug 6, 2012 at 7:10 PM, Dan Magenheimer wrote: > Hmmm.. there's also zbud.c and tmem.c which are critical components > of both zcache and ramster. And there are header files as well which > will need to either be in mm/ or somewhere in include/linux/ > > Is there a reason or rule that mm/ can't have subdirectories? > > Since zcache has at least three .c files plus ramster.c, and > since mm/frontswap.c and mm/cleancache.c are the foundation on > which all of these are built, I was thinking grouping all six > (plus headers) in the same mm/tmem/ subdirectory was a good > way to keep mm/ from continuing to get more cluttered... not counting > new zcache and ramster files, there are now 74 .c files in mm/! > (Personally, I think a directory has too many files in it if > "ls" doesn't fit in a 25x80 window.) > > Thoughts? There's no reason we can't have subdirectories. That said, I really don't see the point of having a separate directory called 'tmem'. It might make sense to have mm/zcache and/or mm/ramster but I suspect you can just fold the core code in mm/zcache.c and mm/ramster.c by slimming down the weird Solaris-like 'tmem' abstractions. Pekka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [RFC/PATCH] zcache/ramster rewrite and promotion
> From: Pekka Enberg [mailto:penb...@kernel.org] > Subject: Re: [RFC/PATCH] zcache/ramster rewrite and promotion > > On Mon, Aug 6, 2012 at 5:07 PM, Dan Magenheimer > wrote: > > I'm OK with placing it wherever kernel developers want to put > > it, as long as the reason is not NIMBY-ness. [1] My preference > > is to keep all the parts together, at least for the review phase, > > but if there is a consensus that it belongs someplace else, > > I will be happy to move it. > > I'd go for core code in mm/zcache.c and mm/ramster.c, and move the > clustering code under net/ramster or drivers/ramster. Hi Pekka -- Thanks for the quick feedback! Hmmm.. there's also zbud.c and tmem.c which are critical components of both zcache and ramster. And there are header files as well which will need to either be in mm/ or somewhere in include/linux/ Is there a reason or rule that mm/ can't have subdirectories? Since zcache has at least three .c files plus ramster.c, and since mm/frontswap.c and mm/cleancache.c are the foundation on which all of these are built, I was thinking grouping all six (plus headers) in the same mm/tmem/ subdirectory was a good way to keep mm/ from continuing to get more cluttered... not counting new zcache and ramster files, there are now 74 .c files in mm/! (Personally, I think a directory has too many files in it if "ls" doesn't fit in a 25x80 window.) Thoughts? Dan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
On Mon, Aug 6, 2012 at 5:07 PM, Dan Magenheimer wrote: > I'm OK with placing it wherever kernel developers want to put > it, as long as the reason is not NIMBY-ness. [1] My preference > is to keep all the parts together, at least for the review phase, > but if there is a consensus that it belongs someplace else, > I will be happy to move it. I'd go for core code in mm/zcache.c and mm/ramster.c, and move the clustering code under net/ramster or drivers/ramster. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [RFC/PATCH] zcache/ramster rewrite and promotion
> From: Pekka Enberg [mailto:penb...@kernel.org] > Subject: Re: [RFC/PATCH] zcache/ramster rewrite and promotion > > Hi Dan, > > On Wed, Aug 1, 2012 at 12:13 AM, Dan Magenheimer > wrote: > > Ramster does the same thing but manages it peer-to-peer across > > multiple systems using kernel sockets. One could argue that > > the dependency on sockets makes it more of a driver than "mm" > > but ramster is "memory management" too, just a bit more exotic. > > How do you configure it? Hi Pekka -- It looks like the build/configuration how-to at https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-v5-120214 is out-of-date and I need to fix some things in it. I'll post a link to it after I update it. > Can we move parts of the network protocol under > net/ramster or something? Ramster is built on top of kernel sockets. Both that networking part and the configuration part of the ramster code are heavily leveraged from ocfs2 and I suspect there is a lot of similarity to gfs code as well. In the code for both of those filesystems I think the network and configuration code lives in the same directory with the file system, so that was the model I was following. I'm OK with placing it wherever kernel developers want to put it, as long as the reason is not NIMBY-ness. [1] My preference is to keep all the parts together, at least for the review phase, but if there is a consensus that it belongs someplace else, I will be happy to move it. Dan [1] http://en.wikipedia.org/wiki/NIMBY -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [RFC/PATCH] zcache/ramster rewrite and promotion
From: Pekka Enberg [mailto:penb...@kernel.org] Subject: Re: [RFC/PATCH] zcache/ramster rewrite and promotion Hi Dan, On Wed, Aug 1, 2012 at 12:13 AM, Dan Magenheimer dan.magenhei...@oracle.com wrote: Ramster does the same thing but manages it peer-to-peer across multiple systems using kernel sockets. One could argue that the dependency on sockets makes it more of a driver than mm but ramster is memory management too, just a bit more exotic. How do you configure it? Hi Pekka -- It looks like the build/configuration how-to at https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-v5-120214 is out-of-date and I need to fix some things in it. I'll post a link to it after I update it. Can we move parts of the network protocol under net/ramster or something? Ramster is built on top of kernel sockets. Both that networking part and the configuration part of the ramster code are heavily leveraged from ocfs2 and I suspect there is a lot of similarity to gfs code as well. In the code for both of those filesystems I think the network and configuration code lives in the same directory with the file system, so that was the model I was following. I'm OK with placing it wherever kernel developers want to put it, as long as the reason is not NIMBY-ness. [1] My preference is to keep all the parts together, at least for the review phase, but if there is a consensus that it belongs someplace else, I will be happy to move it. Dan [1] http://en.wikipedia.org/wiki/NIMBY -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
On Mon, Aug 6, 2012 at 5:07 PM, Dan Magenheimer dan.magenhei...@oracle.com wrote: I'm OK with placing it wherever kernel developers want to put it, as long as the reason is not NIMBY-ness. [1] My preference is to keep all the parts together, at least for the review phase, but if there is a consensus that it belongs someplace else, I will be happy to move it. I'd go for core code in mm/zcache.c and mm/ramster.c, and move the clustering code under net/ramster or drivers/ramster. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [RFC/PATCH] zcache/ramster rewrite and promotion
From: Pekka Enberg [mailto:penb...@kernel.org] Subject: Re: [RFC/PATCH] zcache/ramster rewrite and promotion On Mon, Aug 6, 2012 at 5:07 PM, Dan Magenheimer dan.magenhei...@oracle.com wrote: I'm OK with placing it wherever kernel developers want to put it, as long as the reason is not NIMBY-ness. [1] My preference is to keep all the parts together, at least for the review phase, but if there is a consensus that it belongs someplace else, I will be happy to move it. I'd go for core code in mm/zcache.c and mm/ramster.c, and move the clustering code under net/ramster or drivers/ramster. Hi Pekka -- Thanks for the quick feedback! Hmmm.. there's also zbud.c and tmem.c which are critical components of both zcache and ramster. And there are header files as well which will need to either be in mm/ or somewhere in include/linux/ Is there a reason or rule that mm/ can't have subdirectories? Since zcache has at least three .c files plus ramster.c, and since mm/frontswap.c and mm/cleancache.c are the foundation on which all of these are built, I was thinking grouping all six (plus headers) in the same mm/tmem/ subdirectory was a good way to keep mm/ from continuing to get more cluttered... not counting new zcache and ramster files, there are now 74 .c files in mm/! (Personally, I think a directory has too many files in it if ls doesn't fit in a 25x80 window.) Thoughts? Dan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
On Mon, Aug 6, 2012 at 7:10 PM, Dan Magenheimer dan.magenhei...@oracle.com wrote: Hmmm.. there's also zbud.c and tmem.c which are critical components of both zcache and ramster. And there are header files as well which will need to either be in mm/ or somewhere in include/linux/ Is there a reason or rule that mm/ can't have subdirectories? Since zcache has at least three .c files plus ramster.c, and since mm/frontswap.c and mm/cleancache.c are the foundation on which all of these are built, I was thinking grouping all six (plus headers) in the same mm/tmem/ subdirectory was a good way to keep mm/ from continuing to get more cluttered... not counting new zcache and ramster files, there are now 74 .c files in mm/! (Personally, I think a directory has too many files in it if ls doesn't fit in a 25x80 window.) Thoughts? There's no reason we can't have subdirectories. That said, I really don't see the point of having a separate directory called 'tmem'. It might make sense to have mm/zcache and/or mm/ramster but I suspect you can just fold the core code in mm/zcache.c and mm/ramster.c by slimming down the weird Solaris-like 'tmem' abstractions. Pekka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
Hi Dan, On Wed, Aug 1, 2012 at 12:13 AM, Dan Magenheimer wrote: > Ramster does the same thing but manages it peer-to-peer across > multiple systems using kernel sockets. One could argue that > the dependency on sockets makes it more of a driver than "mm" > but ramster is "memory management" too, just a bit more exotic. How do you configure it? Can we move parts of the network protocol under net/ramster or something? Pekka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
Hi Konrad, > On Tue, Jul 31, 2012 at 11:53:57PM +0300, Pekka Enberg wrote: >> Why on earth would you want to move that under the mm directory? On Wed, Aug 1, 2012 at 12:04 AM, Konrad Rzeszutek Wilk wrote: > If you take aside that problem that it is one big patch instead > of being split up in more reasonable pieces - would you recommend > that it reside in a different directory? > > Or is that it does not make sense b/c it has other components in it - such > as tcp/nodemaneger/hearbeat/etc so it should go under the refactor knife? > > And if you rip out the ramster from this and just concentrate on zcache - > should that go in drivers/mm or mm/tmem/zcache? I definitely think mm/zcache.c makes sense. I hate the fact that it's now riddled with references to "tmem" and "ramster" but that's probably fixable. I also hate the fact that you've now gone and rewritten everything so we lose all the change history zcache has had under staging. As for ramster, it might make sense to have its core in mm/ramster.c and move the TCP weirdness somewhere else. The exact location depends on what kind of userspace ABIs you expose, I suppose. I mean, surely you need to configure the thing somehow? Pekka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
Hi Konrad, On Tue, Jul 31, 2012 at 11:53:57PM +0300, Pekka Enberg wrote: Why on earth would you want to move that under the mm directory? On Wed, Aug 1, 2012 at 12:04 AM, Konrad Rzeszutek Wilk konrad.w...@oracle.com wrote: If you take aside that problem that it is one big patch instead of being split up in more reasonable pieces - would you recommend that it reside in a different directory? Or is that it does not make sense b/c it has other components in it - such as tcp/nodemaneger/hearbeat/etc so it should go under the refactor knife? And if you rip out the ramster from this and just concentrate on zcache - should that go in drivers/mm or mm/tmem/zcache? I definitely think mm/zcache.c makes sense. I hate the fact that it's now riddled with references to tmem and ramster but that's probably fixable. I also hate the fact that you've now gone and rewritten everything so we lose all the change history zcache has had under staging. As for ramster, it might make sense to have its core in mm/ramster.c and move the TCP weirdness somewhere else. The exact location depends on what kind of userspace ABIs you expose, I suppose. I mean, surely you need to configure the thing somehow? Pekka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
Hi Dan, On Wed, Aug 1, 2012 at 12:13 AM, Dan Magenheimer dan.magenhei...@oracle.com wrote: Ramster does the same thing but manages it peer-to-peer across multiple systems using kernel sockets. One could argue that the dependency on sockets makes it more of a driver than mm but ramster is memory management too, just a bit more exotic. How do you configure it? Can we move parts of the network protocol under net/ramster or something? Pekka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [RFC/PATCH] zcache/ramster rewrite and promotion
> From: Pekka Enberg [mailto:penb...@kernel.org] > Sent: Tuesday, July 31, 2012 2:54 PM > > On Tue, Jul 31, 2012 at 11:18 PM, Dan Magenheimer > wrote: > > diffstat vs 3.5: > > drivers/staging/ramster/Kconfig |2 > > drivers/staging/ramster/Makefile |2 > > drivers/staging/zcache/Kconfig|2 > > drivers/staging/zcache/Makefile |2 > > mm/Kconfig|2 > > mm/Makefile |4 > > mm/tmem/Kconfig | 33 > > mm/tmem/Makefile |5 > > mm/tmem/tmem.c| 894 + > > mm/tmem/tmem.h| 259 +++ > > mm/tmem/zbud.c| 1060 +++ > > mm/tmem/zbud.h| 33 > > mm/tmem/zcache-main.c | 1686 + > > mm/tmem/zcache.h | 53 > > mm/tmem/ramster.h | 59 > > mm/tmem/ramster/heartbeat.c | 462 ++ > > mm/tmem/ramster/heartbeat.h | 87 + > > mm/tmem/ramster/masklog.c | 155 ++ > > mm/tmem/ramster/masklog.h | 220 +++ > > mm/tmem/ramster/nodemanager.c | 995 +++ > > mm/tmem/ramster/nodemanager.h | 88 + > > mm/tmem/ramster/r2net.c | 414 ++ > > mm/tmem/ramster/ramster.c | 985 ++ > > mm/tmem/ramster/ramster.h | 161 ++ > > mm/tmem/ramster/ramster_nodemanager.h | 39 > > mm/tmem/ramster/tcp.c | 2253 > > ++ > > mm/tmem/ramster/tcp.h | 159 ++ > > mm/tmem/ramster/tcp_internal.h| 248 +++ > > 28 files changed, 10358 insertions(+), 4 deletions(-) > > So it's basically this commit, right? > > https://oss.oracle.com/git/djm/tmem.git/?p=djm/tmem.git;a=commitdiff;h=22844fe3f52d912247212408294be33 > 0a867937c > > Why on earth would you want to move that under the mm directory? Hi Pekka -- Thanks for your reply and question. MM means "memory management" and zcache manages physical memory to allow more pages of data to be stored in RAM. So it seems a logical place. It's not a block driver, or a network driver, or a device driver, or a filesystem... do you have a different location in the kernel in mind? Zcache does it a bit differently than all the other parts of mm because it needs to; because all the other parts of mm try to maximize the amount of physical memory that is directly addressable by threads but one can't directly address pages that have been compressed. So zcache uses the transcendent memory approach (via cleancache and frontswap) to compress/decompress clean pagecache pages and swap pages "on demand". The tmem design also nicely handles both the fact that the degree of compression is unpredictable and the fact that the fraction of fixed total RAM used for compressed pages vs "normal uncompressed mm" pages needs to be very dynamic. Ramster does the same thing but manages it peer-to-peer across multiple systems using kernel sockets. One could argue that the dependency on sockets makes it more of a driver than "mm" but ramster is "memory management" too, just a bit more exotic. Thanks, Dan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
On Tue, Jul 31, 2012 at 11:53:57PM +0300, Pekka Enberg wrote: > On Tue, Jul 31, 2012 at 11:18 PM, Dan Magenheimer > wrote: > > diffstat vs 3.5: > > drivers/staging/ramster/Kconfig |2 > > drivers/staging/ramster/Makefile |2 > > drivers/staging/zcache/Kconfig|2 > > drivers/staging/zcache/Makefile |2 > > mm/Kconfig|2 > > mm/Makefile |4 > > mm/tmem/Kconfig | 33 > > mm/tmem/Makefile |5 > > mm/tmem/tmem.c| 894 + > > mm/tmem/tmem.h| 259 +++ > > mm/tmem/zbud.c| 1060 +++ > > mm/tmem/zbud.h| 33 > > mm/tmem/zcache-main.c | 1686 + > > mm/tmem/zcache.h | 53 > > mm/tmem/ramster.h | 59 > > mm/tmem/ramster/heartbeat.c | 462 ++ > > mm/tmem/ramster/heartbeat.h | 87 + > > mm/tmem/ramster/masklog.c | 155 ++ > > mm/tmem/ramster/masklog.h | 220 +++ > > mm/tmem/ramster/nodemanager.c | 995 +++ > > mm/tmem/ramster/nodemanager.h | 88 + > > mm/tmem/ramster/r2net.c | 414 ++ > > mm/tmem/ramster/ramster.c | 985 ++ > > mm/tmem/ramster/ramster.h | 161 ++ > > mm/tmem/ramster/ramster_nodemanager.h | 39 > > mm/tmem/ramster/tcp.c | 2253 > > ++ > > mm/tmem/ramster/tcp.h | 159 ++ > > mm/tmem/ramster/tcp_internal.h| 248 +++ > > 28 files changed, 10358 insertions(+), 4 deletions(-) > > So it's basically this commit, right? Yeah, one big RFC patch. > > https://oss.oracle.com/git/djm/tmem.git/?p=djm/tmem.git;a=commitdiff;h=22844fe3f52d912247212408294be330a867937c > > Why on earth would you want to move that under the mm directory? If you take aside that problem that it is one big patch instead of being split up in more reasonable pieces - would you recommend that it reside in a different directory? Or is that it does not make sense b/c it has other components in it - such as tcp/nodemaneger/hearbeat/etc so it should go under the refactor knife? And if you rip out the ramster from this and just concentrate on zcache - should that go in drivers/mm or mm/tmem/zcache? > > Pekka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
On Tue, Jul 31, 2012 at 11:18 PM, Dan Magenheimer wrote: > diffstat vs 3.5: > drivers/staging/ramster/Kconfig |2 > drivers/staging/ramster/Makefile |2 > drivers/staging/zcache/Kconfig|2 > drivers/staging/zcache/Makefile |2 > mm/Kconfig|2 > mm/Makefile |4 > mm/tmem/Kconfig | 33 > mm/tmem/Makefile |5 > mm/tmem/tmem.c| 894 + > mm/tmem/tmem.h| 259 +++ > mm/tmem/zbud.c| 1060 +++ > mm/tmem/zbud.h| 33 > mm/tmem/zcache-main.c | 1686 + > mm/tmem/zcache.h | 53 > mm/tmem/ramster.h | 59 > mm/tmem/ramster/heartbeat.c | 462 ++ > mm/tmem/ramster/heartbeat.h | 87 + > mm/tmem/ramster/masklog.c | 155 ++ > mm/tmem/ramster/masklog.h | 220 +++ > mm/tmem/ramster/nodemanager.c | 995 +++ > mm/tmem/ramster/nodemanager.h | 88 + > mm/tmem/ramster/r2net.c | 414 ++ > mm/tmem/ramster/ramster.c | 985 ++ > mm/tmem/ramster/ramster.h | 161 ++ > mm/tmem/ramster/ramster_nodemanager.h | 39 > mm/tmem/ramster/tcp.c | 2253 > ++ > mm/tmem/ramster/tcp.h | 159 ++ > mm/tmem/ramster/tcp_internal.h| 248 +++ > 28 files changed, 10358 insertions(+), 4 deletions(-) So it's basically this commit, right? https://oss.oracle.com/git/djm/tmem.git/?p=djm/tmem.git;a=commitdiff;h=22844fe3f52d912247212408294be330a867937c Why on earth would you want to move that under the mm directory? Pekka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[RFC/PATCH] zcache/ramster rewrite and promotion
Here finally is the long promised rewrite of zcache (and ramster).
I know that we are concentrating on moving zcache from staging,
and not ramster. However the amount of duplicate code that ramster
used from zcache is astonishing so when I did the rewrite I thought
why not kill two birds with one stone - since both are in the
staging directory.
Of notable interest to the broader mm community, I am proposing, when review
is complete, to place zcache in a new subdirectory of mm, called "tmem"
(short for transcendent memory). Zcache is truly memory management,
not a hardware driver, and it interfaces with mm/swap/vfs through
mm/cleancache.c
and mm/frontswap.c (which possibly should move to the new tmem directory
in the future as well).
This is a major rewrite for zcache, not a sequence of small patches.
So those who are interested in understanding, reviewing, and commenting
in detail on the design and the functioning of the code can find it at:
git://oss.oracle.com/git/djm/tmem.git #zcache-120731
For those who prefer to review and comment line-by-line, it's not clear
yet how best to post the ~10K lines of code to ensure reviewer productivity.
Konrad suggested an IRC talk on Monday to talk about this so we can figure
out what is the proper option.
(If you are not familiar with the tmem terminology, you can review
it here: http://lwn.net/Articles/454795/ )
Some of the highlights of this git branch:
1. Merge of zcache and ramster. Zcache and ramster had a great deal of
duplicate code which is now merged. In essence, zcache *is* ramster
but with no remote machine available, but !CONFIG_RAMSTER will avoid
compiling lots of ramster-specific code.
2. Allocator. Previously, persistent pools used zsmalloc and ephemeral pools
used zbud. Now a completely rewritten zbud is used for both. Notably
this zbud maintains all persistent (frontswap) and ephemeral (cleancache)
pageframes in separate queues in LRU order.
3. Interaction with page allocator. Zbud does no page allocation/freeing,
it is done entirely in zcache where it can be tracked more effectively.
4. Better pre-allocation. Previously, on put, if a new pageframe could not be
pre-allocated, the put would fail, even if the allocator had plenty of
partial pages where the data could be stored; this is now fixed.
5. Ouroboros ("eating its own tail") allocation. If no pageframe can be
allocated
AND no partial pages are available, the least-recently-used ephemeral
pageframe
is reclaimed immediately (including flushing tmem pointers to it) and
re-used.
This ensures that most-recently-used cleancache pages are more likely to
be retained than LRU pages and also that, as in the core mm subsystem,
anonymous pages have a higher priority than clean page cache pages.
6. Zcache and zbud now use debugfs instead of sysfs. Ramster uses debugfs
where possible and sysfs where necessary. (Some ramster configuration
is done from userspace so some sysfs is necessary.)
7. Modularization. As some have observed, the monolithic zcache-main.c code
included zbud code, which has now been separated into its own code module.
Much ramster-specific code in the old ramster zcache-main.c has also been
moved into ramster.c so that it does not get compiled with !CONFIG_RAMSTER.
8. Rebased to 3.5.
Konrad has been suggesting to prepare to "lift" the 2) "Allocator" out as a
separate patch so that it could be used in the zcache1 as part of its promotion
out of staging - if we think that zcache1 needs that. The problem with that is
that
the code has been tested with all the other code together. It is unclear
whether by itself - without the rest of the harness - it would work properly.
And if the time spent finding those bugs (of the lifted code) will be greater
than just dropping in zcache2 as zcache1 and concentrate on promoting that.
The nice-to-have-features that I had in the back of my mind (so after
zcache and ramster have left staging) were:
A. Ouroboros writeback. Since persistent (frontswap) pages may now also be
reclaimed in LRU order, the foundation is in place to properly writeback
these pages back into the swap cache and then the swap disk. This is still
under development and requires some other mm changes which are prototyped
but not yet included with this patch.
B. WasActive patch, requires some mm/frontswap changes previously posted
(but still has a known problem or two).
C. Module capability, see patch posted by Erlangen University. Needs
to be brought up to kernel standards.
If anybody is interested on helping out with these, let me know!
P.S. I've just started tracking down a memory leak, so I don't recommend
benchmarking this zcache-120731 version yet.
Signed-off-by: Dan Magenheimer
diffstat vs 3.5:
drivers/staging/ramster/Kconfig |2
drivers/staging/ramster/Makefile |2
drivers/staging/zcache/Kconfig|2
[RFC/PATCH] zcache/ramster rewrite and promotion
Here finally is the long promised rewrite of zcache (and ramster). I know that we are concentrating on moving zcache from staging, and not ramster. However the amount of duplicate code that ramster used from zcache is astonishing so when I did the rewrite I thought why not kill two birds with one stone - since both are in the staging directory. Of notable interest to the broader mm community, I am proposing, when review is complete, to place zcache in a new subdirectory of mm, called tmem (short for transcendent memory). Zcache is truly memory management, not a hardware driver, and it interfaces with mm/swap/vfs through mm/cleancache.c and mm/frontswap.c (which possibly should move to the new tmem directory in the future as well). This is a major rewrite for zcache, not a sequence of small patches. So those who are interested in understanding, reviewing, and commenting in detail on the design and the functioning of the code can find it at: git://oss.oracle.com/git/djm/tmem.git #zcache-120731 For those who prefer to review and comment line-by-line, it's not clear yet how best to post the ~10K lines of code to ensure reviewer productivity. Konrad suggested an IRC talk on Monday to talk about this so we can figure out what is the proper option. (If you are not familiar with the tmem terminology, you can review it here: http://lwn.net/Articles/454795/ ) Some of the highlights of this git branch: 1. Merge of zcache and ramster. Zcache and ramster had a great deal of duplicate code which is now merged. In essence, zcache *is* ramster but with no remote machine available, but !CONFIG_RAMSTER will avoid compiling lots of ramster-specific code. 2. Allocator. Previously, persistent pools used zsmalloc and ephemeral pools used zbud. Now a completely rewritten zbud is used for both. Notably this zbud maintains all persistent (frontswap) and ephemeral (cleancache) pageframes in separate queues in LRU order. 3. Interaction with page allocator. Zbud does no page allocation/freeing, it is done entirely in zcache where it can be tracked more effectively. 4. Better pre-allocation. Previously, on put, if a new pageframe could not be pre-allocated, the put would fail, even if the allocator had plenty of partial pages where the data could be stored; this is now fixed. 5. Ouroboros (eating its own tail) allocation. If no pageframe can be allocated AND no partial pages are available, the least-recently-used ephemeral pageframe is reclaimed immediately (including flushing tmem pointers to it) and re-used. This ensures that most-recently-used cleancache pages are more likely to be retained than LRU pages and also that, as in the core mm subsystem, anonymous pages have a higher priority than clean page cache pages. 6. Zcache and zbud now use debugfs instead of sysfs. Ramster uses debugfs where possible and sysfs where necessary. (Some ramster configuration is done from userspace so some sysfs is necessary.) 7. Modularization. As some have observed, the monolithic zcache-main.c code included zbud code, which has now been separated into its own code module. Much ramster-specific code in the old ramster zcache-main.c has also been moved into ramster.c so that it does not get compiled with !CONFIG_RAMSTER. 8. Rebased to 3.5. Konrad has been suggesting to prepare to lift the 2) Allocator out as a separate patch so that it could be used in the zcache1 as part of its promotion out of staging - if we think that zcache1 needs that. The problem with that is that the code has been tested with all the other code together. It is unclear whether by itself - without the rest of the harness - it would work properly. And if the time spent finding those bugs (of the lifted code) will be greater than just dropping in zcache2 as zcache1 and concentrate on promoting that. The nice-to-have-features that I had in the back of my mind (so after zcache and ramster have left staging) were: A. Ouroboros writeback. Since persistent (frontswap) pages may now also be reclaimed in LRU order, the foundation is in place to properly writeback these pages back into the swap cache and then the swap disk. This is still under development and requires some other mm changes which are prototyped but not yet included with this patch. B. WasActive patch, requires some mm/frontswap changes previously posted (but still has a known problem or two). C. Module capability, see patch posted by Erlangen University. Needs to be brought up to kernel standards. If anybody is interested on helping out with these, let me know! P.S. I've just started tracking down a memory leak, so I don't recommend benchmarking this zcache-120731 version yet. Signed-off-by: Dan Magenheimer dan.magenhei...@oracle.com diffstat vs 3.5: drivers/staging/ramster/Kconfig |2 drivers/staging/ramster/Makefile |2 drivers/staging/zcache/Kconfig
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
On Tue, Jul 31, 2012 at 11:18 PM, Dan Magenheimer dan.magenhei...@oracle.com wrote: diffstat vs 3.5: drivers/staging/ramster/Kconfig |2 drivers/staging/ramster/Makefile |2 drivers/staging/zcache/Kconfig|2 drivers/staging/zcache/Makefile |2 mm/Kconfig|2 mm/Makefile |4 mm/tmem/Kconfig | 33 mm/tmem/Makefile |5 mm/tmem/tmem.c| 894 + mm/tmem/tmem.h| 259 +++ mm/tmem/zbud.c| 1060 +++ mm/tmem/zbud.h| 33 mm/tmem/zcache-main.c | 1686 + mm/tmem/zcache.h | 53 mm/tmem/ramster.h | 59 mm/tmem/ramster/heartbeat.c | 462 ++ mm/tmem/ramster/heartbeat.h | 87 + mm/tmem/ramster/masklog.c | 155 ++ mm/tmem/ramster/masklog.h | 220 +++ mm/tmem/ramster/nodemanager.c | 995 +++ mm/tmem/ramster/nodemanager.h | 88 + mm/tmem/ramster/r2net.c | 414 ++ mm/tmem/ramster/ramster.c | 985 ++ mm/tmem/ramster/ramster.h | 161 ++ mm/tmem/ramster/ramster_nodemanager.h | 39 mm/tmem/ramster/tcp.c | 2253 ++ mm/tmem/ramster/tcp.h | 159 ++ mm/tmem/ramster/tcp_internal.h| 248 +++ 28 files changed, 10358 insertions(+), 4 deletions(-) So it's basically this commit, right? https://oss.oracle.com/git/djm/tmem.git/?p=djm/tmem.git;a=commitdiff;h=22844fe3f52d912247212408294be330a867937c Why on earth would you want to move that under the mm directory? Pekka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC/PATCH] zcache/ramster rewrite and promotion
On Tue, Jul 31, 2012 at 11:53:57PM +0300, Pekka Enberg wrote: On Tue, Jul 31, 2012 at 11:18 PM, Dan Magenheimer dan.magenhei...@oracle.com wrote: diffstat vs 3.5: drivers/staging/ramster/Kconfig |2 drivers/staging/ramster/Makefile |2 drivers/staging/zcache/Kconfig|2 drivers/staging/zcache/Makefile |2 mm/Kconfig|2 mm/Makefile |4 mm/tmem/Kconfig | 33 mm/tmem/Makefile |5 mm/tmem/tmem.c| 894 + mm/tmem/tmem.h| 259 +++ mm/tmem/zbud.c| 1060 +++ mm/tmem/zbud.h| 33 mm/tmem/zcache-main.c | 1686 + mm/tmem/zcache.h | 53 mm/tmem/ramster.h | 59 mm/tmem/ramster/heartbeat.c | 462 ++ mm/tmem/ramster/heartbeat.h | 87 + mm/tmem/ramster/masklog.c | 155 ++ mm/tmem/ramster/masklog.h | 220 +++ mm/tmem/ramster/nodemanager.c | 995 +++ mm/tmem/ramster/nodemanager.h | 88 + mm/tmem/ramster/r2net.c | 414 ++ mm/tmem/ramster/ramster.c | 985 ++ mm/tmem/ramster/ramster.h | 161 ++ mm/tmem/ramster/ramster_nodemanager.h | 39 mm/tmem/ramster/tcp.c | 2253 ++ mm/tmem/ramster/tcp.h | 159 ++ mm/tmem/ramster/tcp_internal.h| 248 +++ 28 files changed, 10358 insertions(+), 4 deletions(-) So it's basically this commit, right? Yeah, one big RFC patch. https://oss.oracle.com/git/djm/tmem.git/?p=djm/tmem.git;a=commitdiff;h=22844fe3f52d912247212408294be330a867937c Why on earth would you want to move that under the mm directory? If you take aside that problem that it is one big patch instead of being split up in more reasonable pieces - would you recommend that it reside in a different directory? Or is that it does not make sense b/c it has other components in it - such as tcp/nodemaneger/hearbeat/etc so it should go under the refactor knife? And if you rip out the ramster from this and just concentrate on zcache - should that go in drivers/mm or mm/tmem/zcache? Pekka -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [RFC/PATCH] zcache/ramster rewrite and promotion
From: Pekka Enberg [mailto:penb...@kernel.org] Sent: Tuesday, July 31, 2012 2:54 PM On Tue, Jul 31, 2012 at 11:18 PM, Dan Magenheimer dan.magenhei...@oracle.com wrote: diffstat vs 3.5: drivers/staging/ramster/Kconfig |2 drivers/staging/ramster/Makefile |2 drivers/staging/zcache/Kconfig|2 drivers/staging/zcache/Makefile |2 mm/Kconfig|2 mm/Makefile |4 mm/tmem/Kconfig | 33 mm/tmem/Makefile |5 mm/tmem/tmem.c| 894 + mm/tmem/tmem.h| 259 +++ mm/tmem/zbud.c| 1060 +++ mm/tmem/zbud.h| 33 mm/tmem/zcache-main.c | 1686 + mm/tmem/zcache.h | 53 mm/tmem/ramster.h | 59 mm/tmem/ramster/heartbeat.c | 462 ++ mm/tmem/ramster/heartbeat.h | 87 + mm/tmem/ramster/masklog.c | 155 ++ mm/tmem/ramster/masklog.h | 220 +++ mm/tmem/ramster/nodemanager.c | 995 +++ mm/tmem/ramster/nodemanager.h | 88 + mm/tmem/ramster/r2net.c | 414 ++ mm/tmem/ramster/ramster.c | 985 ++ mm/tmem/ramster/ramster.h | 161 ++ mm/tmem/ramster/ramster_nodemanager.h | 39 mm/tmem/ramster/tcp.c | 2253 ++ mm/tmem/ramster/tcp.h | 159 ++ mm/tmem/ramster/tcp_internal.h| 248 +++ 28 files changed, 10358 insertions(+), 4 deletions(-) So it's basically this commit, right? https://oss.oracle.com/git/djm/tmem.git/?p=djm/tmem.git;a=commitdiff;h=22844fe3f52d912247212408294be33 0a867937c Why on earth would you want to move that under the mm directory? Hi Pekka -- Thanks for your reply and question. MM means memory management and zcache manages physical memory to allow more pages of data to be stored in RAM. So it seems a logical place. It's not a block driver, or a network driver, or a device driver, or a filesystem... do you have a different location in the kernel in mind? Zcache does it a bit differently than all the other parts of mm because it needs to; because all the other parts of mm try to maximize the amount of physical memory that is directly addressable by threads but one can't directly address pages that have been compressed. So zcache uses the transcendent memory approach (via cleancache and frontswap) to compress/decompress clean pagecache pages and swap pages on demand. The tmem design also nicely handles both the fact that the degree of compression is unpredictable and the fact that the fraction of fixed total RAM used for compressed pages vs normal uncompressed mm pages needs to be very dynamic. Ramster does the same thing but manages it peer-to-peer across multiple systems using kernel sockets. One could argue that the dependency on sockets makes it more of a driver than mm but ramster is memory management too, just a bit more exotic. Thanks, Dan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
