RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
> From: Dan Magenheimer > Subject: RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages > more efficiently > > > From: Wanpeng Li [mailto:liw...@linux.vnet.ibm.com] > > Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled > > pages more efficiently > > > > Hi Dan, > > > > Some issues against Ramster: > > > > Sure! I am concerned about Konrad's patches adding debug.c as they > add many global variables. They are only required when ZCACHE_DEBUG > is enabled so they may be ok. If not, adding ramster variables > to debug.c may make the problem worse. Oops, I just noticed/remembered that ramster uses BOTH debugfs and sysfs. The sysfs variables are all currently required, i.e. for configuration so should not be tied to debugfs or a DEBUG config option. However, if there is a more acceptable way to implement the function of those sysfs variables, that would be fine. Thanks, Dan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
> From: Wanpeng Li [mailto:liw...@linux.vnet.ibm.com] > Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages > more efficiently > > Hi Dan, > > Some issues against Ramster: > > - Ramster who takes advantage of zcache also should support zero-filled > pages more efficiently, correct? It doesn't handle zero-filled pages well > currently. When you first posted your patchset I took a quick look at ramster and it looked like your patchset should work for ramster also. However I didn't actually run ramster to try it so there may be a bug. If it doesn't work, I would very much appreciate a patch. > - Ramster DebugFS counters are exported in /sys/kernel/mm/, but > zcache/frontswap/cleancache > all are exported in /sys/kernel/debug/, should we unify them? That would be great. > - If ramster also should move DebugFS counters to a single file like > zcache do? Sure! I am concerned about Konrad's patches adding debug.c as they add many global variables. They are only required when ZCACHE_DEBUG is enabled so they may be ok. If not, adding ramster variables to debug.c may make the problem worse. > If you confirm these issues are make sense to fix, I will start coding. ;-) That would be great. Note that I have a how-to for ramster here: https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-120817 If when you are testing you find that this how-to has mistakes, please let me know. Or feel free to add the (corrected) how-to file as a patch in your patchset. Thanks very much, Wanpeng, for your great contributions! (Ric, since you have expressed interest in ramster, if you try it and find corrections to the how-to file above, your input would be very much appreciated also!) Dan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
cc Bob On 04/07/2013 05:03 PM, Wanpeng Li wrote: On Wed, Apr 03, 2013 at 06:16:20PM +0800, Wanpeng Li wrote: Changelog: v5 -> v6: * shove variables in debug.c and in debug.h just have an extern, spotted by Konrad * update patch description, spotted by Konrad v4 -> v5: * fix compile error, reported by Fengguang, Geert * add check for !is_ephemeral(pool), spotted by Bob v3 -> v4: * handle duplication in page_is_zero_filled, spotted by Bob * fix zcache writeback in dubugfs * fix pers_pageframes|_max isn't exported in debugfs * fix static variable defined in debug.h but used in multiple C files * rebase on Greg's staging-next v2 -> v3: * increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan * replace "zero" or "zero page" by "zero_filled_page", spotted by Dan v1 -> v2: * avoid changing tmem.[ch] entirely, spotted by Dan. * don't accumulate [eph|pers]pageframe and [eph|pers]zpages for zero-filled pages, spotted by Dan * cleanup TODO list * add Dan Acked-by. Hi Dan, Some issues against Ramster: - Ramster who takes advantage of zcache also should support zero-filled pages more efficiently, correct? It doesn't handle zero-filled pages well currently. - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache all are exported in /sys/kernel/debug/, should we unify them? - If ramster also should move DebugFS counters to a single file like zcache do? If you confirm these issues are make sense to fix, I will start coding. ;-) Regards, Wanpeng Li Motivation: - Seth Jennings points out compress zero-filled pages with LZO(a lossless data compression algorithm) will waste memory and result in fragmentation. https://lkml.org/lkml/2012/8/14/347 - Dan Magenheimer add "Support zero-filled pages more efficiently" feature in zcache TODO list https://lkml.org/lkml/2013/2/13/503 Design: - For store page, capture zero-filled pages(evicted clean page cache pages and swap pages), but don't compress them, set pampd which store zpage address to 0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled case and take advantage of tmem infrastructure to transform handle-tuple(pool id, object id, and an index) to a pampd. Twice compress zero-filled pages will contribute to one zcache_[eph|pers]_pageframes count accumulated. - For load page, traverse tmem hierachical to transform handle-tuple to pampd and identify zero-filled case by pampd equal to 0x2 when filesystem reads file pages or a page needs to be swapped in, then refill the page to zero and return. Test: dd if=/dev/zero of=zerofile bs=1MB count=500 vmtouch -t zerofile vmtouch -e zerofile formula: - fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes) * 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE) - memory zcache occupy = zcache_[eph|pers]_zbytes Result: without zero-filled awareness: - fragmentation level: 98% - memory zcache occupy: 238MB with zero-filled awareness: - fragmentation level: 0% - memory zcache occupy: 0MB Wanpeng Li (3): staging: zcache: fix static variables defined in debug.h but used in mutiple C files staging: zcache: introduce zero-filled page stat count staging: zcache: clean TODO list drivers/staging/zcache/TODO |3 +- drivers/staging/zcache/debug.c | 35 +++ drivers/staging/zcache/debug.h | 79 - drivers/staging/zcache/zcache-main.c |4 ++ 4 files changed, 88 insertions(+), 33 deletions(-) -- 1.7.5.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majord...@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: mailto:"d...@kvack.org;> em...@kvack.org -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
cc Bob On 04/07/2013 05:03 PM, Wanpeng Li wrote: On Wed, Apr 03, 2013 at 06:16:20PM +0800, Wanpeng Li wrote: Changelog: v5 - v6: * shove variables in debug.c and in debug.h just have an extern, spotted by Konrad * update patch description, spotted by Konrad v4 - v5: * fix compile error, reported by Fengguang, Geert * add check for !is_ephemeral(pool), spotted by Bob v3 - v4: * handle duplication in page_is_zero_filled, spotted by Bob * fix zcache writeback in dubugfs * fix pers_pageframes|_max isn't exported in debugfs * fix static variable defined in debug.h but used in multiple C files * rebase on Greg's staging-next v2 - v3: * increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan * replace zero or zero page by zero_filled_page, spotted by Dan v1 - v2: * avoid changing tmem.[ch] entirely, spotted by Dan. * don't accumulate [eph|pers]pageframe and [eph|pers]zpages for zero-filled pages, spotted by Dan * cleanup TODO list * add Dan Acked-by. Hi Dan, Some issues against Ramster: - Ramster who takes advantage of zcache also should support zero-filled pages more efficiently, correct? It doesn't handle zero-filled pages well currently. - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache all are exported in /sys/kernel/debug/, should we unify them? - If ramster also should move DebugFS counters to a single file like zcache do? If you confirm these issues are make sense to fix, I will start coding. ;-) Regards, Wanpeng Li Motivation: - Seth Jennings points out compress zero-filled pages with LZO(a lossless data compression algorithm) will waste memory and result in fragmentation. https://lkml.org/lkml/2012/8/14/347 - Dan Magenheimer add Support zero-filled pages more efficiently feature in zcache TODO list https://lkml.org/lkml/2013/2/13/503 Design: - For store page, capture zero-filled pages(evicted clean page cache pages and swap pages), but don't compress them, set pampd which store zpage address to 0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled case and take advantage of tmem infrastructure to transform handle-tuple(pool id, object id, and an index) to a pampd. Twice compress zero-filled pages will contribute to one zcache_[eph|pers]_pageframes count accumulated. - For load page, traverse tmem hierachical to transform handle-tuple to pampd and identify zero-filled case by pampd equal to 0x2 when filesystem reads file pages or a page needs to be swapped in, then refill the page to zero and return. Test: dd if=/dev/zero of=zerofile bs=1MB count=500 vmtouch -t zerofile vmtouch -e zerofile formula: - fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes) * 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE) - memory zcache occupy = zcache_[eph|pers]_zbytes Result: without zero-filled awareness: - fragmentation level: 98% - memory zcache occupy: 238MB with zero-filled awareness: - fragmentation level: 0% - memory zcache occupy: 0MB Wanpeng Li (3): staging: zcache: fix static variables defined in debug.h but used in mutiple C files staging: zcache: introduce zero-filled page stat count staging: zcache: clean TODO list drivers/staging/zcache/TODO |3 +- drivers/staging/zcache/debug.c | 35 +++ drivers/staging/zcache/debug.h | 79 - drivers/staging/zcache/zcache-main.c |4 ++ 4 files changed, 88 insertions(+), 33 deletions(-) -- 1.7.5.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majord...@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: a href=mailto:d...@kvack.org; em...@kvack.org /a -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
From: Wanpeng Li [mailto:liw...@linux.vnet.ibm.com] Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently Hi Dan, Some issues against Ramster: - Ramster who takes advantage of zcache also should support zero-filled pages more efficiently, correct? It doesn't handle zero-filled pages well currently. When you first posted your patchset I took a quick look at ramster and it looked like your patchset should work for ramster also. However I didn't actually run ramster to try it so there may be a bug. If it doesn't work, I would very much appreciate a patch. - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache all are exported in /sys/kernel/debug/, should we unify them? That would be great. - If ramster also should move DebugFS counters to a single file like zcache do? Sure! I am concerned about Konrad's patches adding debug.c as they add many global variables. They are only required when ZCACHE_DEBUG is enabled so they may be ok. If not, adding ramster variables to debug.c may make the problem worse. If you confirm these issues are make sense to fix, I will start coding. ;-) That would be great. Note that I have a how-to for ramster here: https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-120817 If when you are testing you find that this how-to has mistakes, please let me know. Or feel free to add the (corrected) how-to file as a patch in your patchset. Thanks very much, Wanpeng, for your great contributions! (Ric, since you have expressed interest in ramster, if you try it and find corrections to the how-to file above, your input would be very much appreciated also!) Dan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
From: Dan Magenheimer Subject: RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently From: Wanpeng Li [mailto:liw...@linux.vnet.ibm.com] Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently Hi Dan, Some issues against Ramster: Sure! I am concerned about Konrad's patches adding debug.c as they add many global variables. They are only required when ZCACHE_DEBUG is enabled so they may be ok. If not, adding ramster variables to debug.c may make the problem worse. Oops, I just noticed/remembered that ramster uses BOTH debugfs and sysfs. The sysfs variables are all currently required, i.e. for configuration so should not be tied to debugfs or a DEBUG config option. However, if there is a more acceptable way to implement the function of those sysfs variables, that would be fine. Thanks, Dan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
