[PATCH 1/3] mm/page_isolation: return last tested pfn rather than failure indicator

2015-11-08 Thread Joonsoo Kim
This is preparation step to report test failed pfn in new tracepoint to analyze cma allocation failure problem. There is no functional change in this patch. Signed-off-by: Joonsoo Kim --- mm/page_isolation.c | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm

[PATCH 2/2] mm/page_ref: add tracepoint to track down page reference manipulation

2015-11-08 Thread Joonsoo Kim
m is hard to find without this page reference tracepoint facility. Enabling this feature bloat kernel text 20 KB in my configuration. textdata bss dec hex filename 120412722223424 1507328 15772024 f0a978 vmlinux_disabled 12064844 2225920 1507328 15798092

[PATCH 1/2] mm: introduce page reference manipulation functions

2015-11-08 Thread Joonsoo Kim
. With this facility, we can easily find reason of CMA allocation failure. There is no functional change in this patch. Signed-off-by: Joonsoo Kim --- arch/mips/mm/gup.c| 2 +- arch/powerpc/mm/mmu_context_hash64.c | 3 +- arch/powerpc/mm/pgtable_64.c

Re: [PATCH] arm64: Increase the max granular size

2015-11-05 Thread Joonsoo Kim
2015-11-05 19:32 GMT+09:00 Catalin Marinas : > On Thu, Nov 05, 2015 at 01:40:14PM +0900, Joonsoo Kim wrote: >> On Tue, Sep 22, 2015 at 07:59:48PM +0200, Robert Richter wrote: >> > From: Tirumalesh Chalamarla >> > >> > Increase the standard cacheline si

Re: [PATCH 3/5] mm, page_owner: copy page owner info during migration

2015-11-05 Thread Joonsoo Kim
On Thu, Nov 05, 2015 at 09:17:33AM +0100, Vlastimil Babka wrote: > On 11/05/2015 09:10 AM, Joonsoo Kim wrote: > > On Wed, Nov 04, 2015 at 04:00:59PM +0100, Vlastimil Babka wrote: > >> +void __copy_page_owner(struct page *oldpage, struct page *newpage) > >> +{ >

Re: [PATCH 1/5] mm, page_owner: print migratetype of a page, not pageblock

2015-11-05 Thread Joonsoo Kim
On Thu, Nov 05, 2015 at 09:15:01AM +0100, Vlastimil Babka wrote: > On 11/05/2015 09:09 AM, Joonsoo Kim wrote: > > On Wed, Nov 04, 2015 at 04:00:57PM +0100, Vlastimil Babka wrote: > >> The information in /sys/kernel/debug/page_owner includes the migratetype > >> declared

Re: [PATCH 3/5] mm, page_owner: copy page owner info during migration

2015-11-05 Thread Joonsoo Kim
On Wed, Nov 04, 2015 at 04:00:59PM +0100, Vlastimil Babka wrote: > The page_owner mechanism stores gfp_flags of an allocation and stack trace > that lead to it. During page migration, the original information is > essentially replaced by the allocation of free page as the migration target. > Arguab

Re: [PATCH 1/5] mm, page_owner: print migratetype of a page, not pageblock

2015-11-05 Thread Joonsoo Kim
On Wed, Nov 04, 2015 at 04:00:57PM +0100, Vlastimil Babka wrote: > The information in /sys/kernel/debug/page_owner includes the migratetype > declared during the page allocation via gfp_flags. This is also checked > against > the pageblock's migratetype, and reported as Fallback allocation if thes

Re: [PATCH] arm64: Increase the max granular size

2015-11-04 Thread Joonsoo Kim
On Tue, Sep 22, 2015 at 07:59:48PM +0200, Robert Richter wrote: > From: Tirumalesh Chalamarla > > Increase the standard cacheline size to avoid having locks in the same > cacheline. > > Cavium's ThunderX core implements cache lines of 128 byte size. With > current granulare size of 64 bytes (L1_

Re: [PATCH] arm64: Increase the max granular size

2015-11-04 Thread Joonsoo Kim
On Wed, Nov 04, 2015 at 03:39:10PM +, Catalin Marinas wrote: > On Wed, Nov 04, 2015 at 09:28:34AM -0600, Christoph Lameter wrote: > > On Wed, 4 Nov 2015, Catalin Marinas wrote: > > > > > BTW, assuming L1_CACHE_BYTES is 512 (I don't ever see this happening but > > > just in theory), we potentia

Re: [PATCH v2 9/9] mm/compaction: new threshold for compaction depleted zone

2015-10-14 Thread Joonsoo Kim
On Wed, Oct 14, 2015 at 02:28:31PM +0200, Vlastimil Babka wrote: > On 08/24/2015 04:19 AM, Joonsoo Kim wrote: > > Now, compaction algorithm become powerful. Migration scanner traverses > > whole zone range. So, old threshold for depleted zone which is designed > > to imitate

Re: [PATCH v2 9/9] mm/compaction: new threshold for compaction depleted zone

2015-10-14 Thread Joonsoo Kim
On Wed, Oct 14, 2015 at 02:28:31PM +0200, Vlastimil Babka wrote: > On 08/24/2015 04:19 AM, Joonsoo Kim wrote: > > Now, compaction algorithm become powerful. Migration scanner traverses > > whole zone range. So, old threshold for depleted zone which is designed > > to imitate

Re: [PATCH v4 8/8] zram: enable contextless compression alg in zram

2015-10-14 Thread Joonsoo Kim
On Thu, Oct 15, 2015 at 11:05:49AM +0900, Sergey Senozhatsky wrote: > On (10/14/15 16:38), Joonsoo Kim wrote: > [..] > > @@ -352,6 +352,7 @@ static ssize_t comp_algorithm_show(struct device *dev, > > size_t sz; > > struct zram *zram = dev_to_zram(dev); > >

Re: [PATCH v2 7/9] mm/compaction: redesign compaction

2015-10-14 Thread Joonsoo Kim
On Wed, Oct 14, 2015 at 01:38:07PM +0200, Vlastimil Babka wrote: > On 08/24/2015 04:19 AM, Joonsoo Kim wrote: > > [...] > > > This patch solves these problems mentioned in above. > > Freepage scanner is largely changed to scan zone from zone_start_pfn > > to zon

Re: [PATCH v4 7/8] zram: use crypto contextless compression API to (de)compress

2015-10-14 Thread Joonsoo Kim
On Thu, Oct 15, 2015 at 09:38:41AM +0900, Sergey Senozhatsky wrote: > Hi, > > On (10/14/15 16:38), Joonsoo Kim wrote: > [..] > > Creates virtual block devices called /dev/zramX (X = 0, 1, ...). > > @@ -18,9 +17,8 @@ config ZRAM > > config ZRAM_LZ4_COM

Re: [PATCH v4 8/8] zram: enable contextless compression alg in zram

2015-10-14 Thread Joonsoo Kim
On Thu, Oct 15, 2015 at 09:29:03AM +0900, Sergey Senozhatsky wrote: > Hi, > > On (10/14/15 16:38), Joonsoo Kim wrote: > [..] > > static const char * const backends[] = { > > "lzo", > > -#ifdef CONFIG_ZRAM_LZ4_COMPRESS > > "lz4",

[PATCH v4 3/8] crypto/lz4: support contextless compressiona API

2015-10-14 Thread Joonsoo Kim
Now, contextless compression API is introduced and it can reduce memory overhead in some cases. All compression algorithm will support it. Signed-off-by: Joonsoo Kim --- crypto/lz4.c | 77 1 file changed, 67 insertions(+), 10

[PATCH v4 2/8] crypto/lzo: support contextless compression API

2015-10-14 Thread Joonsoo Kim
Now, contextless compression API is introduced and it can reduce memory overhead in some cases. All compression algorithm will support it. Signed-off-by: Joonsoo Kim --- crypto/Kconfig | 1 + crypto/lzo.c | 81 -- 2 files changed, 69

[PATCH v4 4/8] crypto/deflate: support contextless compression API

2015-10-14 Thread Joonsoo Kim
Now, contextless compression API is introduced and it can reduce memory overhead in some cases. All compression algorithm will support it. Signed-off-by: Joonsoo Kim --- crypto/deflate.c | 96 ++-- 1 file changed, 86 insertions(+), 10

[PATCH v4 8/8] zram: enable contextless compression alg in zram

2015-10-14 Thread Joonsoo Kim
this patch is applied, there is no way to know current compression algorithm so it's really bad thing. Please let me know proper solution if someone have better idea. Signed-off-by: Joonsoo Kim --- Documentation/blockdev/zram.txt | 29 +++-- drivers/block/zram/Kc

[PATCH v4 6/8] zram: pass zstrm down to decompression path

2015-10-14 Thread Joonsoo Kim
compression backend require zstrm for decompression or NULL if it does not. Acked-by: Minchan Kim Signed-off-by: Sergey Senozhatsky Signed-off-by: Joonsoo Kim --- drivers/block/zram/zcomp.c| 3 ++- drivers/block/zram/zcomp.h| 3 ++- drivers/block/zram/zram_drv.c | 20

[PATCH v4 5/8] zram: make stream find and release functions static

2015-10-14 Thread Joonsoo Kim
: Sergey Senozhatsky Signed-off-by: Joonsoo Kim --- drivers/block/zram/zcomp.c| 27 +-- drivers/block/zram/zcomp.h| 8 ++-- drivers/block/zram/zram_drv.c | 6 +++--- 3 files changed, 34 insertions(+), 7 deletions(-) diff --git a/drivers/block/zram/zcomp.c b

[PATCH v4 1/8] crypto/compress: introduce contextless compression and remove unused pcomp

2015-10-14 Thread Joonsoo Kim
zram in following patches. Signed-off-by: Joonsoo Kim --- crypto/Kconfig | 17 +- crypto/Makefile| 3 +- crypto/ccompress.c | 95 + crypto/pcompress.c | 115 --- crypto/zlib.c

[PATCH v4 7/8] zram: use crypto contextless compression API to (de)compress

2015-10-14 Thread Joonsoo Kim
doesn't support zlib yet. Following patch will change this interface and enable various compression algorithm. Signed-off-by: Joonsoo Kim --- drivers/block/zram/Kconfig| 8 +++--- drivers/block/zram/Makefile | 4 +-- drivers/block/zram/zcomp.c| 63 ---

[PATCH v4 0/8] zram: introduce contextless compression API and use it on zram

2015-10-14 Thread Joonsoo Kim
aration, we can save memory in some cases and get the best performance. This patchset solves Sergey's concern perfectly and provides possibility to use various compression algorithm in zram. Thanks. Joonsoo Kim (6): crypto/compress: introduce contextless compression and remove unused

Re: [PATCH v2 8/9] mm/compaction: don't use higher order freepage than compaction aims at

2015-09-30 Thread Joonsoo Kim
On Fri, Sep 25, 2015 at 01:41:48PM +0200, Vlastimil Babka wrote: > On 08/24/2015 04:19 AM, Joonsoo Kim wrote: > > Purpose of compaction is to make high order page. To achive this purpose, > > it is the best strategy that compaction migrates contiguous used pages > > to fragme

Re: [PATCH v2 5/9] mm/compaction: allow to scan nonmovable pageblock when depleted state

2015-09-30 Thread Joonsoo Kim
On Fri, Sep 25, 2015 at 12:32:02PM +0200, Vlastimil Babka wrote: > On 08/24/2015 04:19 AM, Joonsoo Kim wrote: > > [...] > > > > > Because we just allow freepage scanner to scan non-movable pageblock > > in very limited situation, more scanning events happen. But

Re: [PATCH v2 4/9] mm/compaction: remove compaction deferring

2015-09-30 Thread Joonsoo Kim
On Fri, Sep 25, 2015 at 11:33:38AM +0200, Vlastimil Babka wrote: > On 08/24/2015 04:19 AM, Joonsoo Kim wrote: > > Now, we have a way to determine compaction depleted state and compaction > > activity will be limited according this state and depletion depth so > > compaction o

Re: [PATCH v2 2/9] mm/compaction: introduce compaction depleted state on zone

2015-09-30 Thread Joonsoo Kim
Hello, Vlastimil. First of all, thanks for review! On Fri, Sep 25, 2015 at 10:04:49AM +0200, Vlastimil Babka wrote: > On 08/24/2015 04:19 AM, Joonsoo Kim wrote: > > Further compaction attempt is deferred when some of compaction attempts > > already fails. But, after some numb

Re: [PATCH v3 9/9] zram: use crypto decompress_noctx API

2015-09-24 Thread Joonsoo Kim
On Mon, Sep 21, 2015 at 02:29:18PM +0900, Sergey Senozhatsky wrote: > On (09/18/15 14:19), Joonsoo Kim wrote: > > -/* Never return NULL, may sleep */ > > +/* May return NULL, may sleep */ > > struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp) > > {

Re: [PATCH v3 9/9] zram: use crypto decompress_noctx API

2015-09-24 Thread Joonsoo Kim
On Mon, Sep 21, 2015 at 04:56:00PM +0900, Sergey Senozhatsky wrote: > On (09/18/15 14:19), Joonsoo Kim wrote: > [..] > > + /* > > +* Prepare to use crypto decompress_noctx API. One tfm is required > > +* to initialize crypto algorithm properly and fetch correspon

Re: [PATCH v3 9/9] zram: use crypto decompress_noctx API

2015-09-24 Thread Joonsoo Kim
On Mon, Sep 21, 2015 at 12:51:28PM +0900, Minchan Kim wrote: > On Fri, Sep 18, 2015 at 02:19:24PM +0900, Joonsoo Kim wrote: > > Crypto subsystem now supports decompress_noctx API that requires > > special tfm_noctx. This tfm can be shared by multiple concurrent > > decompress

Re: [PATCH v3 8/9] zram: use crypto API for compression

2015-09-24 Thread Joonsoo Kim
On Mon, Sep 21, 2015 at 12:45:12PM +0900, Minchan Kim wrote: > Hi Joonsoo, > > On Fri, Sep 18, 2015 at 02:19:23PM +0900, Joonsoo Kim wrote: > > Until now, zram uses compression algorithm through direct call > > to core algorithm function, but, it has drawback that we need t

Re: [PATCH v3 8/9] zram: use crypto API for compression

2015-09-24 Thread Joonsoo Kim
On Mon, Sep 21, 2015 at 02:19:03PM +0900, Sergey Senozhatsky wrote: > On (09/18/15 14:19), Joonsoo Kim wrote: > [..] > > -static struct zcomp_backend *find_backend(const char *compress) > > +static const char *find_backend(const char *compress) > > { > > int i

Re: [PATCH v3 0/9] zram: introduce crypto decompress noctx API and use it on zram

2015-09-24 Thread Joonsoo Kim
On Mon, Sep 21, 2015 at 12:58:12PM +0900, Minchan Kim wrote: > On Fri, Sep 18, 2015 at 02:19:15PM +0900, Joonsoo Kim wrote: > > This patchset makes zram to use crypto API in order to support > > more compression algorithm. > > > > The reason we need to support vairou

Re: [PATCH v3 1/9] crypto: introduce decompression API that can be called via sharable tfm object

2015-09-24 Thread Joonsoo Kim
On Mon, Sep 21, 2015 at 02:38:59PM +0900, Sergey Senozhatsky wrote: > On (09/18/15 14:19), Joonsoo Kim wrote: > [..] > > @@ -61,7 +61,8 @@ static struct crypto_alg alg = { > > .cra_module = THIS_MODULE, > > .cra_u = { .compress =

Re: [PATCH v3 1/9] crypto: introduce decompression API that can be called via sharable tfm object

2015-09-24 Thread Joonsoo Kim
On Tue, Sep 22, 2015 at 08:43:46PM +0800, Herbert Xu wrote: > On Fri, Sep 18, 2015 at 02:19:16PM +0900, Joonsoo Kim wrote: > > Until now, tfm object embeds (de)compression context in it and > > (de)compression in crypto API requires tfm object to use > > this context

Re: [PATCH v3 1/9] crypto: introduce decompression API that can be called via sharable tfm object

2015-09-24 Thread Joonsoo Kim
On Mon, Sep 21, 2015 at 03:18:17PM +0900, Sergey Senozhatsky wrote: > On (09/18/15 14:19), Joonsoo Kim wrote: > [..] > > static int __init lzo_mod_init(void) > > diff --git a/include/linux/crypto.h b/include/linux/crypto.h > > index e71cb70..31152b1 100644 > >

Re: [PATCH 12/12] mm, page_alloc: Only enforce watermarks for order-0 allocations

2015-09-17 Thread Joonsoo Kim
On Wed, Sep 09, 2015 at 01:39:01PM +0100, Mel Gorman wrote: > On Tue, Sep 08, 2015 at 05:26:13PM +0900, Joonsoo Kim wrote: > > 2015-08-24 21:30 GMT+09:00 Mel Gorman : > > > The primary purpose of watermarks is to ensure that reclaim can always > > > make forward prog

Re: [PATCH 07/12] mm, page_alloc: Distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd

2015-09-17 Thread Joonsoo Kim
On Wed, Sep 09, 2015 at 01:22:03PM +0100, Mel Gorman wrote: > On Tue, Sep 08, 2015 at 03:49:58PM +0900, Joonsoo Kim wrote: > > 2015-08-24 21:09 GMT+09:00 Mel Gorman : > > > __GFP_WAIT has been used to identify atomic context in callers that hold > > > spinlocks or

Re: [PATCH 11/12] mm, page_alloc: Reserve pageblocks for high-order atomic allocations on demand

2015-09-17 Thread Joonsoo Kim
On Wed, Sep 09, 2015 at 01:32:39PM +0100, Mel Gorman wrote: > On Tue, Sep 08, 2015 at 05:01:06PM +0900, Joonsoo Kim wrote: > > 2015-08-24 21:29 GMT+09:00 Mel Gorman : > > > > > > > > > +/* > > > + * Reserve a pageblock for exclusive use of high-order

[PATCH v3 3/9] crypyo/lz4: support decompress_noctx

2015-09-17 Thread Joonsoo Kim
lz4's decompression doesn't requires any scratch buffer so it doesn't need tfm context. Hence, it can support crypto compression noctx API and this patch implements it. Signed-off-by: Joonsoo Kim --- crypto/lz4.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff

[PATCH v3 8/9] zram: use crypto API for compression

2015-09-17 Thread Joonsoo Kim
llowing patch support to use crypto decompress_noctx API and would restore the performance as is. Signed-off-by: Joonsoo Kim --- drivers/block/zram/Kconfig | 8 +++ drivers/block/zram/Makefile| 4 +--- drivers/block/zram/zcomp.c | 54 +++---

[PATCH v3 5/9] crypto/842: support decompress_noctx

2015-09-17 Thread Joonsoo Kim
842's decompression doesn't requires any scratch buffer so it doesn't need tfm context. Hence, it can support crypto compression noctx API and this patch implements it. Signed-off-by: Joonsoo Kim --- crypto/842.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff

[PATCH v3 6/9] zram: make stream find and release functions static

2015-09-17 Thread Joonsoo Kim
Signed-off-by: Joonsoo Kim --- drivers/block/zram/zcomp.c| 27 +-- drivers/block/zram/zcomp.h| 8 ++-- drivers/block/zram/zram_drv.c | 6 +++--- 3 files changed, 34 insertions(+), 7 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram

[PATCH v3 1/9] crypto: introduce decompression API that can be called via sharable tfm object

2015-09-17 Thread Joonsoo Kim
ple tfm if decompression doesn't require any context to operate. Signed-off-by: Joonsoo Kim --- crypto/842.c | 3 ++- crypto/compress.c | 36 crypto/crypto_null.c | 3 ++- crypto/deflate.c | 3 ++- crypto/lz4.c | 3 ++-

[PATCH v3 4/9] crypto/lz4hc: support decompress_noctx

2015-09-17 Thread Joonsoo Kim
lz4hc's decompression doesn't requires any scratch buffer so it doesn't need tfm context. Hence, it can support crypto compression noctx API and this patch implements it. Signed-off-by: Joonsoo Kim --- crypto/lz4hc.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-

[PATCH v3 9/9] zram: use crypto decompress_noctx API

2015-09-17 Thread Joonsoo Kim
x Read10411701.88 6426911.62 9423894.38 Re-read 10017386.62 6428218.88 11000063.50 Signed-off-by: Joonsoo Kim --- drivers/block/zram/zcomp.c | 24 +++- drivers/block/zram/zcomp.h | 1 + 2 files changed, 24 insertions(+), 1 deletion(-) diff --git

[PATCH v3 7/9] zram: pass zstrm down to decompression path

2015-09-17 Thread Joonsoo Kim
compression backend require zstrm for decompression or NULL if it does not. Signed-off-by: Sergey Senozhatsky Signed-off-by: Joonsoo Kim --- drivers/block/zram/zcomp.c| 3 ++- drivers/block/zram/zcomp.h| 3 ++- drivers/block/zram/zram_drv.c | 20 3 files changed, 20

[PATCH v3 2/9] crypto/lzo: support decompress_noctx

2015-09-17 Thread Joonsoo Kim
lzo's decompression doesn't requires any scratch buffer so it doesn't need tfm context. Hence, it can support crypto compression noctx API and this patch implements it. Signed-off-by: Joonsoo Kim --- crypto/lzo.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff

[PATCH v3 0/9] zram: introduce crypto decompress noctx API and use it on zram

2015-09-17 Thread Joonsoo Kim
/sec zram-base vs zram-crypto vs zram-crypto-noctx Read10411701.88 6426911.62 9423894.38 Re-read 10017386.62 6428218.88 1163.50 Thanks. Joonsoo Kim (7): crypto: introduce decompression API that can be called via sharable tfm object crypto/lzo: s

[PATCH] mm/slab: fix unexpected index mapping result of kmalloc_size(INDEX_NODE+1)

2015-09-17 Thread Joonsoo Kim
blem of kmalloc_size(INDEX_NODE + 1) and removes the BUG by adding 'size >= 256' check to guarantee that all necessary small sized slabs are initialized regardless sequence of slab size in mapping table. Fixes: e33660165c90 ("slab: Use common kmalloc_index/kmalloc_size...") Rep

Re: [PATCH 12/12] mm, page_alloc: Only enforce watermarks for order-0 allocations

2015-09-08 Thread Joonsoo Kim
2015-08-24 21:30 GMT+09:00 Mel Gorman : > The primary purpose of watermarks is to ensure that reclaim can always > make forward progress in PF_MEMALLOC context (kswapd and direct reclaim). > These assume that order-0 allocations are all that is necessary for > forward progress. > > High-order water

Re: [PATCH 11/12] mm, page_alloc: Reserve pageblocks for high-order atomic allocations on demand

2015-09-08 Thread Joonsoo Kim
2015-08-24 21:29 GMT+09:00 Mel Gorman : > High-order watermark checking exists for two reasons -- kswapd high-order > awareness and protection for high-order atomic requests. Historically the > kernel depended on MIGRATE_RESERVE to preserve min_free_kbytes as high-order > free pages for as long as

Re: [PATCH 07/12] mm, page_alloc: Distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd

2015-09-07 Thread Joonsoo Kim
2015-08-24 21:09 GMT+09:00 Mel Gorman : > __GFP_WAIT has been used to identify atomic context in callers that hold > spinlocks or are in interrupts. They are expected to be high priority and > have access one of two watermarks lower than "min" which can be referred > to as the "atomic reserve". __G

Re: [PATCH 3/3] mm, compaction: disginguish contended status in tracepoint

2015-09-06 Thread Joonsoo Kim
On Thu, Aug 27, 2015 at 05:24:04PM +0200, Vlastimil Babka wrote: > Compaction returns prematurely with COMPACT_PARTIAL when contended or has > fatal > signal pending. This is ok for the callers, but might be misleading in the > traces, as the usual reason to return COMPACT_PARTIAL is that we think

Re: [PATCH v2 1/9] mm/compaction: skip useless pfn when updating cached pfn

2015-09-06 Thread Joonsoo Kim
On Mon, Aug 24, 2015 at 11:07:12AM +0200, Vlastimil Babka wrote: > On 08/24/2015 04:19 AM, Joonsoo Kim wrote: > >Cached pfn is used to determine the start position of scanner > >at next compaction run. Current cached pfn points the skipped pageblock > >so we uselessly checks

Re: [PATCH 0/2] prepare zram for crypto api-powered zcomp

2015-09-06 Thread Joonsoo Kim
On Tue, Sep 01, 2015 at 11:12:39AM +0900, Sergey Senozhatsky wrote: > On (09/01/15 11:06), Minchan Kim wrote: > > Thanks. > > > > I want to include this patchset in Joonsoo's crypto support patch > > if you don't mind. Hello, Sorry for long delay. I will work crypto support with this patchset. S

[PATCH v2 2/9] mm/compaction: introduce compaction depleted state on zone

2015-08-23 Thread Joonsoo Kim
d and this threshold is also adjusted to that change. In this patch, only state definition is implemented. There is no action for this new state so no functional change. But, following patch will add some handling for this new state. Signed-off-by: Joonsoo Kim --- include/linux/mmzone.h | 3 ++

[PATCH v2 6/9] mm/compaction: manage separate skip-bits for migration and free scanner

2015-08-23 Thread Joonsoo Kim
will be mentioned on following patch. Please refer it. Signed-off-by: Joonsoo Kim --- include/linux/mmzone.h | 3 --- include/linux/pageblock-flags.h | 37 +++-- mm/compaction.c | 25 - mm/page_al

[PATCH v2 5/9] mm/compaction: allow to scan nonmovable pageblock when depleted state

2015-08-23 Thread Joonsoo Kim
le pageblock in very limited situation, more scanning events happen. But, allowing in very limited situation results in a very important benefit that memory isn't fragmented more than before. Fragmentation effect is measured on following patch so please refer it. Signed-off-by: Joonsoo Kim --- in

[PATCH v2 7/9] mm/compaction: redesign compaction

2015-08-23 Thread Joonsoo Kim
sting depletion check with this new algorithm. Signed-off-by: Joonsoo Kim --- mm/compaction.c | 145 ++-- 1 file changed, 77 insertions(+), 68 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index a259608..ca4d6d1 100644 --- a/mm

[PATCH v2 8/9] mm/compaction: don't use higher order freepage than compaction aims at

2015-08-23 Thread Joonsoo Kim
han compaction order. Signed-off-by: Joonsoo Kim --- mm/compaction.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/mm/compaction.c b/mm/compaction.c index ca4d6d1..e61ee77 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -455,6 +455,7 @@ static unsigned long isolate_fr

[PATCH v2 9/9] mm/compaction: new threshold for compaction depleted zone

2015-08-23 Thread Joonsoo Kim
45 compact_migrate_scanned 69714502 2307433 compact_free_scanned20243121 2325295 This looks like reasonable trade-off. Signed-off-by: Joonsoo Kim --- mm/compaction.c | 19 --- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/mm/compaction.c b/mm/

[PATCH v2 4/9] mm/compaction: remove compaction deferring

2015-08-23 Thread Joonsoo Kim
activity limit. Various functions are renamed and tracepoint outputs are changed due to this removing. Signed-off-by: Joonsoo Kim --- include/linux/compaction.h| 14 +-- include/linux/mmzone.h| 3 +- include/trace/events/compaction.h | 30 ++--- mm/compaction.c

[PATCH v2 3/9] mm/compaction: limit compaction activity in compaction depleted state

2015-08-23 Thread Joonsoo Kim
ring. Signed-off-by: Joonsoo Kim --- mm/compaction.c | 41 + mm/internal.h | 1 + 2 files changed, 42 insertions(+) diff --git a/mm/compaction.c b/mm/compaction.c index de96e9d..c6b8277 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -130,6 +13

[PATCH v2 1/9] mm/compaction: skip useless pfn when updating cached pfn

2015-08-23 Thread Joonsoo Kim
k, we don't need to do this check. Signed-off-by: Joonsoo Kim --- mm/compaction.c | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 6ef2fdf..c2d3d6a 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -261,10 +261

[PATCH v2 0/9] mm/compaction: redesign compaction

2015-08-23 Thread Joonsoo Kim
ll decreases but just a little. In conclusion, this series solves three limitations of current compaction algorithm successfully. :) This patchset is based on linux-next-20150515 and not targeted to merge. After merge window is finished, I will rebase it to recent kernel and send v3. Feel free to com

[PATCH] mm/compaction: correct to flush migrated pages if pageblock skip happens

2015-08-20 Thread Joonsoo Kim
() before entering actual isolation. This patch implements it and fix the problem. Signed-off-by: Joonsoo Kim --- mm/compaction.c | 30 +++--- mm/internal.h | 1 + 2 files changed, 16 insertions(+), 15 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c inde

Re: [PATCH v2 1/8] crypto: support (de)compression API that doesn't require tfm object

2015-08-20 Thread Joonsoo Kim
On Thu, Aug 20, 2015 at 03:50:35PM +0800, Herbert Xu wrote: > On Thu, Aug 20, 2015 at 04:52:17PM +0900, Joonsoo Kim wrote: > > > > Hmm... I guess there is no problem. crypto_alg object fetched by > > crypto_get_comp() introduced in this patch could be hardware device > &

Re: [PATCH v2 1/8] crypto: support (de)compression API that doesn't require tfm object

2015-08-20 Thread Joonsoo Kim
On Thu, Aug 20, 2015 at 02:47:28PM +0800, Herbert Xu wrote: > On Thu, Aug 20, 2015 at 03:34:57PM +0900, Joonsoo Kim wrote: > > Until now, tfm object embeds (de)compression context in it and > > (de)compression in crypto API requires tfm object to use > > this context

[PATCH v2 2/8] crypto/lzo: support decompress_noctx

2015-08-19 Thread Joonsoo Kim
lzo's decompression doesn't requires any scratch buffer so it doesn't need tfm context. Hence, it can support crypto compression noctx API and this patch implements it. Signed-off-by: Joonsoo Kim --- crypto/lzo.c | 18 +- 1 file changed, 17 insertions(+), 1 de

[PATCH v2 8/8] zram: use decompress_noctx

2015-08-19 Thread Joonsoo Kim
2 9423894.38 Re-read 10017386.62 6428218.88 11000063.50 Signed-off-by: Joonsoo Kim --- drivers/block/zram/zcomp.c| 28 +++- drivers/block/zram/zcomp.h| 9 - drivers/block/zram/zram_drv.c | 9 + 3 files changed, 40 insertions(+), 6

[PATCH v2 5/8] crypto/842: support decompress_noctx

2015-08-19 Thread Joonsoo Kim
842's decompression doesn't requires any scratch buffer so it doesn't need tfm context. Hence, it can support crypto compression noctx API and this patch implements it. Signed-off-by: Joonsoo Kim --- crypto/842.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff

[PATCH v2 6/8] zram: change zcomp_compress interface

2015-08-19 Thread Joonsoo Kim
zram regards zstrm's buffer as compression destination buffer, but, it is not intuitive and there is no document about it. Providing destination buffer to zcomp_compress() directly seems more intuitive interface to me so this patch changes zcomp_compress interface. Signed-off-by: Joonso

[PATCH v2 4/8] crypto/lz4hc: support decompress_noctx

2015-08-19 Thread Joonsoo Kim
lz4hc's decompression doesn't requires any scratch buffer so it doesn't need tfm context. Hence, it can support crypto compression noctx API and this patch implements it. Signed-off-by: Joonsoo Kim --- crypto/lz4hc.c | 17 - 1 file changed, 16 insertions(+), 1 de

[PATCH v2 7/8] zram: use crypto API for compression

2015-08-19 Thread Joonsoo Kim
tch support to use crypto noctx API and would restore the performance as is. Signed-off-by: Joonsoo Kim --- drivers/block/zram/Kconfig | 13 +-- drivers/block/zram/Makefile| 4 +- drivers/block/zram/zcomp.c | 84 +-

[PATCH v2 3/8] crypyo/lz4: support decompress_noctx

2015-08-19 Thread Joonsoo Kim
lz4's decompression doesn't requires any scratch buffer so it doesn't need tfm context. Hence, it can support crypto compression noctx API and this patch implements it. Signed-off-by: Joonsoo Kim --- crypto/lz4.c | 17 - 1 file changed, 16 insertions(+), 1 de

[PATCH v2 1/8] crypto: support (de)compression API that doesn't require tfm object

2015-08-19 Thread Joonsoo Kim
ssion function via crypto_alg, instead of tfm object. crypto_alg is required to get appropriate (de)compression function pointer. This can reduce overhead of maintaining multiple tfm if (de)compression doesn't require any context to operate. Signed-off-by: Joonsoo Kim --- crypto/842.c

[PATCH v2 0/8] zram: introduce crypto compress noctx API and use it on zram

2015-08-19 Thread Joonsoo Kim
/sec zram-base vs zram-crypto vs zram-crypto-noctx Read10411701.88 6426911.62 9423894.38 Re-read 10017386.62 6428218.88 1163.50 Thanks. Joonsoo Kim (8): crypto: support (de)compression API that doesn't require tfm object crypto/l

Re: [PATCH 1/2] crypto: export crypto_alg_list and rwsem

2015-08-13 Thread Joonsoo Kim
On Thu, Aug 13, 2015 at 03:29:18PM +0800, Herbert Xu wrote: > On Thu, Aug 13, 2015 at 04:30:31PM +0900, Joonsoo Kim wrote: > > > > How about introducing new functions to search supported algorithm in > > kernel-side? As crypto API is used in more places, this interface > &

Re: [PATCH 2/2] zram: use crypto API to compress the page

2015-08-13 Thread Joonsoo Kim
On Thu, Aug 13, 2015 at 03:32:33PM +0800, Herbert Xu wrote: > On Thu, Aug 13, 2015 at 04:19:41PM +0900, Joonsoo Kim wrote: > > > > If that optimization is really needed for the case that doesn't need > > tfm except fetching function pointer, we can implement sharable t

Re: [PATCH 1/2] crypto: export crypto_alg_list and rwsem

2015-08-13 Thread Joonsoo Kim
On Thu, Aug 13, 2015 at 02:38:23PM +0800, Herbert Xu wrote: > On Thu, Aug 13, 2015 at 03:37:55PM +0900, Joonsoo Kim wrote: > > > > Is there any way to access netlink interface and get the output from > > kernel-side? I'd like to show information through > >

Re: [PATCH 2/2] zram: use crypto API to compress the page

2015-08-13 Thread Joonsoo Kim
On Thu, Aug 13, 2015 at 12:51:13PM +0900, Sergey Senozhatsky wrote: > On (08/13/15 11:24), Joonsoo Kim wrote: > > Until now, zram uses compression algorithm through direct call > > to core algorithm function, but, it has drawback that we need to add > > compression algorith

Re: [PATCH 1/2] crypto: export crypto_alg_list and rwsem

2015-08-12 Thread Joonsoo Kim
On Thu, Aug 13, 2015 at 11:19:54AM +0800, Herbert Xu wrote: > On Thu, Aug 13, 2015 at 11:24:13AM +0900, Joonsoo Kim wrote: > > Until now, zram uses compression algorithm through direct call > > to core algorithm function, but, it has drawback that we need to add > > compressi

[PATCH 1/2] crypto: export crypto_alg_list and rwsem

2015-08-12 Thread Joonsoo Kim
hm in cryto subsystem. To implement it on zram-side, crypto_alg_list and rwsem should be exported so this patch do it. Signed-off-by: Joonsoo Kim --- crypto/internal.h | 2 -- include/linux/crypto.h | 4 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/crypto/internal.h b/c

[PATCH 2/2] zram: use crypto API to compress the page

2015-08-12 Thread Joonsoo Kim
patch will help that situation. Signed-off-by: Joonsoo Kim --- drivers/block/zram/Kconfig | 13 +-- drivers/block/zram/Makefile| 4 +- drivers/block/zram/zcomp.c | 87 ++ drivers/block/zram/zcomp.h | 34 + drive

Re: [PATCH] zram: fix possible race when checking idle_strm

2015-08-11 Thread Joonsoo Kim
On Mon, Aug 10, 2015 at 11:16:59AM +0900, Sergey Senozhatsky wrote: > Hello Joonsoo, > > On (08/10/15 09:32), Joonsoo Kim wrote: > > > on the other hand... it's actually > > > > > > wait_event() is > > > > > > if (condition) > &

Re: [PATCH] zram: fix possible race when checking idle_strm

2015-08-11 Thread Joonsoo Kim
On Tue, Aug 11, 2015 at 08:26:33AM +0900, Minchan Kim wrote: > Hi Joonsoo, > > On Mon, Aug 10, 2015 at 09:32:30AM +0900, Joonsoo Kim wrote: > > On Fri, Aug 07, 2015 at 06:58:16PM +0900, Sergey Senozhatsky wrote: > > > On (08/07/15 18:14), Sergey Senozhatsky wrote: >

Re: [PATCH v2] mm/slub: don't wait for high-order page allocation

2015-08-09 Thread Joonsoo Kim
On Fri, Aug 07, 2015 at 05:05:01PM +0200, Michal Hocko wrote: > On Fri 07-08-15 11:10:03, Joonsoo Kim wrote: > [...] > > diff --git a/mm/slub.c b/mm/slub.c > > index 257283f..52b9025 100644 > > --- a/mm/slub.c > > +++ b/mm/slub.c > > @@ -1364,6 +1364,8 @@ stat

Re: [PATCH] zram: fix possible race when checking idle_strm

2015-08-09 Thread Joonsoo Kim
On Fri, Aug 07, 2015 at 11:49:04PM +0900, Minchan Kim wrote: > Hi Joonsoo, > > On Fri, Aug 07, 2015 at 05:03:29PM +0900, Joonsoo Kim wrote: > > Currently, when we enter the wait state due to lack of idle stream, > > we check idle_strm list without holding the lock in expand

Re: [PATCH] zram: fix possible race when checking idle_strm

2015-08-09 Thread Joonsoo Kim
On Fri, Aug 07, 2015 at 06:58:16PM +0900, Sergey Senozhatsky wrote: > On (08/07/15 18:14), Sergey Senozhatsky wrote: > > hm... I need to think about it more. > > > > we do wake_up every time we put stream back to the list > > > > zcomp_strm_multi_release(): > > > > spin_lock(&zs->strm_lo

[PATCH] zram: fix possible race when checking idle_strm

2015-08-07 Thread Joonsoo Kim
e any error related to this race, it should be fixed. To fix it, we should check idle_strm with holding the lock and this patch implements it. Signed-off-by: Joonsoo Kim --- drivers/block/zram/zcomp.c | 15 +-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/

[PATCH v2] mm/slub: don't wait for high-order page allocation

2015-08-06 Thread Joonsoo Kim
er occupies 150 MB memory. Workload: hackbench -g 20 -l 1000 Average result by 10 runs (Base va Patched) elapsed_time(s): 4.3468 vs 2.9838 compact_stall: 461.7 vs 73.6 pgmigrate_success: 28315.9 vs 7256.1 Signed-off-by: Joonsoo Kim --- mm/slub.c | 2 ++ 1 file changed, 2 insertions(+) diff

Re: [PATCH] mm/slub: don't wait for high-order page allocation

2015-08-06 Thread Joonsoo Kim
On Tue, Aug 04, 2015 at 03:15:26PM +0200, Michal Hocko wrote: > On Fri 31-07-15 10:09:50, Joonsoo Kim wrote: > > Almost description is copied from commit fb05e7a89f50 > > ("net: don't wait for order-3 page allocation"). > > > > I saw excessive direct m

Re: [PATCH 10/10] mm, page_alloc: Only enforce watermarks for order-0 allocations

2015-07-31 Thread Joonsoo Kim
On Fri, Jul 31, 2015 at 08:19:07AM +0100, Mel Gorman wrote: > On Fri, Jul 31, 2015 at 03:08:38PM +0900, Joonsoo Kim wrote: > > On Mon, Jul 20, 2015 at 09:00:19AM +0100, Mel Gorman wrote: > > > From: Mel Gorman > > > > > > The primary purpose of watermarks

Re: [PATCH 09/10] mm, page_alloc: Reserve pageblocks for high-order atomic allocations on demand

2015-07-31 Thread Joonsoo Kim
On Fri, Jul 31, 2015 at 09:25:13AM +0200, Vlastimil Babka wrote: > On 07/31/2015 09:11 AM, Mel Gorman wrote: > >On Fri, Jul 31, 2015 at 02:54:07PM +0900, Joonsoo Kim wrote: > >>Hello, Mel. > >> > >>On Mon, Jul 20, 2015 at 09:00:18AM +0100, Mel Gorman wrote: &g

Re: [PATCH 09/10] mm, page_alloc: Reserve pageblocks for high-order atomic allocations on demand

2015-07-31 Thread Joonsoo Kim
On Fri, Jul 31, 2015 at 08:11:13AM +0100, Mel Gorman wrote: > On Fri, Jul 31, 2015 at 02:54:07PM +0900, Joonsoo Kim wrote: > > Hello, Mel. > > > > On Mon, Jul 20, 2015 at 09:00:18AM +0100, Mel Gorman wrote: > > > From: Mel Gorman > > > > > > High

Re: [RFC PATCH 00/10] Remove zonelist cache and high-order watermark checking

2015-07-30 Thread Joonsoo Kim
On Mon, Jul 20, 2015 at 09:00:09AM +0100, Mel Gorman wrote: > From: Mel Gorman > > This series started with the idea to move LRU lists to pgdat but this > part was more important to start with. It was written against 4.2-rc1 but > applies to 4.2-rc3. > > The zonelist cache has been around for a

Re: [PATCH 10/10] mm, page_alloc: Only enforce watermarks for order-0 allocations

2015-07-30 Thread Joonsoo Kim
On Mon, Jul 20, 2015 at 09:00:19AM +0100, Mel Gorman wrote: > From: Mel Gorman > > The primary purpose of watermarks is to ensure that reclaim can always > make forward progress in PF_MEMALLOC context (kswapd and direct reclaim). > These assume that order-0 allocations are all that is necessary f

<    3   4   5   6   7   8   9   10   11   12   >