Re: [PATCH] crypto/async_pq: use __free_page() instead of put_page()

[Joonsoo Kim]

2016-03-01 3:04 GMT+09:00 Dan Williams :
> On Mon, Feb 29, 2016 at 1:33 AM, Arnd Bergmann  wrote:
>> The addition of tracepoints to the page reference tracking had an
>> unfortunate side-effect in at least one driver that calls put_page
>> from its exit function, resulting in a link error:
>>
>> `.exit.text' referenced in section `__jump_table' of crypto/built-in.o: 
>> defined in discarded section `.exit.text' of crypto/built-in.o
>>
>> From a cursory look at that this driver, it seems that it may be
>> doing the wrong thing here anyway, as the page gets allocated
>> using 'alloc_page()', and should be freed using '__free_page()'
>> rather than 'put_page()'.
>>
>> With this patch, I no longer get any other build errors from the
>> page_ref patch, so hopefully we can assume that it's always wrong
>> to call any of those functions from __exit code, and that no other
>> driver does it.
>>
>> Fixes: 0f80830dd044 ("mm/page_ref: add tracepoint to track down page 
>> reference manipulation")
>> Signed-off-by: Arnd Bergmann 
>
> Acked-by: Dan Williams 
>
> Vinod, will you take this one?

Problematic patch ("mm/page_ref: ~~~") is not yet merged one. It is on mmotm
and this fix should go together with it or before it. I think that
handling this fix by
Andrew is easier to all.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] [RFC] mm/page_ref, crypto/async_pq: don't put_page from __exit

[Joonsoo Kim]

2016-02-29 6:57 GMT+09:00 Arnd Bergmann :
> The addition of tracepoints to the page reference tracking had an
> unfortunate side-effect in at least one driver that calls put_page
> from its exit function, resulting in a link error:
>
> `.exit.text' referenced in section `__jump_table' of crypto/built-in.o: 
> defined in discarded section `.exit.text' of crypto/built-in.o
>
> I could not come up with a nice solution that ignores __jump_table
> entries in discarded code, so we probably now have to treat this
> as something a driver is not allowed to do. Removing the __exit
> annotation avoids the problem in this particular driver, but the
> same problem could come back any time in other code.
>
> On a related problem regarding the runtime patching for SMP
> operations on ARM uniprocessor systems, we resorted to not
> drop the .exit section at link time, but that doesn't seem
> appropriate here.
>
> Signed-off-by: Arnd Bergmann 
> Fixes: 0f80830dd044 ("mm/page_ref: add tracepoint to track down page 
> reference manipulation")
> ---
>  crypto/async_tx/async_pq.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/crypto/async_tx/async_pq.c b/crypto/async_tx/async_pq.c
> index c0748bbd4c08..be167145aa55 100644
> --- a/crypto/async_tx/async_pq.c
> +++ b/crypto/async_tx/async_pq.c
> @@ -442,7 +442,7 @@ static int __init async_pq_init(void)
> return -ENOMEM;
>  }
>
> -static void __exit async_pq_exit(void)
> +static void async_pq_exit(void)
>  {
> put_page(pq_scribble_page);
>  }

Hello, Arnd.

I think that we can avoid this error by using __free_page().
It would not be inlined so calling it would have no problem.

Could you test it, please?

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 04/10] crypto/compress: add asynchronous compression support

[Joonsoo Kim]

2016-02-04 12:28 GMT+09:00 Herbert Xu :
> On Thu, Feb 04, 2016 at 11:25:27AM +0800, Li, Weigang wrote:
>>
>> Please can you advise how to get the acomp patch accepted?
>
> Can you do a posting of these patches without scomp so we can
> evaluate the effects?
>

Do you think not to merge scomp? Please let me know your overall
plan about this.?

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 04/10] crypto/compress: add asynchronous compression support

[Joonsoo Kim]

On Fri, Jan 29, 2016 at 06:09:01PM +0800, Herbert Xu wrote:
> On Thu, Jan 28, 2016 at 12:19:42PM +0900, Joonsoo Kim wrote:
> >
> > I have tested asynchronous compression APIs in zram and I saw
> > regression. Atomic allocation and setting up SG lists are culprit
> > for this regression. Moreover, zram optimizes linearisation
> 
> So which is it, atomic allocations or setting up SG lists? There
> is nothing in acomp that requires you to do an atomic allocation.

Atomic allocation are called for linearisation when needed. Zram's
compressed content is usually stored in two physically separate pages
so linearisation is needed. See scomp_map().

Setting up SG lists means that to use acomp, sg_init_table(),
sg_set_page() are need to be called by zram unlike the case just
passing the pointer based buffer.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 04/10] crypto/compress: add asynchronous compression support

[Joonsoo Kim]

Hello, Herbert.

On Wed, Jan 27, 2016 at 04:09:26PM +0800, Herbert Xu wrote:
> On Wed, Jan 27, 2016 at 04:03:55PM +0800, Herbert Xu wrote:
> > On Wed, Jan 27, 2016 at 03:59:05PM +0800, Li, Weigang wrote:
> > >
> > > The acomp is also SG-based, while scomp only accepts flat buffer.
> > 
> > Right, but do we need a pointer-based scomp at all? IPComp would
> > certainly be better off with an SG-based interface.  Any other
> > users of compression are presumably dealing with large amounts
> > of data where an SG interface would make more sense.
> > 
> > A pointer interface makes sense for shash because you may be hashing
> > 16 bytes at a time.  Nobody sane is going to be compressing 16 bytes,
> > or are they?

Hmm... I'm not an expert on this area so below of my analysis would be
wrong.

Some of compression example in kernel do compression with PAGE_SIZE and
compressed size is naturally less than PAGE_SIZE. There are many cases
that compressed size is below than 100 bytes. To keep and handle data,
they somtimes use kmalloced buffer and I guess it isn't suitable for
SG-based interface. Is it okay to use SG-based interface
if kmalloced object covers two pages?

And, even, someone uses vmalloced buffer that's also not suitable for
SG-based interface. For large amount data case, vmalloced buffer is
more suitable and it needs pointer interface.

> Note that I'm fine with keeping an scomp interface underneath
> for those algorithms where the best way to handle SG input is
> to linearise things.  But I would prefer that this interface is
> not exposed to kernel users unless it is absolutely required.

I have tested asynchronous compression APIs in zram and I saw
regression. Atomic allocation and setting up SG lists are culprit
for this regression. Moreover, zram optimizes linearisation
to get the best performance so it has two Kconfig options. One of
them cannot be support in the general layer. Not supporting pointer
based APIs unavoidably causes regression to zram in this case.

And, S/W compression algorithms that exists in kernel
are pointer based so it's natural to support it first
in crypto compression. That will help existing users to change
their direct library call to crypto compression without any regression.
They may be happy to change it because they just could get more
algorithm support without any loss.

I think that supporting pointer-based interface has some merits
mentioned above. However, I'm not sure what's the benefit if we only
support SG-based interface and it's bigger than above.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 07/10] crypto/lz4hc: support new compression APIs

[Joonsoo Kim]

Now, new compression APIs are introduced and it has some benefits.
Let's support it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |  1 +
 crypto/lz4hc.c | 91 +++---
 2 files changed, 82 insertions(+), 10 deletions(-)

diff --git a/crypto/Kconfig b/crypto/Kconfig
index 72ab0d7..2641a60 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1529,6 +1529,7 @@ config CRYPTO_LZ4HC
select CRYPTO_ALGAPI
select LZ4HC_COMPRESS
select LZ4_DECOMPRESS
+   select CRYPTO_COMPRESS2
help
  This is the LZ4 high compression mode algorithm.
 
diff --git a/crypto/lz4hc.c b/crypto/lz4hc.c
index a1d3b5b..569a8a7 100644
--- a/crypto/lz4hc.c
+++ b/crypto/lz4hc.c
@@ -23,36 +23,53 @@
 #include 
 #include 
 
+#include 
+
 struct lz4hc_ctx {
void *lz4hc_comp_mem;
 };
 
+static void *lz4hc_alloc_ctx(struct crypto_scomp *tfm)
+{
+   void *ctx;
+
+   ctx = vmalloc(LZ4HC_MEM_COMPRESS);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   return ctx;
+}
+
 static int lz4hc_init(struct crypto_tfm *tfm)
 {
struct lz4hc_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   ctx->lz4hc_comp_mem = vmalloc(LZ4HC_MEM_COMPRESS);
-   if (!ctx->lz4hc_comp_mem)
+   ctx->lz4hc_comp_mem = lz4hc_alloc_ctx(NULL);
+   if (IS_ERR(ctx->lz4hc_comp_mem))
return -ENOMEM;
 
return 0;
 }
 
+static void lz4hc_free_ctx(struct crypto_scomp *tfm, void *ctx)
+{
+   vfree(ctx);
+}
+
 static void lz4hc_exit(struct crypto_tfm *tfm)
 {
struct lz4hc_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   vfree(ctx->lz4hc_comp_mem);
+   lz4hc_free_ctx(NULL, ctx->lz4hc_comp_mem);
 }
 
-static int lz4hc_compress_crypto(struct crypto_tfm *tfm, const u8 *src,
-   unsigned int slen, u8 *dst, unsigned int *dlen)
+static int __lz4hc_compress_crypto(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
-   struct lz4hc_ctx *ctx = crypto_tfm_ctx(tfm);
size_t tmp_len = *dlen;
int err;
 
-   err = lz4hc_compress(src, slen, dst, _len, ctx->lz4hc_comp_mem);
+   err = lz4hc_compress(src, slen, dst, _len, ctx);
 
if (err < 0)
return -EINVAL;
@@ -61,8 +78,23 @@ static int lz4hc_compress_crypto(struct crypto_tfm *tfm, 
const u8 *src,
return 0;
 }
 
-static int lz4hc_decompress_crypto(struct crypto_tfm *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int *dlen)
+static int lz4hc_scompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __lz4hc_compress_crypto(src, slen, dst, dlen, ctx);
+}
+
+static int lz4hc_compress_crypto(struct crypto_tfm *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct lz4hc_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   return __lz4hc_compress_crypto(src, slen, dst, dlen,
+   ctx->lz4hc_comp_mem);
+}
+
+static int __lz4hc_decompress_crypto(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
int err;
size_t tmp_len = *dlen;
@@ -76,6 +108,18 @@ static int lz4hc_decompress_crypto(struct crypto_tfm *tfm, 
const u8 *src,
return err;
 }
 
+static int lz4hc_sdecompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __lz4hc_decompress_crypto(src, slen, dst, dlen, NULL);
+}
+
+static int lz4hc_decompress_crypto(struct crypto_tfm *tfm, const u8 *src,
+ unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   return __lz4hc_decompress_crypto(src, slen, dst, dlen, NULL);
+}
+
 static struct crypto_alg alg_lz4hc = {
.cra_name   = "lz4hc",
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -89,14 +133,41 @@ static struct crypto_alg alg_lz4hc = {
.coa_decompress = lz4hc_decompress_crypto } }
 };
 
+static struct scomp_alg scomp = {
+   .alloc_ctx  = lz4hc_alloc_ctx,
+   .free_ctx   = lz4hc_free_ctx,
+   .compress   = lz4hc_scompress,
+   .decompress = lz4hc_sdecompress,
+   .base   = {
+   .cra_name   = "lz4hc",
+   .cra_driver_name= "lz4hc-scomp",
+   .cra_flags  = CRYPTO_ALG_TYPE_SCOMPRESS |
+   CRYPTO_SCOMP_DECOMP_NOCTX,
+   .cra_module = THIS_MODULE,
+   }
+};
+
 static int __init lz4hc_mod_init(void)
 {
-   return crypto_register_alg(_lz4hc);
+   int ret;
+
+   ret = crypto_register_alg(_lz4hc);
+   if (ret)
+   retur

[PATCH v2 04/10] crypto/compress: add asynchronous compression support

[Joonsoo Kim]

From: Weigang Li <weigang...@intel.com>

Now, asynchronous compression APIs are supported. There is no asynchronous
compression driver now but this APIs can be used as front-end to
synchronous compression algorithm. In this case, scatterlist would be
linearlized when needed so it would cause some overhead.

Signed-off-by: Weigang Li <weigang...@intel.com>
Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |   3 +-
 crypto/Makefile|   3 +-
 crypto/acompress.c | 164 
 crypto/scompress.c | 170 +
 include/crypto/compress.h  | 253 +
 include/crypto/internal/compress.h |   4 +
 include/linux/crypto.h |   2 +
 7 files changed, 596 insertions(+), 3 deletions(-)
 create mode 100644 crypto/acompress.c
 create mode 100644 include/crypto/internal/compress.h

diff --git a/crypto/Kconfig b/crypto/Kconfig
index 7159520..f22f4e9 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -84,7 +84,7 @@ config CRYPTO_RNG_DEFAULT
tristate
select CRYPTO_DRBG_MENU
 
-config CRYPTO_SCOMPRESS
+config CRYPTO_COMPRESS2
tristate
select CRYPTO_ALGAPI2
 
@@ -1503,7 +1503,6 @@ config CRYPTO_LZO
select CRYPTO_ALGAPI
select LZO_COMPRESS
select LZO_DECOMPRESS
-   select SCOMPRESS
help
  This is the LZO algorithm.
 
diff --git a/crypto/Makefile b/crypto/Makefile
index 16ef796..9157d69 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -28,7 +28,8 @@ crypto_hash-y += ahash.o
 crypto_hash-y += shash.o
 obj-$(CONFIG_CRYPTO_HASH2) += crypto_hash.o
 
-obj-$(CONFIG_CRYPTO_SCOMPRESS) += scompress.o
+crypto_compress-y += scompress.o acompress.o
+obj-$(CONFIG_CRYPTO_COMPRESS2) += crypto_compress.o
 obj-$(CONFIG_CRYPTO_AKCIPHER2) += akcipher.o
 
 $(obj)/rsapubkey-asn1.o: $(obj)/rsapubkey-asn1.c $(obj)/rsapubkey-asn1.h
diff --git a/crypto/acompress.c b/crypto/acompress.c
new file mode 100644
index 000..ddaa5a0
--- /dev/null
+++ b/crypto/acompress.c
@@ -0,0 +1,164 @@
+/*
+ * Asynchronous compression operations
+ *
+ * Copyright (c) 2015, Intel Corporation
+ * Authors: Weigang Li <weigang...@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include "internal.h"
+
+const struct crypto_type crypto_acomp_type;
+
+#ifdef CONFIG_NET
+static int crypto_acomp_report(struct sk_buff *skb, struct crypto_alg *alg)
+{
+   struct crypto_report_comp racomp;
+
+   strncpy(racomp.type, "acomp", sizeof(racomp.type));
+
+   if (nla_put(skb, CRYPTOCFGA_REPORT_COMPRESS,
+   sizeof(struct crypto_report_comp), ))
+   goto nla_put_failure;
+   return 0;
+
+nla_put_failure:
+   return -EMSGSIZE;
+}
+#else
+static int crypto_acomp_report(struct sk_buff *skb, struct crypto_alg *alg)
+{
+   return -ENOSYS;
+}
+#endif
+
+static void crypto_acomp_show(struct seq_file *m, struct crypto_alg *alg)
+   __attribute__ ((unused));
+
+static void crypto_acomp_show(struct seq_file *m, struct crypto_alg *alg)
+{
+   seq_puts(m, "type : acomp\n");
+}
+
+static void crypto_acomp_exit_tfm(struct crypto_tfm *tfm)
+{
+   struct crypto_acomp *acomp = __crypto_acomp_tfm(tfm);
+   struct acomp_alg *alg = crypto_acomp_alg(acomp);
+
+   alg->exit(acomp);
+}
+
+static int crypto_acomp_init_tfm(struct crypto_tfm *tfm)
+{
+   struct crypto_acomp *acomp = __crypto_acomp_tfm(tfm);
+   struct acomp_alg *alg = crypto_acomp_alg(acomp);
+
+   if (tfm->__crt_alg->cra_type != _acomp_type)
+   return crypto_init_scomp_ops_async(tfm);
+
+   acomp->compress = alg->compress;
+   acomp->decompress = alg->decompress;
+
+   if (alg->exit)
+   acomp->base.exit = crypto_acomp_exit_tfm;
+
+   if (alg->init)
+   return alg->init(acomp);
+
+   return 0;
+}
+
+static unsigned int crypto_acomp_extsize(struct crypto_alg *alg)
+{
+   if (alg->cra_type == _acomp_type)
+   return alg->cra_ctxsize;
+
+   return sizeof(void *);
+}
+
+const struct crypto_type crypto_acomp_type = {
+   .extsize = crypto_acomp_extsize,
+   .init_tfm = crypto_acomp_init_tfm,
+#ifdef CONFIG_PROC_FS
+   .show = crypto_acomp_show,
+#endif
+   .report = crypto_acomp_report,
+   .maskclear = ~CRYPTO_ALG_TYPE_MASK,
+   .maskset = CRYPTO_ALG_TYPE_ACOMPRESS_MASK,
+   .type = CRYPTO_ALG_TYPE_ACOMPRESS,
+   .tfmsize = offsetof(

[PATCH v2 09/10] crypto/deflate: support new compression APIs

[Joonsoo Kim]

Now, new compression APIs are introduced and it has some benefits.
Let's support it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig   |   1 +
 crypto/deflate.c | 110 ++-
 2 files changed, 101 insertions(+), 10 deletions(-)

diff --git a/crypto/Kconfig b/crypto/Kconfig
index 351b859..728f88e 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1492,6 +1492,7 @@ config CRYPTO_DEFLATE
select CRYPTO_ALGAPI
select ZLIB_INFLATE
select ZLIB_DEFLATE
+   select CRYPTO_COMPRESS2
help
  This is the Deflate algorithm (RFC1951), specified for use in
  IPSec with the IPCOMP protocol (RFC3173, RFC2394).
diff --git a/crypto/deflate.c b/crypto/deflate.c
index 95d8d37..1f8f633 100644
--- a/crypto/deflate.c
+++ b/crypto/deflate.c
@@ -33,6 +33,8 @@
 #include 
 #include 
 
+#include 
+
 #define DEFLATE_DEF_LEVEL  Z_DEFAULT_COMPRESSION
 #define DEFLATE_DEF_WINBITS11
 #define DEFLATE_DEF_MEMLEVEL   MAX_MEM_LEVEL
@@ -101,9 +103,8 @@ static void deflate_decomp_exit(struct deflate_ctx *ctx)
vfree(ctx->decomp_stream.workspace);
 }
 
-static int deflate_init(struct crypto_tfm *tfm)
+static int __deflate_init(void *ctx)
 {
-   struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
int ret;
 
ret = deflate_comp_init(ctx);
@@ -116,19 +117,54 @@ out:
return ret;
 }
 
-static void deflate_exit(struct crypto_tfm *tfm)
+static void *deflate_alloc_ctx(struct crypto_scomp *tfm)
+{
+   void *ctx;
+   int ret;
+
+   ctx = kzalloc(sizeof(struct deflate_ctx), GFP_KERNEL);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   ret = __deflate_init(ctx);
+   if (ret) {
+   kfree(ctx);
+   return ERR_PTR(ret);
+   }
+
+   return ctx;
+}
+
+static int deflate_init(struct crypto_tfm *tfm)
 {
struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
 
+   return __deflate_init(ctx);
+}
+
+static void __deflate_exit(void *ctx)
+{
deflate_comp_exit(ctx);
deflate_decomp_exit(ctx);
 }
 
-static int deflate_compress(struct crypto_tfm *tfm, const u8 *src,
-   unsigned int slen, u8 *dst, unsigned int *dlen)
+static void deflate_free_ctx(struct crypto_scomp *tfm, void *ctx)
+{
+   __deflate_exit(ctx);
+}
+
+static void deflate_exit(struct crypto_tfm *tfm)
+{
+   struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   __deflate_exit(ctx);
+}
+
+static int __deflate_compress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
int ret = 0;
-   struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
+   struct deflate_ctx *dctx = ctx;
struct z_stream_s *stream = >comp_stream;
 
ret = zlib_deflateReset(stream);
@@ -153,12 +189,26 @@ out:
return ret;
 }
 
-static int deflate_decompress(struct crypto_tfm *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int *dlen)
+static int deflate_compress(struct crypto_tfm *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
+
+   return __deflate_compress(src, slen, dst, dlen, dctx);
+}
+
+static int deflate_scompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __deflate_compress(src, slen, dst, dlen, ctx);
+}
+
+static int __deflate_decompress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
 
int ret = 0;
-   struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
+   struct deflate_ctx *dctx = ctx;
struct z_stream_s *stream = >decomp_stream;
 
ret = zlib_inflateReset(stream);
@@ -194,6 +244,20 @@ out:
return ret;
 }
 
+static int deflate_decompress(struct crypto_tfm *tfm, const u8 *src,
+ unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
+
+   return __deflate_decompress(src, slen, dst, dlen, dctx);
+}
+
+static int deflate_sdecompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __deflate_decompress(src, slen, dst, dlen, ctx);
+}
+
 static struct crypto_alg alg = {
.cra_name   = "deflate",
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -206,14 +270,40 @@ static struct crypto_alg alg = {
.coa_decompress = deflate_decompress } }
 };
 
+static struct scomp_alg scomp = {
+   .alloc_ctx  = deflate_alloc_ctx,
+   .free_ctx   = deflate_free_ctx,
+   .compress   = deflate_scompress,
+   .decompress = deflate_s

[PATCH v2 06/10] crypto/lz4: support new compression APIs

[Joonsoo Kim]

Now, new compression APIs are introduced and it has some benefits.
Let's support it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |  1 +
 crypto/lz4.c   | 91 +++---
 2 files changed, 82 insertions(+), 10 deletions(-)

diff --git a/crypto/Kconfig b/crypto/Kconfig
index b4b485c..72ab0d7 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1520,6 +1520,7 @@ config CRYPTO_LZ4
select CRYPTO_ALGAPI
select LZ4_COMPRESS
select LZ4_DECOMPRESS
+   select CRYPTO_COMPRESS2
help
  This is the LZ4 algorithm.
 
diff --git a/crypto/lz4.c b/crypto/lz4.c
index aefbcea..728c6c4 100644
--- a/crypto/lz4.c
+++ b/crypto/lz4.c
@@ -24,35 +24,53 @@
 #include 
 #include 
 
+#include 
+
 struct lz4_ctx {
void *lz4_comp_mem;
 };
 
+static void *lz4_alloc_ctx(struct crypto_scomp *tfm)
+{
+   void *ctx;
+
+   ctx = vmalloc(LZ4_MEM_COMPRESS);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   return ctx;
+}
+
 static int lz4_init(struct crypto_tfm *tfm)
 {
struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   ctx->lz4_comp_mem = vmalloc(LZ4_MEM_COMPRESS);
-   if (!ctx->lz4_comp_mem)
+   ctx->lz4_comp_mem = lz4_alloc_ctx(NULL);
+   if (IS_ERR(ctx->lz4_comp_mem))
return -ENOMEM;
 
return 0;
 }
 
+static void lz4_free_ctx(struct crypto_scomp *tfm, void *ctx)
+{
+   vfree(ctx);
+}
+
 static void lz4_exit(struct crypto_tfm *tfm)
 {
struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
-   vfree(ctx->lz4_comp_mem);
+
+   lz4_free_ctx(NULL, ctx->lz4_comp_mem);
 }
 
-static int lz4_compress_crypto(struct crypto_tfm *tfm, const u8 *src,
-   unsigned int slen, u8 *dst, unsigned int *dlen)
+static int __lz4_compress_crypto(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
-   struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
size_t tmp_len = *dlen;
int err;
 
-   err = lz4_compress(src, slen, dst, _len, ctx->lz4_comp_mem);
+   err = lz4_compress(src, slen, dst, _len, ctx);
 
if (err < 0)
return -EINVAL;
@@ -61,8 +79,22 @@ static int lz4_compress_crypto(struct crypto_tfm *tfm, const 
u8 *src,
return 0;
 }
 
-static int lz4_decompress_crypto(struct crypto_tfm *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int *dlen)
+static int lz4_scompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __lz4_compress_crypto(src, slen, dst, dlen, ctx);
+}
+
+static int lz4_compress_crypto(struct crypto_tfm *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   return __lz4_compress_crypto(src, slen, dst, dlen, ctx->lz4_comp_mem);
+}
+
+static int __lz4_decompress_crypto(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
int err;
size_t tmp_len = *dlen;
@@ -76,6 +108,18 @@ static int lz4_decompress_crypto(struct crypto_tfm *tfm, 
const u8 *src,
return err;
 }
 
+static int lz4_sdecompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __lz4_decompress_crypto(src, slen, dst, dlen, NULL);
+}
+
+static int lz4_decompress_crypto(struct crypto_tfm *tfm, const u8 *src,
+ unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   return __lz4_decompress_crypto(src, slen, dst, dlen, NULL);
+}
+
 static struct crypto_alg alg_lz4 = {
.cra_name   = "lz4",
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -89,14 +133,41 @@ static struct crypto_alg alg_lz4 = {
.coa_decompress = lz4_decompress_crypto } }
 };
 
+static struct scomp_alg scomp = {
+   .alloc_ctx  = lz4_alloc_ctx,
+   .free_ctx   = lz4_free_ctx,
+   .compress   = lz4_scompress,
+   .decompress = lz4_sdecompress,
+   .base   = {
+   .cra_name   = "lz4",
+   .cra_driver_name= "lz4-scomp",
+   .cra_flags  = CRYPTO_ALG_TYPE_SCOMPRESS |
+   CRYPTO_SCOMP_DECOMP_NOCTX,
+   .cra_module = THIS_MODULE,
+   }
+};
+
 static int __init lz4_mod_init(void)
 {
-   return crypto_register_alg(_lz4);
+   int ret;
+
+   ret = crypto_register_alg(_lz4);
+   if (ret)
+   return ret;
+
+   ret = crypto_register_scomp();
+   if (ret) {
+   crypto_unregister_alg(_lz4);
+   return ret;
+   }
+
+   return ret;
 }
 
 static 

[PATCH v2 08/10] crypto/842: support new compression APIs

[Joonsoo Kim]

Now, new compression APIs are introduced and it has some benefits.
Let's support it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/842.c   | 85 +++---
 crypto/Kconfig |  1 +
 2 files changed, 83 insertions(+), 3 deletions(-)

diff --git a/crypto/842.c b/crypto/842.c
index 98e387e..47cf7e5 100644
--- a/crypto/842.c
+++ b/crypto/842.c
@@ -32,10 +32,46 @@
 #include 
 #include 
 
+#include 
+
 struct crypto842_ctx {
-   char wmem[SW842_MEM_COMPRESS];  /* working memory for compress */
+   void *wmem; /* working memory for compress */
 };
 
+static void *crypto842_alloc_ctx(struct crypto_scomp *tfm)
+{
+   void *ctx;
+
+   ctx = kmalloc(SW842_MEM_COMPRESS, GFP_KERNEL);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   return ctx;
+}
+
+static int crypto842_init(struct crypto_tfm *tfm)
+{
+   struct crypto842_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   ctx->wmem = crypto842_alloc_ctx(NULL);
+   if (IS_ERR(ctx->wmem))
+   return -ENOMEM;
+
+   return 0;
+}
+
+static void crypto842_free_ctx(struct crypto_scomp *tfm, void *ctx)
+{
+   kfree(ctx);
+}
+
+static void crypto842_exit(struct crypto_tfm *tfm)
+{
+   struct crypto842_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   crypto842_free_ctx(NULL, ctx->wmem);
+}
+
 static int crypto842_compress(struct crypto_tfm *tfm,
  const u8 *src, unsigned int slen,
  u8 *dst, unsigned int *dlen)
@@ -45,6 +81,13 @@ static int crypto842_compress(struct crypto_tfm *tfm,
return sw842_compress(src, slen, dst, dlen, ctx->wmem);
 }
 
+static int crypto842_scompress(struct crypto_scomp *tfm,
+ const u8 *src, unsigned int slen,
+ u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return sw842_compress(src, slen, dst, dlen, ctx);
+}
+
 static int crypto842_decompress(struct crypto_tfm *tfm,
const u8 *src, unsigned int slen,
u8 *dst, unsigned int *dlen)
@@ -52,27 +95,63 @@ static int crypto842_decompress(struct crypto_tfm *tfm,
return sw842_decompress(src, slen, dst, dlen);
 }
 
+static int crypto842_sdecompress(struct crypto_scomp *tfm,
+   const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return sw842_decompress(src, slen, dst, dlen);
+}
+
 static struct crypto_alg alg = {
.cra_name   = "842",
.cra_driver_name= "842-generic",
.cra_priority   = 100,
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
-   .cra_ctxsize= sizeof(struct crypto842_ctx),
.cra_module = THIS_MODULE,
+   .cra_init   = crypto842_init,
+   .cra_exit   = crypto842_exit,
.cra_u  = { .compress = {
.coa_compress   = crypto842_compress,
.coa_decompress = crypto842_decompress } }
 };
 
+static struct scomp_alg scomp = {
+   .alloc_ctx  = crypto842_alloc_ctx,
+   .free_ctx   = crypto842_free_ctx,
+   .compress   = crypto842_scompress,
+   .decompress = crypto842_sdecompress,
+   .base   = {
+   .cra_name   = "842",
+   .cra_driver_name= "842-scomp",
+   .cra_priority   = 100,
+   .cra_flags  = CRYPTO_ALG_TYPE_SCOMPRESS |
+   CRYPTO_SCOMP_DECOMP_NOCTX,
+   .cra_module = THIS_MODULE,
+   }
+};
+
 static int __init crypto842_mod_init(void)
 {
-   return crypto_register_alg();
+   int ret;
+
+   ret = crypto_register_alg();
+   if (ret)
+   return ret;
+
+   ret = crypto_register_scomp();
+   if (ret) {
+   crypto_unregister_alg();
+   return ret;
+   }
+
+   return ret;
 }
 module_init(crypto842_mod_init);
 
 static void __exit crypto842_mod_exit(void)
 {
crypto_unregister_alg();
+   crypto_unregister_scomp();
 }
 module_exit(crypto842_mod_exit);
 
diff --git a/crypto/Kconfig b/crypto/Kconfig
index 2641a60..351b859 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1512,6 +1512,7 @@ config CRYPTO_842
select CRYPTO_ALGAPI
select 842_COMPRESS
select 842_DECOMPRESS
+   select CRYPTO_COMPRESS2
help
  This is the 842 algorithm.
 
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 10/10] crypto/testmgr: add new compression APIs test

[Joonsoo Kim]

New compression APIs are supported now so we need test cases.
This patch implements it based on previous compression test framework.
Almost changes are straight forward.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig   |   1 +
 crypto/testmgr.c | 227 ---
 2 files changed, 216 insertions(+), 12 deletions(-)

diff --git a/crypto/Kconfig b/crypto/Kconfig
index 728f88e..4b9d796 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -118,6 +118,7 @@ config CRYPTO_MANAGER2
select CRYPTO_HASH2
select CRYPTO_BLKCIPHER2
select CRYPTO_AKCIPHER2
+   select CRYPTO_COMPRESS2
 
 config CRYPTO_USER
tristate "Userspace cryptographic algorithm configuration"
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index 086aa4d..d72ab6a 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -32,6 +32,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "internal.h"
 
@@ -1205,12 +1206,14 @@ static int test_skcipher(struct crypto_skcipher *tfm, 
int enc,
return 0;
 }
 
-static int test_comp(struct crypto_comp *tfm, struct comp_testvec *ctemplate,
-struct comp_testvec *dtemplate, int ctcount, int dtcount)
+static int test_comp(struct crypto_tfm *tfm, void *ctx, int type,
+   struct comp_testvec *ctemplate, struct comp_testvec *dtemplate,
+   int ctcount, int dtcount)
 {
-   const char *algo = crypto_tfm_alg_driver_name(crypto_comp_tfm(tfm));
+   const char *algo = crypto_tfm_alg_driver_name(tfm);
unsigned int i;
char result[COMP_BUF_SIZE];
+   struct scatterlist src, dst;
int ret;
 
for (i = 0; i < ctcount; i++) {
@@ -1220,8 +1223,33 @@ static int test_comp(struct crypto_comp *tfm, struct 
comp_testvec *ctemplate,
memset(result, 0, sizeof (result));
 
ilen = ctemplate[i].inlen;
-   ret = crypto_comp_compress(tfm, ctemplate[i].input,
-  ilen, result, );
+
+   switch (type) {
+   case 0:
+   ret = crypto_comp_compress(crypto_comp_cast(tfm),
+   ctemplate[i].input, ilen,
+   result, );
+   break;
+
+   case 1:
+   ret = crypto_scomp_compress(crypto_scomp_cast(tfm),
+   ctemplate[i].input, ilen,
+   result, , ctx);
+   break;
+
+   case 2:
+   sg_init_one(, ctemplate[i].input, ilen);
+   sg_init_one(, result, dlen);
+   acomp_request_set_comp(ctx, , , ilen, dlen);
+   ret = crypto_acomp_compress(ctx);
+   dlen = ((struct acomp_req *)ctx)->out_len;
+   break;
+
+   default:
+   ret = 1;
+   break;
+   }
+
if (ret) {
printk(KERN_ERR "alg: comp: compression failed "
   "on test %d for %s: ret=%d\n", i + 1, algo,
@@ -1253,8 +1281,32 @@ static int test_comp(struct crypto_comp *tfm, struct 
comp_testvec *ctemplate,
memset(result, 0, sizeof (result));
 
ilen = dtemplate[i].inlen;
-   ret = crypto_comp_decompress(tfm, dtemplate[i].input,
-ilen, result, );
+   switch (type) {
+   case 0:
+   ret = crypto_comp_decompress(crypto_comp_cast(tfm),
+   dtemplate[i].input, ilen,
+   result, );
+   break;
+
+   case 1:
+   ret = crypto_scomp_decompress(crypto_scomp_cast(tfm),
+   dtemplate[i].input, ilen,
+   result, , ctx);
+   break;
+
+   case 2:
+   sg_init_one(, dtemplate[i].input, ilen);
+   sg_init_one(, result, dlen);
+   acomp_request_set_comp(ctx, , , ilen, dlen);
+   ret = crypto_acomp_decompress(ctx);
+   dlen = ((struct acomp_req *)ctx)->out_len;
+   break;
+
+   default:
+   ret = 1;
+   break;
+   }
+
if (ret) {
printk(KERN_ERR "alg: comp: decompression failed "
   "on test %d for %s: ret=%d\n", i + 1, algo,
@@ -1446,7 +1498,8 @@ static int alg_test_comp(const struct alg_test_desc 
*desc, const char *driver,
   

[PATCH v2 05/10] crypto/lzo: support new compression APIs

[Joonsoo Kim]

Now, new compression APIs are introduced and it has some benefits.
Let's support it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |  1 +
 crypto/lzo.c   | 95 ++
 2 files changed, 83 insertions(+), 13 deletions(-)

diff --git a/crypto/Kconfig b/crypto/Kconfig
index f22f4e9..b4b485c 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1503,6 +1503,7 @@ config CRYPTO_LZO
select CRYPTO_ALGAPI
select LZO_COMPRESS
select LZO_DECOMPRESS
+   select CRYPTO_COMPRESS2
help
  This is the LZO algorithm.
 
diff --git a/crypto/lzo.c b/crypto/lzo.c
index 4b3e925..94cd7a4 100644
--- a/crypto/lzo.c
+++ b/crypto/lzo.c
@@ -23,39 +23,56 @@
 #include 
 #include 
 
+#include 
+
 struct lzo_ctx {
void *lzo_comp_mem;
 };
 
+static void *lzo_alloc_ctx(struct crypto_scomp *tfm)
+{
+   void *ctx;
+
+   ctx = kmalloc(LZO1X_MEM_COMPRESS,
+   GFP_KERNEL | __GFP_NOWARN | __GFP_REPEAT);
+   if (!ctx)
+   ctx = vmalloc(LZO1X_MEM_COMPRESS);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   return ctx;
+}
+
 static int lzo_init(struct crypto_tfm *tfm)
 {
struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   ctx->lzo_comp_mem = kmalloc(LZO1X_MEM_COMPRESS,
-   GFP_KERNEL | __GFP_NOWARN | __GFP_REPEAT);
-   if (!ctx->lzo_comp_mem)
-   ctx->lzo_comp_mem = vmalloc(LZO1X_MEM_COMPRESS);
-   if (!ctx->lzo_comp_mem)
+   ctx->lzo_comp_mem = lzo_alloc_ctx(NULL);
+   if (IS_ERR(ctx->lzo_comp_mem))
return -ENOMEM;
 
return 0;
 }
 
+static void lzo_free_ctx(struct crypto_scomp *tfm, void *ctx)
+{
+   kvfree(ctx);
+}
+
 static void lzo_exit(struct crypto_tfm *tfm)
 {
struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   kvfree(ctx->lzo_comp_mem);
+   lzo_free_ctx(NULL, ctx->lzo_comp_mem);
 }
 
-static int lzo_compress(struct crypto_tfm *tfm, const u8 *src,
-   unsigned int slen, u8 *dst, unsigned int *dlen)
+static int __lzo_compress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
-   struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
size_t tmp_len = *dlen; /* size_t(ulong) <-> uint on 64 bit */
int err;
 
-   err = lzo1x_1_compress(src, slen, dst, _len, ctx->lzo_comp_mem);
+   err = lzo1x_1_compress(src, slen, dst, _len, ctx);
 
if (err != LZO_E_OK)
return -EINVAL;
@@ -64,8 +81,22 @@ static int lzo_compress(struct crypto_tfm *tfm, const u8 
*src,
return 0;
 }
 
-static int lzo_decompress(struct crypto_tfm *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int *dlen)
+static int lzo_compress(struct crypto_tfm *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   return __lzo_compress(src, slen, dst, dlen, ctx->lzo_comp_mem);
+}
+
+static int lzo_scompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __lzo_compress(src, slen, dst, dlen, ctx);
+}
+
+static int __lzo_decompress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen)
 {
int err;
size_t tmp_len = *dlen; /* size_t(ulong) <-> uint on 64 bit */
@@ -77,7 +108,18 @@ static int lzo_decompress(struct crypto_tfm *tfm, const u8 
*src,
 
*dlen = tmp_len;
return 0;
+}
+
+static int lzo_decompress(struct crypto_tfm *tfm, const u8 *src,
+ unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   return __lzo_decompress(src, slen, dst, dlen);
+}
 
+static int lzo_sdecompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __lzo_decompress(src, slen, dst, dlen);
 }
 
 static struct crypto_alg alg = {
@@ -92,14 +134,41 @@ static struct crypto_alg alg = {
.coa_decompress = lzo_decompress } }
 };
 
+static struct scomp_alg scomp = {
+   .alloc_ctx  = lzo_alloc_ctx,
+   .free_ctx   = lzo_free_ctx,
+   .compress   = lzo_scompress,
+   .decompress = lzo_sdecompress,
+   .base   = {
+   .cra_name   = "lzo",
+   .cra_driver_name= "lzo-scomp",
+   .cra_flags  = CRYPTO_ALG_TYPE_SCOMPRESS |
+   CRYPTO_SCOMP_DECOMP_NOCTX,
+   .cra_module = THIS_MODULE,
+   }
+};
+
 static int __init lzo_mod_init(void)
 {
-   return crypto_register_alg();
+   int ret;
+
+   ret = crypto_register_alg();
+   if (re

[PATCH v2 01/10] crypto/compress: remove unused pcomp interface

[Joonsoo Kim]

It is unused now, so remove it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |  19 --
 crypto/Makefile|   2 -
 crypto/pcompress.c | 115 ---
 crypto/testmgr.c   | 223 --
 crypto/testmgr.h   | 144 --
 crypto/zlib.c  | 381 -
 include/crypto/compress.h  | 145 --
 include/crypto/internal/compress.h |  28 ---
 include/linux/crypto.h |   1 -
 9 files changed, 1058 deletions(-)
 delete mode 100644 crypto/pcompress.c
 delete mode 100644 crypto/zlib.c
 delete mode 100644 include/crypto/compress.h
 delete mode 100644 include/crypto/internal/compress.h

diff --git a/crypto/Kconfig b/crypto/Kconfig
index 7240821..c80d34f 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -84,15 +84,6 @@ config CRYPTO_RNG_DEFAULT
tristate
select CRYPTO_DRBG_MENU
 
-config CRYPTO_PCOMP
-   tristate
-   select CRYPTO_PCOMP2
-   select CRYPTO_ALGAPI
-
-config CRYPTO_PCOMP2
-   tristate
-   select CRYPTO_ALGAPI2
-
 config CRYPTO_AKCIPHER2
tristate
select CRYPTO_ALGAPI2
@@ -122,7 +113,6 @@ config CRYPTO_MANAGER2
select CRYPTO_AEAD2
select CRYPTO_HASH2
select CRYPTO_BLKCIPHER2
-   select CRYPTO_PCOMP2
select CRYPTO_AKCIPHER2
 
 config CRYPTO_USER
@@ -1504,15 +1494,6 @@ config CRYPTO_DEFLATE
 
  You will most probably want this if using IPSec.
 
-config CRYPTO_ZLIB
-   tristate "Zlib compression algorithm"
-   select CRYPTO_PCOMP
-   select ZLIB_INFLATE
-   select ZLIB_DEFLATE
-   select NLATTR
-   help
- This is the zlib algorithm.
-
 config CRYPTO_LZO
tristate "LZO compression algorithm"
select CRYPTO_ALGAPI
diff --git a/crypto/Makefile b/crypto/Makefile
index 2acdbbd..ffe18c9 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -28,7 +28,6 @@ crypto_hash-y += ahash.o
 crypto_hash-y += shash.o
 obj-$(CONFIG_CRYPTO_HASH2) += crypto_hash.o
 
-obj-$(CONFIG_CRYPTO_PCOMP2) += pcompress.o
 obj-$(CONFIG_CRYPTO_AKCIPHER2) += akcipher.o
 
 $(obj)/rsapubkey-asn1.o: $(obj)/rsapubkey-asn1.c $(obj)/rsapubkey-asn1.h
@@ -99,7 +98,6 @@ obj-$(CONFIG_CRYPTO_SALSA20) += salsa20_generic.o
 obj-$(CONFIG_CRYPTO_CHACHA20) += chacha20_generic.o
 obj-$(CONFIG_CRYPTO_POLY1305) += poly1305_generic.o
 obj-$(CONFIG_CRYPTO_DEFLATE) += deflate.o
-obj-$(CONFIG_CRYPTO_ZLIB) += zlib.o
 obj-$(CONFIG_CRYPTO_MICHAEL_MIC) += michael_mic.o
 obj-$(CONFIG_CRYPTO_CRC32C) += crc32c_generic.o
 obj-$(CONFIG_CRYPTO_CRC32) += crc32.o
diff --git a/crypto/pcompress.c b/crypto/pcompress.c
deleted file mode 100644
index 7a13b40..000
--- a/crypto/pcompress.c
+++ /dev/null
@@ -1,115 +0,0 @@
-/*
- * Cryptographic API.
- *
- * Partial (de)compression operations.
- *
- * Copyright 2008 Sony Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program.
- * If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#include 
-#include 
-
-#include "internal.h"
-
-
-static int crypto_pcomp_init(struct crypto_tfm *tfm, u32 type, u32 mask)
-{
-   return 0;
-}
-
-static int crypto_pcomp_init_tfm(struct crypto_tfm *tfm)
-{
-   return 0;
-}
-
-#ifdef CONFIG_NET
-static int crypto_pcomp_report(struct sk_buff *skb, struct crypto_alg *alg)
-{
-   struct crypto_report_comp rpcomp;
-
-   strncpy(rpcomp.type, "pcomp", sizeof(rpcomp.type));
-   if (nla_put(skb, CRYPTOCFGA_REPORT_COMPRESS,
-   sizeof(struct crypto_report_comp), ))
-   goto nla_put_failure;
-   return 0;
-
-nla_put_failure:
-   return -EMSGSIZE;
-}
-#else
-static int crypto_pcomp_report(struct sk_buff *skb, struct crypto_alg *alg)
-{
-   return -ENOSYS;
-}
-#endif
-
-static void crypto_pcomp_show(struct seq_file *m, struct crypto_alg *alg)
-   __attribute__ ((unused));
-static void crypto_pcomp_show(struct seq_file *m, struct crypto_alg *alg)
-{
-   seq_printf(m, "type : pcomp\n");
-}
-
-static const struct crypto_type crypto_pcomp_type = {
-   .extsize= crypto_alg_extsize,
-   .init   = crypto_pcomp_init,
-   .init_tfm   = crypto_pcomp_init_tfm,
-#ifdef CONFIG_PROC_FS
-   .show   = crypto_pco

[PATCH v2 03/10] crypto/compress: introduce sychronuous compression API

[Joonsoo Kim]

This introduces new compression APIs. Major change is that APIs are
stateless. Instead of previous implementation, tfm objects doesn't
embedded any context so we can de/compress concurrently with one tfm
object. Instead, this de/compression context is coupled with the request.
This architecture change will make APIs more flexible and we can naturally
use asynchronous APIs as front-end of synchronous compression algorithm.

Moreover, thanks to this change, we can decompress without context buffer
if algorithm supports it. You can check it by crypto_scomp_decomp_noctx()
and in this case we can achieve maximum parallelism without
memory overhead caused by context buffer.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig|   5 ++
 crypto/Makefile   |   1 +
 crypto/scompress.c| 114 ++
 include/crypto/compress.h |  93 +
 include/linux/crypto.h|   1 +
 5 files changed, 214 insertions(+)
 create mode 100644 crypto/scompress.c
 create mode 100644 include/crypto/compress.h

diff --git a/crypto/Kconfig b/crypto/Kconfig
index c80d34f..7159520 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -84,6 +84,10 @@ config CRYPTO_RNG_DEFAULT
tristate
select CRYPTO_DRBG_MENU
 
+config CRYPTO_SCOMPRESS
+   tristate
+   select CRYPTO_ALGAPI2
+
 config CRYPTO_AKCIPHER2
tristate
select CRYPTO_ALGAPI2
@@ -1499,6 +1503,7 @@ config CRYPTO_LZO
select CRYPTO_ALGAPI
select LZO_COMPRESS
select LZO_DECOMPRESS
+   select SCOMPRESS
help
  This is the LZO algorithm.
 
diff --git a/crypto/Makefile b/crypto/Makefile
index ffe18c9..16ef796 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -28,6 +28,7 @@ crypto_hash-y += ahash.o
 crypto_hash-y += shash.o
 obj-$(CONFIG_CRYPTO_HASH2) += crypto_hash.o
 
+obj-$(CONFIG_CRYPTO_SCOMPRESS) += scompress.o
 obj-$(CONFIG_CRYPTO_AKCIPHER2) += akcipher.o
 
 $(obj)/rsapubkey-asn1.o: $(obj)/rsapubkey-asn1.c $(obj)/rsapubkey-asn1.h
diff --git a/crypto/scompress.c b/crypto/scompress.c
new file mode 100644
index 000..7c9955b
--- /dev/null
+++ b/crypto/scompress.c
@@ -0,0 +1,114 @@
+/*
+ * Cryptographic API.
+ *
+ * Synchronous compression operations.
+ *
+ * Copyright 2015 LG Electronics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.
+ * If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+
+#include "internal.h"
+
+
+static int crypto_scomp_init(struct crypto_tfm *tfm, u32 type, u32 mask)
+{
+   return 0;
+}
+
+static int crypto_scomp_init_tfm(struct crypto_tfm *tfm)
+{
+   return 0;
+}
+
+#ifdef CONFIG_NET
+static int crypto_scomp_report(struct sk_buff *skb, struct crypto_alg *alg)
+{
+   struct crypto_report_comp rcomp;
+
+   strncpy(rcomp.type, "scomp", sizeof(rcomp.type));
+   if (nla_put(skb, CRYPTOCFGA_REPORT_COMPRESS,
+   sizeof(struct crypto_report_comp), ))
+   goto nla_put_failure;
+   return 0;
+
+nla_put_failure:
+   return -EMSGSIZE;
+}
+#else
+static int crypto_scomp_report(struct sk_buff *skb, struct crypto_alg *alg)
+{
+   return -ENOSYS;
+}
+#endif
+
+static void crypto_scomp_show(struct seq_file *m, struct crypto_alg *alg)
+   __attribute__ ((unused));
+static void crypto_scomp_show(struct seq_file *m, struct crypto_alg *alg)
+{
+   seq_puts(m, "type : scomp\n");
+}
+
+static const struct crypto_type crypto_scomp_type = {
+   .extsize= crypto_alg_extsize,
+   .init   = crypto_scomp_init,
+   .init_tfm   = crypto_scomp_init_tfm,
+#ifdef CONFIG_PROC_FS
+   .show   = crypto_scomp_show,
+#endif
+   .report = crypto_scomp_report,
+   .maskclear  = ~CRYPTO_ALG_TYPE_MASK,
+   .maskset= CRYPTO_ALG_TYPE_MASK,
+   .type   = CRYPTO_ALG_TYPE_SCOMPRESS,
+   .tfmsize= offsetof(struct crypto_scomp, base),
+};
+
+struct crypto_scomp *crypto_alloc_scomp(const char *alg_name, u32 type,
+   u32 mask)
+{
+   return crypto_alloc_tfm(alg_name, _scomp_type, type, mask);
+}
+EXPORT_SYMBOL_GPL(crypto_alloc_scomp);
+
+int crypto_register_scomp(struct scomp_alg *alg)
+{
+   struct crypto_alg *base = >base;
+
+   base->

[PATCH v2 02/10] crypto: add algorithm type specific flag, CRYPTO_ALG_PRIVATE

[Joonsoo Kim]

In following patch, new synchronous compression APIs will be
introduced and it needs one flags to determine whether context buffer is
needed or not for decompression. It can be implemented by flag in it's own
algorithm structure definition but because there is a room in general
crypto_alg flag, this patch reuses it to reduce complexity. It possibly
can be used for other algorithm type.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 include/linux/crypto.h | 5 +
 1 file changed, 5 insertions(+)

diff --git a/include/linux/crypto.h b/include/linux/crypto.h
index ab2a745..96530a1 100644
--- a/include/linux/crypto.h
+++ b/include/linux/crypto.h
@@ -101,6 +101,11 @@
 #define CRYPTO_ALG_INTERNAL0x2000
 
 /*
+ * Use this flag as algorithm type specific one.
+ */
+#define CRYPTO_ALG_PRIVATE 0x4000
+
+/*
  * Transform masks and values (for crt_flags).
  */
 #define CRYPTO_TFM_REQ_MASK0x000fff00
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 2/7] crypto/compress: introduce sychronuous compression API

[Joonsoo Kim]

This introduces new compression APIs. Major change is that APIs are
stateless. Instead of previous implementation, tfm objects doesn't
embedded any context so we can de/compress concurrently with one tfm
object. Instead, thsi de/compression context is coupled with the request.
This architecture change will make APIs more flexible.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig|   5 ++
 crypto/Makefile   |   1 +
 crypto/scompress.c| 114 ++
 include/crypto/compress.h |  76 +++
 include/linux/crypto.h|   1 +
 5 files changed, 197 insertions(+)
 create mode 100644 crypto/scompress.c
 create mode 100644 include/crypto/compress.h

diff --git a/crypto/Kconfig b/crypto/Kconfig
index c80d34f..7159520 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -84,6 +84,10 @@ config CRYPTO_RNG_DEFAULT
tristate
select CRYPTO_DRBG_MENU
 
+config CRYPTO_SCOMPRESS
+   tristate
+   select CRYPTO_ALGAPI2
+
 config CRYPTO_AKCIPHER2
tristate
select CRYPTO_ALGAPI2
@@ -1499,6 +1503,7 @@ config CRYPTO_LZO
select CRYPTO_ALGAPI
select LZO_COMPRESS
select LZO_DECOMPRESS
+   select SCOMPRESS
help
  This is the LZO algorithm.
 
diff --git a/crypto/Makefile b/crypto/Makefile
index ffe18c9..16ef796 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -28,6 +28,7 @@ crypto_hash-y += ahash.o
 crypto_hash-y += shash.o
 obj-$(CONFIG_CRYPTO_HASH2) += crypto_hash.o
 
+obj-$(CONFIG_CRYPTO_SCOMPRESS) += scompress.o
 obj-$(CONFIG_CRYPTO_AKCIPHER2) += akcipher.o
 
 $(obj)/rsapubkey-asn1.o: $(obj)/rsapubkey-asn1.c $(obj)/rsapubkey-asn1.h
diff --git a/crypto/scompress.c b/crypto/scompress.c
new file mode 100644
index 000..7c9955b
--- /dev/null
+++ b/crypto/scompress.c
@@ -0,0 +1,114 @@
+/*
+ * Cryptographic API.
+ *
+ * Synchronous compression operations.
+ *
+ * Copyright 2015 LG Electronics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.
+ * If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+
+#include "internal.h"
+
+
+static int crypto_scomp_init(struct crypto_tfm *tfm, u32 type, u32 mask)
+{
+   return 0;
+}
+
+static int crypto_scomp_init_tfm(struct crypto_tfm *tfm)
+{
+   return 0;
+}
+
+#ifdef CONFIG_NET
+static int crypto_scomp_report(struct sk_buff *skb, struct crypto_alg *alg)
+{
+   struct crypto_report_comp rcomp;
+
+   strncpy(rcomp.type, "scomp", sizeof(rcomp.type));
+   if (nla_put(skb, CRYPTOCFGA_REPORT_COMPRESS,
+   sizeof(struct crypto_report_comp), ))
+   goto nla_put_failure;
+   return 0;
+
+nla_put_failure:
+   return -EMSGSIZE;
+}
+#else
+static int crypto_scomp_report(struct sk_buff *skb, struct crypto_alg *alg)
+{
+   return -ENOSYS;
+}
+#endif
+
+static void crypto_scomp_show(struct seq_file *m, struct crypto_alg *alg)
+   __attribute__ ((unused));
+static void crypto_scomp_show(struct seq_file *m, struct crypto_alg *alg)
+{
+   seq_puts(m, "type : scomp\n");
+}
+
+static const struct crypto_type crypto_scomp_type = {
+   .extsize= crypto_alg_extsize,
+   .init   = crypto_scomp_init,
+   .init_tfm   = crypto_scomp_init_tfm,
+#ifdef CONFIG_PROC_FS
+   .show   = crypto_scomp_show,
+#endif
+   .report = crypto_scomp_report,
+   .maskclear  = ~CRYPTO_ALG_TYPE_MASK,
+   .maskset= CRYPTO_ALG_TYPE_MASK,
+   .type   = CRYPTO_ALG_TYPE_SCOMPRESS,
+   .tfmsize= offsetof(struct crypto_scomp, base),
+};
+
+struct crypto_scomp *crypto_alloc_scomp(const char *alg_name, u32 type,
+   u32 mask)
+{
+   return crypto_alloc_tfm(alg_name, _scomp_type, type, mask);
+}
+EXPORT_SYMBOL_GPL(crypto_alloc_scomp);
+
+int crypto_register_scomp(struct scomp_alg *alg)
+{
+   struct crypto_alg *base = >base;
+
+   base->cra_type = _scomp_type;
+   base->cra_flags &= ~CRYPTO_ALG_TYPE_MASK;
+   base->cra_flags |= CRYPTO_ALG_TYPE_SCOMPRESS;
+
+   return crypto_register_alg(base);
+}
+EXPORT_SYMBOL_GPL(crypto_register_scomp);
+
+int crypto_unregister_scomp(struct scomp_alg *alg)
+{
+   return crypto_unregister_alg(>base);
+}
+EXPORT_SYMBOL_GPL(crypto_unr

[PATCH RFC 1/7] crypto/compress: remove unused pcomp interface

[Joonsoo Kim]

It is unused now, so remove it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |  19 --
 crypto/Makefile|   2 -
 crypto/pcompress.c | 115 ---
 crypto/testmgr.c   | 223 --
 crypto/testmgr.h   | 142 --
 crypto/zlib.c  | 381 -
 include/crypto/compress.h  | 145 --
 include/crypto/internal/compress.h |  28 ---
 include/linux/crypto.h |   1 -
 9 files changed, 1056 deletions(-)
 delete mode 100644 crypto/pcompress.c
 delete mode 100644 crypto/zlib.c
 delete mode 100644 include/crypto/compress.h
 delete mode 100644 include/crypto/internal/compress.h

diff --git a/crypto/Kconfig b/crypto/Kconfig
index 7240821..c80d34f 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -84,15 +84,6 @@ config CRYPTO_RNG_DEFAULT
tristate
select CRYPTO_DRBG_MENU
 
-config CRYPTO_PCOMP
-   tristate
-   select CRYPTO_PCOMP2
-   select CRYPTO_ALGAPI
-
-config CRYPTO_PCOMP2
-   tristate
-   select CRYPTO_ALGAPI2
-
 config CRYPTO_AKCIPHER2
tristate
select CRYPTO_ALGAPI2
@@ -122,7 +113,6 @@ config CRYPTO_MANAGER2
select CRYPTO_AEAD2
select CRYPTO_HASH2
select CRYPTO_BLKCIPHER2
-   select CRYPTO_PCOMP2
select CRYPTO_AKCIPHER2
 
 config CRYPTO_USER
@@ -1504,15 +1494,6 @@ config CRYPTO_DEFLATE
 
  You will most probably want this if using IPSec.
 
-config CRYPTO_ZLIB
-   tristate "Zlib compression algorithm"
-   select CRYPTO_PCOMP
-   select ZLIB_INFLATE
-   select ZLIB_DEFLATE
-   select NLATTR
-   help
- This is the zlib algorithm.
-
 config CRYPTO_LZO
tristate "LZO compression algorithm"
select CRYPTO_ALGAPI
diff --git a/crypto/Makefile b/crypto/Makefile
index 2acdbbd..ffe18c9 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -28,7 +28,6 @@ crypto_hash-y += ahash.o
 crypto_hash-y += shash.o
 obj-$(CONFIG_CRYPTO_HASH2) += crypto_hash.o
 
-obj-$(CONFIG_CRYPTO_PCOMP2) += pcompress.o
 obj-$(CONFIG_CRYPTO_AKCIPHER2) += akcipher.o
 
 $(obj)/rsapubkey-asn1.o: $(obj)/rsapubkey-asn1.c $(obj)/rsapubkey-asn1.h
@@ -99,7 +98,6 @@ obj-$(CONFIG_CRYPTO_SALSA20) += salsa20_generic.o
 obj-$(CONFIG_CRYPTO_CHACHA20) += chacha20_generic.o
 obj-$(CONFIG_CRYPTO_POLY1305) += poly1305_generic.o
 obj-$(CONFIG_CRYPTO_DEFLATE) += deflate.o
-obj-$(CONFIG_CRYPTO_ZLIB) += zlib.o
 obj-$(CONFIG_CRYPTO_MICHAEL_MIC) += michael_mic.o
 obj-$(CONFIG_CRYPTO_CRC32C) += crc32c_generic.o
 obj-$(CONFIG_CRYPTO_CRC32) += crc32.o
diff --git a/crypto/pcompress.c b/crypto/pcompress.c
deleted file mode 100644
index 7a13b40..000
--- a/crypto/pcompress.c
+++ /dev/null
@@ -1,115 +0,0 @@
-/*
- * Cryptographic API.
- *
- * Partial (de)compression operations.
- *
- * Copyright 2008 Sony Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program.
- * If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#include 
-#include 
-
-#include "internal.h"
-
-
-static int crypto_pcomp_init(struct crypto_tfm *tfm, u32 type, u32 mask)
-{
-   return 0;
-}
-
-static int crypto_pcomp_init_tfm(struct crypto_tfm *tfm)
-{
-   return 0;
-}
-
-#ifdef CONFIG_NET
-static int crypto_pcomp_report(struct sk_buff *skb, struct crypto_alg *alg)
-{
-   struct crypto_report_comp rpcomp;
-
-   strncpy(rpcomp.type, "pcomp", sizeof(rpcomp.type));
-   if (nla_put(skb, CRYPTOCFGA_REPORT_COMPRESS,
-   sizeof(struct crypto_report_comp), ))
-   goto nla_put_failure;
-   return 0;
-
-nla_put_failure:
-   return -EMSGSIZE;
-}
-#else
-static int crypto_pcomp_report(struct sk_buff *skb, struct crypto_alg *alg)
-{
-   return -ENOSYS;
-}
-#endif
-
-static void crypto_pcomp_show(struct seq_file *m, struct crypto_alg *alg)
-   __attribute__ ((unused));
-static void crypto_pcomp_show(struct seq_file *m, struct crypto_alg *alg)
-{
-   seq_printf(m, "type : pcomp\n");
-}
-
-static const struct crypto_type crypto_pcomp_type = {
-   .extsize= crypto_alg_extsize,
-   .init   = crypto_pcomp_init,
-   .init_tfm   = crypto_pcomp_init_tfm,
-#ifdef CONFIG_PROC_FS
-   .show   = crypto_pco

[PATCH RFC 4/7] crypto/lzo: support SCOMPRESS alg type

[Joonsoo Kim]

For proof of concept, SCOMPRESS alg type is implemented for lzo.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |  1 +
 crypto/lzo.c   | 94 ++
 2 files changed, 82 insertions(+), 13 deletions(-)

diff --git a/crypto/Kconfig b/crypto/Kconfig
index f22f4e9..b4b485c 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1503,6 +1503,7 @@ config CRYPTO_LZO
select CRYPTO_ALGAPI
select LZO_COMPRESS
select LZO_DECOMPRESS
+   select CRYPTO_COMPRESS2
help
  This is the LZO algorithm.
 
diff --git a/crypto/lzo.c b/crypto/lzo.c
index 4b3e925..47998e9 100644
--- a/crypto/lzo.c
+++ b/crypto/lzo.c
@@ -23,39 +23,56 @@
 #include 
 #include 
 
+#include 
+
 struct lzo_ctx {
void *lzo_comp_mem;
 };
 
+static void *lzo_alloc_ctx(struct crypto_scomp *tfm)
+{
+   void *ctx;
+
+   ctx = kmalloc(LZO1X_MEM_COMPRESS,
+   GFP_KERNEL | __GFP_NOWARN | __GFP_REPEAT);
+   if (!ctx)
+   ctx = vmalloc(LZO1X_MEM_COMPRESS);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   return ctx;
+}
+
 static int lzo_init(struct crypto_tfm *tfm)
 {
struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   ctx->lzo_comp_mem = kmalloc(LZO1X_MEM_COMPRESS,
-   GFP_KERNEL | __GFP_NOWARN | __GFP_REPEAT);
-   if (!ctx->lzo_comp_mem)
-   ctx->lzo_comp_mem = vmalloc(LZO1X_MEM_COMPRESS);
-   if (!ctx->lzo_comp_mem)
+   ctx->lzo_comp_mem = lzo_alloc_ctx(NULL);
+   if (IS_ERR(ctx->lzo_comp_mem))
return -ENOMEM;
 
return 0;
 }
 
+static void lzo_free_ctx(struct crypto_scomp *tfm, void *ctx)
+{
+   kvfree(ctx);
+}
+
 static void lzo_exit(struct crypto_tfm *tfm)
 {
struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   kvfree(ctx->lzo_comp_mem);
+   lzo_free_ctx(NULL, ctx->lzo_comp_mem);
 }
 
-static int lzo_compress(struct crypto_tfm *tfm, const u8 *src,
-   unsigned int slen, u8 *dst, unsigned int *dlen)
+static int __lzo_compress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
-   struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
size_t tmp_len = *dlen; /* size_t(ulong) <-> uint on 64 bit */
int err;
 
-   err = lzo1x_1_compress(src, slen, dst, _len, ctx->lzo_comp_mem);
+   err = lzo1x_1_compress(src, slen, dst, _len, ctx);
 
if (err != LZO_E_OK)
return -EINVAL;
@@ -64,8 +81,22 @@ static int lzo_compress(struct crypto_tfm *tfm, const u8 
*src,
return 0;
 }
 
-static int lzo_decompress(struct crypto_tfm *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int *dlen)
+static int lzo_compress(struct crypto_tfm *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   return __lzo_compress(src, slen, dst, dlen, ctx->lzo_comp_mem);
+}
+
+static int lzo_scompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __lzo_compress(src, slen, dst, dlen, ctx);
+}
+
+static int __lzo_decompress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen)
 {
int err;
size_t tmp_len = *dlen; /* size_t(ulong) <-> uint on 64 bit */
@@ -77,7 +108,18 @@ static int lzo_decompress(struct crypto_tfm *tfm, const u8 
*src,
 
*dlen = tmp_len;
return 0;
+}
+
+static int lzo_decompress(struct crypto_tfm *tfm, const u8 *src,
+ unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   return __lzo_decompress(src, slen, dst, dlen);
+}
 
+static int lzo_sdecompress(struct crypto_scomp *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen, void *ctx)
+{
+   return __lzo_decompress(src, slen, dst, dlen);
 }
 
 static struct crypto_alg alg = {
@@ -92,14 +134,40 @@ static struct crypto_alg alg = {
.coa_decompress = lzo_decompress } }
 };
 
+static struct scomp_alg scomp = {
+   .alloc_ctx  = lzo_alloc_ctx,
+   .free_ctx   = lzo_free_ctx,
+   .compress   = lzo_scompress,
+   .decompress = lzo_sdecompress,
+   .base   = {
+   .cra_name   = "lzo",
+   .cra_driver_name= "lzo-scomp",
+   .cra_flags  = CRYPTO_ALG_TYPE_SCOMPRESS,
+   .cra_module = THIS_MODULE,
+   }
+};
+
 static int __init lzo_mod_init(void)
 {
-   return crypto_register_alg();
+   int ret;
+
+   ret = crypto_register_alg();
+   if (ret)
+   return ret;
+
+   ret = crypto_register_scomp();
+   if (ret) {

[PATCH RFC 0/7] Introduce new async/sync compression APIs

[Joonsoo Kim]

Happy New Year!

This patchset is to introduce new compression APIs. It supports async/sync
compression although there is no async compression driver yet. But, async
APIs can be used as front-end to sync compression algorithm. Major change
is that now APIs are stateless. Instead of previous implementation, tfm
objects doesn't embedded any context so we can de/compress concurrently
with one tfm object. Instead, this de/compression context is coupled with
the request. This architecture change will make APIs more flexible.

This is just a RFC so commit description isn't sufficient. If Herbert
confirms that this is what he expect, I will respin the series with proper
commit description and adding more algorithm support.

Thanks.

Joonsoo Kim (6):
  crypto/compress: remove unused pcomp interface
  crypto/compress: introduce sychronuous compression API
  crypto/lzo: support SCOMPRESS alg type
  crypto: testmgr: add scompress/acompress test
  crypto/compress: add algorithm type specific flag, DECOMP_NOCTX
  crypto/lzo: add CRYPTO_ALG_SCOMPRESS_DECOMP_NOCTX cra_flags

Weigang Li (1):
  crypto: add asynchronous compression support

 crypto/Kconfig |  19 +-
 crypto/Makefile|   4 +-
 crypto/acompress.c | 164 
 crypto/lzo.c   |  95 +++--
 crypto/pcompress.c | 115 ---
 crypto/scompress.c | 283 +++
 crypto/testmgr.c   | 376 +++-
 crypto/testmgr.h   | 142 --
 crypto/zlib.c  | 381 -
 include/crypto/compress.h  | 354 +-
 include/crypto/internal/compress.h |  32 +---
 include/linux/crypto.h |  10 +-
 12 files changed, 969 insertions(+), 1006 deletions(-)
 create mode 100644 crypto/acompress.c
 delete mode 100644 crypto/pcompress.c
 create mode 100644 crypto/scompress.c
 delete mode 100644 crypto/zlib.c

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 7/7] crypto/lzo: add CRYPTO_ALG_SCOMPRESS_DECOMP_NOCTX cra_flags

[Joonsoo Kim]

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/lzo.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/crypto/lzo.c b/crypto/lzo.c
index 47998e9..90afbf9 100644
--- a/crypto/lzo.c
+++ b/crypto/lzo.c
@@ -142,7 +142,8 @@ static struct scomp_alg scomp = {
.base   = {
.cra_name   = "lzo",
.cra_driver_name= "lzo-scomp",
-   .cra_flags  = CRYPTO_ALG_TYPE_SCOMPRESS,
+   .cra_flags  = CRYPTO_ALG_TYPE_SCOMPRESS |
+   CRYPTO_ALG_SCOMPRESS_DECOMP_NOCTX,
.cra_module = THIS_MODULE,
}
 };
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 6/7] crypto/compress: add algorithm type specific flag, DECOMP_NOCTX

[Joonsoo Kim]

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/testmgr.c  | 2 ++
 include/crypto/compress.h | 8 
 include/linux/crypto.h| 6 ++
 3 files changed, 16 insertions(+)

diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index bd0c639..acd7428 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -1289,6 +1289,8 @@ static int test_comp(struct crypto_tfm *tfm, void *ctx, 
int type,
break;
 
case 1:
+   if (crypto_scomp_decomp_noctx(crypto_scomp_cast(tfm)))
+   ctx = NULL;
ret = crypto_scomp_decompress(crypto_scomp_cast(tfm),
dtemplate[i].input, ilen,
result, , ctx);
diff --git a/include/crypto/compress.h b/include/crypto/compress.h
index f1e91c5..21c698e 100644
--- a/include/crypto/compress.h
+++ b/include/crypto/compress.h
@@ -2,6 +2,8 @@
 #define _CRYPTO_COMPRESS_H
 #include 
 
+#define CRYPTO_ALG_SCOMPRESS_DECOMP_NOCTX CRYPTO_ALG_PRIVATE
+
 struct crypto_scomp {
struct crypto_tfm base;
 };
@@ -71,6 +73,12 @@ static inline int crypto_scomp_decompress(struct 
crypto_scomp *tfm,
dst, dlen, ctx);
 }
 
+static inline bool crypto_scomp_decomp_noctx(struct crypto_scomp *tfm)
+{
+   return crypto_scomp_tfm(tfm)->__crt_alg->cra_flags &
+   CRYPTO_ALG_SCOMPRESS_DECOMP_NOCTX;
+}
+
 extern int crypto_register_scomp(struct scomp_alg *alg);
 extern int crypto_unregister_scomp(struct scomp_alg *alg);
 
diff --git a/include/linux/crypto.h b/include/linux/crypto.h
index 86baa61..a8a522c 100644
--- a/include/linux/crypto.h
+++ b/include/linux/crypto.h
@@ -104,6 +104,12 @@
 #define CRYPTO_ALG_INTERNAL0x2000
 
 /*
+ * Use this flag as type specific one. For example, it would be used
+ * to check if context is needed or not in CRYPTO_ALG_TYPE_SCOMPRESS.
+ */
+#define CRYPTO_ALG_PRIVATE 0x4000
+
+/*
  * Transform masks and values (for crt_flags).
  */
 #define CRYPTO_TFM_REQ_MASK0x000fff00
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 5/7] crypto: testmgr: add scompress/acompress test

[Joonsoo Kim]

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig   |   1 +
 crypto/testmgr.c | 173 ---
 2 files changed, 165 insertions(+), 9 deletions(-)

diff --git a/crypto/Kconfig b/crypto/Kconfig
index b4b485c..f487502 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -118,6 +118,7 @@ config CRYPTO_MANAGER2
select CRYPTO_HASH2
select CRYPTO_BLKCIPHER2
select CRYPTO_AKCIPHER2
+   select CRYPTO_COMPRESS2
 
 config CRYPTO_USER
tristate "Userspace cryptographic algorithm configuration"
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index 086aa4d..bd0c639 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -32,6 +32,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "internal.h"
 
@@ -1205,12 +1206,14 @@ static int test_skcipher(struct crypto_skcipher *tfm, 
int enc,
return 0;
 }
 
-static int test_comp(struct crypto_comp *tfm, struct comp_testvec *ctemplate,
-struct comp_testvec *dtemplate, int ctcount, int dtcount)
+static int test_comp(struct crypto_tfm *tfm, void *ctx, int type,
+   struct comp_testvec *ctemplate, struct comp_testvec *dtemplate,
+   int ctcount, int dtcount)
 {
-   const char *algo = crypto_tfm_alg_driver_name(crypto_comp_tfm(tfm));
+   const char *algo = crypto_tfm_alg_driver_name(tfm);
unsigned int i;
char result[COMP_BUF_SIZE];
+   struct scatterlist src, dst;
int ret;
 
for (i = 0; i < ctcount; i++) {
@@ -1220,8 +1223,33 @@ static int test_comp(struct crypto_comp *tfm, struct 
comp_testvec *ctemplate,
memset(result, 0, sizeof (result));
 
ilen = ctemplate[i].inlen;
-   ret = crypto_comp_compress(tfm, ctemplate[i].input,
-  ilen, result, );
+
+   switch (type) {
+   case 0:
+   ret = crypto_comp_compress(crypto_comp_cast(tfm),
+   ctemplate[i].input, ilen,
+   result, );
+   break;
+
+   case 1:
+   ret = crypto_scomp_compress(crypto_scomp_cast(tfm),
+   ctemplate[i].input, ilen,
+   result, , ctx);
+   break;
+
+   case 2:
+   sg_init_one(, ctemplate[i].input, ilen);
+   sg_init_one(, result, dlen);
+   acomp_request_set_comp(ctx, , , ilen, dlen);
+   ret = crypto_acomp_compress(ctx);
+   dlen = ((struct acomp_req *)ctx)->out_len;
+   break;
+
+   default:
+   ret = 1;
+   break;
+   }
+
if (ret) {
printk(KERN_ERR "alg: comp: compression failed "
   "on test %d for %s: ret=%d\n", i + 1, algo,
@@ -1253,8 +1281,32 @@ static int test_comp(struct crypto_comp *tfm, struct 
comp_testvec *ctemplate,
memset(result, 0, sizeof (result));
 
ilen = dtemplate[i].inlen;
-   ret = crypto_comp_decompress(tfm, dtemplate[i].input,
-ilen, result, );
+   switch (type) {
+   case 0:
+   ret = crypto_comp_decompress(crypto_comp_cast(tfm),
+   dtemplate[i].input, ilen,
+   result, );
+   break;
+
+   case 1:
+   ret = crypto_scomp_decompress(crypto_scomp_cast(tfm),
+   dtemplate[i].input, ilen,
+   result, , ctx);
+   break;
+
+   case 2:
+   sg_init_one(, dtemplate[i].input, ilen);
+   sg_init_one(, result, dlen);
+   acomp_request_set_comp(ctx, , , ilen, dlen);
+   ret = crypto_acomp_decompress(ctx);
+   dlen = ((struct acomp_req *)ctx)->out_len;
+   break;
+
+   default:
+   ret = 1;
+   break;
+   }
+
if (ret) {
printk(KERN_ERR "alg: comp: decompression failed "
   "on test %d for %s: ret=%d\n", i + 1, algo,
@@ -1446,7 +1498,8 @@ static int alg_test_comp(const struct alg_test_desc 
*desc, const char *driver,
return PTR_ERR(tfm);
}
 
-   err = test_comp(tfm, desc->suite.comp.comp.vecs,
+   err = test_comp(crypto_comp_tfm(tfm), NULL, 0,
+ 

[PATCH RFC 3/7] crypto: add asynchronous compression support

[Joonsoo Kim]

From: Weigang Li <weigang...@intel.com>

Now, asynchronous compression APIs are supported. There is no asynchronous
compression driver now but this APIs can be used as front-end to
synchronous compression algorithm. In this case, scatterlist would be
linearlized so it would cause some overhead.

Signed-off-by: Weigang Li <weigang...@intel.com>
Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |   3 +-
 crypto/Makefile|   3 +-
 crypto/acompress.c | 164 +
 crypto/scompress.c | 169 ++
 include/crypto/compress.h  | 241 +
 include/crypto/internal/compress.h |   4 +
 include/linux/crypto.h |   2 +
 7 files changed, 583 insertions(+), 3 deletions(-)
 create mode 100644 crypto/acompress.c
 create mode 100644 include/crypto/internal/compress.h

diff --git a/crypto/Kconfig b/crypto/Kconfig
index 7159520..f22f4e9 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -84,7 +84,7 @@ config CRYPTO_RNG_DEFAULT
tristate
select CRYPTO_DRBG_MENU
 
-config CRYPTO_SCOMPRESS
+config CRYPTO_COMPRESS2
tristate
select CRYPTO_ALGAPI2
 
@@ -1503,7 +1503,6 @@ config CRYPTO_LZO
select CRYPTO_ALGAPI
select LZO_COMPRESS
select LZO_DECOMPRESS
-   select SCOMPRESS
help
  This is the LZO algorithm.
 
diff --git a/crypto/Makefile b/crypto/Makefile
index 16ef796..9157d69 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -28,7 +28,8 @@ crypto_hash-y += ahash.o
 crypto_hash-y += shash.o
 obj-$(CONFIG_CRYPTO_HASH2) += crypto_hash.o
 
-obj-$(CONFIG_CRYPTO_SCOMPRESS) += scompress.o
+crypto_compress-y += scompress.o acompress.o
+obj-$(CONFIG_CRYPTO_COMPRESS2) += crypto_compress.o
 obj-$(CONFIG_CRYPTO_AKCIPHER2) += akcipher.o
 
 $(obj)/rsapubkey-asn1.o: $(obj)/rsapubkey-asn1.c $(obj)/rsapubkey-asn1.h
diff --git a/crypto/acompress.c b/crypto/acompress.c
new file mode 100644
index 000..744286b
--- /dev/null
+++ b/crypto/acompress.c
@@ -0,0 +1,164 @@
+/*
+ * Asynchronous compression operations
+ *
+ * Copyright (c) 2015, Intel Corporation
+ * Authors: Weigang Li <weigang...@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include "internal.h"
+
+const struct crypto_type crypto_acomp_type;
+
+#ifdef CONFIG_NET
+static int crypto_acomp_report(struct sk_buff *skb, struct crypto_alg *alg)
+{
+   struct crypto_report_comp racomp;
+
+   strncpy(racomp.type, "acomp", sizeof(racomp.type));
+
+   if (nla_put(skb, CRYPTOCFGA_REPORT_COMPRESS,
+   sizeof(struct crypto_report_comp), ))
+   goto nla_put_failure;
+   return 0;
+
+nla_put_failure:
+   return -EMSGSIZE;
+}
+#else
+static int crypto_acomp_report(struct sk_buff *skb, struct crypto_alg *alg)
+{
+   return -ENOSYS;
+}
+#endif
+
+static void crypto_acomp_show(struct seq_file *m, struct crypto_alg *alg)
+   __attribute__ ((unused));
+
+static void crypto_acomp_show(struct seq_file *m, struct crypto_alg *alg)
+{
+   seq_puts(m, "type : acomp\n");
+}
+
+static void crypto_acomp_exit_tfm(struct crypto_tfm *tfm)
+{
+   struct crypto_acomp *acomp = __crypto_acomp_tfm(tfm);
+   struct acomp_alg *alg = crypto_acomp_alg(acomp);
+
+   alg->exit(acomp);
+}
+
+static int crypto_acomp_init_tfm(struct crypto_tfm *tfm)
+{
+   struct crypto_acomp *acomp = __crypto_acomp_tfm(tfm);
+   struct acomp_alg *alg = crypto_acomp_alg(acomp);
+
+   if (tfm->__crt_alg->cra_type != _acomp_type)
+   return crypto_init_scomp_ops_async(tfm);
+
+   acomp->compress = alg->compress;
+   acomp->decompress = alg->decompress;
+
+   if (alg->exit)
+   acomp->base.exit = crypto_acomp_exit_tfm;
+
+   if (alg->init)
+   return alg->init(acomp);
+
+   return 0;
+}
+
+static unsigned int crypto_acomp_extsize(struct crypto_alg *alg)
+{
+   if (alg->cra_type == _acomp_type)
+   return alg->cra_ctxsize;
+
+   return sizeof(void *);
+}
+
+const struct crypto_type crypto_acomp_type = {
+   .extsize = crypto_acomp_extsize,
+   .init_tfm = crypto_acomp_init_tfm,
+#ifdef CONFIG_PROC_FS
+   .show = crypto_acomp_show,
+#endif
+   .report = crypto_acomp_report,
+   .maskclear = ~CRYPTO_ALG_TYPE_MASK,
+   .maskset = CRYPTO_ALG_TYPE_ACOMPRESS_MASK,
+   .type = CRYPTO_ALG_TYPE_ACOMPRESS,
+   .tfmsize = offsetof(struct crypto_acomp, base)

RE: [PATCH] crypto: add asynchronous compression support

[Joonsoo Kim]

Hello, Herbert.

> -Original Message-
> From: Herbert Xu [mailto:herb...@gondor.apana.org.au]
> Sent: Thursday, November 19, 2015 6:43 PM
> To: Li, Weigang
> Cc: linux-crypto@vger.kernel.org; Struk, Tadeusz; Joonsoo Kim; Sergey
> Senozhatsky
> Subject: Re: [PATCH] crypto: add asynchronous compression support
> 
> On Thu, Nov 19, 2015 at 05:52:41AM +, Li, Weigang wrote:
> >
> > After sync-up with Joonsoo Kim, we think it may be not feasible for a
> s/w implementation of the sg-list based asynchronous interface, we propose
> separate interfaces (patches) for acomp & ccomp. The reasons are:
> > 1. to support sg-list in the ccomp (like what shash/ahash did), the
> partial update is required, some algorithms do not support partial update
> (i.e., lzo), that means:
> 
> No this is not true.  For the ones that don't support partial
> updates you can always linearise the input and then feed it in
> as one chunk.  Because the overall interface you're proposing
> does not allow partial updates the underlying implementation
> doesn't need to do it either.  Only linearisation is necessary.

Linearization would be enough to use sg-list but it has a problem.
Linearization needs sleepable function such as vmap() and it makes
sync (de)compression in atomic context impossible. Currently, zram
did sync (de)compression in atomic context. It uses map_vm_area() which
isn't sleepable function to linearize two separate pages. This is possible
because zram already knows that maximum number of spread pages is just two
and have allocated vm area in advance. But, if we implement linearization
in general API, we can't be sure of request input size so we need
sleepable function, vmap().

And, this sleep could degrade performance.

> > 2. the format of output buffer (sg-list) will be different, e.g., the
> lzo need contain the "length" info for each block in the output sg-list in
> order to de-compression, while zlib doesn't need, then it is difficult to
> have a single async sg-list i/f.
> 
> I have no idea what you mean here.  Please explain.
> 
> > 3. to compress a sg-list buffer, the lzo also requires an intermediate
> buffer to save the output of a block, and copy it back to the sg-list
> output buffer, it will introduce the complexity and cost, we don't see
> value for sg-list support in a s/w compression.
> 
> Such an intermediate buffer would only  be needed if the SG list is
> actually non-linear.  So I don't see this as an issue.

Intermediate buffer size could vary greatly so it would be allocated and
Freed whenever requested. This could affect performance.

I think that supporting unified API has more loss than gain.
I'm not expert on this area so please let me know what I missed.

Thanks.

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 0/8] zram: introduce contextless compression API and use it on zram

[Joonsoo Kim]

This patchset introduce contextless compression API and use it on zram in
order to support more compression algorithm without more overhead.

The reason we need to support vairous compression algorithms is that
zram's performance is highly depend on workload and compression algorithm
and architecture. Every compression algorithm has it's own strong point.
For example, zlib is the best for compression ratio, but, it's
(de)compression speed is rather slow. Against my expectation, in my kernel
build test with zram swap, in low-memory condition on x86, zlib shows best
performance than others. In this case, I guess that compression ratio is
the most important factor.

Kernel build elapsed time in QEMU 8 CPUs 448 MB with zram swap
lzo : deflate
188.3 s : 181.9 s
(3% improvement)

Unlike this situation, on ARM, maybe fast (de)compression speed is
the most important because it's computation speed is slower than x86.

Anyway, there is a concern from Sergey to use crypto API in zram. Current
crypto API has a limitation that always require tfm object to (de)compress
something because some of (de)compression function requires scratch buffer
embedded on tfm even if some of (de)compression function doesn't
require it. Due to above reason, using crypto API rather than calling
compression library directly causes more memory footprint and this is
why zram doesn't use crypto API before.

To solve this problem, this patchset introduce contextless compression API.
This new compression API doesn't require user to use tfm one by one.
Tfm is only used for distinguishing compression algorithm and maybe
keeping algorithm parameter so can be used concurrently. Context is now
separate from tfm and user needs to allocate and manage it separately.
With this separation, we can save memory in some cases and get the best
performance.

This patchset solves Sergey's concern perfectly and provides possibility
to use various compression algorithm in zram.

Thanks.

Joonsoo Kim (6):
  crypto/compress: introduce contextless compression and remove unused
pcomp
  crypto/lzo: support contextless compression API
  crypto/lz4: support contextless compressiona API
  crypto/deflate: support contextless compression API
  zram: use crypto contextless compression API to (de)compress
  zram: enable contextless compression alg in zram

Sergey Senozhatsky (2):
  zram: make stream find and release functions static
  zram: pass zstrm down to decompression path

 Documentation/blockdev/zram.txt|  29 ++-
 crypto/Kconfig |  18 +-
 crypto/Makefile|   3 +-
 crypto/ccompress.c |  95 +
 crypto/deflate.c   |  96 +-
 crypto/lz4.c   |  77 +++-
 crypto/lzo.c   |  81 ++--
 crypto/pcompress.c | 115 ---
 crypto/zlib.c  | 381 -
 drivers/block/zram/Kconfig |  13 +-
 drivers/block/zram/Makefile|   4 +-
 drivers/block/zram/zcomp.c | 104 ++
 drivers/block/zram/zcomp.h |  32 ++--
 drivers/block/zram/zram_drv.c  |  33 +++-
 include/crypto/compress.h  | 118 
 include/crypto/internal/compress.h |  28 ---
 include/linux/crypto.h |   2 +-
 17 files changed, 488 insertions(+), 741 deletions(-)
 create mode 100644 crypto/ccompress.c
 delete mode 100644 crypto/pcompress.c
 delete mode 100644 crypto/zlib.c
 delete mode 100644 include/crypto/internal/compress.h

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 3/8] crypto/lz4: support contextless compressiona API

[Joonsoo Kim]

Now, contextless compression API is introduced and it can reduce
memory overhead in some cases. All compression algorithm will
support it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/lz4.c | 77 
 1 file changed, 67 insertions(+), 10 deletions(-)

diff --git a/crypto/lz4.c b/crypto/lz4.c
index aefbcea..9720409 100644
--- a/crypto/lz4.c
+++ b/crypto/lz4.c
@@ -23,36 +23,53 @@
 #include 
 #include 
 #include 
+#include 
 
 struct lz4_ctx {
void *lz4_comp_mem;
 };
 
+static void *lz4_alloc_context(struct crypto_ccomp *tfm)
+{
+   void *ctx;
+
+   ctx = vmalloc(LZ4_MEM_COMPRESS);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   return ctx;
+}
+
 static int lz4_init(struct crypto_tfm *tfm)
 {
struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   ctx->lz4_comp_mem = vmalloc(LZ4_MEM_COMPRESS);
-   if (!ctx->lz4_comp_mem)
+   ctx->lz4_comp_mem = lz4_alloc_context(NULL);
+   if (IS_ERR(ctx->lz4_comp_mem))
return -ENOMEM;
 
return 0;
 }
 
+static void lz4_free_context(struct crypto_ccomp *tfm, void *ctx)
+{
+   vfree(ctx);
+}
+
 static void lz4_exit(struct crypto_tfm *tfm)
 {
struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
-   vfree(ctx->lz4_comp_mem);
+
+   lz4_free_context(NULL, ctx->lz4_comp_mem);
 }
 
-static int lz4_compress_crypto(struct crypto_tfm *tfm, const u8 *src,
-   unsigned int slen, u8 *dst, unsigned int *dlen)
+static int __lz4_compress_crypto(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
-   struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
size_t tmp_len = *dlen;
int err;
 
-   err = lz4_compress(src, slen, dst, _len, ctx->lz4_comp_mem);
+   err = lz4_compress(src, slen, dst, _len, ctx);
 
if (err < 0)
return -EINVAL;
@@ -61,8 +78,16 @@ static int lz4_compress_crypto(struct crypto_tfm *tfm, const 
u8 *src,
return 0;
 }
 
-static int lz4_decompress_crypto(struct crypto_tfm *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int *dlen)
+static int lz4_compress_crypto(struct crypto_tfm *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct lz4_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   return __lz4_compress_crypto(src, slen, dst, dlen, ctx->lz4_comp_mem);
+}
+
+static int __lz4_decompress_crypto(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
int err;
size_t tmp_len = *dlen;
@@ -76,6 +101,12 @@ static int lz4_decompress_crypto(struct crypto_tfm *tfm, 
const u8 *src,
return err;
 }
 
+static int lz4_decompress_crypto(struct crypto_tfm *tfm, const u8 *src,
+ unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   return __lz4_decompress_crypto(src, slen, dst, dlen, NULL);
+}
+
 static struct crypto_alg alg_lz4 = {
.cra_name   = "lz4",
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -89,14 +120,40 @@ static struct crypto_alg alg_lz4 = {
.coa_decompress = lz4_decompress_crypto } }
 };
 
+static struct ccomp_alg ccomp = {
+   .alloc_context  = lz4_alloc_context,
+   .free_context   = lz4_free_context,
+   .compress   = __lz4_compress_crypto,
+   .decompress = __lz4_decompress_crypto,
+   .flags  = CCOMP_TYPE_DECOMP_NOCTX,
+   .base   = {
+   .cra_name   = "lz4",
+   .cra_flags  = CRYPTO_ALG_TYPE_CCOMPRESS,
+   .cra_module = THIS_MODULE,
+   }
+};
+
 static int __init lz4_mod_init(void)
 {
-   return crypto_register_alg(_lz4);
+   int ret;
+
+   ret = crypto_register_alg(_lz4);
+   if (ret)
+   return ret;
+
+   ret = crypto_register_ccomp();
+   if (ret) {
+   crypto_unregister_alg(_lz4);
+   return ret;
+   }
+
+   return ret;
 }
 
 static void __exit lz4_mod_fini(void)
 {
crypto_unregister_alg(_lz4);
+   crypto_unregister_ccomp();
 }
 
 module_init(lz4_mod_init);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 4/8] crypto/deflate: support contextless compression API

[Joonsoo Kim]

Now, contextless compression API is introduced and it can reduce
memory overhead in some cases. All compression algorithm will
support it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/deflate.c | 96 ++--
 1 file changed, 86 insertions(+), 10 deletions(-)

diff --git a/crypto/deflate.c b/crypto/deflate.c
index 95d8d37..7070438 100644
--- a/crypto/deflate.c
+++ b/crypto/deflate.c
@@ -32,6 +32,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #define DEFLATE_DEF_LEVEL  Z_DEFAULT_COMPRESSION
 #define DEFLATE_DEF_WINBITS11
@@ -101,9 +102,8 @@ static void deflate_decomp_exit(struct deflate_ctx *ctx)
vfree(ctx->decomp_stream.workspace);
 }
 
-static int deflate_init(struct crypto_tfm *tfm)
+static int __deflate_init(void *ctx)
 {
-   struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
int ret;
 
ret = deflate_comp_init(ctx);
@@ -116,19 +116,54 @@ out:
return ret;
 }
 
-static void deflate_exit(struct crypto_tfm *tfm)
+static void *deflate_alloc_context(struct crypto_ccomp *tfm)
+{
+   void *ctx;
+   int ret;
+
+   ctx = kzalloc(sizeof(struct deflate_ctx), GFP_KERNEL);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   ret = __deflate_init(ctx);
+   if (ret) {
+   kfree(ctx);
+   return ERR_PTR(ret);
+   }
+
+   return ctx;
+}
+
+static int deflate_init(struct crypto_tfm *tfm)
 {
struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
 
+   return __deflate_init(ctx);
+}
+
+static void __deflate_exit(void *ctx)
+{
deflate_comp_exit(ctx);
deflate_decomp_exit(ctx);
 }
 
-static int deflate_compress(struct crypto_tfm *tfm, const u8 *src,
-   unsigned int slen, u8 *dst, unsigned int *dlen)
+static void deflate_free_context(struct crypto_ccomp *tfm, void *ctx)
+{
+   __deflate_exit(ctx);
+}
+
+static void deflate_exit(struct crypto_tfm *tfm)
+{
+   struct deflate_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   __deflate_exit(ctx);
+}
+
+static int __deflate_compress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
int ret = 0;
-   struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
+   struct deflate_ctx *dctx = ctx;
struct z_stream_s *stream = >comp_stream;
 
ret = zlib_deflateReset(stream);
@@ -153,12 +188,20 @@ out:
return ret;
 }
 
-static int deflate_decompress(struct crypto_tfm *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int *dlen)
+static int deflate_compress(struct crypto_tfm *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
+
+   return __deflate_compress(src, slen, dst, dlen, dctx);
+}
+
+static int __deflate_decompress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
 
int ret = 0;
-   struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
+   struct deflate_ctx *dctx = ctx;
struct z_stream_s *stream = >decomp_stream;
 
ret = zlib_inflateReset(stream);
@@ -194,6 +237,14 @@ out:
return ret;
 }
 
+static int deflate_decompress(struct crypto_tfm *tfm, const u8 *src,
+ unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct deflate_ctx *dctx = crypto_tfm_ctx(tfm);
+
+   return __deflate_compress(src, slen, dst, dlen, dctx);
+}
+
 static struct crypto_alg alg = {
.cra_name   = "deflate",
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -206,14 +257,39 @@ static struct crypto_alg alg = {
.coa_decompress = deflate_decompress } }
 };
 
+static struct ccomp_alg ccomp = {
+   .alloc_context  = deflate_alloc_context,
+   .free_context   = deflate_free_context,
+   .compress   = __deflate_compress,
+   .decompress = __deflate_decompress,
+   .base   = {
+   .cra_name   = "deflate",
+   .cra_flags  = CRYPTO_ALG_TYPE_CCOMPRESS,
+   .cra_module = THIS_MODULE,
+   }
+};
+
 static int __init deflate_mod_init(void)
 {
-   return crypto_register_alg();
+   int ret;
+
+   ret = crypto_register_alg();
+   if (ret)
+   return ret;
+
+   ret = crypto_register_ccomp();
+   if (ret) {
+   crypto_unregister_alg();
+   return ret;
+   }
+
+   return ret;
 }
 
 static void __exit deflate_mod_fini(void)
 {
crypto_unregister_alg();
+   crypto_unregister_ccomp();
 }
 
 module_init(deflate_mod_init);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...

[PATCH v4 1/8] crypto/compress: introduce contextless compression and remove unused pcomp

[Joonsoo Kim]

Until now, tfm for crypto compression embeds (de)compression context.
According to (de)compression algorithm, requirement on context varys but
to be safe to use (de)compression API, tfm should be passed to
(de)compression API one by one.

This causes unbearable overhead in some cases. For example, if someone
want to decompress 100 data in parallel with LZO, 100 tfms are needed and
it causes large memory overhead. But, LZO algorithm doesn't require
context in decompress so if we directly use decompress on LZO lib we don't
need 100 contexts in tfms thus there is no memory overhead.

This patch tries to fix this problem by introducing new compression API,
contextless compression. Although above problem can be solved by some
modifications on existing compression API, but, crypto maintainer, Herbert
recommends introducing new API because existing compression API
is obsolete.

This new compression API doesn't require user to use tfm one by one.
Tfm is only used for distinguish compression algorithm and maybe keeping
algorithm parameter so can be used concurrently. Context is now separate
from tfm and user needs to allocate and manage it separately.

crypto_ccomp_decomp_noctx() is provided to distinguish compression
algorithm which doesn't require context in decompression. If context isn't
needed, user can decompress something without passing allocated context
and can get maximum performance without additional memory overhead.

This API will be used in zram in following patches.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |  17 +-
 crypto/Makefile|   3 +-
 crypto/ccompress.c |  95 +
 crypto/pcompress.c | 115 ---
 crypto/zlib.c  | 381 -
 include/crypto/compress.h  | 118 
 include/crypto/internal/compress.h |  28 ---
 include/linux/crypto.h |   2 +-
 8 files changed, 134 insertions(+), 625 deletions(-)
 create mode 100644 crypto/ccompress.c
 delete mode 100644 crypto/pcompress.c
 delete mode 100644 crypto/zlib.c
 delete mode 100644 include/crypto/internal/compress.h

diff --git a/crypto/Kconfig b/crypto/Kconfig
index fc93444..cfc42e6 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -84,14 +84,8 @@ config CRYPTO_RNG_DEFAULT
tristate
select CRYPTO_DRBG_MENU
 
-config CRYPTO_PCOMP
+config CRYPTO_CCOMPRESS
tristate
-   select CRYPTO_PCOMP2
-   select CRYPTO_ALGAPI
-
-config CRYPTO_PCOMP2
-   tristate
-   select CRYPTO_ALGAPI2
 
 config CRYPTO_AKCIPHER2
tristate
@@ -1497,15 +1491,6 @@ config CRYPTO_DEFLATE
 
  You will most probably want this if using IPSec.
 
-config CRYPTO_ZLIB
-   tristate "Zlib compression algorithm"
-   select CRYPTO_PCOMP
-   select ZLIB_INFLATE
-   select ZLIB_DEFLATE
-   select NLATTR
-   help
- This is the zlib algorithm.
-
 config CRYPTO_LZO
tristate "LZO compression algorithm"
select CRYPTO_ALGAPI
diff --git a/crypto/Makefile b/crypto/Makefile
index e2c5981..93adbfe 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -28,7 +28,7 @@ crypto_hash-y += ahash.o
 crypto_hash-y += shash.o
 obj-$(CONFIG_CRYPTO_HASH2) += crypto_hash.o
 
-obj-$(CONFIG_CRYPTO_PCOMP2) += pcompress.o
+obj-$(CONFIG_CRYPTO_CCOMPRESS) += ccompress.o
 obj-$(CONFIG_CRYPTO_AKCIPHER2) += akcipher.o
 
 $(obj)/rsakey-asn1.o: $(obj)/rsakey-asn1.c $(obj)/rsakey-asn1.h
@@ -94,7 +94,6 @@ obj-$(CONFIG_CRYPTO_SALSA20) += salsa20_generic.o
 obj-$(CONFIG_CRYPTO_CHACHA20) += chacha20_generic.o
 obj-$(CONFIG_CRYPTO_POLY1305) += poly1305_generic.o
 obj-$(CONFIG_CRYPTO_DEFLATE) += deflate.o
-obj-$(CONFIG_CRYPTO_ZLIB) += zlib.o
 obj-$(CONFIG_CRYPTO_MICHAEL_MIC) += michael_mic.o
 obj-$(CONFIG_CRYPTO_CRC32C) += crc32c_generic.o
 obj-$(CONFIG_CRYPTO_CRC32) += crc32.o
diff --git a/crypto/ccompress.c b/crypto/ccompress.c
new file mode 100644
index 000..1983372
--- /dev/null
+++ b/crypto/ccompress.c
@@ -0,0 +1,95 @@
+/*
+ * Cryptographic API.
+ *
+ * Contextless (de)compression operations.
+ *
+ * Copyright 2015 LG Electronics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.
+ * If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include "internal.h"
+
+
+static int crypto_ccomp_init(struct crypto_tfm *tfm, u32 type, u32 m

[PATCH v4 2/8] crypto/lzo: support contextless compression API

[Joonsoo Kim]

Now, contextless compression API is introduced and it can reduce
memory overhead in some cases. All compression algorithm will
support it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/Kconfig |  1 +
 crypto/lzo.c   | 81 --
 2 files changed, 69 insertions(+), 13 deletions(-)

diff --git a/crypto/Kconfig b/crypto/Kconfig
index cfc42e6..d25746f 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1494,6 +1494,7 @@ config CRYPTO_DEFLATE
 config CRYPTO_LZO
tristate "LZO compression algorithm"
select CRYPTO_ALGAPI
+   select CRYPTO_CCOMPRESS
select LZO_COMPRESS
select LZO_DECOMPRESS
help
diff --git a/crypto/lzo.c b/crypto/lzo.c
index 4b3e925..a6ae752 100644
--- a/crypto/lzo.c
+++ b/crypto/lzo.c
@@ -22,40 +22,56 @@
 #include 
 #include 
 #include 
+#include 
 
 struct lzo_ctx {
void *lzo_comp_mem;
 };
 
+static void *lzo_alloc_context(struct crypto_ccomp *tfm)
+{
+   void *ctx;
+
+   ctx = kmalloc(LZO1X_MEM_COMPRESS,
+   GFP_KERNEL | __GFP_NOWARN | __GFP_REPEAT);
+   if (!ctx)
+   ctx = vmalloc(LZO1X_MEM_COMPRESS);
+   if (!ctx)
+   return ERR_PTR(-ENOMEM);
+
+   return ctx;
+}
+
 static int lzo_init(struct crypto_tfm *tfm)
 {
struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   ctx->lzo_comp_mem = kmalloc(LZO1X_MEM_COMPRESS,
-   GFP_KERNEL | __GFP_NOWARN | __GFP_REPEAT);
-   if (!ctx->lzo_comp_mem)
-   ctx->lzo_comp_mem = vmalloc(LZO1X_MEM_COMPRESS);
-   if (!ctx->lzo_comp_mem)
+   ctx->lzo_comp_mem = lzo_alloc_context(NULL);
+   if (IS_ERR(ctx->lzo_comp_mem))
return -ENOMEM;
 
return 0;
 }
 
+static void lzo_free_context(struct crypto_ccomp *tfm, void *ctx)
+{
+   kvfree(ctx);
+}
+
 static void lzo_exit(struct crypto_tfm *tfm)
 {
struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
 
-   kvfree(ctx->lzo_comp_mem);
+   lzo_free_context(NULL, ctx->lzo_comp_mem);
 }
 
-static int lzo_compress(struct crypto_tfm *tfm, const u8 *src,
-   unsigned int slen, u8 *dst, unsigned int *dlen)
+static int __lzo_compress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
-   struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
size_t tmp_len = *dlen; /* size_t(ulong) <-> uint on 64 bit */
int err;
 
-   err = lzo1x_1_compress(src, slen, dst, _len, ctx->lzo_comp_mem);
+   err = lzo1x_1_compress(src, slen, dst, _len, ctx);
 
if (err != LZO_E_OK)
return -EINVAL;
@@ -64,8 +80,16 @@ static int lzo_compress(struct crypto_tfm *tfm, const u8 
*src,
return 0;
 }
 
-static int lzo_decompress(struct crypto_tfm *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int *dlen)
+static int lzo_compress(struct crypto_tfm *tfm, const u8 *src,
+   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   struct lzo_ctx *ctx = crypto_tfm_ctx(tfm);
+
+   return __lzo_compress(src, slen, dst, dlen, ctx->lzo_comp_mem);
+}
+
+static int __lzo_decompress(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen, void *ctx)
 {
int err;
size_t tmp_len = *dlen; /* size_t(ulong) <-> uint on 64 bit */
@@ -77,7 +101,12 @@ static int lzo_decompress(struct crypto_tfm *tfm, const u8 
*src,
 
*dlen = tmp_len;
return 0;
+}
 
+static int lzo_decompress(struct crypto_tfm *tfm, const u8 *src,
+ unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+   return __lzo_decompress(src, slen, dst, dlen, NULL);
 }
 
 static struct crypto_alg alg = {
@@ -92,14 +121,40 @@ static struct crypto_alg alg = {
.coa_decompress = lzo_decompress } }
 };
 
+static struct ccomp_alg ccomp = {
+   .alloc_context  = lzo_alloc_context,
+   .free_context   = lzo_free_context,
+   .compress   = __lzo_compress,
+   .decompress = __lzo_decompress,
+   .flags  = CCOMP_TYPE_DECOMP_NOCTX,
+   .base   = {
+   .cra_name   = "lzo",
+   .cra_flags  = CRYPTO_ALG_TYPE_CCOMPRESS,
+   .cra_module = THIS_MODULE,
+   }
+};
+
 static int __init lzo_mod_init(void)
 {
-   return crypto_register_alg();
+   int ret;
+
+   ret = crypto_register_alg();
+   if (ret)
+   return ret;
+
+   ret = crypto_register_ccomp();
+   if (ret) {
+   crypto_unregister_alg();
+   return ret;
+   }
+
+   return ret;
 }
 
 static void __exit lzo_mod_fini(void)
 {
crypto_unregister_alg();
+   crypto_unregister_ccomp();
 }
 
 module_init(lzo_mod_init);
-- 
1.9.1

[PATCH v4 7/8] zram: use crypto contextless compression API to (de)compress

[Joonsoo Kim]

Until now, zram uses compression algorithm through direct call
to core algorithm function, but, it has drawback that we need to add
compression algorithm manually to zram if needed. Without this work,
we cannot utilize various compression algorithms in the system.
Moreover, to add new compression algorithm, we need to know how to use it
and this is somewhat time-consuming.

When I tested new algorithms such as zlib, these problems make me hard
to test them. To prevent these problem in the future, I think that
using crypto API for compression is better approach and this patch
implements it.

The reason we need to support vairous compression algorithms is that
zram's performance is highly depend on workload and compression algorithm
and architecture. Every compression algorithm has it's own strong point.
For example, zlib is the best for compression ratio, but, it's
(de)compression speed is rather slow. Against my expectation, in my kernel
build test with zram swap, in low-memory condition on x86, zlib shows best
performance than others. In this case, I guess that compression ratio is
the most important factor.

Kernel build elapsed time in QEMU 8 CPUs 448 MB with zram swap
lzo : deflate
188.3 s : 181.9 s
(3% improvement)

Unlike this situation, on ARM, maybe fast (de)compression speed is
the most important because it's computation speed is slower than x86.

We can't expect what algorithm is the best fit for one's system, because
it needs too complex calculation. We need to test it in case by case and
easy to use new compression algorithm by this patch will help
that situation.

Note that this patch just convert zram to use crypto contextless
compression API but doesn't change compression algorithm selection
interface in zram. So it doesn't support zlib yet. Following patch
will change this interface and enable various compression algorithm.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 drivers/block/zram/Kconfig|  8 +++---
 drivers/block/zram/Makefile   |  4 +--
 drivers/block/zram/zcomp.c| 63 ---
 drivers/block/zram/zcomp.h| 21 +++
 drivers/block/zram/zram_drv.c |  6 ++---
 5 files changed, 52 insertions(+), 50 deletions(-)

diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig
index 386ba3d..7619bed 100644
--- a/drivers/block/zram/Kconfig
+++ b/drivers/block/zram/Kconfig
@@ -1,8 +1,7 @@
 config ZRAM
tristate "Compressed RAM block device support"
depends on BLOCK && SYSFS && ZSMALLOC
-   select LZO_COMPRESS
-   select LZO_DECOMPRESS
+   select CRYPTO_LZO
default n
help
  Creates virtual block devices called /dev/zramX (X = 0, 1, ...).
@@ -18,9 +17,8 @@ config ZRAM
 config ZRAM_LZ4_COMPRESS
bool "Enable LZ4 algorithm support"
depends on ZRAM
-   select LZ4_COMPRESS
-   select LZ4_DECOMPRESS
+   select CRYPTO_LZ4
default n
help
  This option enables LZ4 compression algorithm support. Compression
- algorithm can be changed using `comp_algorithm' device attribute.
\ No newline at end of file
+ algorithm can be changed using `comp_algorithm' device attribute.
diff --git a/drivers/block/zram/Makefile b/drivers/block/zram/Makefile
index be0763f..9e2b79e 100644
--- a/drivers/block/zram/Makefile
+++ b/drivers/block/zram/Makefile
@@ -1,5 +1,3 @@
-zram-y :=  zcomp_lzo.o zcomp.o zram_drv.o
-
-zram-$(CONFIG_ZRAM_LZ4_COMPRESS) += zcomp_lz4.o
+zram-y :=  zcomp.o zram_drv.o
 
 obj-$(CONFIG_ZRAM) +=  zram.o
diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index e3016da..9be83db 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -15,10 +15,6 @@
 #include 
 
 #include "zcomp.h"
-#include "zcomp_lzo.h"
-#ifdef CONFIG_ZRAM_LZ4_COMPRESS
-#include "zcomp_lz4.h"
-#endif
 
 /*
  * single zcomp_strm backend
@@ -43,19 +39,20 @@ struct zcomp_strm_multi {
wait_queue_head_t strm_wait;
 };
 
-static struct zcomp_backend *backends[] = {
-   _lzo,
+static const char * const backends[] = {
+   "lzo",
 #ifdef CONFIG_ZRAM_LZ4_COMPRESS
-   _lz4,
+   "lz4",
 #endif
NULL
 };
 
-static struct zcomp_backend *find_backend(const char *compress)
+static const char *find_backend(const char *compress)
 {
int i = 0;
while (backends[i]) {
-   if (sysfs_streq(compress, backends[i]->name))
+   if (sysfs_streq(compress, backends[i]) &&
+   crypto_has_comp(backends[i], 0, 0))
break;
i++;
}
@@ -65,7 +62,7 @@ static struct zcomp_backend *find_backend(const char 
*compress)
 static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm)
 {
if (zstrm->private)
-   comp->backend->destroy(zstrm->private);
+

[PATCH v4 8/8] zram: enable contextless compression alg in zram

[Joonsoo Kim]

Now, zram uses contextless compression API and there is no reason
to limit compression algorithm through hard-wired string. This patch
remove it so enable all contextless compression algorithm in the
system.

After this patch, available compression algorithm for zram can be
retrieved by searching contextless compression in /proc/crypto.

cat /proc/crypto | grep ccomp -B 7
name : lz4
[snip...]
type : ccomp

Previous interface comp_algorithm attr will remain to set new algorithm.
Read from previous interface also works for compatibility but it will
be wrong.

Note that after this patch is applied, there is no way to know current
compression algorithm so it's really bad thing. Please let me know proper
solution if someone have better idea.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 Documentation/blockdev/zram.txt | 29 +++--
 drivers/block/zram/Kconfig  |  9 -
 drivers/block/zram/zcomp.c  | 17 +++--
 drivers/block/zram/zram_drv.c   |  1 +
 4 files changed, 31 insertions(+), 25 deletions(-)

diff --git a/Documentation/blockdev/zram.txt b/Documentation/blockdev/zram.txt
index 5bda503..7a165c2 100644
--- a/Documentation/blockdev/zram.txt
+++ b/Documentation/blockdev/zram.txt
@@ -81,15 +81,32 @@ dynamic max_comp_streams. Only multi stream backend 
supports dynamic
 max_comp_streams adjustment.
 
 3) Select compression algorithm
-   Using comp_algorithm device attribute one can see available and
-   currently selected (shown in square brackets) compression algorithms,
-   change selected compression algorithm (once the device is initialised
-   there is no way to change compression algorithm).
+   You can find available compression algorithms by searching contextless
+   compression algorithm (type: ccomp) in /proc/crypto.
+   Using comp_algorithm device attribute one can change selected
+   compression algorithm (once the device is initialised there is no way
+   to change compression algorithm).
 
Examples:
#show supported compression algorithms
-   cat /sys/block/zram0/comp_algorithm
-   lzo [lz4]
+   cat /proc/crypto | grep ccomp -B 7
+   name : lz4
+   driver   : lz4-generic
+   module   : kernel
+   priority : 0
+   refcnt   : 1
+   selftest : passed
+   internal : no
+   type : ccomp
+   --
+   name : lzo
+   driver   : lzo-generic
+   module   : kernel
+   priority : 0
+   refcnt   : 1
+   selftest : passed
+   internal : no
+   type : ccomp
 
#select lzo compression algorithm
echo lzo > /sys/block/zram0/comp_algorithm
diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig
index 7619bed..36ec96f 100644
--- a/drivers/block/zram/Kconfig
+++ b/drivers/block/zram/Kconfig
@@ -13,12 +13,3 @@ config ZRAM
  disks and maybe many more.
 
  See zram.txt for more information.
-
-config ZRAM_LZ4_COMPRESS
-   bool "Enable LZ4 algorithm support"
-   depends on ZRAM
-   select CRYPTO_LZ4
-   default n
-   help
- This option enables LZ4 compression algorithm support. Compression
- algorithm can be changed using `comp_algorithm' device attribute.
diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 9be83db..f91c0659 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -41,22 +41,16 @@ struct zcomp_strm_multi {
 
 static const char * const backends[] = {
"lzo",
-#ifdef CONFIG_ZRAM_LZ4_COMPRESS
"lz4",
-#endif
NULL
 };
 
 static const char *find_backend(const char *compress)
 {
-   int i = 0;
-   while (backends[i]) {
-   if (sysfs_streq(compress, backends[i]) &&
-   crypto_has_comp(backends[i], 0, 0))
-   break;
-   i++;
-   }
-   return backends[i];
+   if (crypto_has_comp(compress, 0, 0))
+   return compress;
+
+   return NULL;
 }
 
 static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm)
@@ -277,6 +271,9 @@ ssize_t zcomp_available_show(const char *comp, char *buf)
int i = 0;
 
while (backends[i]) {
+   if (!crypto_has_comp(backends[i], 0, 0))
+   continue;
+
if (!strcmp(comp, backends[i]))
sz += scnprintf(buf + sz, PAGE_SIZE - sz - 2,
"[%s] ", backends[i]);
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 6f04fb2..6b4cf85 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -352,6 +352,7 @@ static ssize_t comp_algorithm_show(struct device *dev,
size_t sz;
struct zram *zram = dev_to_zram(dev);
 
+   

[PATCH v4 5/8] zram: make stream find and release functions static

[Joonsoo Kim]

From: Sergey Senozhatsky <sergey.senozhat...@gmail.com>

Hide (make static) zstrm find and release function and introduce
zcomp_compress_begin()/zcomp_compress_end(). We will have begin
and end functions around compression (this patch) and decompression
(next patch). So the work flow is evolving to:

zstrm = foo_begin();
foo(zstrm);
foo_end(zstrm);

where foo is compress or decompress zcomp functions.

This patch is a preparation to make crypto API-powered zcomp
possible. The reasoning is that some crypto compression backends
require zstrm for decompression.

Acked-by: Minchan Kim <minc...@kernel.org>
Signed-off-by: Sergey Senozhatsky <sergey.senozhat...@gmail.com>
Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 drivers/block/zram/zcomp.c| 27 +--
 drivers/block/zram/zcomp.h|  8 ++--
 drivers/block/zram/zram_drv.c |  6 +++---
 3 files changed, 34 insertions(+), 7 deletions(-)

diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 5cb13ca..52d5b04 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -296,16 +296,39 @@ bool zcomp_set_max_streams(struct zcomp *comp, int 
num_strm)
return comp->set_max_streams(comp, num_strm);
 }
 
-struct zcomp_strm *zcomp_strm_find(struct zcomp *comp)
+static struct zcomp_strm *zcomp_strm_find(struct zcomp *comp)
 {
return comp->strm_find(comp);
 }
 
-void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm)
+static void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm)
 {
comp->strm_release(comp, zstrm);
 }
 
+/* Never return NULL, may sleep */
+struct zcomp_strm *zcomp_compress_begin(struct zcomp *comp)
+{
+   return zcomp_strm_find(comp);
+}
+
+void zcomp_compress_end(struct zcomp *comp, struct zcomp_strm *zstrm)
+{
+   zcomp_strm_release(comp, zstrm);
+}
+
+/* May return NULL, may sleep */
+struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp)
+{
+   return NULL;
+}
+
+void zcomp_decompress_end(struct zcomp *comp, struct zcomp_strm *zstrm)
+{
+   if (zstrm)
+   zcomp_strm_release(comp, zstrm);
+}
+
 int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
const unsigned char *src, size_t *dst_len)
 {
diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
index 46e2b9f..616e013 100644
--- a/drivers/block/zram/zcomp.h
+++ b/drivers/block/zram/zcomp.h
@@ -56,12 +56,16 @@ bool zcomp_available_algorithm(const char *comp);
 struct zcomp *zcomp_create(const char *comp, int max_strm);
 void zcomp_destroy(struct zcomp *comp);
 
-struct zcomp_strm *zcomp_strm_find(struct zcomp *comp);
-void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm);
+
+struct zcomp_strm *zcomp_compress_begin(struct zcomp *comp);
+void zcomp_compress_end(struct zcomp *comp, struct zcomp_strm *zstrm);
 
 int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
const unsigned char *src, size_t *dst_len);
 
+struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp);
+void zcomp_decompress_end(struct zcomp *comp, struct zcomp_strm *zstrm);
+
 int zcomp_decompress(struct zcomp *comp, const unsigned char *src,
size_t src_len, unsigned char *dst);
 
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 3e8d8ff..83a06f2 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -673,7 +673,7 @@ static int zram_bvec_write(struct zram *zram, struct 
bio_vec *bvec, u32 index,
goto out;
}
 
-   zstrm = zcomp_strm_find(zram->comp);
+   zstrm = zcomp_compress_begin(zram->comp);
user_mem = kmap_atomic(page);
 
if (is_partial_io(bvec)) {
@@ -744,7 +744,7 @@ static int zram_bvec_write(struct zram *zram, struct 
bio_vec *bvec, u32 index,
memcpy(cmem, src, clen);
}
 
-   zcomp_strm_release(zram->comp, zstrm);
+   zcomp_compress_end(zram->comp, zstrm);
zstrm = NULL;
zs_unmap_object(meta->mem_pool, handle);
 
@@ -764,7 +764,7 @@ static int zram_bvec_write(struct zram *zram, struct 
bio_vec *bvec, u32 index,
atomic64_inc(>stats.pages_stored);
 out:
if (zstrm)
-   zcomp_strm_release(zram->comp, zstrm);
+   zcomp_compress_end(zram->comp, zstrm);
if (is_partial_io(bvec))
kfree(uncmem);
return ret;
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 8/8] zram: enable contextless compression alg in zram

[Joonsoo Kim]

On Thu, Oct 15, 2015 at 09:29:03AM +0900, Sergey Senozhatsky wrote:
> Hi,
> 
> On (10/14/15 16:38), Joonsoo Kim wrote:
> [..]
> >  static const char * const backends[] = {
> > "lzo",
> > -#ifdef CONFIG_ZRAM_LZ4_COMPRESS
> > "lz4",
> > -#endif
> > NULL
> >  };
> >  
> >  static const char *find_backend(const char *compress)
> >  {
> > -   int i = 0;
> > -   while (backends[i]) {
> > -   if (sysfs_streq(compress, backends[i]) &&
> > -   crypto_has_comp(backends[i], 0, 0))
> > -   break;
> > -   i++;
> > -   }
> > -   return backends[i];
> > +   if (crypto_has_comp(compress, 0, 0))
> > +   return compress;
> > +
> > +   return NULL;
> >  }
> >  
> >  static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm)
> > @@ -277,6 +271,9 @@ ssize_t zcomp_available_show(const char *comp, char 
> > *buf)
> > int i = 0;
> >  
> > while (backends[i]) {
> > +   if (!crypto_has_comp(backends[i], 0, 0))
> > +   continue;
> > +
> 
> hm... this sort of looks a bit `unnatural' to me. we have two _independent_
> sets -- what zram supports and what crypto supports. that's why you have
> to do extra work and consult crypto. can we return back the old scheme:
> use ifdef CONFIG in backends, but replace CONFIG_ZRAM with CONFIG_CRYPTO?
> 
> e.g.
> 
>   static const char * const backends[] = {
>   "lzo",
>   #ifdef CONFIG_CRYPTO_LZ4
>   "lz4",
>   #endif
>   NULL
>   };
> 
> 
> so you can remove `crypto_has_comp(backends[i], 0, 0)' from
> zcomp_available_show(), because zram will support *only* what
> crypto supports.

Hello, Sergey.

Okay. I will change it in next spin.

Anyway, now I noticed that crypto_has_comp() is not proper API to
check contextless compression algorithm. I will change it, too.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 7/8] zram: use crypto contextless compression API to (de)compress

[Joonsoo Kim]

On Thu, Oct 15, 2015 at 09:38:41AM +0900, Sergey Senozhatsky wrote:
> Hi,
> 
> On (10/14/15 16:38), Joonsoo Kim wrote:
> [..]
> >   Creates virtual block devices called /dev/zramX (X = 0, 1, ...).
> > @@ -18,9 +17,8 @@ config ZRAM
> >  config ZRAM_LZ4_COMPRESS
> > bool "Enable LZ4 algorithm support"
> > depends on ZRAM
> > -   select LZ4_COMPRESS
> > -   select LZ4_DECOMPRESS
> > +   select CRYPTO_LZ4
> > default n
> > help
> >   This option enables LZ4 compression algorithm support. Compression
> > - algorithm can be changed using `comp_algorithm' device attribute.
> > \ No newline at end of file
> > + algorithm can be changed using `comp_algorithm' device attribute.
> > diff --git a/drivers/block/zram/Makefile b/drivers/block/zram/Makefile
> > index be0763f..9e2b79e 100644
> > --- a/drivers/block/zram/Makefile
> > +++ b/drivers/block/zram/Makefile
> > @@ -1,5 +1,3 @@
> > -zram-y :=  zcomp_lzo.o zcomp.o zram_drv.o
> > -
> > -zram-$(CONFIG_ZRAM_LZ4_COMPRESS) += zcomp_lz4.o
> > +zram-y :=  zcomp.o zram_drv.o
> 
> + git rm zcomp_lzo.* zcomp_lz4.*:)

Will do. :)

> 
> 
> >  obj-$(CONFIG_ZRAM) +=  zram.o
> > diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
> > index e3016da..9be83db 100644
> > --- a/drivers/block/zram/zcomp.c
> > +++ b/drivers/block/zram/zcomp.c
> > @@ -15,10 +15,6 @@
> >  #include 
> >  
> >  #include "zcomp.h"
> > -#include "zcomp_lzo.h"
> > -#ifdef CONFIG_ZRAM_LZ4_COMPRESS
> > -#include "zcomp_lz4.h"
> > -#endif
> >  
> >  /*
> >   * single zcomp_strm backend
> > @@ -43,19 +39,20 @@ struct zcomp_strm_multi {
> > wait_queue_head_t strm_wait;
> >  };
> >  
> > -static struct zcomp_backend *backends[] = {
> > -   _lzo,
> > +static const char * const backends[] = {
> > +   "lzo",
> >  #ifdef CONFIG_ZRAM_LZ4_COMPRESS
> 
> a nitpick -- this CONFIG option does not exist at this point.

Will fix.

> 
> > -   _lz4,
> > +   "lz4",
> >  #endif
> > NULL
> >  };
> >  
> > -static struct zcomp_backend *find_backend(const char *compress)
> > +static const char *find_backend(const char *compress)
> >  {
> > int i = 0;
> > while (backends[i]) {
> > -   if (sysfs_streq(compress, backends[i]->name))
> > +   if (sysfs_streq(compress, backends[i]) &&
> > +   crypto_has_comp(backends[i], 0, 0))
> 
> again (the same as in my previous email). I'd rather prefer to have
> FOO backends in ` const char * const backends[]' array only when the
> corresponding CONFIG_CRYPTO_FOO option has been selected. otherwise you
> have to find an intersection of two independent sets. sort of unreasonable
> to me.

Okay.

> 
> > break;
> > i++;
> > }
> > @@ -65,7 +62,7 @@ static struct zcomp_backend *find_backend(const char 
> > *compress)
> >  static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm)
> >  {
> > if (zstrm->private)
> > -   comp->backend->destroy(zstrm->private);
> > +   crypto_ccomp_free_context(comp->tfm, zstrm->private);
> > free_pages((unsigned long)zstrm->buffer, 1);
> > kfree(zstrm);
> >  }
> > @@ -80,7 +77,13 @@ static struct zcomp_strm *zcomp_strm_alloc(struct zcomp 
> > *comp)
> > if (!zstrm)
> > return NULL;
> >  
> > -   zstrm->private = comp->backend->create();
> > +   zstrm->private = crypto_ccomp_alloc_context(comp->tfm);
> > +   if (IS_ERR(zstrm->private)) {
> > +   zstrm->private = NULL;
> > +   zcomp_strm_free(comp, zstrm);
> > +   return NULL;
> > +   }
> > +
> > /*
> >  * allocate 2 pages. 1 for compressed data, plus 1 extra for the
> >  * case when compressed size is larger than the original one
> > @@ -274,12 +277,12 @@ ssize_t zcomp_available_show(const char *comp, char 
> > *buf)
> > int i = 0;
> >  
> > while (backends[i]) {
> > -   if (!strcmp(comp, backends[i]->name))
> > +   if (!strcmp(comp, backends[i]))
> > sz += scnprintf(buf + sz, PAGE_SIZE - sz - 2,
> > -   "[%s] ", backends[i]->name);
> > +   "[%s] ", backends[i]);
> > el

Re: [PATCH v4 8/8] zram: enable contextless compression alg in zram

[Joonsoo Kim]

On Thu, Oct 15, 2015 at 11:05:49AM +0900, Sergey Senozhatsky wrote:
> On (10/14/15 16:38), Joonsoo Kim wrote:
> [..]
> > @@ -352,6 +352,7 @@ static ssize_t comp_algorithm_show(struct device *dev,
> > size_t sz;
> > struct zram *zram = dev_to_zram(dev);
> >  
> > +   deprecated_attr_warn("comp_algorithm");
> > down_read(>init_lock);
> > sz = zcomp_available_show(zram->compressor, buf);
> > up_read(>init_lock);
> 
> oh, one more thing.
> 
> deprecated_attr_warn() should come with 
> `Documentation/ABI/obsolete/sysfs-block-zram' update.
> something like:
> 
> diff --git a/Documentation/ABI/obsolete/sysfs-block-zram 
> b/Documentation/ABI/obsolete/sysfs-block-zram
> index 720ea92..ad5b59d 100644
> --- a/Documentation/ABI/obsolete/sysfs-block-zram
> +++ b/Documentation/ABI/obsolete/sysfs-block-zram
> @@ -117,3 +117,14 @@ Description:
> Downgraded to write-only node: so it's possible to set new
> value only; its current value is stored in zram/mm_stat
> node.
> +
> +What:  /sys/block/zram/comp_algorithm
> +Date:  XXX
> +Contact:   XXX
> +Description:
> +   The comp_algorithm file is read/write and provides information
> +   on available, currently selected compression algorithm (read
> +   operation) and lets one to change the compression algorithm
> +   (write operation).
> +   Downgraded to write-only node: use `/proc/crypto' to get the
> +   list of supported compression algorithms.
> 
> 
> ---
> 
> 
> And I guess Cc `Jonathan Corbet <cor...@lwn.net>' (doc maintainer) and
> 'linux-...@vger.kernel.org' will be the right thing to do here.

Okay. I will do it in next spin.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 8/9] zram: use crypto API for compression

[Joonsoo Kim]

On Mon, Sep 21, 2015 at 02:19:03PM +0900, Sergey Senozhatsky wrote:
> On (09/18/15 14:19), Joonsoo Kim wrote:
> [..]
> > -static struct zcomp_backend *find_backend(const char *compress)
> > +static const char *find_backend(const char *compress)
> >  {
> > int i = 0;
> > while (backends[i]) {
> > -   if (sysfs_streq(compress, backends[i]->name))
> > +   if (sysfs_streq(compress, backends[i]) &&
> > +   crypto_has_comp(compress, 0, 0))
> 
> ok, just for note. zcomp does sysfs_streq(), because sysfs data
> usually contain a trailing new line, crypto_has_comp() does strcmp().

Okay. Will check.

> 
> 
> >  int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
> > -   const unsigned char *src, size_t *dst_len)
> > +   const unsigned char *src, unsigned int *dst_len)
> >  {
> > -   return comp->backend->compress(src, zstrm->buffer, dst_len,
> > -   zstrm->private);
> > +   *dst_len = PAGE_SIZE << 1;
> > +
> 
> hm... wouldn't it be better to say crypto api that we have a PAGE_SIZE
> buffer instead of PAGE_SIZE << 1, so in case of buffer overrun (or
> whatever is the correct term here) it will stop compression earlier
> (well, possibly)? zram will drop compressed data larger than PAGE_SIZE
> anyway, so if passing a smaller buffer size can save us CPU time then
> let's do it.

It can be implemented and maybe good way to go. But, in this patchset,
it isn't needed. It is better to do in separate patch.

> 
> > +   return crypto_comp_compress(zstrm->tfm, src, PAGE_SIZE,
> > +   zstrm->buffer, dst_len);
> >  }
> >  
> >  int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm,
> > const unsigned char *src,
> > -   size_t src_len, unsigned char *dst)
> > +   unsigned int src_len, unsigned char *dst)
> >  {
> > -   return comp->backend->decompress(src, src_len, dst);
> > +   unsigned int size = PAGE_SIZE;
> > +
> > +   return crypto_comp_decompress(zstrm->tfm, src, src_len, dst, );
> 
>    tab?
> 
> >  }
> >  
> >  void zcomp_destroy(struct zcomp *comp)
> > @@ -359,7 +365,7 @@ void zcomp_destroy(struct zcomp *comp)
> >  struct zcomp *zcomp_create(const char *compress, int max_strm)
> >  {
> > struct zcomp *comp;
> > -   struct zcomp_backend *backend;
> > +   const char *backend;
> 
> rebase against the current linux-next. this and the next patch do not
> apply cleanly. the function was touched recently: '+int error'.

Okay.

> 
> >  
> > backend = find_backend(compress);
> > if (!backend)
> > diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
> > index 4c09c01..4f9df8e 100644
> > --- a/drivers/block/zram/zcomp.h
> > +++ b/drivers/block/zram/zcomp.h
> > @@ -11,38 +11,22 @@
> >  #define _ZCOMP_H_
> >  
> >  #include 
> > +#include 
> >  
> >  struct zcomp_strm {
> > +   struct crypto_comp *tfm;
> > +
> > /* compression/decompression buffer */
> > void *buffer;
> > -   /*
> > -* The private data of the compression stream, only compression
> > -* stream backend can touch this (e.g. compression algorithm
> > -* working memory)
> > -*/
> > -   void *private;
> > +
> > /* used in multi stream backend, protected by backend strm_lock */
> > struct list_head list;
> >  };
> >  
> > -/* static compression backend */
> > -struct zcomp_backend {
> > -   int (*compress)(const unsigned char *src, unsigned char *dst,
> > -   size_t *dst_len, void *private);
> > -
> > -   int (*decompress)(const unsigned char *src, size_t src_len,
> > -   unsigned char *dst);
> > -
> > -   void *(*create)(void);
> > -   void (*destroy)(void *private);
> > -
> > -   const char *name;
> > -};
> > -
> >  /* dynamic per-device compression frontend */
> >  struct zcomp {
> > void *stream;
> > -   struct zcomp_backend *backend;
> > +   const char *backend;
> 
>   ^^ what for?

Will remove.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 8/9] zram: use crypto API for compression

[Joonsoo Kim]

On Mon, Sep 21, 2015 at 12:45:12PM +0900, Minchan Kim wrote:
> Hi Joonsoo,
> 
> On Fri, Sep 18, 2015 at 02:19:23PM +0900, Joonsoo Kim wrote:
> > Until now, zram uses compression algorithm through direct call
> > to core algorithm function, but, it has drawback that we need to add
> > compression algorithm manually to zram if needed. Without this work,
> > we cannot utilize various compression algorithms in the system.
> > Moreover, to add new compression algorithm, we need to know how to use it
> > and this is somewhat time-consuming.
> > 
> > When I tested new algorithms such as zlib, these problems make me hard
> > to test them. To prevent these problem in the future, I think that
> > using crypto API for compression is better approach and this patch
> > implements it.
> > 
> > The reason we need to support vairous compression algorithms is that
> > zram's performance is highly depend on workload and compression algorithm
> > and architecture. Every compression algorithm has it's own strong point.
> > For example, zlib is the best for compression ratio, but, it's
> > (de)compression speed is rather slow. Against my expectation, in my kernel
> > build test with zram swap, in low-memory condition on x86, zlib shows best
> > performance than others. In this case, I guess that compression ratio is
> > the most important factor. Unlike this situation, on ARM, maybe fast
> > (de)compression speed is the most important because it's computation speed
> > is slower than x86.
> > 
> > We can't expect what algorithm is the best fit for one's system, because
> > it needs too complex calculation. We need to test it in case by case and
> > easy to use new compression algorithm by this patch will help
> > that situation.
> > 
> > There is one problem that crypto API requires tfm object to (de)compress
> > something and zram abstract it on zstrm which is very limited resource.
> > If number of zstrm is set to low and is contended, zram's performace could
> > be down-graded due to this change. But, following patch support to use
> > crypto decompress_noctx API and would restore the performance as is.
> > 
> > Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
> > ---
> >  drivers/block/zram/Kconfig |  8 +++
> >  drivers/block/zram/Makefile|  4 +---
> >  drivers/block/zram/zcomp.c | 54 
> > +++---
> >  drivers/block/zram/zcomp.h | 30 ++-
> >  drivers/block/zram/zcomp_lz4.c | 47 
> >  drivers/block/zram/zcomp_lz4.h | 17 -
> >  drivers/block/zram/zcomp_lzo.c | 47 
> >  drivers/block/zram/zcomp_lzo.h | 17 -
> >  drivers/block/zram/zram_drv.c  |  6 ++---
> >  9 files changed, 44 insertions(+), 186 deletions(-)
> >  delete mode 100644 drivers/block/zram/zcomp_lz4.c
> >  delete mode 100644 drivers/block/zram/zcomp_lz4.h
> >  delete mode 100644 drivers/block/zram/zcomp_lzo.c
> >  delete mode 100644 drivers/block/zram/zcomp_lzo.h
> > 
> > diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig
> > index 386ba3d..7619bed 100644
> > --- a/drivers/block/zram/Kconfig
> > +++ b/drivers/block/zram/Kconfig
> > @@ -1,8 +1,7 @@
> >  config ZRAM
> > tristate "Compressed RAM block device support"
> > depends on BLOCK && SYSFS && ZSMALLOC
> > -   select LZO_COMPRESS
> > -   select LZO_DECOMPRESS
> > +   select CRYPTO_LZO
> > default n
> > help
> >   Creates virtual block devices called /dev/zramX (X = 0, 1, ...).
> > @@ -18,9 +17,8 @@ config ZRAM
> >  config ZRAM_LZ4_COMPRESS
> > bool "Enable LZ4 algorithm support"
> > depends on ZRAM
> > -   select LZ4_COMPRESS
> > -   select LZ4_DECOMPRESS
> > +   select CRYPTO_LZ4
> 
> It is out of my expectation.
> 
> When I heard crypto support for zRAM first, I imagined zram user can
> use any crypto compressor algorithm easily if system has the algorithm.
> IOW, I expect zRAM shouldn't bind CONFIG_CRYPTO_ALGONAME statically
> except the default one(ie, CONFIG_CRYPTO_LZO) like zswap and It seems
> you did in eariler version.
> 
> You seem to have a trouble to adapt crypto to current comp_algorithm
> because crypto doesn't export any API to enumerate algorithm name
> while zram should export supporting algorithm name via comp_algorithm
> and I heard crypto maintainer doesn't want to export it. Instead,
> user can use /proc/crypto to know what kinds of compressor

Re: [PATCH v3 1/9] crypto: introduce decompression API that can be called via sharable tfm object

[Joonsoo Kim]

On Mon, Sep 21, 2015 at 02:38:59PM +0900, Sergey Senozhatsky wrote:
> On (09/18/15 14:19), Joonsoo Kim wrote:
> [..]
> > @@ -61,7 +61,8 @@ static struct crypto_alg alg = {
> > .cra_module = THIS_MODULE,
> > .cra_u  = { .compress = {
> > .coa_compress   = crypto842_compress,
> > -   .coa_decompress = crypto842_decompress } }
> > +   .coa_decompress = crypto842_decompress,
> > +   .coa_decompress_noctx   = NULL } }
> >  };
> >  
> >  static int __init crypto842_mod_init(void)
> > diff --git a/crypto/compress.c b/crypto/compress.c
> > index c33f076..abb36a8 100644
> > --- a/crypto/compress.c
> > +++ b/crypto/compress.c
> > @@ -33,12 +33,21 @@ static int crypto_decompress(struct crypto_tfm *tfm,
> >dlen);
> >  }
> >  
> > +static int crypto_decompress_noctx(struct crypto_tfm *tfm,
> > +   const u8 *src, unsigned int slen,
> > +   u8 *dst, unsigned int *dlen)
> > +{
> > +   return tfm->__crt_alg->cra_compress.coa_decompress_noctx(src, slen,
> > +   dst, dlen);
> > +}
> 
> 
> hm... well... sorry, if irrelevant.
> if the algorithm can have a _noctx() decompression function, does it
> automatically guarantee that this algorithm never dereferences a passed
> `struct crypto_tfm *tfm' pointer in its decompress function? in other words,
> can we simply pass that `shared tfm pointer' to the existing decompress
> function instead of defining new symbols, new callbacks, etc.?
> 
>   cot_decompress_noctx()  ==  cot_decompress(shared_ftm) ?
> 
> just a thought.

Will think it if I implement next version in this way. Due to Herbert
comment, interface will be changed so much in next version.

> 
> [..]
> > +struct crypto_comp *crypto_alloc_comp_noctx(const char *alg_name,
> > +   u32 type, u32 mask)
> > +{
> > +   struct crypto_comp *comp;
> > +   struct crypto_tfm *tfm;
> > +   struct compress_tfm *ops;
> > +
> > +   comp = crypto_alloc_comp(alg_name, type, mask);
> > +   if (IS_ERR(comp))
> > +   return comp;
> > +
> > +   tfm = crypto_comp_tfm(comp);
> > +   if (!tfm->__crt_alg->cra_compress.coa_decompress_noctx) {
> > +   crypto_free_comp(comp);
> > +   return ERR_PTR(-EINVAL);
> 
>   -ENOMEM?

No, it's failure isn't related to NOMEM. Just not support it.

Thanks.

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 9/9] zram: use crypto decompress_noctx API

[Joonsoo Kim]

On Mon, Sep 21, 2015 at 04:56:00PM +0900, Sergey Senozhatsky wrote:
> On (09/18/15 14:19), Joonsoo Kim wrote:
> [..]
> > +   /*
> > +* Prepare to use crypto decompress_noctx API. One tfm is required
> > +* to initialize crypto algorithm properly and fetch corresponding
> > +* function pointer. But, it is sharable for multiple concurrent
> > +* decompress users.
> > +*/
> 
> if comp algorithm require zstrm for both compression and decompression,
> then there seems to be no need in allocating sharable ->tfm_noctx, we
> will never use it.
> 

Yes, in that case, NULL will be returned. I should describe it on
somewhere.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 0/9] zram: introduce crypto decompress noctx API and use it on zram

[Joonsoo Kim]

On Mon, Sep 21, 2015 at 12:58:12PM +0900, Minchan Kim wrote:
> On Fri, Sep 18, 2015 at 02:19:15PM +0900, Joonsoo Kim wrote:
> > This patchset makes zram to use crypto API in order to support
> > more compression algorithm.
> > 
> > The reason we need to support vairous compression algorithms is that
> > zram's performance is highly depend on workload and compression algorithm
> > and architecture. Every compression algorithm has it's own strong point.
> > For example, zlib is the best for compression ratio, but, it's
> > (de)compression speed is rather slow. Against my expectation, in my kernel
> > build test with zram swap, in low-memory condition on x86, zlib shows best
> > performance than others. In this case, I guess that compression ratio is
> > the most important factor. Unlike this situation, on ARM, maybe fast
> > (de)compression speed is the most important because it's computation speed
> > is slower than x86.
> 
> Fair enough. lzo and lz4 cannot cover all of workloads so surely, there are
> workloads other compressor algorithm can win. As well, we could support
> H/W compressor with crypto support so I think crypto is a way to go.

Hello, Minchan.

Good!

> 
> One thing I have a concern is I don't want to bind some of compressor
> algorithm to zram statically. User can see what kinds of crypto compressor
> support in his system via /proc/crypto and use it dynamically.
> I hope zram supports it.

Okay. I will handle it next version.

> > 
> > Anyway, there is a concern from Sergey to use crypto API in zram. Current
> > crypto API has a limitation that always require tfm object to (de)compress
> > something because some of (de)compression function requires scratch buffer
> > embedded on tfm even if some of (de)compression function doesn't
> > require it. Due to above reason, using crypto API rather than calling
> 
> Nice catch from Sergey!
> 
> > compression library directly causes more memory footprint and this is
> > why zram doesn't use crypto API before.
> > 
> > In this patchset, crypto compress noctx API is introduced to reduce memory
> > footprint caused by maintaining multiple tfm and zram uses it. Before
> > noctx API, zram's performace is down-graded if tfm is insufficient. But,
> > after applying noctx API, performace is restored.
> > 
> > This addresses Sergey's concern perfectly and provides possibility to use
> > various compression algorithm in zram.
> > 
> > Following is zram's read performance number.
> > 
> > * iozone -t 4 -R -r 16K -s 60M -I +Z -i 0 -i 1
> > * max_stream is set to 1
> > * Output is in Kbytes/sec
> > 
> > zram-base vs zram-crypto vs zram-crypto-noctx
> > 
> > Read10411701.88 6426911.62  9423894.38 
> > Re-read 10017386.62 6428218.88  1163.50 
> 
> Another patch and number we need to merge this patchset is zlib support
> patchset and number you had experiement.

Okay. Will include.

Thanks.

> > 
> > Thanks.
> > 
> > Joonsoo Kim (7):
> >   crypto: introduce decompression API that can be called via sharable
> > tfm object
> >   crypto/lzo: support decompress_noctx
> >   crypyo/lz4: support decompress_noctx
> >   crypto/lz4hc: support decompress_noctx
> >   crypto/842: support decompress_noctx
> >   zram: use crypto API for compression
> >   zram: use crypto decompress_noctx API
> > 
> > Sergey Senozhatsky (2):
> >   zram: make stream find and release functions static
> >   zram: pass zstrm down to decompression path
> > 
> >  crypto/842.c   |   9 +++-
> >  crypto/compress.c  |  36 +++
> >  crypto/crypto_null.c   |   3 +-
> >  crypto/deflate.c   |   3 +-
> >  crypto/lz4.c   |   9 +++-
> >  crypto/lz4hc.c |   9 +++-
> >  crypto/lzo.c   |   9 +++-
> >  drivers/block/zram/Kconfig |   8 ++--
> >  drivers/block/zram/Makefile|   4 +-
> >  drivers/block/zram/zcomp.c | 102 
> > +++--
> >  drivers/block/zram/zcomp.h |  42 +++--
> >  drivers/block/zram/zcomp_lz4.c |  47 ---
> >  drivers/block/zram/zcomp_lz4.h |  17 ---
> >  drivers/block/zram/zcomp_lzo.c |  47 ---
> >  drivers/block/zram/zcomp_lzo.h |  17 ---
> >  drivers/block/zram/zram_drv.c  |  32 +
> >  include/linux/crypto.h |  20 
> >  17 files changed, 211 insertions(+), 203 deletions(-)
> >  delete mode 100644 drivers/block/zr

Re: [PATCH v3 1/9] crypto: introduce decompression API that can be called via sharable tfm object

[Joonsoo Kim]

On Tue, Sep 22, 2015 at 08:43:46PM +0800, Herbert Xu wrote:
> On Fri, Sep 18, 2015 at 02:19:16PM +0900, Joonsoo Kim wrote:
> > Until now, tfm object embeds (de)compression context in it and
> > (de)compression in crypto API requires tfm object to use
> > this context. But, there are some algorithms that doesn't need
> > such context to operate. Therefore, this patch introduce new crypto
> > decompression API that calls decompression function via sharable tfm
> > object. Concurrent calls to decompress_noctx function through sharable
> > tfm object will be okay because caller don't need any context in tfm and
> > tfm is only used for fetching function pointer to decompress_noctx
> > function. This can reduce overhead of maintaining multiple tfm
> > if decompression doesn't require any context to operate.
> > 
> > Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
> 
> Have you looked into include/crypto/compress.h?

Okay. Now, I have looked it.

> 
> The compress type itself is obsolete and should be replaced with
> the pcomp interface.
> 
> However, we made some mistakes with the pcomp interface.  First
> of all it doesn't have a non-partial interface like the digest
> function for crypto_shash.
> 
> But the biggest problem is that it should be completely stateless
> but is not.  IOW we should not store anything in the tfm that
> is per-request.  That should all go into the request structure.

I'm not a expert on this area but I have different opinion.
IIUC, what partial compression aims at is to support partial data
compression. It isn't for stateless compression and they are
different. Current implementation would work well for it's own
purpose.

And, making it stateless looks not good to me. If we make it
stateless, we should include algorithm's state in request object.
It enforces much memory to request object and, unlike crypto_shash
which can be allocated in stack, user needs to manage life time of
request objects and it requires additional complexity to
compression user.

Moreover, to use sharable tfm concurrently from my original intend,
many request objects would be needed and if it cannot be allocated
in the stack, it requires much memory. It's not suitable
for my purpose.

If compression is obsolete, what I think the best is leaving pcomp
as is and implementing sharable tfm in pcomp, And then, make lzo
and others support pcomp interface and sharable tfm.

Maybe, I'm wrong so please correct me.

Thanks.

> Fortunately it seems that pcomp doesn't actually have any users
> so we can just rip it out and start from scratch and redo it
> properly.
> 
> So to recap, please abandon any efforts on the crypto_compress
> interface as it is old and obsolete.  Instead reshape crypto_pcomp
> into a proper stateless interface which should then give you what
> you want.
> 
> When you do so, just keep in mind that we need to find a way to
> support IPComp.  That means the ability to allocate requests in
> thread context and then use them to compress/decompress in IRQ
> context.
> 
> Cheers,
> -- 
> Email: Herbert Xu <herb...@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 1/9] crypto: introduce decompression API that can be called via sharable tfm object

[Joonsoo Kim]

On Mon, Sep 21, 2015 at 03:18:17PM +0900, Sergey Senozhatsky wrote:
> On (09/18/15 14:19), Joonsoo Kim wrote:
> [..]
> >  static int __init lzo_mod_init(void)
> > diff --git a/include/linux/crypto.h b/include/linux/crypto.h
> > index e71cb70..31152b1 100644
> > --- a/include/linux/crypto.h
> > +++ b/include/linux/crypto.h
> > @@ -355,6 +355,8 @@ struct compress_alg {
> > unsigned int slen, u8 *dst, unsigned int *dlen);
> > int (*coa_decompress)(struct crypto_tfm *tfm, const u8 *src,
> >   unsigned int slen, u8 *dst, unsigned int *dlen);
> > +   int (*coa_decompress_noctx)(const u8 *src, unsigned int slen,
> > +   u8 *dst, unsigned int *dlen);
> >  };
> >  
> >  
> > @@ -538,6 +540,9 @@ struct compress_tfm {
> > int (*cot_decompress)(struct crypto_tfm *tfm,
> >   const u8 *src, unsigned int slen,
> >   u8 *dst, unsigned int *dlen);
> > +   int (*cot_decompress_noctx)(struct crypto_tfm *tfm,
> > +   const u8 *src, unsigned int slen,
> > +   u8 *dst, unsigned int *dlen);
> >  };
> >  
> >  #define crt_ablkcipher crt_u.ablkcipher
> > @@ -1836,6 +1841,14 @@ static inline void crypto_free_comp(struct 
> > crypto_comp *tfm)
> > crypto_free_tfm(crypto_comp_tfm(tfm));
> >  }
> >  
> > +struct crypto_comp *crypto_alloc_comp_noctx(const char *alg_name,
> > +   u32 type, u32 mask);
> > +
> 
> this should be EXPORT_SYMBOL_GPL().
> 

Will do in next version.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 9/9] zram: use crypto decompress_noctx API

[Joonsoo Kim]

On Mon, Sep 21, 2015 at 12:51:28PM +0900, Minchan Kim wrote:
> On Fri, Sep 18, 2015 at 02:19:24PM +0900, Joonsoo Kim wrote:
> > Crypto subsystem now supports decompress_noctx API that requires
> > special tfm_noctx. This tfm can be shared by multiple concurrent
> > decompress user because this API doesn't rely on this tfm object
> > except to fetch decompress function pointer.
> > 
> > Until changing to use crypto API, zram doesn't require any zstrm
> > on decompress so decompress is parallelized unlimitedly. But, previous
> > patch make zram to use crypto API and this requires one zstrm on every
> > decompress users so, in some zstrm contended situations, zram's
> > performance would be degraded.
> > 
> > This patch makes zram use decompress_noctx API and restore zram's
> > performance as the time that zram doesn't use crypto API.
> > 
> > Following is zram's read performance number.
> > 
> > * iozone -t 4 -R -r 16K -s 60M -I +Z -i 0 -i 1
> > * max_stream is set to 1
> > * Output is in Kbytes/sec
> > 
> > zram-base vs zram-crypto vs zram-crypto-noctx
> > 
> > Read    10411701.88 6426911.62  9423894.38
> > Re-read 10017386.62 6428218.88  1163.50
> > 
> > Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
> > ---
> >  drivers/block/zram/zcomp.c | 24 +++-
> >  drivers/block/zram/zcomp.h |  1 +
> >  2 files changed, 24 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
> > index c2ed7c8..a020200 100644
> > --- a/drivers/block/zram/zcomp.c
> > +++ b/drivers/block/zram/zcomp.c
> > @@ -319,9 +319,12 @@ void zcomp_compress_end(struct zcomp *comp, struct 
> > zcomp_strm *zstrm)
> > zcomp_strm_release(comp, zstrm);
> >  }
> >  
> > -/* Never return NULL, may sleep */
> > +/* May return NULL, may sleep */
> >  struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp)
> >  {
> > +   if (comp->tfm_noctx)
> > +   return NULL;
> 
> Hmm,, I understand why returns NULL but it seems to be awkward to use NULL
> to express meaningful semantic and following functions relies on.
> If I have a time, I will think over.

I also think that it's not good thing but I can't think any better
idea. Please let me know if you have better one.

> 
> > +
> > return zcomp_strm_find(comp);
> >  }
> >  
> > @@ -346,11 +349,18 @@ int zcomp_decompress(struct zcomp *comp, struct 
> > zcomp_strm *zstrm,
> >  {
> > unsigned int size = PAGE_SIZE;
> >  
> > +   if (!zstrm) {
> > +   return crypto_comp_decompress_noctx(comp->tfm_noctx,
> > +   src, src_len, dst, );
> > +   }
> > +
> > return crypto_comp_decompress(zstrm->tfm, src, src_len, dst, );
> >  }
> >  
> >  void zcomp_destroy(struct zcomp *comp)
> >  {
> > +   if (comp->tfm_noctx)
> > +   crypto_free_comp_noctx(comp->tfm_noctx);
> > comp->destroy(comp);
> > kfree(comp);
> >  }
> > @@ -366,6 +376,7 @@ struct zcomp *zcomp_create(const char *compress, int 
> > max_strm)
> >  {
> > struct zcomp *comp;
> > const char *backend;
> > +   struct crypto_comp *tfm;
> >  
> > backend = find_backend(compress);
> > if (!backend)
> > @@ -384,5 +395,16 @@ struct zcomp *zcomp_create(const char *compress, int 
> > max_strm)
> > kfree(comp);
> > return ERR_PTR(-ENOMEM);
> > }
> > +
> > +   /*
> > +* Prepare to use crypto decompress_noctx API. One tfm is required
> > +* to initialize crypto algorithm properly and fetch corresponding
> > +* function pointer. But, it is sharable for multiple concurrent
> > +* decompress users.
> > +*/
> 
> Please comment out that this API will return NULL if the compressor doesn't
> support noctx mode.

Will do.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 9/9] zram: use crypto decompress_noctx API

[Joonsoo Kim]

On Mon, Sep 21, 2015 at 02:29:18PM +0900, Sergey Senozhatsky wrote:
> On (09/18/15 14:19), Joonsoo Kim wrote:
> > -/* Never return NULL, may sleep */
> > +/* May return NULL, may sleep */
> >  struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp)
> >  {
> > +   if (comp->tfm_noctx)
> > +   return NULL;
> > +
> > return zcomp_strm_find(comp);
> >  }
> >  
> > @@ -346,11 +349,18 @@ int zcomp_decompress(struct zcomp *comp, struct 
> > zcomp_strm *zstrm,
> >  {
> > unsigned int size = PAGE_SIZE;
> >  
> > +   if (!zstrm) {
> > +   return crypto_comp_decompress_noctx(comp->tfm_noctx,
> > +   src, src_len, dst, );
> > +   }
> 
> unneeded braces.

It's more readable for me. But, if you don't like it, I will remove brace.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 0/9] zram: introduce crypto decompress noctx API and use it on zram

[Joonsoo Kim]

This patchset makes zram to use crypto API in order to support
more compression algorithm.

The reason we need to support vairous compression algorithms is that
zram's performance is highly depend on workload and compression algorithm
and architecture. Every compression algorithm has it's own strong point.
For example, zlib is the best for compression ratio, but, it's
(de)compression speed is rather slow. Against my expectation, in my kernel
build test with zram swap, in low-memory condition on x86, zlib shows best
performance than others. In this case, I guess that compression ratio is
the most important factor. Unlike this situation, on ARM, maybe fast
(de)compression speed is the most important because it's computation speed
is slower than x86.

Anyway, there is a concern from Sergey to use crypto API in zram. Current
crypto API has a limitation that always require tfm object to (de)compress
something because some of (de)compression function requires scratch buffer
embedded on tfm even if some of (de)compression function doesn't
require it. Due to above reason, using crypto API rather than calling
compression library directly causes more memory footprint and this is
why zram doesn't use crypto API before.

In this patchset, crypto compress noctx API is introduced to reduce memory
footprint caused by maintaining multiple tfm and zram uses it. Before
noctx API, zram's performace is down-graded if tfm is insufficient. But,
after applying noctx API, performace is restored.

This addresses Sergey's concern perfectly and provides possibility to use
various compression algorithm in zram.

Following is zram's read performance number.

* iozone -t 4 -R -r 16K -s 60M -I +Z -i 0 -i 1
* max_stream is set to 1
* Output is in Kbytes/sec

zram-base vs zram-crypto vs zram-crypto-noctx

Read10411701.88 6426911.62  9423894.38 
Re-read 10017386.62 6428218.88  1163.50 

Thanks.

Joonsoo Kim (7):
  crypto: introduce decompression API that can be called via sharable
tfm object
  crypto/lzo: support decompress_noctx
  crypyo/lz4: support decompress_noctx
  crypto/lz4hc: support decompress_noctx
  crypto/842: support decompress_noctx
  zram: use crypto API for compression
  zram: use crypto decompress_noctx API

Sergey Senozhatsky (2):
  zram: make stream find and release functions static
  zram: pass zstrm down to decompression path

 crypto/842.c   |   9 +++-
 crypto/compress.c  |  36 +++
 crypto/crypto_null.c   |   3 +-
 crypto/deflate.c   |   3 +-
 crypto/lz4.c   |   9 +++-
 crypto/lz4hc.c |   9 +++-
 crypto/lzo.c   |   9 +++-
 drivers/block/zram/Kconfig |   8 ++--
 drivers/block/zram/Makefile|   4 +-
 drivers/block/zram/zcomp.c | 102 +++--
 drivers/block/zram/zcomp.h |  42 +++--
 drivers/block/zram/zcomp_lz4.c |  47 ---
 drivers/block/zram/zcomp_lz4.h |  17 ---
 drivers/block/zram/zcomp_lzo.c |  47 ---
 drivers/block/zram/zcomp_lzo.h |  17 ---
 drivers/block/zram/zram_drv.c  |  32 +
 include/linux/crypto.h |  20 
 17 files changed, 211 insertions(+), 203 deletions(-)
 delete mode 100644 drivers/block/zram/zcomp_lz4.c
 delete mode 100644 drivers/block/zram/zcomp_lz4.h
 delete mode 100644 drivers/block/zram/zcomp_lzo.c
 delete mode 100644 drivers/block/zram/zcomp_lzo.h

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 8/9] zram: use crypto API for compression

[Joonsoo Kim]

Until now, zram uses compression algorithm through direct call
to core algorithm function, but, it has drawback that we need to add
compression algorithm manually to zram if needed. Without this work,
we cannot utilize various compression algorithms in the system.
Moreover, to add new compression algorithm, we need to know how to use it
and this is somewhat time-consuming.

When I tested new algorithms such as zlib, these problems make me hard
to test them. To prevent these problem in the future, I think that
using crypto API for compression is better approach and this patch
implements it.

The reason we need to support vairous compression algorithms is that
zram's performance is highly depend on workload and compression algorithm
and architecture. Every compression algorithm has it's own strong point.
For example, zlib is the best for compression ratio, but, it's
(de)compression speed is rather slow. Against my expectation, in my kernel
build test with zram swap, in low-memory condition on x86, zlib shows best
performance than others. In this case, I guess that compression ratio is
the most important factor. Unlike this situation, on ARM, maybe fast
(de)compression speed is the most important because it's computation speed
is slower than x86.

We can't expect what algorithm is the best fit for one's system, because
it needs too complex calculation. We need to test it in case by case and
easy to use new compression algorithm by this patch will help
that situation.

There is one problem that crypto API requires tfm object to (de)compress
something and zram abstract it on zstrm which is very limited resource.
If number of zstrm is set to low and is contended, zram's performace could
be down-graded due to this change. But, following patch support to use
crypto decompress_noctx API and would restore the performance as is.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 drivers/block/zram/Kconfig |  8 +++
 drivers/block/zram/Makefile|  4 +---
 drivers/block/zram/zcomp.c | 54 +++---
 drivers/block/zram/zcomp.h | 30 ++-
 drivers/block/zram/zcomp_lz4.c | 47 
 drivers/block/zram/zcomp_lz4.h | 17 -
 drivers/block/zram/zcomp_lzo.c | 47 
 drivers/block/zram/zcomp_lzo.h | 17 -
 drivers/block/zram/zram_drv.c  |  6 ++---
 9 files changed, 44 insertions(+), 186 deletions(-)
 delete mode 100644 drivers/block/zram/zcomp_lz4.c
 delete mode 100644 drivers/block/zram/zcomp_lz4.h
 delete mode 100644 drivers/block/zram/zcomp_lzo.c
 delete mode 100644 drivers/block/zram/zcomp_lzo.h

diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig
index 386ba3d..7619bed 100644
--- a/drivers/block/zram/Kconfig
+++ b/drivers/block/zram/Kconfig
@@ -1,8 +1,7 @@
 config ZRAM
tristate "Compressed RAM block device support"
depends on BLOCK && SYSFS && ZSMALLOC
-   select LZO_COMPRESS
-   select LZO_DECOMPRESS
+   select CRYPTO_LZO
default n
help
  Creates virtual block devices called /dev/zramX (X = 0, 1, ...).
@@ -18,9 +17,8 @@ config ZRAM
 config ZRAM_LZ4_COMPRESS
bool "Enable LZ4 algorithm support"
depends on ZRAM
-   select LZ4_COMPRESS
-   select LZ4_DECOMPRESS
+   select CRYPTO_LZ4
default n
help
  This option enables LZ4 compression algorithm support. Compression
- algorithm can be changed using `comp_algorithm' device attribute.
\ No newline at end of file
+ algorithm can be changed using `comp_algorithm' device attribute.
diff --git a/drivers/block/zram/Makefile b/drivers/block/zram/Makefile
index be0763f..9e2b79e 100644
--- a/drivers/block/zram/Makefile
+++ b/drivers/block/zram/Makefile
@@ -1,5 +1,3 @@
-zram-y :=  zcomp_lzo.o zcomp.o zram_drv.o
-
-zram-$(CONFIG_ZRAM_LZ4_COMPRESS) += zcomp_lz4.o
+zram-y :=  zcomp.o zram_drv.o
 
 obj-$(CONFIG_ZRAM) +=  zram.o
diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 3456d5a..c2ed7c8 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -15,10 +15,6 @@
 #include 
 
 #include "zcomp.h"
-#include "zcomp_lzo.h"
-#ifdef CONFIG_ZRAM_LZ4_COMPRESS
-#include "zcomp_lz4.h"
-#endif
 
 /*
  * single zcomp_strm backend
@@ -43,19 +39,20 @@ struct zcomp_strm_multi {
wait_queue_head_t strm_wait;
 };
 
-static struct zcomp_backend *backends[] = {
-   _lzo,
+static const char * const backends[] = {
+   "lzo",
 #ifdef CONFIG_ZRAM_LZ4_COMPRESS
-   _lz4,
+   "lz4",
 #endif
NULL
 };
 
-static struct zcomp_backend *find_backend(const char *compress)
+static const char *find_backend(const char *compress)
 {
int i = 0;
while (backends[i]) {
-   if (sysfs_streq(compress, backends[i]->name))
+ 

[PATCH v3 7/9] zram: pass zstrm down to decompression path

[Joonsoo Kim]

From: Sergey Senozhatsky <sergey.senozhat...@gmail.com>

Introduce zcomp_decompress_begin()/zcomp_decompress_end() as a
preparation for crypto API-powered zcomp.

Change zcomp_decompress() signature to require zstrm parameter.

Unlike zcomp_compress_begin(), zcomp_decompress_begin() may return
zstrm if currently selected compression backend require zstrm for
decompression or NULL if it does not.

Signed-off-by: Sergey Senozhatsky <sergey.senozhat...@gmail.com>
Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 drivers/block/zram/zcomp.c|  3 ++-
 drivers/block/zram/zcomp.h|  3 ++-
 drivers/block/zram/zram_drv.c | 20 
 3 files changed, 20 insertions(+), 6 deletions(-)

diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index fc13bf2..3456d5a 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -336,7 +336,8 @@ int zcomp_compress(struct zcomp *comp, struct zcomp_strm 
*zstrm,
zstrm->private);
 }
 
-int zcomp_decompress(struct zcomp *comp, const unsigned char *src,
+int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm,
+   const unsigned char *src,
size_t src_len, unsigned char *dst)
 {
return comp->backend->decompress(src, src_len, dst);
diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
index 616e013..4c09c01 100644
--- a/drivers/block/zram/zcomp.h
+++ b/drivers/block/zram/zcomp.h
@@ -66,7 +66,8 @@ int zcomp_compress(struct zcomp *comp, struct zcomp_strm 
*zstrm,
 struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp);
 void zcomp_decompress_end(struct zcomp *comp, struct zcomp_strm *zstrm);
 
-int zcomp_decompress(struct zcomp *comp, const unsigned char *src,
+int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm,
+   const unsigned char *src,
size_t src_len, unsigned char *dst);
 
 bool zcomp_set_max_streams(struct zcomp *comp, int num_strm);
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 20c41ed..55e09db1 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -560,7 +560,8 @@ static void zram_free_page(struct zram *zram, size_t index)
zram_set_obj_size(meta, index, 0);
 }
 
-static int zram_decompress_page(struct zram *zram, char *mem, u32 index)
+static int zram_decompress_page(struct zram *zram, struct zcomp_strm *zstrm,
+   char *mem, u32 index)
 {
int ret = 0;
unsigned char *cmem;
@@ -582,7 +583,7 @@ static int zram_decompress_page(struct zram *zram, char 
*mem, u32 index)
if (size == PAGE_SIZE)
copy_page(mem, cmem);
else
-   ret = zcomp_decompress(zram->comp, cmem, size, mem);
+   ret = zcomp_decompress(zram->comp, zstrm, cmem, size, mem);
zs_unmap_object(meta->mem_pool, handle);
bit_spin_unlock(ZRAM_ACCESS, >table[index].value);
 
@@ -602,6 +603,7 @@ static int zram_bvec_read(struct zram *zram, struct bio_vec 
*bvec,
struct page *page;
unsigned char *user_mem, *uncmem = NULL;
struct zram_meta *meta = zram->meta;
+   struct zcomp_strm *zstrm = NULL;
page = bvec->bv_page;
 
bit_spin_lock(ZRAM_ACCESS, >table[index].value);
@@ -617,6 +619,7 @@ static int zram_bvec_read(struct zram *zram, struct bio_vec 
*bvec,
/* Use  a temporary buffer to decompress the page */
uncmem = kmalloc(PAGE_SIZE, GFP_NOIO);
 
+   zstrm = zcomp_decompress_begin(zram->comp);
user_mem = kmap_atomic(page);
if (!is_partial_io(bvec))
uncmem = user_mem;
@@ -627,7 +630,7 @@ static int zram_bvec_read(struct zram *zram, struct bio_vec 
*bvec,
goto out_cleanup;
}
 
-   ret = zram_decompress_page(zram, uncmem, index);
+   ret = zram_decompress_page(zram, zstrm, uncmem, index);
/* Should NEVER happen. Return bio error if it does. */
if (unlikely(ret))
goto out_cleanup;
@@ -636,10 +639,14 @@ static int zram_bvec_read(struct zram *zram, struct 
bio_vec *bvec,
memcpy(user_mem + bvec->bv_offset, uncmem + offset,
bvec->bv_len);
 
+   zcomp_decompress_end(zram->comp, zstrm);
+   zstrm = NULL;
+
flush_dcache_page(page);
ret = 0;
 out_cleanup:
kunmap_atomic(user_mem);
+   zcomp_decompress_end(zram->comp, zstrm);
if (is_partial_io(bvec))
kfree(uncmem);
return ret;
@@ -659,6 +666,7 @@ static int zram_bvec_write(struct zram *zram, struct 
bio_vec *bvec, u32 index,
 
page = bvec->bv_page;
if (is_partial_io(bvec)) {
+   struct zcomp_strm *zstrm;
/*
 * This is a partial IO. We need to read the full page
 * before to write the changes.
@@ -668,7 +676,1

[PATCH v3 4/9] crypto/lz4hc: support decompress_noctx

[Joonsoo Kim]

lz4hc's decompression doesn't requires any scratch buffer so
it doesn't need tfm context. Hence, it can support
crypto compression noctx API and this patch implements it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/lz4hc.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/crypto/lz4hc.c b/crypto/lz4hc.c
index 0cb38a7..0797065 100644
--- a/crypto/lz4hc.c
+++ b/crypto/lz4hc.c
@@ -76,6 +76,12 @@ static int lz4hc_decompress_crypto(struct crypto_tfm *tfm, 
const u8 *src,
return err;
 }
 
+static int lz4hc_decompress_noctx(const u8 *src, unsigned int slen,
+ u8 *dst, unsigned int *dlen)
+{
+   return lz4hc_decompress_crypto(NULL, src, slen, dst, dlen);
+}
+
 static struct crypto_alg alg_lz4hc = {
.cra_name   = "lz4hc",
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -87,7 +93,7 @@ static struct crypto_alg alg_lz4hc = {
.cra_u  = { .compress = {
.coa_compress   = lz4hc_compress_crypto,
.coa_decompress = lz4hc_decompress_crypto,
-   .coa_decompress_noctx   = NULL } }
+   .coa_decompress_noctx   = lz4hc_decompress_noctx } }
 };
 
 static int __init lz4hc_mod_init(void)
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 9/9] zram: use crypto decompress_noctx API

[Joonsoo Kim]

Crypto subsystem now supports decompress_noctx API that requires
special tfm_noctx. This tfm can be shared by multiple concurrent
decompress user because this API doesn't rely on this tfm object
except to fetch decompress function pointer.

Until changing to use crypto API, zram doesn't require any zstrm
on decompress so decompress is parallelized unlimitedly. But, previous
patch make zram to use crypto API and this requires one zstrm on every
decompress users so, in some zstrm contended situations, zram's
performance would be degraded.

This patch makes zram use decompress_noctx API and restore zram's
performance as the time that zram doesn't use crypto API.

Following is zram's read performance number.

* iozone -t 4 -R -r 16K -s 60M -I +Z -i 0 -i 1
* max_stream is set to 1
* Output is in Kbytes/sec

zram-base vs zram-crypto vs zram-crypto-noctx

Read10411701.88 6426911.62  9423894.38
Re-read 10017386.62 6428218.88  1163.50

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 drivers/block/zram/zcomp.c | 24 +++-
 drivers/block/zram/zcomp.h |  1 +
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index c2ed7c8..a020200 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -319,9 +319,12 @@ void zcomp_compress_end(struct zcomp *comp, struct 
zcomp_strm *zstrm)
zcomp_strm_release(comp, zstrm);
 }
 
-/* Never return NULL, may sleep */
+/* May return NULL, may sleep */
 struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp)
 {
+   if (comp->tfm_noctx)
+   return NULL;
+
return zcomp_strm_find(comp);
 }
 
@@ -346,11 +349,18 @@ int zcomp_decompress(struct zcomp *comp, struct 
zcomp_strm *zstrm,
 {
unsigned int size = PAGE_SIZE;
 
+   if (!zstrm) {
+   return crypto_comp_decompress_noctx(comp->tfm_noctx,
+   src, src_len, dst, );
+   }
+
return crypto_comp_decompress(zstrm->tfm, src, src_len, dst, );
 }
 
 void zcomp_destroy(struct zcomp *comp)
 {
+   if (comp->tfm_noctx)
+   crypto_free_comp_noctx(comp->tfm_noctx);
comp->destroy(comp);
kfree(comp);
 }
@@ -366,6 +376,7 @@ struct zcomp *zcomp_create(const char *compress, int 
max_strm)
 {
struct zcomp *comp;
const char *backend;
+   struct crypto_comp *tfm;
 
backend = find_backend(compress);
if (!backend)
@@ -384,5 +395,16 @@ struct zcomp *zcomp_create(const char *compress, int 
max_strm)
kfree(comp);
return ERR_PTR(-ENOMEM);
}
+
+   /*
+* Prepare to use crypto decompress_noctx API. One tfm is required
+* to initialize crypto algorithm properly and fetch corresponding
+* function pointer. But, it is sharable for multiple concurrent
+* decompress users.
+*/
+   tfm = crypto_alloc_comp_noctx(compress, 0, 0);
+   if (!IS_ERR(tfm))
+   comp->tfm_noctx = tfm;
+
return comp;
 }
diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
index 4f9df8e..c76d8e4 100644
--- a/drivers/block/zram/zcomp.h
+++ b/drivers/block/zram/zcomp.h
@@ -26,6 +26,7 @@ struct zcomp_strm {
 /* dynamic per-device compression frontend */
 struct zcomp {
void *stream;
+   struct crypto_comp *tfm_noctx;
const char *backend;
 
struct zcomp_strm *(*strm_find)(struct zcomp *comp);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 6/9] zram: make stream find and release functions static

[Joonsoo Kim]

From: Sergey Senozhatsky <sergey.senozhat...@gmail.com>

Hide (make static) zstrm find and release function and introduce
zcomp_compress_begin()/zcomp_compress_end(). We will have begin
and end functions around compression (this patch) and decompression
(next patch). So the work flow is evolving to:

zstrm = foo_begin();
foo(zstrm);
foo_end(zstrm);

where foo is compress or decompress zcomp functions.

This patch is a preparation to make crypto API-powered zcomp
possible. The reasoning is that some crypto compression backends
require zstrm for decompression.

Signed-off-by: Sergey Senozhatsky <sergey.senozhat...@gmail.com>
Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 drivers/block/zram/zcomp.c| 27 +--
 drivers/block/zram/zcomp.h|  8 ++--
 drivers/block/zram/zram_drv.c |  6 +++---
 3 files changed, 34 insertions(+), 7 deletions(-)

diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 965d1af..fc13bf2 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -296,16 +296,39 @@ bool zcomp_set_max_streams(struct zcomp *comp, int 
num_strm)
return comp->set_max_streams(comp, num_strm);
 }
 
-struct zcomp_strm *zcomp_strm_find(struct zcomp *comp)
+static struct zcomp_strm *zcomp_strm_find(struct zcomp *comp)
 {
return comp->strm_find(comp);
 }
 
-void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm)
+static void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm)
 {
comp->strm_release(comp, zstrm);
 }
 
+/* Never return NULL, may sleep */
+struct zcomp_strm *zcomp_compress_begin(struct zcomp *comp)
+{
+   return zcomp_strm_find(comp);
+}
+
+void zcomp_compress_end(struct zcomp *comp, struct zcomp_strm *zstrm)
+{
+   zcomp_strm_release(comp, zstrm);
+}
+
+/* May return NULL, may sleep */
+struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp)
+{
+   return NULL;
+}
+
+void zcomp_decompress_end(struct zcomp *comp, struct zcomp_strm *zstrm)
+{
+   if (zstrm)
+   zcomp_strm_release(comp, zstrm);
+}
+
 int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
const unsigned char *src, size_t *dst_len)
 {
diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
index 46e2b9f..616e013 100644
--- a/drivers/block/zram/zcomp.h
+++ b/drivers/block/zram/zcomp.h
@@ -56,12 +56,16 @@ bool zcomp_available_algorithm(const char *comp);
 struct zcomp *zcomp_create(const char *comp, int max_strm);
 void zcomp_destroy(struct zcomp *comp);
 
-struct zcomp_strm *zcomp_strm_find(struct zcomp *comp);
-void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm);
+
+struct zcomp_strm *zcomp_compress_begin(struct zcomp *comp);
+void zcomp_compress_end(struct zcomp *comp, struct zcomp_strm *zstrm);
 
 int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
const unsigned char *src, size_t *dst_len);
 
+struct zcomp_strm *zcomp_decompress_begin(struct zcomp *comp);
+void zcomp_decompress_end(struct zcomp *comp, struct zcomp_strm *zstrm);
+
 int zcomp_decompress(struct zcomp *comp, const unsigned char *src,
size_t src_len, unsigned char *dst);
 
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 9fa15bb..20c41ed 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -673,7 +673,7 @@ static int zram_bvec_write(struct zram *zram, struct 
bio_vec *bvec, u32 index,
goto out;
}
 
-   zstrm = zcomp_strm_find(zram->comp);
+   zstrm = zcomp_compress_begin(zram->comp);
user_mem = kmap_atomic(page);
 
if (is_partial_io(bvec)) {
@@ -744,7 +744,7 @@ static int zram_bvec_write(struct zram *zram, struct 
bio_vec *bvec, u32 index,
memcpy(cmem, src, clen);
}
 
-   zcomp_strm_release(zram->comp, zstrm);
+   zcomp_compress_end(zram->comp, zstrm);
zstrm = NULL;
zs_unmap_object(meta->mem_pool, handle);
 
@@ -764,7 +764,7 @@ static int zram_bvec_write(struct zram *zram, struct 
bio_vec *bvec, u32 index,
atomic64_inc(>stats.pages_stored);
 out:
if (zstrm)
-   zcomp_strm_release(zram->comp, zstrm);
+   zcomp_compress_end(zram->comp, zstrm);
if (is_partial_io(bvec))
kfree(uncmem);
return ret;
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 5/9] crypto/842: support decompress_noctx

[Joonsoo Kim]

842's decompression doesn't requires any scratch buffer so
it doesn't need tfm context. Hence, it can support
crypto compression noctx API and this patch implements it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/842.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/crypto/842.c b/crypto/842.c
index 1b6cdab..a933f71 100644
--- a/crypto/842.c
+++ b/crypto/842.c
@@ -52,6 +52,12 @@ static int crypto842_decompress(struct crypto_tfm *tfm,
return sw842_decompress(src, slen, dst, dlen);
 }
 
+static int crypto842_decompress_noctx(const u8 *src, unsigned int slen,
+ u8 *dst, unsigned int *dlen)
+{
+   return sw842_decompress(src, slen, dst, dlen);
+}
+
 static struct crypto_alg alg = {
.cra_name   = "842",
.cra_driver_name= "842-generic",
@@ -62,7 +68,7 @@ static struct crypto_alg alg = {
.cra_u  = { .compress = {
.coa_compress   = crypto842_compress,
.coa_decompress = crypto842_decompress,
-   .coa_decompress_noctx   = NULL } }
+   .coa_decompress_noctx   = crypto842_decompress_noctx } }
 };
 
 static int __init crypto842_mod_init(void)
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 3/9] crypyo/lz4: support decompress_noctx

[Joonsoo Kim]

lz4's decompression doesn't requires any scratch buffer so
it doesn't need tfm context. Hence, it can support
crypto compression noctx API and this patch implements it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/lz4.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/crypto/lz4.c b/crypto/lz4.c
index d38ce2a..589fa251 100644
--- a/crypto/lz4.c
+++ b/crypto/lz4.c
@@ -76,6 +76,12 @@ static int lz4_decompress_crypto(struct crypto_tfm *tfm, 
const u8 *src,
return err;
 }
 
+static int lz4_decompress_noctx(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen)
+{
+   return lz4_decompress_crypto(NULL, src, slen, dst, dlen);
+}
+
 static struct crypto_alg alg_lz4 = {
.cra_name   = "lz4",
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -87,7 +93,7 @@ static struct crypto_alg alg_lz4 = {
.cra_u  = { .compress = {
.coa_compress   = lz4_compress_crypto,
.coa_decompress = lz4_decompress_crypto,
-   .coa_decompress_noctx   = NULL } }
+   .coa_decompress_noctx   = lz4_decompress_noctx } }
 };
 
 static int __init lz4_mod_init(void)
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 2/9] crypto/lzo: support decompress_noctx

[Joonsoo Kim]

lzo's decompression doesn't requires any scratch buffer so
it doesn't need tfm context. Hence, it can support
crypto compression noctx API and this patch implements it.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/lzo.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/crypto/lzo.c b/crypto/lzo.c
index ec0f7b3..3cc0ce7 100644
--- a/crypto/lzo.c
+++ b/crypto/lzo.c
@@ -80,6 +80,12 @@ static int lzo_decompress(struct crypto_tfm *tfm, const u8 
*src,
 
 }
 
+static int lzo_decompress_noctx(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen)
+{
+   return lzo_decompress(NULL, src, slen, dst, dlen);
+}
+
 static struct crypto_alg alg = {
.cra_name   = "lzo",
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -90,7 +96,7 @@ static struct crypto_alg alg = {
.cra_u  = { .compress = {
.coa_compress   = lzo_compress,
.coa_decompress = lzo_decompress,
-   .coa_decompress_noctx   = NULL } }
+   .coa_decompress_noctx   = lzo_decompress_noctx } }
 };
 
 static int __init lzo_mod_init(void)
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 1/9] crypto: introduce decompression API that can be called via sharable tfm object

[Joonsoo Kim]

Until now, tfm object embeds (de)compression context in it and
(de)compression in crypto API requires tfm object to use
this context. But, there are some algorithms that doesn't need
such context to operate. Therefore, this patch introduce new crypto
decompression API that calls decompression function via sharable tfm
object. Concurrent calls to decompress_noctx function through sharable
tfm object will be okay because caller don't need any context in tfm and
tfm is only used for fetching function pointer to decompress_noctx
function. This can reduce overhead of maintaining multiple tfm
if decompression doesn't require any context to operate.

Signed-off-by: Joonsoo Kim <iamjoonsoo@lge.com>
---
 crypto/842.c   |  3 ++-
 crypto/compress.c  | 36 
 crypto/crypto_null.c   |  3 ++-
 crypto/deflate.c   |  3 ++-
 crypto/lz4.c   |  3 ++-
 crypto/lz4hc.c |  3 ++-
 crypto/lzo.c   |  3 ++-
 include/linux/crypto.h | 20 
 8 files changed, 68 insertions(+), 6 deletions(-)

diff --git a/crypto/842.c b/crypto/842.c
index 98e387e..1b6cdab 100644
--- a/crypto/842.c
+++ b/crypto/842.c
@@ -61,7 +61,8 @@ static struct crypto_alg alg = {
.cra_module = THIS_MODULE,
.cra_u  = { .compress = {
.coa_compress   = crypto842_compress,
-   .coa_decompress = crypto842_decompress } }
+   .coa_decompress = crypto842_decompress,
+   .coa_decompress_noctx   = NULL } }
 };
 
 static int __init crypto842_mod_init(void)
diff --git a/crypto/compress.c b/crypto/compress.c
index c33f076..abb36a8 100644
--- a/crypto/compress.c
+++ b/crypto/compress.c
@@ -33,12 +33,21 @@ static int crypto_decompress(struct crypto_tfm *tfm,
   dlen);
 }
 
+static int crypto_decompress_noctx(struct crypto_tfm *tfm,
+   const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen)
+{
+   return tfm->__crt_alg->cra_compress.coa_decompress_noctx(src, slen,
+   dst, dlen);
+}
+
 int crypto_init_compress_ops(struct crypto_tfm *tfm)
 {
struct compress_tfm *ops = >crt_compress;
 
ops->cot_compress = crypto_compress;
ops->cot_decompress = crypto_decompress;
+   ops->cot_decompress_noctx = NULL;
 
return 0;
 }
@@ -46,3 +55,30 @@ int crypto_init_compress_ops(struct crypto_tfm *tfm)
 void crypto_exit_compress_ops(struct crypto_tfm *tfm)
 {
 }
+
+struct crypto_comp *crypto_alloc_comp_noctx(const char *alg_name,
+   u32 type, u32 mask)
+{
+   struct crypto_comp *comp;
+   struct crypto_tfm *tfm;
+   struct compress_tfm *ops;
+
+   comp = crypto_alloc_comp(alg_name, type, mask);
+   if (IS_ERR(comp))
+   return comp;
+
+   tfm = crypto_comp_tfm(comp);
+   if (!tfm->__crt_alg->cra_compress.coa_decompress_noctx) {
+   crypto_free_comp(comp);
+   return ERR_PTR(-EINVAL);
+   }
+
+   ops = >crt_compress;
+
+   /* Only allow noctx ops to comp_noctx */
+   ops->cot_compress = NULL;
+   ops->cot_decompress = NULL;
+   ops->cot_decompress_noctx = crypto_decompress_noctx;
+
+   return comp;
+}
diff --git a/crypto/crypto_null.c b/crypto/crypto_null.c
index 941c9a4..3560b75 100644
--- a/crypto/crypto_null.c
+++ b/crypto/crypto_null.c
@@ -146,7 +146,8 @@ static struct crypto_alg null_algs[3] = { {
.cra_module =   THIS_MODULE,
.cra_u  =   { .compress = {
.coa_compress   =   null_compress,
-   .coa_decompress =   null_compress } }
+   .coa_decompress =   null_compress,
+   .coa_decompress_noctx   =   NULL } }
 } };
 
 MODULE_ALIAS_CRYPTO("compress_null");
diff --git a/crypto/deflate.c b/crypto/deflate.c
index 95d8d37..c0b0a40 100644
--- a/crypto/deflate.c
+++ b/crypto/deflate.c
@@ -203,7 +203,8 @@ static struct crypto_alg alg = {
.cra_exit   = deflate_exit,
.cra_u  = { .compress = {
.coa_compress   = deflate_compress,
-   .coa_decompress = deflate_decompress } }
+   .coa_decompress = deflate_decompress,
+   .coa_decompress_noctx   = NULL } }
 };
 
 static int __init deflate_mod_init(void)
diff --git a/crypto/lz4.c b/crypto/lz4.c
index aefbcea..d38ce2a 100644
--- a/crypto/lz4.c
+++ b/crypto/lz4.c
@@ -86,7 +86,8 @@ static struct crypto_alg alg_lz4 = {
.cra_exit   = lz4_exit,
.cra_u  = { .compress = {
.coa_compress   = lz4_compress_crypto,
-   .coa_decompress = lz4_decompress_crypto } }
+   .coa_decompress = lz4_decompress_crypto,
+   

[PATCH v2 8/8] zram: use decompress_noctx

[Joonsoo Kim]

Crypto subsystem supports noctx API which doesn't require tfm.
To get tfm in zram, we need zstrm and it is limited resource.
If we uses noctx API, we don't need to get zstrm so that
we get much better performance when zstrm is insufficient.

This patch restores zram's performance to the time that zram
doesn't use crypto subsystem.

Following is zram's read performance number.

* iozone -t 4 -R -r 16K -s 60M -I +Z -i 0 -i 1
* max_stream is set to 1
* Output is in Kbytes/sec

zram-base vs zram-crypto vs zram-crypto-noctx

Read10411701.88 6426911.62  9423894.38
Re-read 10017386.62 6428218.88  1163.50

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 drivers/block/zram/zcomp.c| 28 +++-
 drivers/block/zram/zcomp.h|  9 -
 drivers/block/zram/zram_drv.c |  9 +
 3 files changed, 40 insertions(+), 6 deletions(-)

diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index d2734f2..86b0c9b 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -291,8 +291,12 @@ bool zcomp_set_max_streams(struct zcomp *comp, int 
num_strm)
return comp-set_max_streams(comp, num_strm);
 }
 
-struct zcomp_strm *zcomp_strm_find(struct zcomp *comp)
+struct zcomp_strm *zcomp_strm_find(struct zcomp *comp, bool decomp)
 {
+   /* We don't need decompression context so zstrm neither */
+   if (decomp  test_bit(ZCOMP_DECOMPRESS_NOCTX, comp-flags))
+   return NULL;
+
return comp-strm_find(comp);
 }
 
@@ -307,6 +311,11 @@ int zcomp_compress(struct zcomp *comp, struct zcomp_strm 
*zstrm,
 {
*dst_len = PAGE_SIZE  1;
 
+   if (!zstrm) {
+   return crypto_comp_compress_noctx(comp-alg, src, PAGE_SIZE,
+   dst, dst_len);
+   }
+
return crypto_comp_compress(zstrm-tfm, src, PAGE_SIZE, dst, dst_len);
 }
 
@@ -316,12 +325,18 @@ int zcomp_decompress(struct zcomp *comp, struct 
zcomp_strm *zstrm,
 {
unsigned int size = PAGE_SIZE;
 
+   if (!zstrm) {
+   return crypto_comp_decompress_noctx(comp-alg, src, src_len,
+   dst, size);
+   }
+
return crypto_comp_decompress(zstrm-tfm, src, src_len, dst, size);
 }
 
 void zcomp_destroy(struct zcomp *comp)
 {
comp-destroy(comp);
+   crypto_put_comp(comp-alg);
kfree(comp);
 }
 
@@ -344,12 +359,23 @@ struct zcomp *zcomp_create(const char *compress, int 
max_strm)
return ERR_PTR(-ENOMEM);
 
comp-name = compress;
+   comp-alg = crypto_get_comp(compress, 0, 0);
+   if (!comp-alg) {
+   kfree(comp);
+   return ERR_PTR(-EINVAL);
+   }
+
+   if (crypto_has_compress_noctx(comp-alg))
+   set_bit(ZCOMP_COMPRESS_NOCTX, comp-flags);
+   if (crypto_has_decompress_noctx(comp-alg))
+   set_bit(ZCOMP_DECOMPRESS_NOCTX, comp-flags);
 
if (max_strm  1)
zcomp_strm_multi_create(comp, max_strm);
else
zcomp_strm_single_create(comp);
if (!comp-stream) {
+   crypto_put_comp(comp-alg);
kfree(comp);
return ERR_PTR(-ENOMEM);
}
diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
index c47db4d..6c137c8 100644
--- a/drivers/block/zram/zcomp.h
+++ b/drivers/block/zram/zcomp.h
@@ -13,6 +13,11 @@
 #include linux/crypto.h
 #include linux/mutex.h
 
+enum {
+   ZCOMP_COMPRESS_NOCTX,
+   ZCOMP_DECOMPRESS_NOCTX,
+};
+
 struct zcomp_strm {
struct crypto_comp *tfm;
 
@@ -27,6 +32,8 @@ struct zcomp_strm {
 struct zcomp {
void *stream;
const char *name;
+   struct crypto_alg *alg;
+   unsigned long flags;
 
struct zcomp_strm *(*strm_find)(struct zcomp *comp);
void (*strm_release)(struct zcomp *comp, struct zcomp_strm *zstrm);
@@ -39,7 +46,7 @@ ssize_t zcomp_available_show(const char *comp, char *buf);
 struct zcomp *zcomp_create(const char *comp, int max_strm);
 void zcomp_destroy(struct zcomp *comp);
 
-struct zcomp_strm *zcomp_strm_find(struct zcomp *comp);
+struct zcomp_strm *zcomp_strm_find(struct zcomp *comp, bool decomp);
 void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm);
 
 int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index b3044d3..8283bd3 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -623,7 +623,7 @@ static int zram_bvec_read(struct zram *zram, struct bio_vec 
*bvec,
/* Use  a temporary buffer to decompress the page */
uncmem = kmalloc(PAGE_SIZE, GFP_NOIO);
 
-   zstrm = zcomp_strm_find(zram-comp);
+   zstrm = zcomp_strm_find(zram-comp, true);
user_mem = kmap_atomic(page);
if (!is_partial_io(bvec))
uncmem

[PATCH v2 7/8] zram: use crypto API for compression

[Joonsoo Kim]

Until now, zram uses compression algorithm through direct call
to core algorithm function, but, it has drawback that we need to add
compression algorithm manually to zram if needed. Without this work,
we cannot utilize various compression algorithms in the system.
Moreover, to add new compression algorithm, we need to know how to use it
and this is somewhat time-consuming.

When I tested new algorithms such as zlib, these problems make me hard
to test them. To prevent these problem in the future, I think that
using crypto API for compression is better approach and this patch
implements it.

The reason we need to support vairous compression algorithms is that
zram's performance is highly depend on workload and compression algorithm
and architecture. Every compression algorithm has it's own strong point.
For example, zlib is the best for compression ratio, but, it's
(de)compression speed is rather slow. Against my expectation, in my kernel
build test with zram swap, in low-memory condition on x86, zlib shows best
performance than others. In this case, I guess that compression ratio is
the most important factor. Unlike this situation, on ARM, maybe fast
(de)compression speed is the most important because it's computation speed
is slower than x86.

We can't expect what algorithm is the best fit for one's system, because
it needs too complex calculation. We need to test it in case by case and
easy to use new compression algorithm by this patch will help
that situation.

There is one problem that crypto API requires tfm object to (de)compress
something and zram abstract it on zstrm which is very limited resource.
If number of zstrm is set to low and is contended, zram's performace could
be down-graded due to this change. But, following patch support to use
crypto noctx API and would restore the performance as is.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 drivers/block/zram/Kconfig | 13 +--
 drivers/block/zram/Makefile|  4 +-
 drivers/block/zram/zcomp.c | 84 +-
 drivers/block/zram/zcomp.h | 35 +-
 drivers/block/zram/zcomp_lz4.c | 47 ---
 drivers/block/zram/zcomp_lz4.h | 17 -
 drivers/block/zram/zcomp_lzo.c | 47 ---
 drivers/block/zram/zcomp_lzo.h | 17 -
 drivers/block/zram/zram_drv.c  | 26 -
 9 files changed, 71 insertions(+), 219 deletions(-)
 delete mode 100644 drivers/block/zram/zcomp_lz4.c
 delete mode 100644 drivers/block/zram/zcomp_lz4.h
 delete mode 100644 drivers/block/zram/zcomp_lzo.c
 delete mode 100644 drivers/block/zram/zcomp_lzo.h

diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig
index 386ba3d..36ec96f 100644
--- a/drivers/block/zram/Kconfig
+++ b/drivers/block/zram/Kconfig
@@ -1,8 +1,7 @@
 config ZRAM
tristate Compressed RAM block device support
depends on BLOCK  SYSFS  ZSMALLOC
-   select LZO_COMPRESS
-   select LZO_DECOMPRESS
+   select CRYPTO_LZO
default n
help
  Creates virtual block devices called /dev/zramX (X = 0, 1, ...).
@@ -14,13 +13,3 @@ config ZRAM
  disks and maybe many more.
 
  See zram.txt for more information.
-
-config ZRAM_LZ4_COMPRESS
-   bool Enable LZ4 algorithm support
-   depends on ZRAM
-   select LZ4_COMPRESS
-   select LZ4_DECOMPRESS
-   default n
-   help
- This option enables LZ4 compression algorithm support. Compression
- algorithm can be changed using `comp_algorithm' device attribute.
\ No newline at end of file
diff --git a/drivers/block/zram/Makefile b/drivers/block/zram/Makefile
index be0763f..9e2b79e 100644
--- a/drivers/block/zram/Makefile
+++ b/drivers/block/zram/Makefile
@@ -1,5 +1,3 @@
-zram-y :=  zcomp_lzo.o zcomp.o zram_drv.o
-
-zram-$(CONFIG_ZRAM_LZ4_COMPRESS) += zcomp_lz4.o
+zram-y :=  zcomp.o zram_drv.o
 
 obj-$(CONFIG_ZRAM) +=  zram.o
diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 2ad504b..d2734f2 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -13,12 +13,9 @@
 #include linux/slab.h
 #include linux/wait.h
 #include linux/sched.h
+#include linux/crypto.h
 
 #include zcomp.h
-#include zcomp_lzo.h
-#ifdef CONFIG_ZRAM_LZ4_COMPRESS
-#include zcomp_lz4.h
-#endif
 
 /*
  * single zcomp_strm backend
@@ -43,36 +40,34 @@ struct zcomp_strm_multi {
wait_queue_head_t strm_wait;
 };
 
-static struct zcomp_backend *backends[] = {
-   zcomp_lzo,
-#ifdef CONFIG_ZRAM_LZ4_COMPRESS
-   zcomp_lz4,
+static const char * const backends[] = {
+   lzo,
+#ifdef CONFIG_CRYPTO_LZ4
+   lz4,
+#endif
+#ifdef CONFIG_CRYPTO_LZ4HC
+   lz4hc,
+#endif
+#ifdef CONFIG_CRYPTO_DEFLATE
+   deflate,
+#endif
+#ifdef CONFIG_CRYPTO_842
+   842,
 #endif
NULL
 };
 
-static struct zcomp_backend *find_backend(const char *compress)
-{
-   int i = 0;
-   while (backends[i

[PATCH v2 6/8] zram: change zcomp_compress interface

[Joonsoo Kim]

zram regards zstrm's buffer as compression destination buffer, but,
it is not intuitive and there is no document about it. Providing
destination buffer to zcomp_compress() directly seems more intuitive
interface to me so this patch changes zcomp_compress interface.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 drivers/block/zram/zcomp.c| 5 ++---
 drivers/block/zram/zcomp.h| 2 +-
 drivers/block/zram/zram_drv.c | 2 +-
 3 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 965d1af..2ad504b 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -307,10 +307,9 @@ void zcomp_strm_release(struct zcomp *comp, struct 
zcomp_strm *zstrm)
 }
 
 int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
-   const unsigned char *src, size_t *dst_len)
+   const unsigned char *src, unsigned char *dst, size_t *dst_len)
 {
-   return comp-backend-compress(src, zstrm-buffer, dst_len,
-   zstrm-private);
+   return comp-backend-compress(src, dst, dst_len, zstrm-private);
 }
 
 int zcomp_decompress(struct zcomp *comp, const unsigned char *src,
diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
index 46e2b9f..b2388e0 100644
--- a/drivers/block/zram/zcomp.h
+++ b/drivers/block/zram/zcomp.h
@@ -60,7 +60,7 @@ struct zcomp_strm *zcomp_strm_find(struct zcomp *comp);
 void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm);
 
 int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
-   const unsigned char *src, size_t *dst_len);
+   const unsigned char *src, unsigned char *dst, size_t *dst_len);
 
 int zcomp_decompress(struct zcomp *comp, const unsigned char *src,
size_t src_len, unsigned char *dst);
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index b088ca9..4801e4d 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -701,7 +701,7 @@ static int zram_bvec_write(struct zram *zram, struct 
bio_vec *bvec, u32 index,
goto out;
}
 
-   ret = zcomp_compress(zram-comp, zstrm, uncmem, clen);
+   ret = zcomp_compress(zram-comp, zstrm, uncmem, zstrm-buffer, clen);
if (!is_partial_io(bvec)) {
kunmap_atomic(user_mem);
user_mem = NULL;
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 2/8] crypto/lzo: support decompress_noctx

[Joonsoo Kim]

lzo's decompression doesn't requires any scratch buffer so
it doesn't need tfm context. Hence, it can support
crypto compression noctx API and this patch implements it.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 crypto/lzo.c | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/crypto/lzo.c b/crypto/lzo.c
index 9ca516b..f1844dd 100644
--- a/crypto/lzo.c
+++ b/crypto/lzo.c
@@ -80,6 +80,22 @@ static int lzo_decompress(struct crypto_tfm *tfm, const u8 
*src,
 
 }
 
+static int lzo_decompress_noctx(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen)
+{
+   int err;
+   size_t tmp_len = *dlen; /* size_t(ulong) - uint on 64 bit */
+
+   err = lzo1x_decompress_safe(src, slen, dst, tmp_len);
+
+   if (err != LZO_E_OK)
+   return -EINVAL;
+
+   *dlen = tmp_len;
+   return 0;
+
+}
+
 static struct crypto_alg alg = {
.cra_name   = lzo,
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -91,7 +107,7 @@ static struct crypto_alg alg = {
.coa_compress   = lzo_compress,
.coa_decompress = lzo_decompress,
.coa_compress_noctx = NULL,
-   .coa_decompress_noctx   = NULL } }
+   .coa_decompress_noctx   = lzo_decompress_noctx } }
 };
 
 static int __init lzo_mod_init(void)
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 5/8] crypto/842: support decompress_noctx

[Joonsoo Kim]

842's decompression doesn't requires any scratch buffer so
it doesn't need tfm context. Hence, it can support
crypto compression noctx API and this patch implements it.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 crypto/842.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/crypto/842.c b/crypto/842.c
index b6503ea..9acb595 100644
--- a/crypto/842.c
+++ b/crypto/842.c
@@ -52,6 +52,12 @@ static int crypto842_decompress(struct crypto_tfm *tfm,
return sw842_decompress(src, slen, dst, dlen);
 }
 
+static int crypto842_decompress_noctx(const u8 *src, unsigned int slen,
+ u8 *dst, unsigned int *dlen)
+{
+   return sw842_decompress(src, slen, dst, dlen);
+}
+
 static struct crypto_alg alg = {
.cra_name   = 842,
.cra_driver_name= 842-generic,
@@ -63,7 +69,7 @@ static struct crypto_alg alg = {
.coa_compress   = crypto842_compress,
.coa_decompress = crypto842_decompress,
.coa_compress_noctx = NULL,
-   .coa_decompress_noctx   = NULL } }
+   .coa_decompress_noctx   = crypto842_decompress_noctx } }
 };
 
 static int __init crypto842_mod_init(void)
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 3/8] crypyo/lz4: support decompress_noctx

[Joonsoo Kim]

lz4's decompression doesn't requires any scratch buffer so
it doesn't need tfm context. Hence, it can support
crypto compression noctx API and this patch implements it.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 crypto/lz4.c | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/crypto/lz4.c b/crypto/lz4.c
index 1848435..aa23da3 100644
--- a/crypto/lz4.c
+++ b/crypto/lz4.c
@@ -76,6 +76,21 @@ static int lz4_decompress_crypto(struct crypto_tfm *tfm, 
const u8 *src,
return err;
 }
 
+static int lz4_decompress_noctx(const u8 *src, unsigned int slen,
+   u8 *dst, unsigned int *dlen)
+{
+   int err;
+   size_t tmp_len = *dlen;
+   size_t __slen = slen;
+
+   err = lz4_decompress_unknownoutputsize(src, __slen, dst, tmp_len);
+   if (err  0)
+   return -EINVAL;
+
+   *dlen = tmp_len;
+   return err;
+}
+
 static struct crypto_alg alg_lz4 = {
.cra_name   = lz4,
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -88,7 +103,7 @@ static struct crypto_alg alg_lz4 = {
.coa_compress   = lz4_compress_crypto,
.coa_decompress = lz4_decompress_crypto,
.coa_compress_noctx = NULL,
-   .coa_decompress_noctx   = NULL } }
+   .coa_decompress_noctx   = lz4_decompress_noctx } }
 };
 
 static int __init lz4_mod_init(void)
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 4/8] crypto/lz4hc: support decompress_noctx

[Joonsoo Kim]

lz4hc's decompression doesn't requires any scratch buffer so
it doesn't need tfm context. Hence, it can support
crypto compression noctx API and this patch implements it.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 crypto/lz4hc.c | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/crypto/lz4hc.c b/crypto/lz4hc.c
index bcf0baa..a529620 100644
--- a/crypto/lz4hc.c
+++ b/crypto/lz4hc.c
@@ -76,6 +76,21 @@ static int lz4hc_decompress_crypto(struct crypto_tfm *tfm, 
const u8 *src,
return err;
 }
 
+static int lz4hc_decompress_noctx(const u8 *src, unsigned int slen,
+ u8 *dst, unsigned int *dlen)
+{
+   int err;
+   size_t tmp_len = *dlen;
+   size_t __slen = slen;
+
+   err = lz4_decompress_unknownoutputsize(src, __slen, dst, tmp_len);
+   if (err  0)
+   return -EINVAL;
+
+   *dlen = tmp_len;
+   return err;
+}
+
 static struct crypto_alg alg_lz4hc = {
.cra_name   = lz4hc,
.cra_flags  = CRYPTO_ALG_TYPE_COMPRESS,
@@ -88,7 +103,7 @@ static struct crypto_alg alg_lz4hc = {
.coa_compress   = lz4hc_compress_crypto,
.coa_decompress = lz4hc_decompress_crypto,
.coa_compress_noctx = NULL,
-   .coa_decompress_noctx   = NULL } }
+   .coa_decompress_noctx   = lz4hc_decompress_noctx } }
 };
 
 static int __init lz4hc_mod_init(void)
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 1/8] crypto: support (de)compression API that doesn't require tfm object

[Joonsoo Kim]

On Thu, Aug 20, 2015 at 03:50:35PM +0800, Herbert Xu wrote:
 On Thu, Aug 20, 2015 at 04:52:17PM +0900, Joonsoo Kim wrote:
  
  Hmm... I guess there is no problem. crypto_alg object fetched by
  crypto_get_comp() introduced in this patch could be hardware device
  algorithm which is same one that we can eventually fetch from tfm object.
  So, this approach would correctly track the crypto_alg regardless
  it is a hardware one or not. If there is some dependency between
  algorithm and tfm, it can't support _noctx API. Am I missing
  something?
 
 Your approach limits what hardware devices we can support in
 future.  It is fairly common for hardware drivers to only allocate
 resources when a tfm is created.  With your tfmless interface,
 the driver would have to unconditionally allocate resources.

Ah...Okay. thanks for clarifying.

 
 It is essentially a global tfm without the tfm.
 
  Yes, I thought this way before, but, current way is much simpler so
  I try it first. If it is not acceptable, I will implement this
  approach.
 
 Please go with a global tfm.

Okay. Will do that in next spin.

Thanks.

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 0/8] zram: introduce crypto compress noctx API and use it on zram

[Joonsoo Kim]

This patchset makes zram to use crypto API in order to support
more compression algorithm.

The reason we need to support vairous compression algorithms is that
zram's performance is highly depend on workload and compression algorithm
and architecture. Every compression algorithm has it's own strong point.
For example, zlib is the best for compression ratio, but, it's
(de)compression speed is rather slow. Against my expectation, in my kernel
build test with zram swap, in low-memory condition on x86, zlib shows best
performance than others. In this case, I guess that compression ratio is
the most important factor. Unlike this situation, on ARM, maybe fast
(de)compression speed is the most important because it's computation speed
is slower than x86.

Anyway, there is a concern from Sergey to use crypto API in zram. Current
crypto API has a limitation that always require tfm object to (de)compress
something because some of (de)compression function requires scratch buffer
embedded on tfm even if some of (de)compression function doesn't
require it. Due to above reason, using crypto API rather than calling
compression library directly causes more memory footprint and this is
why zram doesn't use crypto API before.

In this patchset, crypto compress noctx API is introduced to reduce memory
footprint caused by maintaining multiple tfm and zram uses it. Before
noctx API, zram's performace is down-graded if tfm is insufficient. But,
after applying noctx API, performace is restored.

This addresses Sergey's concern perfectly and provides possibility to use
various compression algorithm in zram.

Following is zram's read performance number.

* iozone -t 4 -R -r 16K -s 60M -I +Z -i 0 -i 1
* max_stream is set to 1
* Output is in Kbytes/sec

zram-base vs zram-crypto vs zram-crypto-noctx

Read10411701.88 6426911.62  9423894.38 
Re-read 10017386.62 6428218.88  1163.50 

Thanks.

Joonsoo Kim (8):
  crypto: support (de)compression API that doesn't require tfm object
  crypto/lzo: support decompress_noctx
  crypyo/lz4: support decompress_noctx
  crypto/lz4hc: support decompress_noctx
  crypto/842: support decompress_noctx
  zram: change zcomp_compress interface
  zram: use crypto API for compression
  zram: use decompress_noctx

 crypto/842.c   |  10 +++-
 crypto/compress.c  |  20 
 crypto/crypto_null.c   |   4 +-
 crypto/deflate.c   |   4 +-
 crypto/lz4.c   |  19 ++-
 crypto/lz4hc.c |  19 ++-
 crypto/lzo.c   |  20 +++-
 drivers/block/zram/Kconfig |  13 +
 drivers/block/zram/Makefile|   4 +-
 drivers/block/zram/zcomp.c | 113 +
 drivers/block/zram/zcomp.h |  44 +++-
 drivers/block/zram/zcomp_lz4.c |  47 -
 drivers/block/zram/zcomp_lz4.h |  17 ---
 drivers/block/zram/zcomp_lzo.c |  47 -
 drivers/block/zram/zcomp_lzo.h |  17 ---
 drivers/block/zram/zram_drv.c  |  29 +++
 include/linux/crypto.h |  31 +++
 17 files changed, 229 insertions(+), 229 deletions(-)
 delete mode 100644 drivers/block/zram/zcomp_lz4.c
 delete mode 100644 drivers/block/zram/zcomp_lz4.h
 delete mode 100644 drivers/block/zram/zcomp_lzo.c
 delete mode 100644 drivers/block/zram/zcomp_lzo.h

-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 1/8] crypto: support (de)compression API that doesn't require tfm object

[Joonsoo Kim]

Until now, tfm object embeds (de)compression context in it and
(de)compression in crypto API requires tfm object to use
this context. But, there are some algorithms that doesn't need
such context to operate. Therefore, this patch introduce new crypto
compression API that call (de)compression function via crypto_alg,
instead of tfm object. crypto_alg is required to get appropriate
(de)compression function pointer. This can reduce overhead of
maintaining multiple tfm if (de)compression doesn't require
any context to operate.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 crypto/842.c   |  4 +++-
 crypto/compress.c  | 20 
 crypto/crypto_null.c   |  4 +++-
 crypto/deflate.c   |  4 +++-
 crypto/lz4.c   |  4 +++-
 crypto/lz4hc.c |  4 +++-
 crypto/lzo.c   |  4 +++-
 include/linux/crypto.h | 31 +++
 8 files changed, 69 insertions(+), 6 deletions(-)

diff --git a/crypto/842.c b/crypto/842.c
index 98e387e..b6503ea 100644
--- a/crypto/842.c
+++ b/crypto/842.c
@@ -61,7 +61,9 @@ static struct crypto_alg alg = {
.cra_module = THIS_MODULE,
.cra_u  = { .compress = {
.coa_compress   = crypto842_compress,
-   .coa_decompress = crypto842_decompress } }
+   .coa_decompress = crypto842_decompress,
+   .coa_compress_noctx = NULL,
+   .coa_decompress_noctx   = NULL } }
 };
 
 static int __init crypto842_mod_init(void)
diff --git a/crypto/compress.c b/crypto/compress.c
index c33f076..f27eee0 100644
--- a/crypto/compress.c
+++ b/crypto/compress.c
@@ -33,6 +33,26 @@ static int crypto_decompress(struct crypto_tfm *tfm,
   dlen);
 }
 
+struct crypto_alg *crypto_get_comp(const char *alg_name, u32 type, u32 mask)
+{
+   struct crypto_alg *alg;
+
+   type = ~CRYPTO_ALG_TYPE_MASK;
+   type |= CRYPTO_ALG_TYPE_COMPRESS;
+   mask |= CRYPTO_ALG_TYPE_MASK;
+
+   alg = crypto_alg_mod_lookup(alg_name, type, mask);
+   if (!IS_ERR(alg))
+   return alg;
+
+   return NULL;
+}
+
+void crypto_put_comp(struct crypto_alg *alg)
+{
+   crypto_mod_put(alg);
+}
+
 int crypto_init_compress_ops(struct crypto_tfm *tfm)
 {
struct compress_tfm *ops = tfm-crt_compress;
diff --git a/crypto/crypto_null.c b/crypto/crypto_null.c
index 941c9a4..e397d1c 100644
--- a/crypto/crypto_null.c
+++ b/crypto/crypto_null.c
@@ -146,7 +146,9 @@ static struct crypto_alg null_algs[3] = { {
.cra_module =   THIS_MODULE,
.cra_u  =   { .compress = {
.coa_compress   =   null_compress,
-   .coa_decompress =   null_compress } }
+   .coa_decompress =   null_compress,
+   .coa_compress_noctx =   NULL,
+   .coa_decompress_noctx   =   NULL } }
 } };
 
 MODULE_ALIAS_CRYPTO(compress_null);
diff --git a/crypto/deflate.c b/crypto/deflate.c
index 95d8d37..dee4daf 100644
--- a/crypto/deflate.c
+++ b/crypto/deflate.c
@@ -203,7 +203,9 @@ static struct crypto_alg alg = {
.cra_exit   = deflate_exit,
.cra_u  = { .compress = {
.coa_compress   = deflate_compress,
-   .coa_decompress = deflate_decompress } }
+   .coa_decompress = deflate_decompress,
+   .coa_compress_noctx = NULL,
+   .coa_decompress_noctx   = NULL } }
 };
 
 static int __init deflate_mod_init(void)
diff --git a/crypto/lz4.c b/crypto/lz4.c
index aefbcea..1848435 100644
--- a/crypto/lz4.c
+++ b/crypto/lz4.c
@@ -86,7 +86,9 @@ static struct crypto_alg alg_lz4 = {
.cra_exit   = lz4_exit,
.cra_u  = { .compress = {
.coa_compress   = lz4_compress_crypto,
-   .coa_decompress = lz4_decompress_crypto } }
+   .coa_decompress = lz4_decompress_crypto,
+   .coa_compress_noctx = NULL,
+   .coa_decompress_noctx   = NULL } }
 };
 
 static int __init lz4_mod_init(void)
diff --git a/crypto/lz4hc.c b/crypto/lz4hc.c
index a1d3b5b..bcf0baa 100644
--- a/crypto/lz4hc.c
+++ b/crypto/lz4hc.c
@@ -86,7 +86,9 @@ static struct crypto_alg alg_lz4hc = {
.cra_exit   = lz4hc_exit,
.cra_u  = { .compress = {
.coa_compress   = lz4hc_compress_crypto,
-   .coa_decompress = lz4hc_decompress_crypto } }
+   .coa_decompress = lz4hc_decompress_crypto,
+   .coa_compress_noctx = NULL,
+   .coa_decompress_noctx   = NULL } }
 };
 
 static int __init lz4hc_mod_init(void)
diff --git a/crypto/lzo.c b/crypto/lzo.c
index 4b3e925..9ca516b 100644
--- a/crypto/lzo.c
+++ b/crypto/lzo.c
@@ -89,7 +89,9 @@ static struct crypto_alg alg = {
.cra_exit   = lzo_exit,
.cra_u  = { .compress = {
.coa_compress   = lzo_compress

Re: [PATCH v2 1/8] crypto: support (de)compression API that doesn't require tfm object

[Joonsoo Kim]

On Thu, Aug 20, 2015 at 02:47:28PM +0800, Herbert Xu wrote:
 On Thu, Aug 20, 2015 at 03:34:57PM +0900, Joonsoo Kim wrote:
  Until now, tfm object embeds (de)compression context in it and
  (de)compression in crypto API requires tfm object to use
  this context. But, there are some algorithms that doesn't need
  such context to operate. Therefore, this patch introduce new crypto
  compression API that call (de)compression function via crypto_alg,
  instead of tfm object. crypto_alg is required to get appropriate
  (de)compression function pointer. This can reduce overhead of
  maintaining multiple tfm if (de)compression doesn't require
  any context to operate.
  
  Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
 
 This isn't going to fly I'm afraid.  The main purpose of a tfm
 is not to allocate memory but to track the crypto_alg object
 which could be a hardware device.
 
 So you're not going to get away with not allocating it.

Hmm... I guess there is no problem. crypto_alg object fetched by
crypto_get_comp() introduced in this patch could be hardware device
algorithm which is same one that we can eventually fetch from tfm object.
So, this approach would correctly track the crypto_alg regardless
it is a hardware one or not. If there is some dependency between
algorithm and tfm, it can't support _noctx API. Am I missing
something?

 
 What you can do for these contextless algorithms (and by definition
 every compression algorithm is conxteless) is to allocate a system-
 wide tfm that is used by everybody, or at least by every one within
 your subsystem.

Yes, I thought this way before, but, current way is much simpler so
I try it first. If it is not acceptable, I will implement this
approach.

Thanks.

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] zram: use crypto API to compress the page

[Joonsoo Kim]

On Thu, Aug 13, 2015 at 12:51:13PM +0900, Sergey Senozhatsky wrote:
 On (08/13/15 11:24), Joonsoo Kim wrote:
  Until now, zram uses compression algorithm through direct call
  to core algorithm function, but, it has drawback that we need to add
  compression algorithm manually to zram if needed. Without this work,
  we cannot utilize various compression algorithms in the system.
  Moreover, to add new compression algorithm, we need to know how to use it
  and this is somewhat time-consuming.
  
 
 I don't like this change, sorry.
 
 Here is why. One of the main reasons why I haven't implemented it using
 crypto API (and it was an option and I thought about it) was and still
 is the fact that crypto API requires tfm for both compress and decompress
 operations. And this is a show stopper. No matter how you implement it,
 it is slower than what we have by definition.

I don't think it is show stopper. Current implementation that uses zstrm
only for compression seems over-optimized thing to me. Some of other
compression algorithms need scratch-buffer to decompress the contents.
As you know, zlib is one of them. I tested some of other decompression
algorithm such as quicklz, wkdm and they also need scratch-buffer.

And, we cannot support crypto algorithm *module* in current
implementation.

If that optimization is really needed for the case that doesn't need
tfm except fetching function pointer, we can implement sharable tfm
in crypto subsystem.

 
 Now zram_bvec_read() operations depends on:
 
 a) other read operations
because read() path now use limited in size idle stream list
 
 b) write operations
because write() path uses same idle streams list
 
 
 
 Literally, you change zram_bvec_read() from a fast
 
   zram_decompress_page(zram, uncmem, index);
 
 to a slow
 
   zstrm = zcomp_strm_find(zram-comp);
   zram_decompress_page(zram, zstrm, uncmem, index);
   zcomp_strm_release(zram-comp, zstrm);
 
 
 you either slow down both write() and read() if you use a single idle
 stream list for both read and write, or double the amount of memory
 used by idle streams if you decide to use separate read and write
 idle stream lists. I see no real reason to do either of those. I like
 that the existing read() is as fast as probably possible.

Yes, there can be some performace regression. But, by allocating some
more streams, we can easily eliminate regression. Memory overhead
would be really small. I think that gaining flexibility and reducing
management overhead can compensate this small memory overhead.

Thanks.

 
  When I tested new algorithms such as zlib, these problems make me hard
  to test them. To prevent these problem in the future, I think that
  using crypto API to compress the page is better approach and this patch
  implements it.
  
  The reason we need to support vairous compression algorithms is that
  zram's performance is highly depend on workload and compression algorithm
  and architecture. Every compression algorithm has it's own strong point.
  For example, zlib is the best for compression ratio, but, it's
  (de)compression speed is rather slow. Against my expectation, in my kernel
  build test with zram swap, in low-memory condition on x86, zlib shows best
  performance than others. In this case, I guess that compression ratio is
  the most important factor. Unlike this situation, on ARM, maybe fast
  (de)compression speed is the most important because it's computation speed
  is slower than x86.
  
  We can't expect what algorithm is the best fit for one's system, because
  it needs too complex calculation. We need to test it in case by case and
  easy to use new compression algorithm by this patch will help
  that situation.
  
  Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
  ---
   drivers/block/zram/Kconfig | 13 +--
   drivers/block/zram/Makefile|  4 +-
   drivers/block/zram/zcomp.c | 87 
  ++
   drivers/block/zram/zcomp.h | 34 +
   drivers/block/zram/zcomp_lz4.c | 47 ---
   drivers/block/zram/zcomp_lz4.h | 17 -
   drivers/block/zram/zcomp_lzo.c | 47 ---
   drivers/block/zram/zcomp_lzo.h | 17 -
   drivers/block/zram/zram_drv.c  | 28 +-
   9 files changed, 66 insertions(+), 228 deletions(-)
   delete mode 100644 drivers/block/zram/zcomp_lz4.c
   delete mode 100644 drivers/block/zram/zcomp_lz4.h
   delete mode 100644 drivers/block/zram/zcomp_lzo.c
   delete mode 100644 drivers/block/zram/zcomp_lzo.h
  
  diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig
  index 386ba3d..36ec96f 100644
  --- a/drivers/block/zram/Kconfig
  +++ b/drivers/block/zram/Kconfig
  @@ -1,8 +1,7 @@
   config ZRAM
  tristate Compressed RAM block device support
  depends on BLOCK  SYSFS  ZSMALLOC
  -   select LZO_COMPRESS
  -   select LZO_DECOMPRESS
  +   select CRYPTO_LZO
  default n
  help
Creates virtual block devices

Re: [PATCH 1/2] crypto: export crypto_alg_list and rwsem

[Joonsoo Kim]

On Thu, Aug 13, 2015 at 11:19:54AM +0800, Herbert Xu wrote:
 On Thu, Aug 13, 2015 at 11:24:13AM +0900, Joonsoo Kim wrote:
  Until now, zram uses compression algorithm through direct call
  to core algorithm function, but, it has drawback that we need to add
  compression algorithm manually to zram. If we don't do that, we cannot
  utilize various compression algorithms in the system. To improve this
  situation, zram will be changed to use crypto subsystem in following
  patch. There is one problem with this change. Zram has a interface
  that what compression algorithm it can support. Although crypto subsystem
  has /proc interface to search all of crypto algorithm, but, there is
  no way to get just compression algorithm in cryto subsystem. To implement
  it on zram-side, crypto_alg_list and rwsem should be exported so
  this patch do it.
  
  Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
 
 Nack.  We already have a netlink interface that can be used to
 query algorithms and the type information is present in the report.
 The interface is crypto_user and should be used instead of exporting
 the raw list.

Oh... I see.

Is there any way to access netlink interface and get the output from
kernel-side? I'd like to show information through
/sys/block/zramX/comp_algorithm, because some user program can be
broken if we change output of userspace exposed interface.

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] crypto: export crypto_alg_list and rwsem

[Joonsoo Kim]

On Thu, Aug 13, 2015 at 03:29:18PM +0800, Herbert Xu wrote:
 On Thu, Aug 13, 2015 at 04:30:31PM +0900, Joonsoo Kim wrote:
 
  How about introducing new functions to search supported algorithm in
  kernel-side? As crypto API is used in more places, this interface
  would be requested more. Defined list weaken the advantage of strong
  point of generic crypto API.
 
 You're only getting the list so you can immediately reexport
 it to user-space, with no added value whatsoever.  As I said
 we already have an interface for that so you should use it.
 
 If more genuine uses come up then I will reconsider.

Okay. I will think this issue more and come back with better
solution.

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] zram: use crypto API to compress the page

[Joonsoo Kim]

On Thu, Aug 13, 2015 at 03:32:33PM +0800, Herbert Xu wrote:
 On Thu, Aug 13, 2015 at 04:19:41PM +0900, Joonsoo Kim wrote:
  
  If that optimization is really needed for the case that doesn't need
  tfm except fetching function pointer, we can implement sharable tfm
  in crypto subsystem.
 
 I'm happy to consider changes to the crypto compression interface
 as long as it continues to support the existing algorithms and users.

Happy to hear that.
I will think how it can be improved.

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] crypto: export crypto_alg_list and rwsem

[Joonsoo Kim]

On Thu, Aug 13, 2015 at 02:38:23PM +0800, Herbert Xu wrote:
 On Thu, Aug 13, 2015 at 03:37:55PM +0900, Joonsoo Kim wrote:
 
  Is there any way to access netlink interface and get the output from
  kernel-side? I'd like to show information through
  /sys/block/zramX/comp_algorithm, because some user program can be
  broken if we change output of userspace exposed interface.
 
 You could simply deprecate that interface but keep it for existing
 algorithms.  Any new algorithms can then be queried through the
 crypto_user interface.
 
 The other option is to have a defined list of algorithms which is
 independent of the crypto API.  For example, have a look at what
 IPsec does in net/xfrm/xfrm_algo.c.  IPsec needs its own list
 because they come with annotations.

Hmm... they looks suboptimal.

How about introducing new functions to search supported algorithm in
kernel-side? As crypto API is used in more places, this interface
would be requested more. Defined list weaken the advantage of strong
point of generic crypto API.

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] crypto: export crypto_alg_list and rwsem

[Joonsoo Kim]

Until now, zram uses compression algorithm through direct call
to core algorithm function, but, it has drawback that we need to add
compression algorithm manually to zram. If we don't do that, we cannot
utilize various compression algorithms in the system. To improve this
situation, zram will be changed to use crypto subsystem in following
patch. There is one problem with this change. Zram has a interface
that what compression algorithm it can support. Although crypto subsystem
has /proc interface to search all of crypto algorithm, but, there is
no way to get just compression algorithm in cryto subsystem. To implement
it on zram-side, crypto_alg_list and rwsem should be exported so
this patch do it.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 crypto/internal.h  | 2 --
 include/linux/crypto.h | 4 
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/crypto/internal.h b/crypto/internal.h
index 00e42a3..806f17a 100644
--- a/crypto/internal.h
+++ b/crypto/internal.h
@@ -45,8 +45,6 @@ struct crypto_larval {
u32 mask;
 };
 
-extern struct list_head crypto_alg_list;
-extern struct rw_semaphore crypto_alg_sem;
 extern struct blocking_notifier_head crypto_chain;
 
 #ifdef CONFIG_PROC_FS
diff --git a/include/linux/crypto.h b/include/linux/crypto.h
index 81ef938..ab39f4b 100644
--- a/include/linux/crypto.h
+++ b/include/linux/crypto.h
@@ -24,6 +24,10 @@
 #include linux/slab.h
 #include linux/string.h
 #include linux/uaccess.h
+#include linux/rwsem.h
+
+extern struct list_head crypto_alg_list;
+extern struct rw_semaphore crypto_alg_sem;
 
 /*
  * Autoloaded crypto modules should only use a prefixed name to avoid allowing
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] zram: use crypto API to compress the page

[Joonsoo Kim]

Until now, zram uses compression algorithm through direct call
to core algorithm function, but, it has drawback that we need to add
compression algorithm manually to zram if needed. Without this work,
we cannot utilize various compression algorithms in the system.
Moreover, to add new compression algorithm, we need to know how to use it
and this is somewhat time-consuming.

When I tested new algorithms such as zlib, these problems make me hard
to test them. To prevent these problem in the future, I think that
using crypto API to compress the page is better approach and this patch
implements it.

The reason we need to support vairous compression algorithms is that
zram's performance is highly depend on workload and compression algorithm
and architecture. Every compression algorithm has it's own strong point.
For example, zlib is the best for compression ratio, but, it's
(de)compression speed is rather slow. Against my expectation, in my kernel
build test with zram swap, in low-memory condition on x86, zlib shows best
performance than others. In this case, I guess that compression ratio is
the most important factor. Unlike this situation, on ARM, maybe fast
(de)compression speed is the most important because it's computation speed
is slower than x86.

We can't expect what algorithm is the best fit for one's system, because
it needs too complex calculation. We need to test it in case by case and
easy to use new compression algorithm by this patch will help
that situation.

Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com
---
 drivers/block/zram/Kconfig | 13 +--
 drivers/block/zram/Makefile|  4 +-
 drivers/block/zram/zcomp.c | 87 ++
 drivers/block/zram/zcomp.h | 34 +
 drivers/block/zram/zcomp_lz4.c | 47 ---
 drivers/block/zram/zcomp_lz4.h | 17 -
 drivers/block/zram/zcomp_lzo.c | 47 ---
 drivers/block/zram/zcomp_lzo.h | 17 -
 drivers/block/zram/zram_drv.c  | 28 +-
 9 files changed, 66 insertions(+), 228 deletions(-)
 delete mode 100644 drivers/block/zram/zcomp_lz4.c
 delete mode 100644 drivers/block/zram/zcomp_lz4.h
 delete mode 100644 drivers/block/zram/zcomp_lzo.c
 delete mode 100644 drivers/block/zram/zcomp_lzo.h

diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig
index 386ba3d..36ec96f 100644
--- a/drivers/block/zram/Kconfig
+++ b/drivers/block/zram/Kconfig
@@ -1,8 +1,7 @@
 config ZRAM
tristate Compressed RAM block device support
depends on BLOCK  SYSFS  ZSMALLOC
-   select LZO_COMPRESS
-   select LZO_DECOMPRESS
+   select CRYPTO_LZO
default n
help
  Creates virtual block devices called /dev/zramX (X = 0, 1, ...).
@@ -14,13 +13,3 @@ config ZRAM
  disks and maybe many more.
 
  See zram.txt for more information.
-
-config ZRAM_LZ4_COMPRESS
-   bool Enable LZ4 algorithm support
-   depends on ZRAM
-   select LZ4_COMPRESS
-   select LZ4_DECOMPRESS
-   default n
-   help
- This option enables LZ4 compression algorithm support. Compression
- algorithm can be changed using `comp_algorithm' device attribute.
\ No newline at end of file
diff --git a/drivers/block/zram/Makefile b/drivers/block/zram/Makefile
index be0763f..9e2b79e 100644
--- a/drivers/block/zram/Makefile
+++ b/drivers/block/zram/Makefile
@@ -1,5 +1,3 @@
-zram-y :=  zcomp_lzo.o zcomp.o zram_drv.o
-
-zram-$(CONFIG_ZRAM_LZ4_COMPRESS) += zcomp_lz4.o
+zram-y :=  zcomp.o zram_drv.o
 
 obj-$(CONFIG_ZRAM) +=  zram.o
diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 965d1af..3dc9bfa 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -13,12 +13,9 @@
 #include linux/slab.h
 #include linux/wait.h
 #include linux/sched.h
+#include linux/crypto.h
 
 #include zcomp.h
-#include zcomp_lzo.h
-#ifdef CONFIG_ZRAM_LZ4_COMPRESS
-#include zcomp_lz4.h
-#endif
 
 /*
  * single zcomp_strm backend
@@ -43,36 +40,17 @@ struct zcomp_strm_multi {
wait_queue_head_t strm_wait;
 };
 
-static struct zcomp_backend *backends[] = {
-   zcomp_lzo,
-#ifdef CONFIG_ZRAM_LZ4_COMPRESS
-   zcomp_lz4,
-#endif
-   NULL
-};
-
-static struct zcomp_backend *find_backend(const char *compress)
-{
-   int i = 0;
-   while (backends[i]) {
-   if (sysfs_streq(compress, backends[i]-name))
-   break;
-   i++;
-   }
-   return backends[i];
-}
-
 static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm)
 {
-   if (zstrm-private)
-   comp-backend-destroy(zstrm-private);
+   if (zstrm-tfm)
+   crypto_free_comp(zstrm-tfm);
free_pages((unsigned long)zstrm-buffer, 1);
kfree(zstrm);
 }
 
 /*
- * allocate new zcomp_strm structure with -private initialized by
- * backend, return NULL on error
+ * allocate new zcomp_strm structure

Re: [PATCH 01/10] workqueue: add wq_numa_tbl_len and wq_numa_possible_cpumask[]

[JoonSoo Kim]

2013/3/20 Tejun Heo t...@kernel.org:
 Unbound workqueues are going to be NUMA-affine.  Add wq_numa_tbl_len
 and wq_numa_possible_cpumask[] in preparation.  The former is the
 highest NUMA node ID + 1 and the latter is masks of possibles CPUs for
 each NUMA node.

It is better to move this code to topology.c or cpumask.c,
then it can be generally used.

Thanks.

 This patch only introduces these.  Future patches will make use of
 them.

 Signed-off-by: Tejun Heo t...@kernel.org
 ---
  kernel/workqueue.c | 35 ++-
  1 file changed, 34 insertions(+), 1 deletion(-)

 diff --git a/kernel/workqueue.c b/kernel/workqueue.c
 index 775c2f4..9b096e3 100644
 --- a/kernel/workqueue.c
 +++ b/kernel/workqueue.c
 @@ -44,6 +44,7 @@
  #include linux/jhash.h
  #include linux/hashtable.h
  #include linux/rculist.h
 +#include linux/nodemask.h

  #include workqueue_internal.h

 @@ -256,6 +257,11 @@ struct workqueue_struct {

  static struct kmem_cache *pwq_cache;

 +static int wq_numa_tbl_len;/* highest possible NUMA node id + 1 
 */
 +static cpumask_var_t *wq_numa_possible_cpumask;
 +   /* possible CPUs of each node, may
 +  be NULL if init failed */
 +
  static DEFINE_MUTEX(wq_mutex); /* protects workqueues and pools */
  static DEFINE_SPINLOCK(pwq_lock);  /* protects pool_workqueues */
  static DEFINE_SPINLOCK(wq_mayday_lock);/* protects wq-maydays list 
 */
 @@ -4416,7 +4422,7 @@ out_unlock:
  static int __init init_workqueues(void)
  {
 int std_nice[NR_STD_WORKER_POOLS] = { 0, HIGHPRI_NICE_LEVEL };
 -   int i, cpu;
 +   int i, node, cpu;

 /* make sure we have enough bits for OFFQ pool ID */
 BUILD_BUG_ON((1LU  (BITS_PER_LONG - WORK_OFFQ_POOL_SHIFT)) 
 @@ -4429,6 +4435,33 @@ static int __init init_workqueues(void)
 cpu_notifier(workqueue_cpu_up_callback, CPU_PRI_WORKQUEUE_UP);
 hotcpu_notifier(workqueue_cpu_down_callback, CPU_PRI_WORKQUEUE_DOWN);

 +   /* determine NUMA pwq table len - highest node id + 1 */
 +   for_each_node(node)
 +   wq_numa_tbl_len = max(wq_numa_tbl_len, node + 1);
 +
 +   /*
 +* We want masks of possible CPUs of each node which isn't readily
 +* available.  Build one from cpu_to_node() which should have been
 +* fully initialized by now.
 +*/
 +   wq_numa_possible_cpumask = kzalloc(wq_numa_tbl_len *
 +  
 sizeof(wq_numa_possible_cpumask[0]),
 +  GFP_KERNEL);
 +   BUG_ON(!wq_numa_possible_cpumask);
 +
 +   for_each_node(node)
 +   
 BUG_ON(!alloc_cpumask_var_node(wq_numa_possible_cpumask[node],
 +  GFP_KERNEL, node));
 +   for_each_possible_cpu(cpu) {
 +   node = cpu_to_node(cpu);
 +   if (WARN_ON(node == NUMA_NO_NODE)) {
 +   pr_err(workqueue: NUMA node mapping not available 
 for cpu%d, disabling NUMA support\n, cpu);
 +   wq_numa_possible_cpumask = NULL;
 +   break;
 +   }
 +   cpumask_set_cpu(cpu, wq_numa_possible_cpumask[node]);
 +   }
 +
 /* initialize CPU pools */
 for_each_possible_cpu(cpu) {
 struct worker_pool *pool;
 --
 1.8.1.4

 --
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html