Re: [PATCH v7 4/9] crypto: acomp - add support for lzo via scomp

[Herbert Xu]

On Mon, Sep 26, 2016 at 06:27:04PM +0100, Giovanni Cabiddu wrote:
>
> I think the definition of the acomp interface already allows for this.
> If the destination scatterlist inside the request is NULL, the
> algorithm/driver can allocate pages of memory for the output buffers as
> well as the scatterlist. In this case, the destination length, if not zero,
> could be used to specify the maximum size to allocate.
> What do you think?

Sounds good to me.  We should also add a function to free the
resulting SG list along with the allocated pages.

Cheers,
-- 
Email: Herbert Xu 
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7 4/9] crypto: acomp - add support for lzo via scomp

[Giovanni Cabiddu]

On Fri, Sep 23, 2016 at 11:05:18PM +0800, Herbert Xu wrote:
> When I said acomp layer I'm referring specifically to the algorithm
> or driver.  As to your last question it would be the caller's
> responsibility to free that memory.
>
> The use-case is our oldest user, IPcomp.  Most packets are 1500 bytes
> max but we have to allocate 64K of memory to cover the worst case.
> For an algorithm that can deal with SG lists it can easily allocate
> pages of memory as it goes and place them in an SG list.
It is clear now. Thanks.

> Sure if you already have memory allocated then we don't want to
> force you to allocate it again in the algorithm/driver.  But our
> interface should allow the memory to be allocated in the driver.
I think the definition of the acomp interface already allows for this.
If the destination scatterlist inside the request is NULL, the
algorithm/driver can allocate pages of memory for the output buffers as
well as the scatterlist. In this case, the destination length, if not zero,
could be used to specify the maximum size to allocate.
What do you think?

--
Giovanni
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7 4/9] crypto: acomp - add support for lzo via scomp

[Giovanni Cabiddu]

On Thu, Sep 22, 2016 at 05:22:44PM +0800, Herbert Xu wrote:
> I'm suggesting that we have just one set of buffers for all scomp
> algorithms.  After all, a CPU can only process one request at a
> time.
Makes sense. Implemented in v8.

> Yes scomp should just be flat.  A sync algorithm capable of producing
> partial output should use the acomp interface.
I went back to scomp interface in v6.
>
> I think you may have misread my earlier message from June.  What
> I'd like to see is for the acomp layer to allocate the output
> memory, rather than have it provided by the user as is the case
> with the current interface.  The user could provide a maximum to
> prevent crazy cases consuming unlimited memory.
Why would you prefer to have the output buffer allocated in the acomp
layer? Is there a use case for this? Who would free that memory?

We believe that the output buffer should be allocated by the user of the
API. A caller might decide to allocate memory upfront or point the
buffer list to pre-allocate buffers. This would happen in BTRFS where
its block buffers are already allocated for submission to the
compression API.

Thanks,

--
Giovanni
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7 4/9] crypto: acomp - add support for lzo via scomp

[Herbert Xu]

On Tue, Sep 20, 2016 at 12:51:40PM +0100, Giovanni Cabiddu wrote:
> Hi Herbert,
> 
> On Tue, Sep 20, 2016 at 05:26:18PM +0800, Herbert Xu wrote:
> > Rather than duplicating the scratch buffer handling in every scomp
> > algorithm, let's centralize this and put it into scomp.c.
> Are you suggesting to hide the scratch buffers from the scomp implementations 
> and allocate them inside crypto_register_scomp?

I'm suggesting that we have just one set of buffers for all scomp
algorithms.  After all, a CPU can only process one request at a
time.

> Does that mean we have to go back to a scomp_alg with a flat buffer API
> and linearize inside scomp?

Yes scomp should just be flat.  A sync algorithm capable of producing
partial output should use the acomp interface.

> If we take this direction, how do we support DMA from scomp implementation?
> Scratch buffers are allocated using vmalloc.

Huh? If you're doing DMA then you'd be using the acomp interface,
how can you even get into scomp?

I think you may have misread my earlier message from June.  What
I'd like to see is for the acomp layer to allocate the output
memory, rather than have it provided by the user as is the case
with the current interface.  The user could provide a maximum to
prevent crazy cases consuming unlimited memory.

What will happen with scomp is that it will simply use a linear
vmalloc buffer to store the output before copying it into an SG
list of individual pages allocated on the spot.

Cheers,
-- 
Email: Herbert Xu 
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7 4/9] crypto: acomp - add support for lzo via scomp

[Giovanni Cabiddu]

Hi Herbert,

apologies for the duplicate. The previous email didn't get delivered to
the ML.

On Tue, Sep 20, 2016 at 05:26:18PM +0800, Herbert Xu wrote:
> Rather than duplicating the scratch buffer handling in every scomp
> algorithm, let's centralize this and put it into scomp.c.
Are you suggesting to hide the scratch buffers from the scomp
implementations and allocate them inside crypto_register_scomp?
Does that mean we have to go back to a scomp_alg with a flat buffer API
and linearize inside scomp?
If we take this direction, how do we support DMA from scomp
implementation? Scratch buffers are allocated using vmalloc.

Thanks,

--
Giovanni
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7 4/9] crypto: acomp - add support for lzo via scomp

[Herbert Xu]

On Tue, Sep 13, 2016 at 01:49:36PM +0100, Giovanni Cabiddu wrote:
>
> + lzo_src_scratches = crypto_scomp_alloc_scratches(LZO_SCRATCH_SIZE);
> + if (!lzo_src_scratches)
> + return -ENOMEM;

Rather than duplicating the scratch buffer handling in every scomp
algorithm, let's centralize this and put it into scomp.c.

Thanks,
-- 
Email: Herbert Xu 
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html