How driver can mark the algo implementation Unavailable

[Harsh Jain]

Hi All,

PCI based devices can be shutdown from sysfs interface

echo "unbind" > /sys/bus/pci/drivers/cxgb4/unbind

In case device has active Transformation(tfm), Drivers cannot un-register the 
Algorithms because alg->cra_refcnt will be non zero.

Can driver use the "CRYPTO_ALG_DEAD" flag to mark it un-available so that 
crypto_alg_lookup does not allocate new tfm using dead algo.

Regards

Harsh Jain

[PATCH 1/1] crypto:chelsio: Fix memory corruption in DMA Mapped buffers.

[Harsh Jain]

Update PCI Id in "cpl_rx_phys_dsgl" header. In case pci_chan_id and
tx_chan_id are not derived from same queue, H/W can send request
completion indication before completing DMA Transfer.

Herbert, It would be good if fix can be merge to stable tree.
For 4.14 kernel, It requires some update to avoid mege conficts.

Signed-off-by: Harsh Jain 
---
 drivers/crypto/chelsio/chcr_algo.c   | 32 ++--
 drivers/crypto/chelsio/chcr_crypto.h |  2 ++
 2 files changed, 24 insertions(+), 10 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 5c539af..010bbf6 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -367,7 +367,8 @@ static inline void dsgl_walk_init(struct dsgl_walk *walk,
walk->to = (struct phys_sge_pairs *)(dsgl + 1);
 }
 
-static inline void dsgl_walk_end(struct dsgl_walk *walk, unsigned short qid)
+static inline void dsgl_walk_end(struct dsgl_walk *walk, unsigned short qid,
+int pci_chan_id)
 {
struct cpl_rx_phys_dsgl *phys_cpl;
 
@@ -385,6 +386,7 @@ static inline void dsgl_walk_end(struct dsgl_walk *walk, 
unsigned short qid)
phys_cpl->rss_hdr_int.opcode = CPL_RX_PHYS_ADDR;
phys_cpl->rss_hdr_int.qid = htons(qid);
phys_cpl->rss_hdr_int.hash_val = 0;
+   phys_cpl->rss_hdr_int.channel = pci_chan_id;
 }
 
 static inline void dsgl_walk_add_page(struct dsgl_walk *walk,
@@ -718,7 +720,7 @@ static inline void create_wreq(struct chcr_context *ctx,
FILL_WR_RX_Q_ID(ctx->dev->rx_channel_id, qid,
!!lcb, ctx->tx_qidx);
 
-   chcr_req->ulptx.cmd_dest = FILL_ULPTX_CMD_DEST(ctx->dev->tx_channel_id,
+   chcr_req->ulptx.cmd_dest = FILL_ULPTX_CMD_DEST(ctx->tx_chan_id,
   qid);
chcr_req->ulptx.len = htonl((DIV_ROUND_UP(len16, 16) -
 ((sizeof(chcr_req->wreq)) >> 4)));
@@ -1339,16 +1341,23 @@ static int chcr_device_init(struct chcr_context *ctx)
adap->vres.ncrypto_fc);
rxq_perchan = u_ctx->lldi.nrxq / u_ctx->lldi.nchan;
txq_perchan = ntxq / u_ctx->lldi.nchan;
-   rxq_idx = ctx->dev->tx_channel_id * rxq_perchan;
-   rxq_idx += id % rxq_perchan;
-   txq_idx = ctx->dev->tx_channel_id * txq_perchan;
-   txq_idx += id % txq_perchan;
spin_lock(>dev->lock_chcr_dev);
-   ctx->rx_qidx = rxq_idx;
-   ctx->tx_qidx = txq_idx;
+   ctx->tx_chan_id = ctx->dev->tx_channel_id;
ctx->dev->tx_channel_id = !ctx->dev->tx_channel_id;
ctx->dev->rx_channel_id = 0;
spin_unlock(>dev->lock_chcr_dev);
+   rxq_idx = ctx->tx_chan_id * rxq_perchan;
+   rxq_idx += id % rxq_perchan;
+   txq_idx = ctx->tx_chan_id * txq_perchan;
+   txq_idx += id % txq_perchan;
+   ctx->rx_qidx = rxq_idx;
+   ctx->tx_qidx = txq_idx;
+   /* Channel Id used by SGE to forward packet to Host.
+* Same value should be used in cpl_fw6_pld RSS_CH field
+* by FW. Driver programs PCI channel ID to be used in fw
+* at the time of queue allocation with value "pi->tx_chan"
+*/
+   ctx->pci_chan_id = txq_idx / txq_perchan;
}
 out:
return err;
@@ -2503,6 +2512,7 @@ void chcr_add_aead_dst_ent(struct aead_request *req,
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct dsgl_walk dsgl_walk;
unsigned int authsize = crypto_aead_authsize(tfm);
+   struct chcr_context *ctx = a_ctx(tfm);
u32 temp;
 
dsgl_walk_init(_walk, phys_cpl);
@@ -2512,7 +2522,7 @@ void chcr_add_aead_dst_ent(struct aead_request *req,
dsgl_walk_add_page(_walk, IV, >iv_dma);
temp = req->cryptlen + (reqctx->op ? -authsize : authsize);
dsgl_walk_add_sg(_walk, req->dst, temp, req->assoclen);
-   dsgl_walk_end(_walk, qid);
+   dsgl_walk_end(_walk, qid, ctx->pci_chan_id);
 }
 
 void chcr_add_cipher_src_ent(struct ablkcipher_request *req,
@@ -2544,6 +2554,8 @@ void chcr_add_cipher_dst_ent(struct ablkcipher_request 
*req,
 unsigned short qid)
 {
struct chcr_blkcipher_req_ctx *reqctx = ablkcipher_request_ctx(req);
+   struct crypto_ablkcipher *tfm = crypto_ablkcipher_reqtfm(wrparam->req);
+   struct chcr_context *ctx = c_ctx(tfm);
struct dsgl_walk dsgl_walk;
 
dsgl_walk_init(_walk, phys_cpl);
@@ -2552,7 +2564,7 @@ void chcr_add_cipher_dst_ent(struct ablkcipher_request 
*req,
reqctx

[PATCH 0/3] crypto:chelsio: Fixes and cleanup

[Harsh Jain]

It includes Fixes and cleanup .

Harsh Jain (3):
  crypto:chelsio:Return -ENOSPC for transient busy indication.
  crypt:chelsio:Send IV as Immediate for cipher algo
  crypto:chelsio: Remove separate buffer used for DMA map B0 block in
CCM

 drivers/crypto/chelsio/chcr_algo.c   | 303 +++
 drivers/crypto/chelsio/chcr_algo.h   |   3 +-
 drivers/crypto/chelsio/chcr_core.h   |   2 +-
 drivers/crypto/chelsio/chcr_crypto.h |  15 +-
 4 files changed, 140 insertions(+), 183 deletions(-)

-- 
2.1.4

[PATCH 1/3] crypto:chelsio:Return -ENOSPC for transient busy indication.

[Harsh Jain]

Change the return type based on following patch
https://www.mail-archive.com/linux-crypto@vger.kernel.org/msg28552.html

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 56 ++
 1 file changed, 26 insertions(+), 30 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 59fe6631e..db15a9f 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -688,6 +688,7 @@ static int chcr_cipher_fallback(struct crypto_skcipher 
*cipher,
int err;
 
SKCIPHER_REQUEST_ON_STACK(subreq, cipher);
+
skcipher_request_set_tfm(subreq, cipher);
skcipher_request_set_callback(subreq, flags, NULL, NULL);
skcipher_request_set_crypt(subreq, src, dst,
@@ -1113,14 +1114,6 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
goto complete;
}
 
-   if (unlikely(cxgb4_is_crypto_q_full(u_ctx->lldi.ports[0],
-   c_ctx(tfm)->tx_qidx))) {
-   if (!(req->base.flags & CRYPTO_TFM_REQ_MAY_BACKLOG)) {
-   err = -EBUSY;
-   goto unmap;
-   }
-
-   }
if (!reqctx->imm) {
bytes = chcr_sg_ent_in_wr(reqctx->srcsg, reqctx->dstsg, 1,
  CIP_SPACE_LEFT(ablkctx->enckey_len),
@@ -1293,13 +1286,14 @@ static int chcr_aes_encrypt(struct ablkcipher_request 
*req)
 {
struct crypto_ablkcipher *tfm = crypto_ablkcipher_reqtfm(req);
struct sk_buff *skb = NULL;
-   int err;
+   int err, isfull = 0;
struct uld_ctx *u_ctx = ULD_CTX(c_ctx(tfm));
 
if (unlikely(cxgb4_is_crypto_q_full(u_ctx->lldi.ports[0],
c_ctx(tfm)->tx_qidx))) {
+   isfull = 1;
if (!(req->base.flags & CRYPTO_TFM_REQ_MAY_BACKLOG))
-   return -EBUSY;
+   return -ENOSPC;
}
 
err = process_cipher(req, u_ctx->lldi.rxq_ids[c_ctx(tfm)->rx_qidx],
@@ -1309,7 +1303,7 @@ static int chcr_aes_encrypt(struct ablkcipher_request 
*req)
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, c_ctx(tfm)->tx_qidx);
chcr_send_wr(skb);
-   return -EINPROGRESS;
+   return isfull ? -EBUSY : -EINPROGRESS;
 }
 
 static int chcr_aes_decrypt(struct ablkcipher_request *req)
@@ -1317,12 +1311,13 @@ static int chcr_aes_decrypt(struct ablkcipher_request 
*req)
struct crypto_ablkcipher *tfm = crypto_ablkcipher_reqtfm(req);
struct uld_ctx *u_ctx = ULD_CTX(c_ctx(tfm));
struct sk_buff *skb = NULL;
-   int err;
+   int err, isfull = 0;
 
if (unlikely(cxgb4_is_crypto_q_full(u_ctx->lldi.ports[0],
c_ctx(tfm)->tx_qidx))) {
+   isfull = 1;
if (!(req->base.flags & CRYPTO_TFM_REQ_MAY_BACKLOG))
-   return -EBUSY;
+   return -ENOSPC;
}
 
 err = process_cipher(req, u_ctx->lldi.rxq_ids[c_ctx(tfm)->rx_qidx],
@@ -1332,7 +1327,7 @@ static int chcr_aes_decrypt(struct ablkcipher_request 
*req)
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, c_ctx(tfm)->tx_qidx);
chcr_send_wr(skb);
-   return -EINPROGRESS;
+   return isfull ? -EBUSY : -EINPROGRESS;
 }
 
 static int chcr_device_init(struct chcr_context *ctx)
@@ -1574,14 +1569,15 @@ static int chcr_ahash_update(struct ahash_request *req)
u8 remainder = 0, bs;
unsigned int nbytes = req->nbytes;
struct hash_wr_param params;
-   int error;
+   int error, isfull = 0;
 
bs = crypto_tfm_alg_blocksize(crypto_ahash_tfm(rtfm));
u_ctx = ULD_CTX(h_ctx(rtfm));
if (unlikely(cxgb4_is_crypto_q_full(u_ctx->lldi.ports[0],
h_ctx(rtfm)->tx_qidx))) {
+   isfull = 1;
if (!(req->base.flags & CRYPTO_TFM_REQ_MAY_BACKLOG))
-   return -EBUSY;
+   return -ENOSPC;
}
 
if (nbytes + req_ctx->reqlen >= bs) {
@@ -1633,7 +1629,7 @@ static int chcr_ahash_update(struct ahash_request *req)
set_wr_txq(skb, CPL_PRIORITY_DATA, h_ctx(rtfm)->tx_qidx);
chcr_send_wr(skb);
 
-   return -EINPROGRESS;
+   return isfull ? -EBUSY : -EINPROGRESS;
 unmap:
chcr_hash_dma_unmap(_ctx->lldi.pdev->dev, req);
return error;
@@ -1710,15 +1706,16 @@ static int chcr_ahash_finup(struct ahash_request *req)
struct sk_buff *skb;
struct hash_wr_param params;
u8  bs;
-   int error;
+   int error, isfull = 0;
 
bs = crypto_tfm_alg_blocksize(crypto_aha

[PATCH 2/3] crypt:chelsio:Send IV as Immediate for cipher algo

[Harsh Jain]

Send IV in WR as immediate instead of dma mapped entry for cipher.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 49 +++-
 drivers/crypto/chelsio/chcr_algo.h   |  3 +--
 drivers/crypto/chelsio/chcr_core.h   |  2 +-
 drivers/crypto/chelsio/chcr_crypto.h |  3 +--
 4 files changed, 17 insertions(+), 40 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index db15a9f..b2bfeb2 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -638,7 +638,6 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
src = sg_next(src);
srcskip = 0;
}
-
if (sg_dma_len(dst) == dstskip) {
dst = sg_next(dst);
dstskip = 0;
@@ -761,13 +760,13 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
 
nents = sg_nents_xlen(reqctx->dstsg,  wrparam->bytes, CHCR_DST_SG_SIZE,
  reqctx->dst_ofst);
-   dst_size = get_space_for_phys_dsgl(nents + 1);
+   dst_size = get_space_for_phys_dsgl(nents);
kctx_len = roundup(ablkctx->enckey_len, 16);
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, dst_size);
nents = sg_nents_xlen(reqctx->srcsg, wrparam->bytes,
  CHCR_SRC_SG_SIZE, reqctx->src_ofst);
-   temp = reqctx->imm ? roundup(IV + wrparam->req->nbytes, 16) :
-(sgl_len(nents + MIN_CIPHER_SG) * 8);
+   temp = reqctx->imm ? roundup(wrparam->bytes, 16) :
+(sgl_len(nents) * 8);
transhdr_len += temp;
transhdr_len = roundup(transhdr_len, 16);
skb = alloc_skb(SGE_MAX_WR_LEN, flags);
@@ -789,7 +788,7 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
 ablkctx->ciph_mode,
 0, 0, IV >> 1);
chcr_req->sec_cpl.ivgen_hdrlen = FILL_SEC_CPL_IVGEN_HDRLEN(0, 0, 0,
- 0, 0, dst_size);
+ 0, 1, dst_size);
 
chcr_req->key_ctx.ctx_hdr = ablkctx->key_ctx_hdr;
if ((reqctx->op == CHCR_DECRYPT_OP) &&
@@ -819,8 +818,8 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
chcr_add_cipher_dst_ent(wrparam->req, phys_cpl, wrparam, wrparam->qid);
 
atomic_inc(>chcr_stats.cipher_rqst);
-   temp = sizeof(struct cpl_rx_phys_dsgl) + dst_size + kctx_len
-   +(reqctx->imm ? (IV + wrparam->bytes) : 0);
+   temp = sizeof(struct cpl_rx_phys_dsgl) + dst_size + kctx_len + IV
+   + (reqctx->imm ? (wrparam->bytes) : 0);
create_wreq(c_ctx(tfm), chcr_req, &(wrparam->req->base), reqctx->imm, 0,
transhdr_len, temp,
ablkctx->ciph_mode == CHCR_SCMD_CIPHER_MODE_AES_CBC);
@@ -1023,7 +1022,7 @@ static int chcr_update_tweak(struct ablkcipher_request 
*req, u8 *iv,
ret = crypto_cipher_setkey(cipher, key, keylen);
if (ret)
goto out;
-   /*H/W sends the encrypted IV in dsgl when AADIVDROP bit is 0*/
+   crypto_cipher_encrypt_one(cipher, iv, iv);
for (i = 0; i < round8; i++)
gf128mul_x8_ble((le128 *)iv, (le128 *)iv);
 
@@ -1115,7 +1114,7 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
}
 
if (!reqctx->imm) {
-   bytes = chcr_sg_ent_in_wr(reqctx->srcsg, reqctx->dstsg, 1,
+   bytes = chcr_sg_ent_in_wr(reqctx->srcsg, reqctx->dstsg, 0,
  CIP_SPACE_LEFT(ablkctx->enckey_len),
  reqctx->src_ofst, reqctx->dst_ofst);
if ((bytes + reqctx->processed) >= req->nbytes)
@@ -1126,11 +1125,7 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
/*CTR mode counter overfloa*/
bytes  = req->nbytes - reqctx->processed;
}
-   dma_sync_single_for_cpu(_CTX(c_ctx(tfm))->lldi.pdev->dev,
-   reqctx->iv_dma, IV, DMA_BIDIRECTIONAL);
err = chcr_update_cipher_iv(req, fw6_pld, reqctx->iv);
-   dma_sync_single_for_device(_CTX(c_ctx(tfm))->lldi.pdev->dev,
-  reqctx->iv_dma, IV, DMA_BIDIRECTIONAL);
if (err)
goto unmap;
 
@@ -1205,7 +1200,6 @@ static int process_cipher(struct ablkcipher_request *req,
 
dnents = sg_nents_xlen(req->dst, req->nbytes,
   CHCR_DST_SG_SIZE, 0);
-   

[PATCH 3/3] crypto:chelsio: Remove separate buffer used for DMA map B0 block in CCM

[Harsh Jain]

Extends memory required for IV to include B0 Block and DMA map in
single operation.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 198 ---
 drivers/crypto/chelsio/chcr_crypto.h |  12 +--
 2 files changed, 97 insertions(+), 113 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index b2bfeb2..b916c4e 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -203,13 +203,8 @@ static inline void chcr_handle_aead_resp(struct 
aead_request *req,
 int err)
 {
struct chcr_aead_reqctx *reqctx = aead_request_ctx(req);
-   struct crypto_aead *tfm = crypto_aead_reqtfm(req);
-   struct uld_ctx *u_ctx = ULD_CTX(a_ctx(tfm));
 
-   chcr_aead_dma_unmap(_ctx->lldi.pdev->dev, req, reqctx->op);
-   if (reqctx->b0_dma)
-   dma_unmap_single(_ctx->lldi.pdev->dev, reqctx->b0_dma,
-reqctx->b0_len, DMA_BIDIRECTIONAL);
+   chcr_aead_common_exit(req);
if (reqctx->verify == VERIFY_SW) {
chcr_verify_tag(req, input, );
reqctx->verify = VERIFY_HW;
@@ -2178,22 +2173,35 @@ static void chcr_hmac_cra_exit(struct crypto_tfm *tfm)
}
 }
 
-static int chcr_aead_common_init(struct aead_request *req,
-unsigned short op_type)
+inline void chcr_aead_common_exit(struct aead_request *req)
+{
+   struct chcr_aead_reqctx  *reqctx = aead_request_ctx(req);
+   struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+   struct uld_ctx *u_ctx = ULD_CTX(a_ctx(tfm));
+
+   chcr_aead_dma_unmap(_ctx->lldi.pdev->dev, req, reqctx->op);
+}
+
+static int chcr_aead_common_init(struct aead_request *req)
 {
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct chcr_aead_ctx *aeadctx = AEAD_CTX(a_ctx(tfm));
struct chcr_aead_reqctx  *reqctx = aead_request_ctx(req);
-   int error = -EINVAL;
unsigned int authsize = crypto_aead_authsize(tfm);
+   int error = -EINVAL;
 
/* validate key size */
if (aeadctx->enckey_len == 0)
goto err;
-   if (op_type && req->cryptlen < authsize)
+   if (reqctx->op && req->cryptlen < authsize)
goto err;
+   if (reqctx->b0_len)
+   reqctx->scratch_pad = reqctx->iv + IV;
+   else
+   reqctx->scratch_pad = NULL;
+
error = chcr_aead_dma_map(_CTX(a_ctx(tfm))->lldi.pdev->dev, req,
- op_type);
+ reqctx->op);
if (error) {
error = -ENOMEM;
goto err;
@@ -2230,7 +2238,7 @@ static int chcr_aead_fallback(struct aead_request *req, 
unsigned short op_type)
aead_request_set_tfm(subreq, aeadctx->sw_cipher);
aead_request_set_callback(subreq, req->base.flags,
  req->base.complete, req->base.data);
-aead_request_set_crypt(subreq, req->src, req->dst, req->cryptlen,
+   aead_request_set_crypt(subreq, req->src, req->dst, req->cryptlen,
 req->iv);
 aead_request_set_ad(subreq, req->assoclen);
return op_type ? crypto_aead_decrypt(subreq) :
@@ -2239,8 +2247,7 @@ static int chcr_aead_fallback(struct aead_request *req, 
unsigned short op_type)
 
 static struct sk_buff *create_authenc_wr(struct aead_request *req,
 unsigned short qid,
-int size,
-unsigned short op_type)
+int size)
 {
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct chcr_aead_ctx *aeadctx = AEAD_CTX(a_ctx(tfm));
@@ -2264,18 +2271,20 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
if (req->cryptlen == 0)
return NULL;
 
-   reqctx->b0_dma = 0;
+   reqctx->b0_len = 0;
+   error = chcr_aead_common_init(req);
+   if (error)
+   return ERR_PTR(error);
+
if (subtype == CRYPTO_ALG_SUB_TYPE_CBC_NULL ||
-   subtype == CRYPTO_ALG_SUB_TYPE_CTR_NULL) {
+   subtype == CRYPTO_ALG_SUB_TYPE_CTR_NULL) {
null = 1;
assoclen = 0;
+   reqctx->aad_nents = 0;
}
-   error = chcr_aead_common_init(req, op_type);
-   if (error)
-   return ERR_PTR(error);
dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
dnents += sg_nents_xlen(req->dst, req->cryptlen +
-   (op_type ? -authsize : authsize), CHCR_DST_SG_SIZE,
+   (reqctx->op ? -authsize : authsize), CHCR_DST_SG_SIZE,
 

Re: DMA map buffer allocated in ahash_request_ctx

[Harsh Jain]

On Wed, May 9, 2018 at 8:37 PM, Herbert Xu <herb...@gondor.apana.org.au> wrote:
> On Wed, May 09, 2018 at 04:13:12PM +0300, Gilad Ben-Yossef wrote:
>> On Wed, May 9, 2018 at 3:12 PM, Harsh Jain <harshjain.p...@gmail.com> wrote:
>> > Hi Herbert,
>> >
>> > Can we use buffer defined in ahash request context(private space for
>> > each request) to DMA map it to H/W.?
>>
>> Not Herbert but... the ccree driver is doing just and Herbert
>> specifically indicated ahash request contexts are not allowed to be
>> allocated from stack (I asked) so it should be OK.
>
> That's right.  For async algorithms the request must come from
> kmalloc memory.

Thanks Herbert/Gilad.

It means, It's perfectly fine to have array of required(For DMA)
memory in request context structure and DMA map it later on.

>
> Cheers,
> --
> Email: Herbert Xu <herb...@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

DMA map buffer allocated in ahash_request_ctx

[Harsh Jain]

Hi Herbert,

Can we use buffer defined in ahash request context(private space for
each request) to DMA map it to H/W.?

Regards
Harsh Jain

Re: [PATCH] crypto: chelsio: move chcr_ahash_continue declation out of header

[Harsh Jain]

Hi,

Fix for that is already submitted in below patch.

https://patchwork.kernel.org/patch/10292333/

Thanks


On 28-03-2018 19:35, Arnd Bergmann wrote:
> static function declarations don't belong in a header file, as shown
> by this compiler warning:
>
> In file included from /git/arm-soc/drivers/crypto/chelsio/chcr_ipsec.c:66:
> drivers/crypto/chelsio/chcr_crypto.h:343:12: error: 'chcr_ahash_continue' 
> declared 'static' but never defined [-Werror=unused-function]
>
> This moves the declaration into the file that actually needs it.
>
> Fixes: 5110e65536f3 ("crypto: chelsio -Split Hash requests for large scatter 
> gather list")
> Signed-off-by: Arnd Bergmann 
> ---
>  drivers/crypto/chelsio/chcr_algo.c   | 1 +
>  drivers/crypto/chelsio/chcr_crypto.h | 1 -
>  2 files changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/crypto/chelsio/chcr_algo.c 
> b/drivers/crypto/chelsio/chcr_algo.c
> index 4617c7acf4da..ebc96f2fae01 100644
> --- a/drivers/crypto/chelsio/chcr_algo.c
> +++ b/drivers/crypto/chelsio/chcr_algo.c
> @@ -95,6 +95,7 @@ static u32 round_constant[11] = {
>  
>  static int chcr_handle_cipher_resp(struct ablkcipher_request *req,
>  unsigned char *input, int err);
> +static int chcr_ahash_continue(struct ahash_request *req);
>  
>  static inline  struct chcr_aead_ctx *AEAD_CTX(struct chcr_context *ctx)
>  {
> diff --git a/drivers/crypto/chelsio/chcr_crypto.h 
> b/drivers/crypto/chelsio/chcr_crypto.h
> index 71025ea9c3db..c8e8972af283 100644
> --- a/drivers/crypto/chelsio/chcr_crypto.h
> +++ b/drivers/crypto/chelsio/chcr_crypto.h
> @@ -340,5 +340,4 @@ void chcr_add_hash_src_ent(struct ahash_request *req, 
> struct ulptx_sgl *ulptx,
>  struct hash_wr_param *param);
>  int chcr_hash_dma_map(struct device *dev, struct ahash_request *req);
>  void chcr_hash_dma_unmap(struct device *dev, struct ahash_request *req);
> -static int chcr_ahash_continue(struct ahash_request *req);
>  #endif /* __CHCR_CRYPTO_H__ */

[PATCH 1/1] crypto:chelsio - Remove declaration of static function from header

[Harsh Jain]

It fixes compilation warning introduced in commit

Introduced by commit

  5110e65536f3 ("crypto: chelsio -Split Hash requests for large scatter gather 
list")

Reported-by: Stephen Rothwell <s...@canb.auug.org.au>
Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 291 +--
 drivers/crypto/chelsio/chcr_crypto.h |   1 -
 2 files changed, 145 insertions(+), 147 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 4617c7a..752ed9b 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -165,60 +165,6 @@ static int sg_nents_xlen(struct scatterlist *sg, unsigned 
int reqlen,
return nents;
 }
 
-static inline void chcr_handle_ahash_resp(struct ahash_request *req,
- unsigned char *input,
- int err)
-{
-   struct chcr_ahash_req_ctx *reqctx = ahash_request_ctx(req);
-   struct chcr_hctx_per_wr *hctx_wr = >hctx_wr;
-   int digestsize, updated_digestsize;
-   struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
-   struct uld_ctx *u_ctx = ULD_CTX(h_ctx(tfm));
-
-   if (input == NULL)
-   goto out;
-   digestsize = crypto_ahash_digestsize(crypto_ahash_reqtfm(req));
-   updated_digestsize = digestsize;
-   if (digestsize == SHA224_DIGEST_SIZE)
-   updated_digestsize = SHA256_DIGEST_SIZE;
-   else if (digestsize == SHA384_DIGEST_SIZE)
-   updated_digestsize = SHA512_DIGEST_SIZE;
-
-   if (hctx_wr->dma_addr) {
-   dma_unmap_single(_ctx->lldi.pdev->dev, hctx_wr->dma_addr,
-hctx_wr->dma_len, DMA_TO_DEVICE);
-   hctx_wr->dma_addr = 0;
-   }
-   if (hctx_wr->isfinal || ((hctx_wr->processed + reqctx->reqlen) ==
-req->nbytes)) {
-   if (hctx_wr->result == 1) {
-   hctx_wr->result = 0;
-   memcpy(req->result, input + sizeof(struct cpl_fw6_pld),
-  digestsize);
-   } else {
-   memcpy(reqctx->partial_hash,
-  input + sizeof(struct cpl_fw6_pld),
-  updated_digestsize);
-
-   }
-   goto unmap;
-   }
-   memcpy(reqctx->partial_hash, input + sizeof(struct cpl_fw6_pld),
-  updated_digestsize);
-
-   err = chcr_ahash_continue(req);
-   if (err)
-   goto unmap;
-   return;
-unmap:
-   if (hctx_wr->is_sg_map)
-   chcr_hash_dma_unmap(_ctx->lldi.pdev->dev, req);
-
-
-out:
-   req->base.complete(>base, err);
-}
-
 static inline int get_aead_subtype(struct crypto_aead *aead)
 {
struct aead_alg *alg = crypto_aead_alg(aead);
@@ -271,34 +217,6 @@ static inline void chcr_handle_aead_resp(struct 
aead_request *req,
req->base.complete(>base, err);
 }
 
-/*
- * chcr_handle_resp - Unmap the DMA buffers associated with the request
- * @req: crypto request
- */
-int chcr_handle_resp(struct crypto_async_request *req, unsigned char *input,
-int err)
-{
-   struct crypto_tfm *tfm = req->tfm;
-   struct chcr_context *ctx = crypto_tfm_ctx(tfm);
-   struct adapter *adap = padap(ctx->dev);
-
-   switch (tfm->__crt_alg->cra_flags & CRYPTO_ALG_TYPE_MASK) {
-   case CRYPTO_ALG_TYPE_AEAD:
-   chcr_handle_aead_resp(aead_request_cast(req), input, err);
-   break;
-
-   case CRYPTO_ALG_TYPE_ABLKCIPHER:
-err = chcr_handle_cipher_resp(ablkcipher_request_cast(req),
-  input, err);
-   break;
-
-   case CRYPTO_ALG_TYPE_AHASH:
-   chcr_handle_ahash_resp(ahash_request_cast(req), input, err);
-   }
-   atomic_inc(>chcr_stats.complete);
-   return err;
-}
-
 static void get_aes_decrypt_key(unsigned char *dec_key,
   const unsigned char *key,
   unsigned int keylength)
@@ -1784,70 +1702,6 @@ static int chcr_ahash_final(struct ahash_request *req)
return -EINPROGRESS;
 }
 
-static int chcr_ahash_continue(struct ahash_request *req)
-{
-   struct chcr_ahash_req_ctx *reqctx = ahash_request_ctx(req);
-   struct chcr_hctx_per_wr *hctx_wr = >hctx_wr;
-   struct crypto_ahash *rtfm = crypto_ahash_reqtfm(req);
-   struct uld_ctx *u_ctx = NULL;
-   struct sk_buff *skb;
-   struct hash_wr_param params;
-   u8  bs;
-   int error;
-
-   bs = crypto_tfm_alg_blocksize(crypto_ahash_tfm(rtfm));
-   u_ctx = ULD_CTX(h_ctx(rtfm));
-   if (unlikely(cxgb4_is_crypto_q_full(u_ctx->lldi.p

Re: [PATCH v2 0/5] crypto:chelsio: Bug fixes and cleanup

[Harsh Jain]

On Tue, Mar 6, 2018 at 11:22 AM, Herbert Xu <herb...@gondor.apana.org.au> wrote:
> On Tue, Mar 06, 2018 at 10:37:47AM +0530, Harsh Jain wrote:
>> It includes bug fixes and code cleanup.
>>
>> Changes from v1:
>>
>> Remove Redundant soffset initialisation from 2/5.
>
> Hmm, the first series has already been applied.  So if you want
> to change it then you'll need to send patches based on that.
>
Sorry for confusion. This series is not applied yet. By mistake I
replied to wrong patch series to ignore.
> Thanks,
> --
> Email: Herbert Xu <herb...@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: [PATCH 0/5] crypto: chelsio - Cleanup and bug fixes

[Harsh Jain]

Hi Herbert,

Please ignore this series. V2 sent with minor change.


On 11-01-2018 16:45, Harsh Jain wrote:
> This series include cleanup, bug fixes and authenc algo supporting
>  ctr(aes)-sha operation.
>
> Harsh Jain (5):
>   crypto: chelsio - Fix Indentation
>   crypto: chelsio - check for sg null
>   crypto: chelsio - Fix IV updated in XTS operation
>   crypto: chelsio - Add authenc versions of ctr and sha
>   crypto: chelsio - Remove dst sg size zero check
>
>  drivers/crypto/chelsio/chcr_algo.c   | 299 
> ++-
>  drivers/crypto/chelsio/chcr_crypto.h |   7 +-
>  2 files changed, 233 insertions(+), 73 deletions(-)
>

[PATCH v2 1/5] crypto:chelsio: Use kernel round function to align lengths

[Harsh Jain]

Replace DIV_ROUND_UP to roundup or rounddown

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 73 ++
 drivers/crypto/chelsio/chcr_algo.h |  1 -
 2 files changed, 34 insertions(+), 40 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 8a67884..2bef618 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -784,14 +784,14 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
nents = sg_nents_xlen(reqctx->dstsg,  wrparam->bytes, CHCR_DST_SG_SIZE,
  reqctx->dst_ofst);
dst_size = get_space_for_phys_dsgl(nents + 1);
-   kctx_len = (DIV_ROUND_UP(ablkctx->enckey_len, 16) * 16);
+   kctx_len = roundup(ablkctx->enckey_len, 16);
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, dst_size);
nents = sg_nents_xlen(reqctx->srcsg, wrparam->bytes,
  CHCR_SRC_SG_SIZE, reqctx->src_ofst);
-   temp = reqctx->imm ? (DIV_ROUND_UP((IV + wrparam->req->nbytes), 16)
- * 16) : (sgl_len(nents + MIN_CIPHER_SG) * 8);
+   temp = reqctx->imm ? roundup(IV + wrparam->req->nbytes, 16) :
+(sgl_len(nents + MIN_CIPHER_SG) * 8);
transhdr_len += temp;
-   transhdr_len = DIV_ROUND_UP(transhdr_len, 16) * 16;
+   transhdr_len = roundup(transhdr_len, 16);
skb = alloc_skb(SGE_MAX_WR_LEN, flags);
if (!skb) {
error = -ENOMEM;
@@ -1148,7 +1148,7 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
if ((bytes + reqctx->processed) >= req->nbytes)
bytes  = req->nbytes - reqctx->processed;
else
-   bytes = ROUND_16(bytes);
+   bytes = rounddown(bytes, 16);
} else {
/*CTR mode counter overfloa*/
bytes  = req->nbytes - reqctx->processed;
@@ -1234,7 +1234,7 @@ static int process_cipher(struct ablkcipher_request *req,
   CHCR_DST_SG_SIZE, 0);
dnents += 1; // IV
phys_dsgl = get_space_for_phys_dsgl(dnents);
-   kctx_len = (DIV_ROUND_UP(ablkctx->enckey_len, 16) * 16);
+   kctx_len = roundup(ablkctx->enckey_len, 16);
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, phys_dsgl);
reqctx->imm = (transhdr_len + IV + req->nbytes) <=
SGE_MAX_WR_LEN;
@@ -1252,7 +1252,7 @@ static int process_cipher(struct ablkcipher_request *req,
if ((bytes + reqctx->processed) >= req->nbytes)
bytes  = req->nbytes - reqctx->processed;
else
-   bytes = ROUND_16(bytes);
+   bytes = rounddown(bytes, 16);
} else {
bytes = req->nbytes;
}
@@ -1526,10 +1526,10 @@ static struct sk_buff *create_hash_wr(struct 
ahash_request *req,
SGE_MAX_WR_LEN;
nents = sg_nents_xlen(req->src, param->sg_len, CHCR_SRC_SG_SIZE, 0);
nents += param->bfr_len ? 1 : 0;
-   transhdr_len += req_ctx->imm ? (DIV_ROUND_UP((param->bfr_len +
-   param->sg_len), 16) * 16) :
+   transhdr_len += req_ctx->imm ? roundup((param->bfr_len +
+   param->sg_len), 16) :
(sgl_len(nents) * 8);
-   transhdr_len = DIV_ROUND_UP(transhdr_len, 16) * 16;
+   transhdr_len = roundup(transhdr_len, 16);
 
skb = alloc_skb(SGE_MAX_WR_LEN, flags);
if (!skb)
@@ -2124,11 +2124,11 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, dst_size);
reqctx->imm = (transhdr_len + assoclen + IV + req->cryptlen) <
SGE_MAX_WR_LEN;
-   temp = reqctx->imm ? (DIV_ROUND_UP((assoclen + IV + req->cryptlen), 16)
-   * 16) : (sgl_len(reqctx->src_nents + reqctx->aad_nents
+   temp = reqctx->imm ? roundup(assoclen + IV + req->cryptlen, 16)
+   : (sgl_len(reqctx->src_nents + reqctx->aad_nents
+ MIN_GCM_SG) * 8);
transhdr_len += temp;
-   transhdr_len = DIV_ROUND_UP(transhdr_len, 16) * 16;
+   transhdr_len = roundup(transhdr_len, 16);
 
if (chcr_aead_need_fallback(req, dnents, T6_MAX_AAD_SIZE,
transhdr_len, op_type)) {
@@ -2187,9 +2187,8 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
memcpy(chcr_req->key_ctx.key, actx->dec_rrkey,
   aeadctx->enckey_len);
 
-   memcpy(chcr_req-&

[PATCH v2 3/5] crypto:chelsio: Update IV before sending request to HW

[Harsh Jain]

CBC Decryption requires Last Block as IV. In case src/dst buffer
are same last block will be replaced by plain text. This patch copies
the Last Block before sending request to HW.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index bc70f4f..002e0c2 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -854,6 +854,13 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
transhdr_len, temp,
ablkctx->ciph_mode == CHCR_SCMD_CIPHER_MODE_AES_CBC);
reqctx->skb = skb;
+
+   if (reqctx->op && (ablkctx->ciph_mode ==
+  CHCR_SCMD_CIPHER_MODE_AES_CBC))
+   sg_pcopy_to_buffer(wrparam->req->src,
+   sg_nents(wrparam->req->src), wrparam->req->info, 16,
+   reqctx->processed + wrparam->bytes - AES_BLOCK_SIZE);
+
return skb;
 err:
return ERR_PTR(error);
@@ -1077,9 +1084,8 @@ static int chcr_update_cipher_iv(struct 
ablkcipher_request *req,
ret = chcr_update_tweak(req, iv, 0);
else if (subtype == CRYPTO_ALG_SUB_TYPE_CBC) {
if (reqctx->op)
-   sg_pcopy_to_buffer(req->src, sg_nents(req->src), iv,
-  16,
-  reqctx->processed - AES_BLOCK_SIZE);
+   /*Updated before sending last WR*/
+   memcpy(iv, req->info, AES_BLOCK_SIZE);
else
memcpy(iv, _pld->data[2], AES_BLOCK_SIZE);
}
@@ -1107,11 +1113,8 @@ static int chcr_final_cipher_iv(struct 
ablkcipher_request *req,
else if (subtype == CRYPTO_ALG_SUB_TYPE_XTS)
ret = chcr_update_tweak(req, iv, 1);
else if (subtype == CRYPTO_ALG_SUB_TYPE_CBC) {
-   if (reqctx->op)
-   sg_pcopy_to_buffer(req->src, sg_nents(req->src), iv,
-  16,
-  reqctx->processed - AES_BLOCK_SIZE);
-   else
+   /*Already updated for Decrypt*/
+   if (!reqctx->op)
memcpy(iv, _pld->data[2], AES_BLOCK_SIZE);
 
}
-- 
2.1.4

[PATCH v2 4/5] crypto:chelsio: Fix iv passed in fallback path for rfc3686

[Harsh Jain]

We use ctr(aes) to fallback rfc3686(ctr) request. Send updated IV to fallback 
path.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 002e0c2..8507076 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1292,7 +1292,7 @@ static int process_cipher(struct ablkcipher_request *req,
   req->src,
   req->dst,
   req->nbytes,
-  req->info,
+  reqctx->iv,
   op_type);
goto error;
}
-- 
2.1.4

[PATCH v2 5/5] crypto:chelsio:Split Hash requests for large scatter gather list

[Harsh Jain]

Send multiple WRs to H/W when No. of entries received in scatter list
cannot be sent in single request.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 358 ++-
 drivers/crypto/chelsio/chcr_algo.h   |  10 +-
 drivers/crypto/chelsio/chcr_core.h   |   6 +-
 drivers/crypto/chelsio/chcr_crypto.h |  32 +++-
 4 files changed, 298 insertions(+), 108 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 8507076..4617c7a 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -131,6 +131,11 @@ static inline int is_ofld_imm(const struct sk_buff *skb)
return (skb->len <= SGE_MAX_WR_LEN);
 }
 
+static inline void chcr_init_hctx_per_wr(struct chcr_ahash_req_ctx *reqctx)
+{
+   memset(>hctx_wr, 0, sizeof(struct chcr_hctx_per_wr));
+}
+
 static int sg_nents_xlen(struct scatterlist *sg, unsigned int reqlen,
 unsigned int entlen,
 unsigned int skip)
@@ -165,6 +170,7 @@ static inline void chcr_handle_ahash_resp(struct 
ahash_request *req,
  int err)
 {
struct chcr_ahash_req_ctx *reqctx = ahash_request_ctx(req);
+   struct chcr_hctx_per_wr *hctx_wr = >hctx_wr;
int digestsize, updated_digestsize;
struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
struct uld_ctx *u_ctx = ULD_CTX(h_ctx(tfm));
@@ -172,25 +178,43 @@ static inline void chcr_handle_ahash_resp(struct 
ahash_request *req,
if (input == NULL)
goto out;
digestsize = crypto_ahash_digestsize(crypto_ahash_reqtfm(req));
-   if (reqctx->is_sg_map)
-   chcr_hash_dma_unmap(_ctx->lldi.pdev->dev, req);
-   if (reqctx->dma_addr)
-   dma_unmap_single(_ctx->lldi.pdev->dev, reqctx->dma_addr,
-reqctx->dma_len, DMA_TO_DEVICE);
-   reqctx->dma_addr = 0;
updated_digestsize = digestsize;
if (digestsize == SHA224_DIGEST_SIZE)
updated_digestsize = SHA256_DIGEST_SIZE;
else if (digestsize == SHA384_DIGEST_SIZE)
updated_digestsize = SHA512_DIGEST_SIZE;
-   if (reqctx->result == 1) {
-   reqctx->result = 0;
-   memcpy(req->result, input + sizeof(struct cpl_fw6_pld),
-  digestsize);
-   } else {
-   memcpy(reqctx->partial_hash, input + sizeof(struct cpl_fw6_pld),
-  updated_digestsize);
+
+   if (hctx_wr->dma_addr) {
+   dma_unmap_single(_ctx->lldi.pdev->dev, hctx_wr->dma_addr,
+hctx_wr->dma_len, DMA_TO_DEVICE);
+   hctx_wr->dma_addr = 0;
+   }
+   if (hctx_wr->isfinal || ((hctx_wr->processed + reqctx->reqlen) ==
+req->nbytes)) {
+   if (hctx_wr->result == 1) {
+   hctx_wr->result = 0;
+   memcpy(req->result, input + sizeof(struct cpl_fw6_pld),
+  digestsize);
+   } else {
+   memcpy(reqctx->partial_hash,
+  input + sizeof(struct cpl_fw6_pld),
+  updated_digestsize);
+
+   }
+   goto unmap;
}
+   memcpy(reqctx->partial_hash, input + sizeof(struct cpl_fw6_pld),
+  updated_digestsize);
+
+   err = chcr_ahash_continue(req);
+   if (err)
+   goto unmap;
+   return;
+unmap:
+   if (hctx_wr->is_sg_map)
+   chcr_hash_dma_unmap(_ctx->lldi.pdev->dev, req);
+
+
 out:
req->base.complete(>base, err);
 }
@@ -563,7 +587,6 @@ static void  ulptx_walk_add_sg(struct ulptx_walk *walk,
 
if (!len)
return;
-
while (sg && skip) {
if (sg_dma_len(sg) <= skip) {
skip -= sg_dma_len(sg);
@@ -653,6 +676,35 @@ static int generate_copy_rrkey(struct ablk_ctx *ablkctx,
}
return 0;
 }
+
+static int chcr_hash_ent_in_wr(struct scatterlist *src,
+unsigned int minsg,
+unsigned int space,
+unsigned int srcskip)
+{
+   int srclen = 0;
+   int srcsg = minsg;
+   int soffset = 0, sless;
+
+   if (sg_dma_len(src) == srcskip) {
+   src = sg_next(src);
+   srcskip = 0;
+   }
+   while (src && space > (sgl_ent_len[srcsg + 1])) {
+   sless = min_t(unsigned int, sg_dma_len(src) - soffset - srcskip,
+   CHCR_SRC_SG_SIZE);
+   srclen += sless;
+   soffset += sless;
+   srcsg++;
+ 

[PATCH v2 2/5] crypto:chelsio: Fix src buffer dma length

[Harsh Jain]

ulptx header cannot have length > 64k. Adjust length accordingly.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 17 -
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 2bef618..bc70f4f 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -662,7 +662,7 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
 {
int srclen = 0, dstlen = 0;
int srcsg = minsg, dstsg = minsg;
-   int offset = 0, less;
+   int offset = 0, soffset = 0, less, sless = 0;
 
if (sg_dma_len(src) == srcskip) {
src = sg_next(src);
@@ -676,7 +676,9 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
 
while (src && dst &&
   space > (sgl_ent_len[srcsg + 1] + dsgl_ent_len[dstsg])) {
-   srclen += (sg_dma_len(src) - srcskip);
+   sless = min_t(unsigned int, sg_dma_len(src) - srcskip - soffset,
+   CHCR_SRC_SG_SIZE);
+   srclen += sless;
srcsg++;
offset = 0;
while (dst && ((dstsg + 1) <= MAX_DSGL_ENT) &&
@@ -687,15 +689,20 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
 dstskip, CHCR_DST_SG_SIZE);
dstlen += less;
offset += less;
-   if (offset == sg_dma_len(dst)) {
+   if ((offset + dstskip) == sg_dma_len(dst)) {
dst = sg_next(dst);
offset = 0;
}
dstsg++;
dstskip = 0;
}
-   src = sg_next(src);
-   srcskip = 0;
+   soffset += sless;
+   if ((soffset + srcskip) == sg_dma_len(src)) {
+   src = sg_next(src);
+   srcskip = 0;
+   soffset = 0;
+   }
+
}
return min(srclen, dstlen);
 }
-- 
2.1.4

[PATCH v2 0/5] crypto:chelsio: Bug fixes and cleanup

[Harsh Jain]

It includes bug fixes and code cleanup.

Changes from v1:

Remove Redundant soffset initialisation from 2/5.

Harsh Jain (5):
  crypto:chelsio: Use kernel round function to align lengths
  crypto:chelsio: Fix src buffer dma length
  crypto:chelsio: Update IV before sending request to HW
  crypto:chelsio: Fix iv passed in fallback path for rfc3686
  crypto:chelsio:Split Hash requests for large scatter gather list

 drivers/crypto/chelsio/chcr_algo.c   | 465 ---
 drivers/crypto/chelsio/chcr_algo.h   |  11 +-
 drivers/crypto/chelsio/chcr_core.h   |   6 +-
 drivers/crypto/chelsio/chcr_crypto.h |  32 ++-
 4 files changed, 354 insertions(+), 160 deletions(-)

-- 
2.1.4

Re: error in libkcapi 1.0.3 for aead aio

[Harsh Jain]

On 28-02-2018 14:28, Stephan Mueller wrote:
> Am Mittwoch, 28. Februar 2018, 08:34:21 CET schrieb Harsh Jain:
>
> Hi Harsh,
>
>> Try with gdb. AIO(-x 10) works fine with step by step debugging.  Also
>> verified by adding print in "af_alg_async_cb" for received err value. 
>> Driver is sending -74 here.
> I am unable to reproduce the issue. Both commands with sync and async 
> operation work fine with me and using 1.0.3.
Ok. I will try to root cause it or will get some debug info.
>
> Ciao
> Stephan
>
>

[PATCH 4/5] crypto:chelsio: Fix iv passed in fallback path for rfc3686

[Harsh Jain]

We use ctr(aes) to fallback rfc3686(ctr) request. Send updated IV to fallback 
path.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index f9c1970..3c3ca34 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1292,7 +1292,7 @@ static int process_cipher(struct ablkcipher_request *req,
   req->src,
   req->dst,
   req->nbytes,
-  req->info,
+  reqctx->iv,
   op_type);
goto error;
}
-- 
2.1.4

[PATCH 3/5] crypto:chelsio: Update IV before sending request to HW

[Harsh Jain]

CBC Decryption requires Last Block as IV. In case src/dst buffer
are same last block will be replaced by plain text. This patch copies
the Last Block before sending request to HW.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 33f7b90..f9c1970 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -854,6 +854,13 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
transhdr_len, temp,
ablkctx->ciph_mode == CHCR_SCMD_CIPHER_MODE_AES_CBC);
reqctx->skb = skb;
+
+   if (reqctx->op && (ablkctx->ciph_mode ==
+  CHCR_SCMD_CIPHER_MODE_AES_CBC))
+   sg_pcopy_to_buffer(wrparam->req->src,
+   sg_nents(wrparam->req->src), wrparam->req->info, 16,
+   reqctx->processed + wrparam->bytes - AES_BLOCK_SIZE);
+
return skb;
 err:
return ERR_PTR(error);
@@ -1077,9 +1084,8 @@ static int chcr_update_cipher_iv(struct 
ablkcipher_request *req,
ret = chcr_update_tweak(req, iv, 0);
else if (subtype == CRYPTO_ALG_SUB_TYPE_CBC) {
if (reqctx->op)
-   sg_pcopy_to_buffer(req->src, sg_nents(req->src), iv,
-  16,
-  reqctx->processed - AES_BLOCK_SIZE);
+   /*Updated before sending last WR*/
+   memcpy(iv, req->info, AES_BLOCK_SIZE);
else
memcpy(iv, _pld->data[2], AES_BLOCK_SIZE);
}
@@ -1107,11 +1113,8 @@ static int chcr_final_cipher_iv(struct 
ablkcipher_request *req,
else if (subtype == CRYPTO_ALG_SUB_TYPE_XTS)
ret = chcr_update_tweak(req, iv, 1);
else if (subtype == CRYPTO_ALG_SUB_TYPE_CBC) {
-   if (reqctx->op)
-   sg_pcopy_to_buffer(req->src, sg_nents(req->src), iv,
-  16,
-  reqctx->processed - AES_BLOCK_SIZE);
-   else
+   /*Already updated for Decrypt*/
+   if (!reqctx->op)
memcpy(iv, _pld->data[2], AES_BLOCK_SIZE);
 
}
-- 
2.1.4

[PATCH 5/5] crypto:chelsio:Split Hash requests for large scatter gather list

[Harsh Jain]

Send multiple WRs to H/W when No. of entries received in scatter list
cannot be sent in single request.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 358 ++-
 drivers/crypto/chelsio/chcr_algo.h   |  10 +-
 drivers/crypto/chelsio/chcr_core.h   |   6 +-
 drivers/crypto/chelsio/chcr_crypto.h |  32 +++-
 4 files changed, 298 insertions(+), 108 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 3c3ca34..9db1cca 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -131,6 +131,11 @@ static inline int is_ofld_imm(const struct sk_buff *skb)
return (skb->len <= SGE_MAX_WR_LEN);
 }
 
+static inline void chcr_init_hctx_per_wr(struct chcr_ahash_req_ctx *reqctx)
+{
+   memset(>hctx_wr, 0, sizeof(struct chcr_hctx_per_wr));
+}
+
 static int sg_nents_xlen(struct scatterlist *sg, unsigned int reqlen,
 unsigned int entlen,
 unsigned int skip)
@@ -165,6 +170,7 @@ static inline void chcr_handle_ahash_resp(struct 
ahash_request *req,
  int err)
 {
struct chcr_ahash_req_ctx *reqctx = ahash_request_ctx(req);
+   struct chcr_hctx_per_wr *hctx_wr = >hctx_wr;
int digestsize, updated_digestsize;
struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
struct uld_ctx *u_ctx = ULD_CTX(h_ctx(tfm));
@@ -172,25 +178,43 @@ static inline void chcr_handle_ahash_resp(struct 
ahash_request *req,
if (input == NULL)
goto out;
digestsize = crypto_ahash_digestsize(crypto_ahash_reqtfm(req));
-   if (reqctx->is_sg_map)
-   chcr_hash_dma_unmap(_ctx->lldi.pdev->dev, req);
-   if (reqctx->dma_addr)
-   dma_unmap_single(_ctx->lldi.pdev->dev, reqctx->dma_addr,
-reqctx->dma_len, DMA_TO_DEVICE);
-   reqctx->dma_addr = 0;
updated_digestsize = digestsize;
if (digestsize == SHA224_DIGEST_SIZE)
updated_digestsize = SHA256_DIGEST_SIZE;
else if (digestsize == SHA384_DIGEST_SIZE)
updated_digestsize = SHA512_DIGEST_SIZE;
-   if (reqctx->result == 1) {
-   reqctx->result = 0;
-   memcpy(req->result, input + sizeof(struct cpl_fw6_pld),
-  digestsize);
-   } else {
-   memcpy(reqctx->partial_hash, input + sizeof(struct cpl_fw6_pld),
-  updated_digestsize);
+
+   if (hctx_wr->dma_addr) {
+   dma_unmap_single(_ctx->lldi.pdev->dev, hctx_wr->dma_addr,
+hctx_wr->dma_len, DMA_TO_DEVICE);
+   hctx_wr->dma_addr = 0;
+   }
+   if (hctx_wr->isfinal || ((hctx_wr->processed + reqctx->reqlen) ==
+req->nbytes)) {
+   if (hctx_wr->result == 1) {
+   hctx_wr->result = 0;
+   memcpy(req->result, input + sizeof(struct cpl_fw6_pld),
+  digestsize);
+   } else {
+   memcpy(reqctx->partial_hash,
+  input + sizeof(struct cpl_fw6_pld),
+  updated_digestsize);
+
+   }
+   goto unmap;
}
+   memcpy(reqctx->partial_hash, input + sizeof(struct cpl_fw6_pld),
+  updated_digestsize);
+
+   err = chcr_ahash_continue(req);
+   if (err)
+   goto unmap;
+   return;
+unmap:
+   if (hctx_wr->is_sg_map)
+   chcr_hash_dma_unmap(_ctx->lldi.pdev->dev, req);
+
+
 out:
req->base.complete(>base, err);
 }
@@ -563,7 +587,6 @@ static void  ulptx_walk_add_sg(struct ulptx_walk *walk,
 
if (!len)
return;
-
while (sg && skip) {
if (sg_dma_len(sg) <= skip) {
skip -= sg_dma_len(sg);
@@ -653,6 +676,35 @@ static int generate_copy_rrkey(struct ablk_ctx *ablkctx,
}
return 0;
 }
+
+static int chcr_hash_ent_in_wr(struct scatterlist *src,
+unsigned int minsg,
+unsigned int space,
+unsigned int srcskip)
+{
+   int srclen = 0;
+   int srcsg = minsg;
+   int soffset = 0, sless;
+
+   if (sg_dma_len(src) == srcskip) {
+   src = sg_next(src);
+   srcskip = 0;
+   }
+   while (src && space > (sgl_ent_len[srcsg + 1])) {
+   sless = min_t(unsigned int, sg_dma_len(src) - soffset - srcskip,
+   CHCR_SRC_SG_SIZE);
+   srclen += sless;
+   soffset += sless;
+   srcsg++;
+ 

[PATCH 2/5] crypto:chelsio: Fix src buffer dma length

[Harsh Jain]

ulptx header cannot have length > 64k. Adjust length accordingly.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 19 +--
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 2bef618..33f7b90 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -662,7 +662,7 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
 {
int srclen = 0, dstlen = 0;
int srcsg = minsg, dstsg = minsg;
-   int offset = 0, less;
+   int offset = 0, soffset = 0, less, sless = 0;
 
if (sg_dma_len(src) == srcskip) {
src = sg_next(src);
@@ -673,10 +673,12 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
dst = sg_next(dst);
dstskip = 0;
}
-
+   soffset = 0;
while (src && dst &&
   space > (sgl_ent_len[srcsg + 1] + dsgl_ent_len[dstsg])) {
-   srclen += (sg_dma_len(src) - srcskip);
+   sless = min_t(unsigned int, sg_dma_len(src) - srcskip - soffset,
+   CHCR_SRC_SG_SIZE);
+   srclen += sless;
srcsg++;
offset = 0;
while (dst && ((dstsg + 1) <= MAX_DSGL_ENT) &&
@@ -687,15 +689,20 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
 dstskip, CHCR_DST_SG_SIZE);
dstlen += less;
offset += less;
-   if (offset == sg_dma_len(dst)) {
+   if ((offset + dstskip) == sg_dma_len(dst)) {
dst = sg_next(dst);
offset = 0;
}
dstsg++;
dstskip = 0;
}
-   src = sg_next(src);
-   srcskip = 0;
+   soffset += sless;
+   if ((soffset + srcskip) == sg_dma_len(src)) {
+   src = sg_next(src);
+   srcskip = 0;
+   soffset = 0;
+   }
+
}
return min(srclen, dstlen);
 }
-- 
2.1.4

[PATCH 1/5] crypto:chelsio: Use kernel round function to align lengths

[Harsh Jain]

Replace DIV_ROUND_UP to roundup or rounddown

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 73 ++
 drivers/crypto/chelsio/chcr_algo.h |  1 -
 2 files changed, 34 insertions(+), 40 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 8a67884..2bef618 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -784,14 +784,14 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
nents = sg_nents_xlen(reqctx->dstsg,  wrparam->bytes, CHCR_DST_SG_SIZE,
  reqctx->dst_ofst);
dst_size = get_space_for_phys_dsgl(nents + 1);
-   kctx_len = (DIV_ROUND_UP(ablkctx->enckey_len, 16) * 16);
+   kctx_len = roundup(ablkctx->enckey_len, 16);
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, dst_size);
nents = sg_nents_xlen(reqctx->srcsg, wrparam->bytes,
  CHCR_SRC_SG_SIZE, reqctx->src_ofst);
-   temp = reqctx->imm ? (DIV_ROUND_UP((IV + wrparam->req->nbytes), 16)
- * 16) : (sgl_len(nents + MIN_CIPHER_SG) * 8);
+   temp = reqctx->imm ? roundup(IV + wrparam->req->nbytes, 16) :
+(sgl_len(nents + MIN_CIPHER_SG) * 8);
transhdr_len += temp;
-   transhdr_len = DIV_ROUND_UP(transhdr_len, 16) * 16;
+   transhdr_len = roundup(transhdr_len, 16);
skb = alloc_skb(SGE_MAX_WR_LEN, flags);
if (!skb) {
error = -ENOMEM;
@@ -1148,7 +1148,7 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
if ((bytes + reqctx->processed) >= req->nbytes)
bytes  = req->nbytes - reqctx->processed;
else
-   bytes = ROUND_16(bytes);
+   bytes = rounddown(bytes, 16);
} else {
/*CTR mode counter overfloa*/
bytes  = req->nbytes - reqctx->processed;
@@ -1234,7 +1234,7 @@ static int process_cipher(struct ablkcipher_request *req,
   CHCR_DST_SG_SIZE, 0);
dnents += 1; // IV
phys_dsgl = get_space_for_phys_dsgl(dnents);
-   kctx_len = (DIV_ROUND_UP(ablkctx->enckey_len, 16) * 16);
+   kctx_len = roundup(ablkctx->enckey_len, 16);
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, phys_dsgl);
reqctx->imm = (transhdr_len + IV + req->nbytes) <=
SGE_MAX_WR_LEN;
@@ -1252,7 +1252,7 @@ static int process_cipher(struct ablkcipher_request *req,
if ((bytes + reqctx->processed) >= req->nbytes)
bytes  = req->nbytes - reqctx->processed;
else
-   bytes = ROUND_16(bytes);
+   bytes = rounddown(bytes, 16);
} else {
bytes = req->nbytes;
}
@@ -1526,10 +1526,10 @@ static struct sk_buff *create_hash_wr(struct 
ahash_request *req,
SGE_MAX_WR_LEN;
nents = sg_nents_xlen(req->src, param->sg_len, CHCR_SRC_SG_SIZE, 0);
nents += param->bfr_len ? 1 : 0;
-   transhdr_len += req_ctx->imm ? (DIV_ROUND_UP((param->bfr_len +
-   param->sg_len), 16) * 16) :
+   transhdr_len += req_ctx->imm ? roundup((param->bfr_len +
+   param->sg_len), 16) :
(sgl_len(nents) * 8);
-   transhdr_len = DIV_ROUND_UP(transhdr_len, 16) * 16;
+   transhdr_len = roundup(transhdr_len, 16);
 
skb = alloc_skb(SGE_MAX_WR_LEN, flags);
if (!skb)
@@ -2124,11 +2124,11 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, dst_size);
reqctx->imm = (transhdr_len + assoclen + IV + req->cryptlen) <
SGE_MAX_WR_LEN;
-   temp = reqctx->imm ? (DIV_ROUND_UP((assoclen + IV + req->cryptlen), 16)
-   * 16) : (sgl_len(reqctx->src_nents + reqctx->aad_nents
+   temp = reqctx->imm ? roundup(assoclen + IV + req->cryptlen, 16)
+   : (sgl_len(reqctx->src_nents + reqctx->aad_nents
+ MIN_GCM_SG) * 8);
transhdr_len += temp;
-   transhdr_len = DIV_ROUND_UP(transhdr_len, 16) * 16;
+   transhdr_len = roundup(transhdr_len, 16);
 
if (chcr_aead_need_fallback(req, dnents, T6_MAX_AAD_SIZE,
transhdr_len, op_type)) {
@@ -2187,9 +2187,8 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
memcpy(chcr_req->key_ctx.key, actx->dec_rrkey,
   aeadctx->enckey_len);
 
-   memcpy(chcr_req-&

[PATCH 0/5] crypto:chelsio: Bug fixes and cleanup

[Harsh Jain]

It includes bug fixes and code cleanup.

Harsh Jain (5):
  crypto:chelsio: Use kernel round function to align lengths
  crypto:chelsio: Fix src buffer dma length
  crypto:chelsio: Update IV before sending request ot HW
  crypto:chelsio: Fix iv passed in fallback path for rfc3686
  crypto:chelsio:Split Hash requests for large scatter gather list

 drivers/crypto/chelsio/chcr_algo.c   | 467 ---
 drivers/crypto/chelsio/chcr_algo.h   |  11 +-
 drivers/crypto/chelsio/chcr_core.h   |   6 +-
 drivers/crypto/chelsio/chcr_crypto.h |  32 ++-
 4 files changed, 355 insertions(+), 161 deletions(-)

-- 
2.1.4

error in libkcapi 1.0.3 for aead aio

[Harsh Jain]

Hi Stephan,

1 of the test mentioned in test.sh is failing for AEAD AIO operation even 
thought driver is returning EBADMSG(as expected) to af_alg with latest 
cryptodev tree.

Debug log and strace attached.

Command :

strace -o strace.log ../bin/kcapi   -x 10   -c "gcm(aes)" -i 
7815d4b06ae50c9c56e87bd7 -k ea38ac0c9b9998c80e28fb496a2b88d9 -a 
"853f98a750098bec1aa7497e979e78098155c877879556bb51ddeb6374cbaefc" -t 
"c4ce58985b7203094be1d134c1b8ab0b" -q "b03692f86d1b8b39baf2abb255197c98"

Thanks & Regards

Harsh Jain

execve("../bin/kcapi", ["../bin/kcapi", "-x", "10", "-c", "gcm(aes)", "-i", 
"7815d4b06ae50c9c56e87bd7", "-k", "ea38ac0c9b9998c80e28fb496a2b88d9", "-a", 
"853f98a750098bec1aa7497e979e7809"..., "-t", 
"c4ce58985b7203094be1d134c1b8ab0b", "-q", "b03692f86d1b8b39baf2abb255197c98"], 
[/* 22 vars */]) = 0
brk(0)  = 0x12a1000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7f1f6471f000
access("/etc/ld.so.preload", R_OK)  = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=86613, ...}) = 0
mmap(NULL, 86613, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f1f64709000
close(3)= 0
open("/lib64/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\316\0\0\0\0\0\0"..., 
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=174528, ...}) = 0
mmap(NULL, 2268928, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 
0x7f1f642d7000
mprotect(0x7f1f642fc000, 2097152, PROT_NONE) = 0
mmap(0x7f1f644fc000, 20480, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x25000) = 0x7f1f644fc000
close(3)= 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\16\0\0\0\0\0\0"..., 
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=19776, ...}) = 0
mmap(NULL, 2109744, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 
0x7f1f640d3000
mprotect(0x7f1f640d5000, 2097152, PROT_NONE) = 0
mmap(0x7f1f642d5000, 8192, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f1f642d5000
close(3)= 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\34\2\0\0\0\0\0"..., 
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=2118128, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7f1f64708000
mmap(NULL, 3932672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 
0x7f1f63d12000
mprotect(0x7f1f63ec8000, 2097152, PROT_NONE) = 0
mmap(0x7f1f640c8000, 24576, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b6000) = 0x7f1f640c8000
mmap(0x7f1f640ce000, 16896, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f1f640ce000
close(3)= 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7f1f64706000
arch_prctl(ARCH_SET_FS, 0x7f1f64706740) = 0
mprotect(0x7f1f640c8000, 16384, PROT_READ) = 0
mprotect(0x7f1f642d5000, 4096, PROT_READ) = 0
mprotect(0x7f1f644fc000, 16384, PROT_READ) = 0
mprotect(0x6dc000, 4096, PROT_READ) = 0
mprotect(0x7f1f6472, 4096, PROT_READ) = 0
munmap(0x7f1f64709000, 86613)   = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
open("/dev/tty", O_RDWR|O_NONBLOCK) = 3
close(3)= 0
brk(0)  = 0x12a1000
brk(0x12c2000)  = 0x12c2000
brk(0)  = 0x12c2000
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=106070960, ...}) = 0
mmap(NULL, 106070960, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f1f5d7e9000
close(3)= 0
brk(0)  = 0x12c2000
getuid()= 0
getgid()= 0
geteuid()   = 0
getegid()   = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
open("/proc/meminfo", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7f1f6471e000
read(3, "MemTotal:   16415288 kB\nMemF"..., 1024) = 1024
close(3)= 0
munmap(0x7f1f6471e000, 4096)= 0
rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER|SA_RESTART, 0x7f1f63d47250}, 
{SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTOR

Re: [PATCH v3 1/4] crypto: AF_ALG AIO - lock context IV

[Harsh Jain]

On 15-02-2018 17:15, Stephan Mueller wrote:
> Am Donnerstag, 15. Februar 2018, 12:38:12 CET schrieb Harsh Jain:
>
> Hi Harsh,
>
>> On 15-02-2018 12:47, Stephan Mueller wrote:
>>> Am Donnerstag, 15. Februar 2018, 08:03:20 CET schrieb Harsh Jain:
>>>
>>> Hi Harsh,
>>>
>>>> Even after guarantee of serialization, In the end we will get wrong
>>>> result
>>>> as mentioned above. which destination side cannot decrypt it. What I feel
>>>> is scenario of sending 2 of more IOCB in case of AEAD itself is wrong.
>>> Without the inline IV handling, I would concur.
>> Even with Inline IV, We will have 2 Auth Tag. can we authenticate the data
>> with 2 Auth Tags?
> The AAD and the tag are both sent to the kernel like in the inline IV case as 
> part of the data via sendmsg.
>
> Thus, when you use inline IV support, an entire self-contained AEAD cipher 
> operation can be defined with one IOCB. Thus, if you have multiple IOCBs, 
> inline IV allow fully parallel execution of different AEAD requests.
>
> See the following kernel comment:
>
> /*
>  * Encryption operation - The in-place cipher operation is
>  * achieved by the following operation:
>  *
>  * TX SGL: AAD || PT
>  *  |  |
>  *  | copy |
>  *  v  v
>  * RX SGL: AAD || PT || Tag
>
> /*
>  * Decryption operation - To achieve an in-place cipher
>  * operation, the following  SGL structure is used:
>  *
>  * TX SGL: AAD || CT || Tag
>  *  |  | ^
>  *  | copy | | Create SGL link.
>  *  v  v |
>  * RX SGL: AAD || CT +
>  */
>
> Note, the TX SGL is what the caller submitted via sendmsg and the RX SGL is 
> the data the caller obtains via recvmsg.
>
> Hence, in inline IV support, the caller sends the following:
>
> encryption: IV || AAD || PT
>
> decryption: IV || AAD || CT || Tag
>
>>>> We
>>>> should not allow this type of requests for AEAD.
>>> "Not allow" as in "technically block"? As a user would only shoot itself
>>> when he does that not knowing the consequences, I am not in favor of such
>>> an artificial block.
>> Agreed, It may not be right thing to do, but if we allowed it, What he will
>> do with Auth( each processed with final Block) tags received in each
>> request.
> Each tag is returned as part of the recvmsg data. Thus, AEAD cipher 
> operations 
> can commence fully parallel if inline IV handling is used.
>> I personally feels AEAD IV serialization logic is incomplete without partial
>> tag support.
> Could you please elaborate what you mean with "partial tag" support?
Here is the catch, Calculation of tag depends on total payload length atleast 
for shaX, gcm,ccm mode on which I have worked.

If we take an example of shaX. It appends 1 special block at the end of user 
data which includes total input length in bit. Refer "sha1_base_do_finalize"
Suppose we have 32 byte and we break this in 2 IOCB of 16 bytes each.
Expected result : 32 encrypted bytes + sha auth tag considering length 32 bytes.
What we will  get : 16 bytes + sha auth tag considering length 16 bytes + 16 
encrypted bytes + another sha tag considering 16 bytes.



>
> Ciao
> Stephan
>
>

Re: [PATCH v3 1/4] crypto: AF_ALG AIO - lock context IV

[Harsh Jain]

On 15-02-2018 12:47, Stephan Mueller wrote:
> Am Donnerstag, 15. Februar 2018, 08:03:20 CET schrieb Harsh Jain:
>
> Hi Harsh,
>
>> Even after guarantee of serialization, In the end we will get wrong result
>> as mentioned above. which destination side cannot decrypt it. What I feel
>> is scenario of sending 2 of more IOCB in case of AEAD itself is wrong.
> Without the inline IV handling, I would concur.
Even with Inline IV, We will have 2 Auth Tag. can we authenticate the data with 
2 Auth Tags?
>
>> We
>> should not allow this type of requests for AEAD.
> "Not allow" as in "technically block"? As a user would only shoot itself when 
> he does that not knowing the consequences, I am not in favor of such an 
> artificial block.
Agreed, It may not be right thing to do, but if we allowed it, What he will do 
with Auth( each processed with final Block) tags received in each request.

I personally feels AEAD IV serialization logic is incomplete without partial 
tag support.

>> Can you think of any use
>> case it is going to solve?
> Well, I could fathom a use case of this. In FIPS 140-2 (yes, a term not well 
> received by some here), NIST insists for GCM that the IV is handled by the 
> cryptographic implementation.
>
> So, when using GCM for TLS, for example, the GCM implementation would know a 
> bit about how the IV is updated as a session ID. I.e. after the end of one 
> AEAD operation, the IV is written back but modified such to comply with the 
> rules of some higher level proto. Thus, if such a scenarios is implemented by 
> a driver here, multiple IOCBs could be used with such "TLSified" GCM, for 
> example.
>
> And such "TLSification" could be as simple as implementing an IV generator 
> that can be used with every (AEAD) cipher implementation.
>
>> Can receiver decrypt(with 2 IOCB) the same request successfully without
>> knowing  sender has done the operation in 2 request with size "x" each?
>>> Ciao
>>> Stephan
>
>
> Ciao
> Stephan
>
>

Re: [PATCH v3 1/4] crypto: AF_ALG AIO - lock context IV

[Harsh Jain]

On 15-02-2018 11:58, Stephan Mueller wrote:
> Am Donnerstag, 15. Februar 2018, 06:30:36 CET schrieb Harsh Jain:
>
> Hi Harsh,
>
>> On 14-02-2018 18:22, Stephan Mueller wrote:
>>> Am Mittwoch, 14. Februar 2018, 06:43:53 CET schrieb Harsh Jain:
>>>
>>> Hi Harsh,
>>>
>>>> Patch set is working fine with chelsio Driver.
>>> Thank you.
>>>
>>>> Do we really need IV locking mechanism for AEAD algo because AEAD algo's
>>>> don't support Partial mode operation and Driver are not updating(atleast
>>>> Chelsio) IV's on AEAD request completions.
>>> Yes, I think we would need it. It is technically possible to have multiple
>>> IOCBs for AEAD ciphers. Even though your implementation may not write the
>>> IV back, others may do that. At least I do not see a guarantee that the
>>> IV is *not* written back by a driver.
>> There is no  use of writing IV back in AEAD algo till Framework starts
>> supporting Partial mode.
> I agree.
>
>> Even if Driver starts updating IV for AEAD,
>> Multiple IOCB's in both cases will yield wrong results only.
> This would only be the case if the driver would not implicitly or explicitly 
> serialize the requests.
>> Case 1 : If we have AEAD IV serialization  applied,  Encryption will be
>> wrong if same IV gets used.
> Agreed.
>
>> Case 2: If we do not have IV serialization for
>> AEAD. Encryption will be fine but user will have multiple Authentication 
>> tag (that too with final block processed).  Its like 2nd Block encryption
>> is based on IV received from 1st block  and Authentication Tag value is
>> based on 2nd block content only.
> Agreed.
>
> But are we sure that all drivers behave correctly? Before you notified us of 
> the issue, I was not even aware of the fact that this serialization may not 
> be 
> done in the driver. And we only have seen that issue with AF_ALG where we 
> test 
> for multiple concurrent AIO operations.
I am sure other H/W will have similar problem, It's just that we tested it 
first.

>
> Besides, when we do not have the locking for AEAD, what would we gain: one 
> less lock to take vs. guarantee that the AEAD operation is always properly 
> serialized.
Even after guarantee of serialization, In the end we will get wrong result as 
mentioned above. which destination side cannot decrypt it.
What I feel is scenario of sending 2 of more IOCB in case of AEAD itself is 
wrong.  We should not allow this type of requests for AEAD.
Can you think of any use case it is going to solve?
Can receiver decrypt(with 2 IOCB) the same request successfully without knowing 
 sender has done the operation in 2 request with size "x" each?
>
> Ciao
> Stephan
>
>

Re: [PATCH v3 1/4] crypto: AF_ALG AIO - lock context IV

[Harsh Jain]

On 14-02-2018 18:22, Stephan Mueller wrote:
> Am Mittwoch, 14. Februar 2018, 06:43:53 CET schrieb Harsh Jain:
>
> Hi Harsh,
>
>> Patch set is working fine with chelsio Driver.
> Thank you.
>
>> Do we really need IV locking mechanism for AEAD algo because AEAD algo's
>> don't support Partial mode operation and Driver are not updating(atleast
>> Chelsio) IV's on AEAD request completions.
> Yes, I think we would need it. It is technically possible to have multiple 
> IOCBs for AEAD ciphers. Even though your implementation may not write the IV 
> back, others may do that. At least I do not see a guarantee that the IV is 
> *not* written back by a driver.
There is no  use of writing IV back in AEAD algo till Framework starts 
supporting Partial mode.
Even if Driver starts updating IV for AEAD, Multiple IOCB's in both cases will 
yield wrong results only.

Case 1 : If we have AEAD IV serialization  applied,  Encryption will be wrong 
if same IV gets used.
Case 2: If we do not have IV serialization for AEAD. Encryption will be fine 
but user will have multiple Authentication  tag (that too with final block 
processed).  Its like 2nd Block encryption is based on IV received from 1st 
block  and Authentication Tag value is based on 2nd block content only.

>
> In case your driver does not write the IV back and thus does not need to 
> serialize, the driver can report CRYPTO_ALG_SERIALIZES_IV_ACCESS. In this 
> case, the higher level functions would not serialize as the driver serializes 
> the requests (or the driver deems it appropriate that no serialization is 
> needed as is the case with your driver).
>
> Ciao
> Stephan
>
>

Re: [PATCH v3 4/4] crypto: add CRYPTO_TFM_REQ_IV_SERIALIZE flag

[Harsh Jain]

On 10-02-2018 03:34, Stephan Müller wrote:
> Crypto drivers may implement a streamlined serialization support for AIO
> requests that is reported by the CRYPTO_ALG_SERIALIZES_IV_ACCESS flag to
> the crypto user. When the user decides that he wants to send multiple
> AIO requests concurrently and wants the crypto driver to handle the
> serialization, the caller has to set CRYPTO_TFM_REQ_IV_SERIALIZE to notify
> the crypto driver.
Will crypto_alloc_* API takes cares of this flag?. For Kernel Crypto user IV 
Synchronization logic depends on weather tfm allocated supports IV 
Serialisation or not.
>
> Only when this flag is enabled, the crypto driver shall apply its
> serialization logic for handling IV updates between requests. If this
> flag is not provided, the serialization logic shall not be applied by
> the driver as the caller decides that it does not need it (because no
> parallel AIO requests are sent) or that it performs its own
> serialization.
>
> Signed-off-by: Stephan Mueller 
> ---
>  crypto/algif_aead.c | 15 ---
>  crypto/algif_skcipher.c | 15 ---
>  include/linux/crypto.h  |  1 +
>  3 files changed, 25 insertions(+), 6 deletions(-)
>
> diff --git a/crypto/algif_aead.c b/crypto/algif_aead.c
> index 619147792cc9..5ec4dec6e6a1 100644
> --- a/crypto/algif_aead.c
> +++ b/crypto/algif_aead.c
> @@ -66,13 +66,22 @@ static int aead_sendmsg(struct socket *sock, struct 
> msghdr *msg, size_t size)
>  {
>   struct sock *sk = sock->sk;
>   struct alg_sock *ask = alg_sk(sk);
> + struct af_alg_ctx *ctx = ask->private;
>   struct sock *psk = ask->parent;
>   struct alg_sock *pask = alg_sk(psk);
>   struct aead_tfm *aeadc = pask->private;
> - struct crypto_aead *tfm = aeadc->aead;
> - unsigned int ivsize = crypto_aead_ivsize(tfm);
> + struct crypto_aead *aead = aeadc->aead;
> + struct crypto_tfm *tfm = crypto_aead_tfm(aead);
> + unsigned int ivsize = crypto_aead_ivsize(aead);
> + int ret = af_alg_sendmsg(sock, msg, size, ivsize);
> +
> + if (ret < 0)
> + return ret;
>  
> - return af_alg_sendmsg(sock, msg, size, ivsize);
> + if (ctx->iiv == ALG_IV_SERIAL_PROCESSING)
> + tfm->crt_flags |= CRYPTO_TFM_REQ_IV_SERIALIZE;
> +
> + return ret;
>  }
>  
>  static int crypto_aead_copy_sgl(struct crypto_skcipher *null_tfm,
> diff --git a/crypto/algif_skcipher.c b/crypto/algif_skcipher.c
> index cf27dda6a181..fd2a0ba32feb 100644
> --- a/crypto/algif_skcipher.c
> +++ b/crypto/algif_skcipher.c
> @@ -43,12 +43,21 @@ static int skcipher_sendmsg(struct socket *sock, struct 
> msghdr *msg,
>  {
>   struct sock *sk = sock->sk;
>   struct alg_sock *ask = alg_sk(sk);
> + struct af_alg_ctx *ctx = ask->private;
>   struct sock *psk = ask->parent;
>   struct alg_sock *pask = alg_sk(psk);
> - struct crypto_skcipher *tfm = pask->private;
> - unsigned ivsize = crypto_skcipher_ivsize(tfm);
> + struct crypto_skcipher *skc = pask->private;
> + struct crypto_tfm *tfm = crypto_skcipher_tfm(skc);
> + unsigned int ivsize = crypto_skcipher_ivsize(skc);
> + int ret = af_alg_sendmsg(sock, msg, size, ivsize);
> +
> + if (ret < 0)
> + return ret;
>  
> - return af_alg_sendmsg(sock, msg, size, ivsize);
> + if (ctx->iiv == ALG_IV_SERIAL_PROCESSING)
> + tfm->crt_flags |= CRYPTO_TFM_REQ_IV_SERIALIZE;
> +
> + return ret;
>  }
>  
>  static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg,
> diff --git a/include/linux/crypto.h b/include/linux/crypto.h
> index 4860aa2c9be4..4d54f2b30692 100644
> --- a/include/linux/crypto.h
> +++ b/include/linux/crypto.h
> @@ -133,6 +133,7 @@
>  #define CRYPTO_TFM_REQ_WEAK_KEY  0x0100
>  #define CRYPTO_TFM_REQ_MAY_SLEEP 0x0200
>  #define CRYPTO_TFM_REQ_MAY_BACKLOG   0x0400
> +#define CRYPTO_TFM_REQ_IV_SERIALIZE  0x0800
>  #define CRYPTO_TFM_RES_WEAK_KEY  0x0010
>  #define CRYPTO_TFM_RES_BAD_KEY_LEN   0x0020
>  #define CRYPTO_TFM_RES_BAD_KEY_SCHED 0x0040

Re: [PATCH v3 1/4] crypto: AF_ALG AIO - lock context IV

[Harsh Jain]

On 10-02-2018 03:33, Stephan Müller wrote:
> The kernel crypto API requires the caller to set an IV in the request data
> structure. That request data structure shall define one particular cipher
> operation. During the cipher operation, the IV is read by the cipher
> implementation and eventually the potentially updated IV (e.g. in case of
> CBC) is written back to the memory location the request data structure
> points to.
>
> AF_ALG allows setting the IV with a sendmsg request, where the IV is stored
> in the AF_ALG context that is unique to one particular AF_ALG socket. Note
> the analogy: an AF_ALG socket is like a TFM where one recvmsg operation
> uses one request with the TFM from the socket.
>
> AF_ALG these days supports AIO operations with multiple IOCBs. I.e. with
> one recvmsg call, multiple IOVECs can be specified. Each individual IOCB
> (derived from one IOVEC) implies that one request data structure is
> created with the data to be processed by the cipher implementation. The
> IV that was set with the sendmsg call is registered with the request data
> structure before the cipher operation.
>
> In case of an AIO operation, the cipher operation invocation returns
> immediately, queuing the request to the hardware. While the AIO request is
> processed by the hardware, recvmsg processes the next IOVEC for which
> another request is created. Again, the IV buffer from the AF_ALG socket
> context is registered with the new request and the cipher operation is
> invoked.
>
> You may now see that there is a potential race condition regarding the IV
> handling, because there is *no* separate IV buffer for the different
> requests. This is nicely demonstrated with libkcapi using the following
> command which creates an AIO request with two IOCBs each encrypting one
> AES block in CBC mode:
>
> kcapi  -d 2 -x 9  -e -c "cbc(aes)" -k
> 8d7dd9b0170ce0b5f2f8e1aa768e01e91da8bfc67fd486d081b28254c99eb423 -i
> 7fbc02ebf5b93322329df9bfccb635af -p 48981da18e4bb9ef7e2e3162d16b1910
>
> When the first AIO request finishes before the 2nd AIO request is
> processed, the returned value is:
>
> 8b19050f66582cb7f7e4b6c873819b7108afa0eaa7de29bac7d903576b674c32
>
> I.e. two blocks where the IV output from the first request is the IV input
> to the 2nd block.
>
> In case the first AIO request is not completed before the 2nd request
> commences, the result is two identical AES blocks (i.e. both use the same
> IV):
>
> 8b19050f66582cb7f7e4b6c873819b718b19050f66582cb7f7e4b6c873819b71
>
> This inconsistent result may even lead to the conclusion that there can be
> a memory corruption in the IV buffer if both AIO requests write to the IV
> buffer at the same time.
>
> As the AF_ALG interface is used by user space, a mutex provides the
> serialization which puts the caller to sleep in case a previous IOCB
> processing is not yet finished.
>
> If multiple IOCBs arrive that all are blocked, the mutex' FIFO operation
> of processing arriving requests ensures that the blocked IOCBs are
> unblocked in the right order.
Hi Stephen,

Patch set is working fine with chelsio Driver.
Do we really need IV locking mechanism for AEAD algo because AEAD algo's don't 
support Partial mode operation and Driver are not updating(atleast Chelsio) 
IV's on AEAD request completions.
>
> CC:  #4.14
> Fixes: e870456d8e7c8 ("crypto: algif_skcipher - overhaul memory management")
> Fixes: d887c52d6ae43 ("crypto: algif_aead - overhaul memory management")
> Signed-off-by: Stephan Mueller 
> ---
>  crypto/af_alg.c | 31 +++
>  crypto/algif_aead.c | 20 +---
>  crypto/algif_skcipher.c | 12 +---
>  include/crypto/if_alg.h |  5 +
>  4 files changed, 58 insertions(+), 10 deletions(-)
>
> diff --git a/crypto/af_alg.c b/crypto/af_alg.c
> index 5231f421ad00..e7887621aa44 100644
> --- a/crypto/af_alg.c
> +++ b/crypto/af_alg.c
> @@ -1051,6 +1051,8 @@ void af_alg_async_cb(struct crypto_async_request *_req, 
> int err)
>   struct kiocb *iocb = areq->iocb;
>   unsigned int resultlen;
>  
> + af_alg_put_iv(sk);
> +
>   /* Buffer size written by crypto operation. */
>   resultlen = areq->outlen;
>  
> @@ -1175,6 +1177,35 @@ int af_alg_get_rsgl(struct sock *sk, struct msghdr 
> *msg, int flags,
>  }
>  EXPORT_SYMBOL_GPL(af_alg_get_rsgl);
>  
> +/**
> + * af_alg_get_iv
> + *
> + * @sk [in] AF_ALG socket
> + * @return 0 on success, < 0 on error
> + */
> +int af_alg_get_iv(struct sock *sk)
> +{
> + struct alg_sock *ask = alg_sk(sk);
> + struct af_alg_ctx *ctx = ask->private;
> +
> + return mutex_lock_interruptible(>ivlock);
> +}
> +EXPORT_SYMBOL_GPL(af_alg_get_iv);
> +
> +/**
> + * af_alg_put_iv - release lock on IV in case CTX IV is used
> + *
> + * @sk [in] AF_ALG socket
> + */
> +void af_alg_put_iv(struct sock *sk)
> +{
> + struct alg_sock *ask = alg_sk(sk);
> + struct af_alg_ctx *ctx = ask->private;
> +
> + 

Re: [PATCH v2 0/4] crypto: AF_ALG AIO improvements

[Harsh Jain]

On 07-02-2018 13:12, Stephan Müller wrote:
> Hi Herbert,
>
> Herbert, the patch 1 is meant for stable. However, this patch as is
> only applies to the new AF_ALG interface implementation. Though,
> the issue goes back to the first implementation of AIO support.
> Shall I try prepare a patch for the old AF_ALG implementation
> as well?
>
> Changes from v1:
>
> * integrate the inline IV and locking patch into one patch set
>
> * reverse the order of lock context IV patch and inline IV patch --
>   the reason is to allow the first patch to be back-ported to stable
>
> * mark the first patch (locking of the context IV) as applicable to
>   stable as there is an existing inconsistency which was demonstrated
>   by Harsh with the Chelsio driver vs the AES-NI driver
>
> * modify the inline IV patch to have proper unlocking of the mutex
>   in case of errors
>
> * prevent locking if no IV is defined by cipher
>
> * add a patch to allow crypto drivers to report whether they support
>   serialization -- in this case the locking in AF_ALG shall be
>   disabled
>
> * add a patch to inform the crypto drivers that their serialization
>   support should actually be enabled and used because AF_ALG does not
>   serialize the interdependent parallel AIO requests
>
> * streamline the code in patch 1 and 2 slightly
>
> I would like to ask the folks with real AIO hardware (Harsh, Jonathan)
> to test the patches. Especially, is the locking patch should be tested
> by Harsh as you have seen the issue with your hardware.
Sure I will test the patch and let you know.
>
> Thanks.
>
> Stephan Mueller (4):
>   crypto: AF_ALG AIO - lock context IV
>   crypto: AF_ALG - inline IV support
>   crypto: AF_ALG - allow driver to serialize IV access
>   crypto: add CRYPTO_TFM_REQ_PARALLEL flag
>
>  crypto/af_alg.c | 119 
> +++-
>  crypto/algif_aead.c |  86 +---
>  crypto/algif_skcipher.c |  38 ++
>  include/crypto/if_alg.h |  37 ++
>  include/linux/crypto.h  |  16 ++
>  include/uapi/linux/if_alg.h |   6 ++-
>  6 files changed, 249 insertions(+), 53 deletions(-)
>

Re: [PATCH] crypto: AF_ALG AIO - lock context IV

[Harsh Jain]

On 01-02-2018 15:55, Jonathan Cameron wrote:
> On Thu, 1 Feb 2018 12:07:21 +0200
> Gilad Ben-Yossef  wrote:
>
>> On Thu, Feb 1, 2018 at 12:04 PM, Stephan Mueller  wrote:
>>> Am Donnerstag, 1. Februar 2018, 10:35:07 CET schrieb Gilad Ben-Yossef:
>>>
>>> Hi Gilad,
>>>  
> Which works well for the sort of optimization I did and for hardware that
> can do iv dependency tracking itself. If hardware dependency tracking was
> avilable, you would be able to queue up requests with a chained IV
> without taking any special precautions at all. The hardware would
> handle the IV update dependencies.
>
> So in conclusion, Stephan's approach should work and give us a nice
> small patch suitable for stable inclusion.
>
> However, if people know that their setup overhead can be put in parallel
> with previous requests (even when the IV is not yet updated) then they
> will
> probably want to do something inside their own driver and set the flag
> that Stephan is proposing adding to bypass the mutex approach.  
 The patches from Stephan looks good to me, but I think we can do better
 for the long term approach you are discussing.  
>>> What you made me think of is the following: shouldn't we relay the inline IV
>>> flag on to the crypto drivers?
>>>
>>> The reason is to allow a clean implementation of the enabling or disabling 
>>> of
>>> the dependency handling in the driver. Jonathan's driver, for example, 
>>> decides
>>> based on the pointer value of the IV buffer whether it is the same buffer 
>>> and
>>> thus dependency handling is to be applied. This is fragile.
> I agree it's inelegant and a flag would be better than pointer tricks (though
> they are safe currently - we never know what might change in future)
> It was really a minimal example rather than a suggestion of the ultimate
> solution ;)  I was planning on suggesting a flag myself once the basic
> discussion concluded the approach was worthwhile.
>
>>> As AF_ALG knows whether the inline IV with separate IVs per request or the
>>> serialization with one IV buffer for all requests is requested, it should
>>> relay this state on to the drivers. Thus, for example, Jonathan's driver can
>>> be changed to rely on this flag instead on the buffer pointer value to 
>>> decide
>>> whether to enable its dependency handling.  
>> Yes, that is exactly what I was trying to point out :-)
> Agreed - make things explicit rather than basically relying on knowing how
> the above layers are working.
IPSec layer may not get benefit from this because they send complete sg list in 
single request only. They don't need partial mode support.
>
> Thanks,
>
> Jonathan
>

Re: [PATCH] crypto: AF_ALG - inline IV support

[Harsh Jain]

On 21-01-2018 17:44, Stephan Müller wrote:
> Hi Herbert,
>
> I tried to summarize the use cases of the AIO support at [1].
>
> The use case covering the inline IV support is documented in section [2]. It 
> naturally would depend on this patch to be accepted. What is your take on 
> this 
> use case?
>
> What is your take on the issue outlined at [3]? Should the affected drivers 
> be 
> updated?

Till now driver is using content of req->info as IV for cipher operation and It 
treats each request independent of previous request.

>From description given at [1]. Following is the new requirement for Driver.
"The cipher implementations ensure that although the IOVECs cause parallel
invocation of the recvmsg handler, they are serialized such that the
one IV sent to the kernel is updated by the first cipher operation and
used as input to the second cipher operation, and so on. This implies,
for example, that the CBC block chaining operation is applied
for all IOVECs."

To make this work how driver will know that it has to delay the processing of 
current request because previous request is not complete(or IV is not 
available) .

> Thanks
> Stephan
>
> [1] https://github.com/smuellerDD/libkcapi/commit/
> 29132075b8f045f3f92367c0190add81ccf5da11
>
> [2] https://github.com/smuellerDD/libkcapi/commit/
> 29132075b8f045f3f92367c0190add81ccf5da11#diff-
> dce7a00cbc610df94f0dc27eb769d01dR644
>
> [3] https://github.com/smuellerDD/libkcapi/commit/
> 29132075b8f045f3f92367c0190add81ccf5da11#diff-
> dce7a00cbc610df94f0dc27eb769d01dR575
>
>

Re: [PATCH] crypto: chelsio - Delete stray tabs in create_authenc_wr()

[Harsh Jain]

On 22-01-2018 15:51, Dan Carpenter wrote:
> We removed some if statements but left these statements indented too
> far.
HI Dan,

Change already applied to cryptodev tree.

https://www.mail-archive.com/linux-crypto@vger.kernel.org/msg30560.html

>
> Signed-off-by: Dan Carpenter 
>
> diff --git a/drivers/crypto/chelsio/chcr_algo.c 
> b/drivers/crypto/chelsio/chcr_algo.c
> index a9c894bf9c01..34a02d690548 100644
> --- a/drivers/crypto/chelsio/chcr_algo.c
> +++ b/drivers/crypto/chelsio/chcr_algo.c
> @@ -2112,11 +2112,11 @@ static struct sk_buff *create_authenc_wr(struct 
> aead_request *req,
>   error = chcr_aead_common_init(req, op_type);
>   if (error)
>   return ERR_PTR(error);
> - dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
> - dnents += sg_nents_xlen(req->dst, req->cryptlen +
> - (op_type ? -authsize : authsize), CHCR_DST_SG_SIZE,
> - req->assoclen);
> - dnents += MIN_AUTH_SG; // For IV
> + dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
> + dnents += sg_nents_xlen(req->dst, req->cryptlen +
> + (op_type ? -authsize : authsize), CHCR_DST_SG_SIZE,
> + req->assoclen);
> + dnents += MIN_AUTH_SG; // For IV
>  
>   dst_size = get_space_for_phys_dsgl(dnents);
>   kctx_len = (ntohl(KEY_CONTEXT_CTX_LEN_V(aeadctx->key_ctx_hdr)) << 4)

[PATCH 1/1] crypto: chelsio - Fix indentation warning

[Harsh Jain]

Fix Warning introduced in changeset

e1a018e607a3 ("crypto: chelsio - Remove dst sg size zero check")

Reported-by: Stephen Rothwell <s...@canb.auug.org.au>
Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index a9c894b..34a02d6 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -2112,11 +2112,11 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
error = chcr_aead_common_init(req, op_type);
if (error)
return ERR_PTR(error);
-   dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
-   dnents += sg_nents_xlen(req->dst, req->cryptlen +
-   (op_type ? -authsize : authsize), CHCR_DST_SG_SIZE,
-   req->assoclen);
-   dnents += MIN_AUTH_SG; // For IV
+   dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
+   dnents += sg_nents_xlen(req->dst, req->cryptlen +
+   (op_type ? -authsize : authsize), CHCR_DST_SG_SIZE,
+   req->assoclen);
+   dnents += MIN_AUTH_SG; // For IV
 
dst_size = get_space_for_phys_dsgl(dnents);
kctx_len = (ntohl(KEY_CONTEXT_CTX_LEN_V(aeadctx->key_ctx_hdr)) << 4)
-- 
2.1.4

[PATCH 0/5] crypto: chelsio - Cleanup and bug fixes

[Harsh Jain]

This series include cleanup, bug fixes and authenc algo supporting
 ctr(aes)-sha operation.

Harsh Jain (5):
  crypto: chelsio - Fix Indentation
  crypto: chelsio - check for sg null
  crypto: chelsio - Fix IV updated in XTS operation
  crypto: chelsio - Add authenc versions of ctr and sha
  crypto: chelsio - Remove dst sg size zero check

 drivers/crypto/chelsio/chcr_algo.c   | 299 ++-
 drivers/crypto/chelsio/chcr_crypto.h |   7 +-
 2 files changed, 233 insertions(+), 73 deletions(-)

-- 
2.1.4

[PATCH 1/5] crypto: chelsio - Fix Indentation

[Harsh Jain]

Fix inconsistent Indenting.

Reported-by: Dan Carpenter <dan.carpen...@oracle.com>
Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 28 +---
 1 file changed, 13 insertions(+), 15 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index b663b93..f6b1161 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -683,7 +683,7 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
if (srclen <= dstlen)
break;
less = min_t(unsigned int, sg_dma_len(dst) - offset -
-   dstskip, CHCR_DST_SG_SIZE);
+dstskip, CHCR_DST_SG_SIZE);
dstlen += less;
offset += less;
if (offset == sg_dma_len(dst)) {
@@ -694,7 +694,7 @@ static int chcr_sg_ent_in_wr(struct scatterlist *src,
dstskip = 0;
}
src = sg_next(src);
-srcskip = 0;
+   srcskip = 0;
}
return min(srclen, dstlen);
 }
@@ -1142,10 +1142,10 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
bytes = chcr_sg_ent_in_wr(reqctx->srcsg, reqctx->dstsg, 1,
  SPACE_LEFT(ablkctx->enckey_len),
  reqctx->src_ofst, reqctx->dst_ofst);
-   if ((bytes + reqctx->processed) >= req->nbytes)
-   bytes  = req->nbytes - reqctx->processed;
-   else
-   bytes = ROUND_16(bytes);
+   if ((bytes + reqctx->processed) >= req->nbytes)
+   bytes  = req->nbytes - reqctx->processed;
+   else
+   bytes = ROUND_16(bytes);
} else {
/*CTR mode counter overfloa*/
bytes  = req->nbytes - reqctx->processed;
@@ -1246,15 +1246,15 @@ static int process_cipher(struct ablkcipher_request 
*req,
  MIN_CIPHER_SG,
  SPACE_LEFT(ablkctx->enckey_len),
  0, 0);
-   if ((bytes + reqctx->processed) >= req->nbytes)
-   bytes  = req->nbytes - reqctx->processed;
-   else
-   bytes = ROUND_16(bytes);
+   if ((bytes + reqctx->processed) >= req->nbytes)
+   bytes  = req->nbytes - reqctx->processed;
+   else
+   bytes = ROUND_16(bytes);
} else {
bytes = req->nbytes;
}
if (get_cryptoalg_subtype(crypto_ablkcipher_tfm(tfm)) ==
- CRYPTO_ALG_SUB_TYPE_CTR) {
+   CRYPTO_ALG_SUB_TYPE_CTR) {
bytes = adjust_ctr_overflow(req->info, bytes);
}
if (get_cryptoalg_subtype(crypto_ablkcipher_tfm(tfm)) ==
@@ -2399,10 +2399,8 @@ void chcr_add_hash_src_ent(struct ahash_request *req,
ulptx_walk_add_page(_walk, param->bfr_len,
>dma_addr);
ulptx_walk_add_sg(_walk, req->src, param->sg_len,
- 0);
-//reqctx->srcsg = ulp_walk.last_sg;
-//reqctx->src_ofst = ulp_walk.last_sg_len;
-   ulptx_walk_end(_walk);
+ 0);
+   ulptx_walk_end(_walk);
}
 }
 
-- 
2.1.4

[PATCH 3/5] crypto: chelsio - Fix IV updated in XTS operation

[Harsh Jain]

Skip decrypt operation on IV received from HW for last request.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)
 mode change 100644 => 100755 drivers/crypto/chelsio/chcr_algo.c

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
old mode 100644
new mode 100755
index 280377f..27ee920
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1017,7 +1017,8 @@ static unsigned int adjust_ctr_overflow(u8 *iv, u32 bytes)
return bytes;
 }
 
-static int chcr_update_tweak(struct ablkcipher_request *req, u8 *iv)
+static int chcr_update_tweak(struct ablkcipher_request *req, u8 *iv,
+u32 isfinal)
 {
struct crypto_ablkcipher *tfm = crypto_ablkcipher_reqtfm(req);
struct ablk_ctx *ablkctx = ABLK_CTX(c_ctx(tfm));
@@ -1044,7 +1045,8 @@ static int chcr_update_tweak(struct ablkcipher_request 
*req, u8 *iv)
for (i = 0; i < (round % 8); i++)
gf128mul_x_ble((le128 *)iv, (le128 *)iv);
 
-   crypto_cipher_decrypt_one(cipher, iv, iv);
+   if (!isfinal)
+   crypto_cipher_decrypt_one(cipher, iv, iv);
 out:
return ret;
 }
@@ -1065,7 +1067,7 @@ static int chcr_update_cipher_iv(struct 
ablkcipher_request *req,
CTR_RFC3686_IV_SIZE) = cpu_to_be32((reqctx->processed /
AES_BLOCK_SIZE) + 1);
else if (subtype == CRYPTO_ALG_SUB_TYPE_XTS)
-   ret = chcr_update_tweak(req, iv);
+   ret = chcr_update_tweak(req, iv, 0);
else if (subtype == CRYPTO_ALG_SUB_TYPE_CBC) {
if (reqctx->op)
sg_pcopy_to_buffer(req->src, sg_nents(req->src), iv,
@@ -1096,7 +1098,7 @@ static int chcr_final_cipher_iv(struct ablkcipher_request 
*req,
ctr_add_iv(iv, req->info, (reqctx->processed /
   AES_BLOCK_SIZE));
else if (subtype == CRYPTO_ALG_SUB_TYPE_XTS)
-   ret = chcr_update_tweak(req, iv);
+   ret = chcr_update_tweak(req, iv, 1);
else if (subtype == CRYPTO_ALG_SUB_TYPE_CBC) {
if (reqctx->op)
sg_pcopy_to_buffer(req->src, sg_nents(req->src), iv,
-- 
2.1.4

[PATCH 2/5] crypto: chelsio - check for sg null

[Harsh Jain]

Add warning message if sg is NULL after skipping bytes.

Reported-by: Dan Carpenter <dan.carpen...@oracle.com>
Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index f6b1161..280377f 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -574,7 +574,8 @@ static void  ulptx_walk_add_sg(struct ulptx_walk *walk,
skip = 0;
}
}
-   if (walk->nents == 0) {
+   WARN(!sg, "SG should not be null here\n");
+   if (sg && (walk->nents == 0)) {
small = min_t(unsigned int, sg_dma_len(sg) - skip_len, len);
sgmin = min_t(unsigned int, small, CHCR_SRC_SG_SIZE);
walk->sgl->len0 = cpu_to_be32(sgmin);
-- 
2.1.4

[PATCH 5/5] crypto: chelsio - Remove dst sg size zero check

[Harsh Jain]

sg_nents_xlen will take care of zero length sg list.
Remove Destination sg list size zero check.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 43 --
 1 file changed, 13 insertions(+), 30 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 5cc84c4..a9c894b 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -2109,20 +2109,14 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
null = 1;
assoclen = 0;
}
-   dst_size = assoclen + req->cryptlen + (op_type ? -authsize :
-   authsize);
error = chcr_aead_common_init(req, op_type);
if (error)
return ERR_PTR(error);
-   if (dst_size) {
dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
dnents += sg_nents_xlen(req->dst, req->cryptlen +
(op_type ? -authsize : authsize), CHCR_DST_SG_SIZE,
req->assoclen);
dnents += MIN_AUTH_SG; // For IV
-   } else {
-   dnents = 0;
-   }
 
dst_size = get_space_for_phys_dsgl(dnents);
kctx_len = (ntohl(KEY_CONTEXT_CTX_LEN_V(aeadctx->key_ctx_hdr)) << 4)
@@ -2687,8 +2681,6 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_request *req,
sub_type = get_aead_subtype(tfm);
if (sub_type == CRYPTO_ALG_SUB_TYPE_AEAD_RFC4309)
assoclen -= 8;
-   dst_size = assoclen + req->cryptlen + (op_type ? -authsize :
-  authsize);
error = chcr_aead_common_init(req, op_type);
if (error)
return ERR_PTR(error);
@@ -2698,15 +2690,11 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_request *req,
error = aead_ccm_validate_input(op_type, req, aeadctx, sub_type);
if (error)
goto err;
-   if (dst_size) {
-   dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
-   dnents += sg_nents_xlen(req->dst, req->cryptlen
-   + (op_type ? -authsize : authsize),
-   CHCR_DST_SG_SIZE, req->assoclen);
-   dnents += MIN_CCM_SG; // For IV and B0
-   } else {
-   dnents = 0;
-   }
+   dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
+   dnents += sg_nents_xlen(req->dst, req->cryptlen
+   + (op_type ? -authsize : authsize),
+   CHCR_DST_SG_SIZE, req->assoclen);
+   dnents += MIN_CCM_SG; // For IV and B0
dst_size = get_space_for_phys_dsgl(dnents);
kctx_len = ((DIV_ROUND_UP(aeadctx->enckey_len, 16)) << 4) * 2;
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, dst_size);
@@ -2801,19 +2789,14 @@ static struct sk_buff *create_gcm_wr(struct 
aead_request *req,
assoclen = req->assoclen - 8;
 
reqctx->b0_dma = 0;
-   dst_size = assoclen + req->cryptlen + (op_type ? -authsize :  authsize);
error = chcr_aead_common_init(req, op_type);
-   if (error)
-   return  ERR_PTR(error);
-   if (dst_size) {
-   dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
-   dnents += sg_nents_xlen(req->dst,
-   req->cryptlen + (op_type ? -authsize : authsize),
+   if (error)
+   return ERR_PTR(error);
+   dnents = sg_nents_xlen(req->dst, assoclen, CHCR_DST_SG_SIZE, 0);
+   dnents += sg_nents_xlen(req->dst, req->cryptlen +
+   (op_type ? -authsize : authsize),
CHCR_DST_SG_SIZE, req->assoclen);
-   dnents += MIN_GCM_SG; // For IV
-   } else {
-   dnents = 0;
-   }
+   dnents += MIN_GCM_SG; // For IV
dst_size = get_space_for_phys_dsgl(dnents);
kctx_len = ((DIV_ROUND_UP(aeadctx->enckey_len, 16)) << 4) +
AEAD_H_SIZE;
@@ -2850,10 +2833,10 @@ static struct sk_buff *create_gcm_wr(struct 
aead_request *req,
chcr_req->sec_cpl.aadstart_cipherstop_hi = FILL_SEC_CPL_CIPHERSTOP_HI(
assoclen ? 1 : 0, assoclen,
assoclen + IV + 1, 0);
-   chcr_req->sec_cpl.cipherstop_lo_authinsert =
+   chcr_req->sec_cpl.cipherstop_lo_authinsert =
FILL_SEC_CPL_AUTHINSERT(0, assoclen + IV + 1,
temp, temp);
-   chcr_req->sec_cpl.seqno_numivs =
+   chcr_req->sec_cpl.seqno_numivs =
  

[PATCH 4/5] crypto: chelsio - Add authenc versions of ctr and sha

[Harsh Jain]

Add ctr and sha combination of algo in authenc mode.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 215 +++
 drivers/crypto/chelsio/chcr_crypto.h |   7 +-
 2 files changed, 199 insertions(+), 23 deletions(-)
 mode change 100755 => 100644 drivers/crypto/chelsio/chcr_algo.c

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
old mode 100755
new mode 100644
index 27ee920..5cc84c4
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -2090,7 +2090,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
struct cpl_rx_phys_dsgl *phys_cpl;
struct ulptx_sgl *ulptx;
unsigned int transhdr_len;
-   unsigned int dst_size = 0, temp;
+   unsigned int dst_size = 0, temp, subtype = get_aead_subtype(tfm);
unsigned int   kctx_len = 0, dnents;
unsigned int  assoclen = req->assoclen;
unsigned int  authsize = crypto_aead_authsize(tfm);
@@ -2104,7 +2104,8 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
return NULL;
 
reqctx->b0_dma = 0;
-   if (get_aead_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_AEAD_NULL) {
+   if (subtype == CRYPTO_ALG_SUB_TYPE_CBC_NULL ||
+   subtype == CRYPTO_ALG_SUB_TYPE_CTR_NULL) {
null = 1;
assoclen = 0;
}
@@ -2169,16 +2170,23 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
temp & 0xF,
null ? 0 : assoclen + IV + 1,
temp, temp);
+   if (subtype == CRYPTO_ALG_SUB_TYPE_CTR_NULL ||
+   subtype == CRYPTO_ALG_SUB_TYPE_CTR_SHA)
+   temp = CHCR_SCMD_CIPHER_MODE_AES_CTR;
+   else
+   temp = CHCR_SCMD_CIPHER_MODE_AES_CBC;
chcr_req->sec_cpl.seqno_numivs = FILL_SEC_CPL_SCMD0_SEQNO(op_type,
(op_type == CHCR_ENCRYPT_OP) ? 1 : 0,
-   CHCR_SCMD_CIPHER_MODE_AES_CBC,
+   temp,
actx->auth_mode, aeadctx->hmac_ctrl,
IV >> 1);
chcr_req->sec_cpl.ivgen_hdrlen =  FILL_SEC_CPL_IVGEN_HDRLEN(0, 0, 1,
 0, 0, dst_size);
 
chcr_req->key_ctx.ctx_hdr = aeadctx->key_ctx_hdr;
-   if (op_type == CHCR_ENCRYPT_OP)
+   if (op_type == CHCR_ENCRYPT_OP ||
+   subtype == CRYPTO_ALG_SUB_TYPE_CTR_SHA ||
+   subtype == CRYPTO_ALG_SUB_TYPE_CTR_NULL)
memcpy(chcr_req->key_ctx.key, aeadctx->key,
   aeadctx->enckey_len);
else
@@ -2188,7 +2196,16 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
memcpy(chcr_req->key_ctx.key + (DIV_ROUND_UP(aeadctx->enckey_len, 16) <<
4), actx->h_iopad, kctx_len -
(DIV_ROUND_UP(aeadctx->enckey_len, 16) << 4));
-   memcpy(reqctx->iv, req->iv, IV);
+   if (subtype == CRYPTO_ALG_SUB_TYPE_CTR_SHA ||
+   subtype == CRYPTO_ALG_SUB_TYPE_CTR_NULL) {
+   memcpy(reqctx->iv, aeadctx->nonce, CTR_RFC3686_NONCE_SIZE);
+   memcpy(reqctx->iv + CTR_RFC3686_NONCE_SIZE, req->iv,
+   CTR_RFC3686_IV_SIZE);
+   *(__be32 *)(reqctx->iv + CTR_RFC3686_NONCE_SIZE +
+   CTR_RFC3686_IV_SIZE) = cpu_to_be32(1);
+   } else {
+   memcpy(reqctx->iv, req->iv, IV);
+   }
phys_cpl = (struct cpl_rx_phys_dsgl *)((u8 *)(chcr_req + 1) + kctx_len);
ulptx = (struct ulptx_sgl *)((u8 *)(phys_cpl + 1) + dst_size);
chcr_add_aead_dst_ent(req, phys_cpl, assoclen, op_type, qid);
@@ -3216,7 +3233,7 @@ static int chcr_authenc_setkey(struct crypto_aead 
*authenc, const u8 *key,
struct chcr_authenc_ctx *actx = AUTHENC_CTX(aeadctx);
/* it contains auth and cipher key both*/
struct crypto_authenc_keys keys;
-   unsigned int bs;
+   unsigned int bs, subtype;
unsigned int max_authsize = crypto_aead_alg(authenc)->maxauthsize;
int err = 0, i, key_ctx_len = 0;
unsigned char ck_size = 0;
@@ -3245,6 +3262,15 @@ static int chcr_authenc_setkey(struct crypto_aead 
*authenc, const u8 *key,
pr_err("chcr : Unsupported digest size\n");
goto out;
}
+   subtype = get_aead_subtype(authenc);
+   if (subtype == CRYPTO_ALG_SUB_TYPE_CTR_SHA ||
+   subtype == CRYPTO_ALG_SUB_TYPE_CTR_NULL) {
+   if (keys.enckeylen < CTR_RFC3686_NONCE_SIZE)
+   goto out;
+  

[PATCH v2 4/7] crypto:chelsio:Use x8_ble gf multiplication to calculate IV.

[Harsh Jain]

gf128mul_x8_ble() will reduce gf Multiplication iteration by 8.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 11 +--
 drivers/crypto/chelsio/chcr_crypto.h |  1 +
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index e4bf32d..e0ab34a 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -888,9 +888,11 @@ static int chcr_update_tweak(struct ablkcipher_request 
*req, u8 *iv)
int ret, i;
u8 *key;
unsigned int keylen;
+   int round = reqctx->last_req_len / AES_BLOCK_SIZE;
+   int round8 = round / 8;
 
cipher = ablkctx->aes_generic;
-   memcpy(iv, req->info, AES_BLOCK_SIZE);
+   memcpy(iv, reqctx->iv, AES_BLOCK_SIZE);
 
keylen = ablkctx->enckey_len / 2;
key = ablkctx->key + keylen;
@@ -899,7 +901,10 @@ static int chcr_update_tweak(struct ablkcipher_request 
*req, u8 *iv)
goto out;
 
crypto_cipher_encrypt_one(cipher, iv, iv);
-   for (i = 0; i < (reqctx->processed / AES_BLOCK_SIZE); i++)
+   for (i = 0; i < round8; i++)
+   gf128mul_x8_ble((le128 *)iv, (le128 *)iv);
+
+   for (i = 0; i < (round % 8); i++)
gf128mul_x_ble((le128 *)iv, (le128 *)iv);
 
crypto_cipher_decrypt_one(cipher, iv, iv);
@@ -1040,6 +1045,7 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
CRYPTO_ALG_SUB_TYPE_CTR)
bytes = adjust_ctr_overflow(reqctx->iv, bytes);
reqctx->processed += bytes;
+   reqctx->last_req_len = bytes;
wrparam.qid = u_ctx->lldi.rxq_ids[ctx->rx_qidx];
wrparam.req = req;
wrparam.bytes = bytes;
@@ -1132,6 +1138,7 @@ static int process_cipher(struct ablkcipher_request *req,
goto error;
}
reqctx->processed = bytes;
+   reqctx->last_req_len = bytes;
reqctx->dst = reqctx->dstsg;
reqctx->op = op_type;
wrparam.qid = qid;
diff --git a/drivers/crypto/chelsio/chcr_crypto.h 
b/drivers/crypto/chelsio/chcr_crypto.h
index 30af1ee..b3722b3 100644
--- a/drivers/crypto/chelsio/chcr_crypto.h
+++ b/drivers/crypto/chelsio/chcr_crypto.h
@@ -247,6 +247,7 @@ struct chcr_blkcipher_req_ctx {
struct scatterlist *dst;
struct scatterlist *newdstsg;
unsigned int processed;
+   unsigned int last_req_len;
unsigned int op;
short int dst_nents;
u8 iv[CHCR_MAX_CRYPTO_IV_LEN];
-- 
2.1.4

[PATCH v2 2/7] crypto:chelsio: Check error code with IS_ERR macro

[Harsh Jain]

From: Yeshaswi M R Gowda <yesha...@chelsio.com>

Check and return proper error code.

Signed-off-by: Jitendra Lulla <jlu...@chelsio.com>
Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index bdb1014..e4bf32d 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1455,8 +1455,8 @@ static int chcr_ahash_update(struct ahash_request *req)
req_ctx->result = 0;
req_ctx->data_len += params.sg_len + params.bfr_len;
skb = create_hash_wr(req, );
-   if (!skb)
-   return -ENOMEM;
+   if (IS_ERR(skb))
+   return PTR_ERR(skb);
 
if (remainder) {
u8 *temp;
@@ -1519,8 +1519,8 @@ static int chcr_ahash_final(struct ahash_request *req)
params.more = 0;
}
skb = create_hash_wr(req, );
-   if (!skb)
-   return -ENOMEM;
+   if (IS_ERR(skb))
+   return PTR_ERR(skb);
 
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_qidx);
@@ -1570,8 +1570,8 @@ static int chcr_ahash_finup(struct ahash_request *req)
}
 
skb = create_hash_wr(req, );
-   if (!skb)
-   return -ENOMEM;
+   if (IS_ERR(skb))
+   return PTR_ERR(skb);
 
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_qidx);
@@ -1621,8 +1621,8 @@ static int chcr_ahash_digest(struct ahash_request *req)
}
 
skb = create_hash_wr(req, );
-   if (!skb)
-   return -ENOMEM;
+   if (IS_ERR(skb))
+   return PTR_ERR(skb);
 
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_qidx);
-- 
2.1.4

[PATCH v2 5/7] crypto:chelsio:Remove allocation of sg list to implement 2K limit of dsgl header

[Harsh Jain]

Update DMA address index instead of allocating new sg list to impose  2k size 
limit for each entry.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 237 +++
 drivers/crypto/chelsio/chcr_algo.h   |   3 +-
 drivers/crypto/chelsio/chcr_core.h   |   2 +-
 drivers/crypto/chelsio/chcr_crypto.h |   6 -
 4 files changed, 76 insertions(+), 172 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index e0ab34a..b13991d 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -117,6 +117,21 @@ static inline unsigned int sgl_len(unsigned int n)
return (3 * n) / 2 + (n & 1) + 2;
 }
 
+static int dstsg_2k(struct scatterlist *sgl, unsigned int reqlen)
+{
+   int nents = 0;
+   unsigned int less;
+
+   while (sgl && reqlen) {
+   less = min(reqlen, sgl->length);
+   nents += DIV_ROUND_UP(less, CHCR_SG_SIZE);
+   reqlen -= less;
+   sgl = sg_next(sgl);
+   }
+
+   return nents;
+}
+
 static void chcr_verify_tag(struct aead_request *req, u8 *input, int *err)
 {
u8 temp[SHA512_DIGEST_SIZE];
@@ -166,8 +181,6 @@ int chcr_handle_resp(struct crypto_async_request *req, 
unsigned char *input,
kfree_skb(ctx_req.ctx.reqctx->skb);
ctx_req.ctx.reqctx->skb = NULL;
}
-   free_new_sg(ctx_req.ctx.reqctx->newdstsg);
-   ctx_req.ctx.reqctx->newdstsg = NULL;
if (ctx_req.ctx.reqctx->verify == VERIFY_SW) {
chcr_verify_tag(ctx_req.req.aead_req, input,
);
@@ -388,31 +401,41 @@ static void write_phys_cpl(struct cpl_rx_phys_dsgl 
*phys_cpl,
 {
struct phys_sge_pairs *to;
unsigned int len = 0, left_size = sg_param->obsize;
-   unsigned int nents = sg_param->nents, i, j = 0;
+   unsigned int j = 0;
+   int offset, ent_len;
 
phys_cpl->op_to_tid = htonl(CPL_RX_PHYS_DSGL_OPCODE_V(CPL_RX_PHYS_DSGL)
| CPL_RX_PHYS_DSGL_ISRDMA_V(0));
+   to = (struct phys_sge_pairs *)((unsigned char *)phys_cpl +
+  sizeof(struct cpl_rx_phys_dsgl));
+   while (left_size && sg) {
+   len = min_t(u32, left_size, sg_dma_len(sg));
+   offset = 0;
+   while (len) {
+   ent_len =  min_t(u32, len, CHCR_SG_SIZE);
+   to->len[j % 8] = htons(ent_len);
+   to->addr[j % 8] = cpu_to_be64(sg_dma_address(sg) +
+ offset);
+   offset += ent_len;
+   len -= ent_len;
+   j++;
+   if ((j % 8) == 0)
+   to++;
+   }
+   left_size -= min(left_size, sg_dma_len(sg));
+   sg = sg_next(sg);
+   }
phys_cpl->pcirlxorder_to_noofsgentr =
htonl(CPL_RX_PHYS_DSGL_PCIRLXORDER_V(0) |
  CPL_RX_PHYS_DSGL_PCINOSNOOP_V(0) |
  CPL_RX_PHYS_DSGL_PCITPHNTENB_V(0) |
  CPL_RX_PHYS_DSGL_PCITPHNT_V(0) |
  CPL_RX_PHYS_DSGL_DCAID_V(0) |
- CPL_RX_PHYS_DSGL_NOOFSGENTR_V(nents));
+ CPL_RX_PHYS_DSGL_NOOFSGENTR_V(j));
phys_cpl->rss_hdr_int.opcode = CPL_RX_PHYS_ADDR;
phys_cpl->rss_hdr_int.qid = htons(sg_param->qid);
phys_cpl->rss_hdr_int.hash_val = 0;
-   to = (struct phys_sge_pairs *)((unsigned char *)phys_cpl +
-  sizeof(struct cpl_rx_phys_dsgl));
-   for (i = 0; nents && left_size; to++) {
-   for (j = 0; j < 8 && nents && left_size; j++, nents--) {
-   len = min(left_size, sg_dma_len(sg));
-   to->len[j] = htons(len);
-   to->addr[j] = cpu_to_be64(sg_dma_address(sg));
-   left_size -= len;
-   sg = sg_next(sg);
-   }
-   }
+
 }
 
 static inline int map_writesg_phys_cpl(struct device *dev,
@@ -523,31 +546,33 @@ static int generate_copy_rrkey(struct ablk_ctx *ablkctx,
 static int chcr_sg_ent_in_wr(struct scatterlist *src,
 struct scatterlist *dst,
 unsigned int minsg,
-unsigned int space,
-short int *sent,
-short int *dent)
+unsigned int space)
 {
int srclen = 0, dstlen = 0;
int srcsg = minsg, dstsg = 0;
+   int offset = 0, less;
 
-   *sent = 0;
-   *dent = 0;
while (src &

[PATCH v2 6/7] crypto:chelsio:Move DMA un/mapping to chcr from lld cxgb4 driver

[Harsh Jain]

Allow chcr to do DMA mapping/Unmapping instead of lld cxgb4.
It moves "Copy AAD to dst buffer" requirement from driver to
firmware.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 1645 ++
 drivers/crypto/chelsio/chcr_algo.h   |   44 +-
 drivers/crypto/chelsio/chcr_crypto.h |  114 ++-
 drivers/net/ethernet/chelsio/cxgb4/sge.c |8 +-
 4 files changed, 1116 insertions(+), 695 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index b13991d..646dfff 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -70,6 +70,8 @@
 #include "chcr_algo.h"
 #include "chcr_crypto.h"
 
+#define IV AES_BLOCK_SIZE
+
 static inline  struct chcr_aead_ctx *AEAD_CTX(struct chcr_context *ctx)
 {
return ctx->crypto_ctx->aeadctx;
@@ -102,7 +104,7 @@ static inline struct uld_ctx *ULD_CTX(struct chcr_context 
*ctx)
 
 static inline int is_ofld_imm(const struct sk_buff *skb)
 {
-   return (skb->len <= CRYPTO_MAX_IMM_TX_PKT_LEN);
+   return (skb->len <= SGE_MAX_WR_LEN);
 }
 
 /*
@@ -117,21 +119,92 @@ static inline unsigned int sgl_len(unsigned int n)
return (3 * n) / 2 + (n & 1) + 2;
 }
 
-static int dstsg_2k(struct scatterlist *sgl, unsigned int reqlen)
+static int sg_nents_xlen(struct scatterlist *sg, unsigned int reqlen,
+unsigned int entlen,
+unsigned int skip)
 {
int nents = 0;
unsigned int less;
+   unsigned int skip_len = 0;
 
-   while (sgl && reqlen) {
-   less = min(reqlen, sgl->length);
-   nents += DIV_ROUND_UP(less, CHCR_SG_SIZE);
-   reqlen -= less;
-   sgl = sg_next(sgl);
+   while (sg && skip) {
+   if (sg_dma_len(sg) <= skip) {
+   skip -= sg_dma_len(sg);
+   skip_len = 0;
+   sg = sg_next(sg);
+   } else {
+   skip_len = skip;
+   skip = 0;
+   }
}
 
+   while (sg && reqlen) {
+   less = min(reqlen, sg_dma_len(sg) - skip_len);
+   nents += DIV_ROUND_UP(less, entlen);
+   reqlen -= less;
+   skip_len = 0;
+   sg = sg_next(sg);
+   }
return nents;
 }
 
+static inline void chcr_handle_ahash_resp(struct ahash_request *req,
+ unsigned char *input,
+ int err)
+{
+   struct chcr_ahash_req_ctx *reqctx = ahash_request_ctx(req);
+   int digestsize, updated_digestsize;
+   struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+   struct uld_ctx *u_ctx = ULD_CTX(h_ctx(tfm));
+
+   if (input == NULL)
+   goto out;
+   reqctx = ahash_request_ctx(req);
+   digestsize = crypto_ahash_digestsize(crypto_ahash_reqtfm(req));
+   if (reqctx->is_sg_map)
+   chcr_hash_dma_unmap(_ctx->lldi.pdev->dev, req);
+   if (reqctx->dma_addr)
+   dma_unmap_single(_ctx->lldi.pdev->dev, reqctx->dma_addr,
+reqctx->dma_len, DMA_TO_DEVICE);
+   reqctx->dma_addr = 0;
+   updated_digestsize = digestsize;
+   if (digestsize == SHA224_DIGEST_SIZE)
+   updated_digestsize = SHA256_DIGEST_SIZE;
+   else if (digestsize == SHA384_DIGEST_SIZE)
+   updated_digestsize = SHA512_DIGEST_SIZE;
+   if (reqctx->result == 1) {
+   reqctx->result = 0;
+   memcpy(req->result, input + sizeof(struct cpl_fw6_pld),
+  digestsize);
+   } else {
+   memcpy(reqctx->partial_hash, input + sizeof(struct cpl_fw6_pld),
+  updated_digestsize);
+   }
+out:
+   req->base.complete(>base, err);
+
+   }
+
+static inline void chcr_handle_aead_resp(struct aead_request *req,
+unsigned char *input,
+int err)
+{
+   struct chcr_aead_reqctx *reqctx = aead_request_ctx(req);
+   struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+   struct uld_ctx *u_ctx = ULD_CTX(a_ctx(tfm));
+
+
+   chcr_aead_dma_unmap(_ctx->lldi.pdev->dev, req, reqctx->op);
+   if (reqctx->b0_dma)
+   dma_unmap_single(_ctx->lldi.pdev->dev, reqctx->b0_dma,
+reqctx->b0_len, DMA_BIDIRECTIONAL);
+   if (reqctx->verify == VERIFY_SW) {
+   chcr_verify_tag(req, input, );
+   reqctx->verify = VERIFY_HW;
+}
+   req->base.complete(>base, err);
+
+}
 static void chcr_verify_tag(struct aead_request *req, u8 *input, int *err)
 {
u8 temp[SHA512_DIGEST_SIZE];
@@ -

[PATCH v2 3/7] crypto:gf128mul: The x8_ble multiplication functions

[Harsh Jain]

It multiply GF(2^128) elements in the ble format.
It will be used by chelsio driver to speed up gf multiplication.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 crypto/gf128mul.c | 13 +
 include/crypto/gf128mul.h |  2 +-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/crypto/gf128mul.c b/crypto/gf128mul.c
index dc01212..24e6019 100644
--- a/crypto/gf128mul.c
+++ b/crypto/gf128mul.c
@@ -156,6 +156,19 @@ static void gf128mul_x8_bbe(be128 *x)
x->b = cpu_to_be64((b << 8) ^ _tt);
 }
 
+void gf128mul_x8_ble(le128 *r, const le128 *x)
+{
+   u64 a = le64_to_cpu(x->a);
+   u64 b = le64_to_cpu(x->b);
+
+   /* equivalent to gf128mul_table_be[b >> 63] (see crypto/gf128mul.c): */
+   u64 _tt = gf128mul_table_be[a >> 56];
+
+   r->a = cpu_to_le64((a << 8) | (b >> 56));
+   r->b = cpu_to_le64((b << 8) ^ _tt);
+}
+EXPORT_SYMBOL(gf128mul_x8_ble);
+
 void gf128mul_lle(be128 *r, const be128 *b)
 {
be128 p[8];
diff --git a/include/crypto/gf128mul.h b/include/crypto/gf128mul.h
index 0977fb1..fa0a63d 100644
--- a/include/crypto/gf128mul.h
+++ b/include/crypto/gf128mul.h
@@ -227,7 +227,7 @@ struct gf128mul_4k *gf128mul_init_4k_lle(const be128 *g);
 struct gf128mul_4k *gf128mul_init_4k_bbe(const be128 *g);
 void gf128mul_4k_lle(be128 *a, const struct gf128mul_4k *t);
 void gf128mul_4k_bbe(be128 *a, const struct gf128mul_4k *t);
-
+void gf128mul_x8_ble(le128 *r, const le128 *x);
 static inline void gf128mul_free_4k(struct gf128mul_4k *t)
 {
kzfree(t);
-- 
2.1.4

[PATCH v2 7/7] crypto:chelsio: Fix memory leak

[Harsh Jain]

Fix memory leak when device does not support crypto.

Reported-by: Dan Carpenter <dan.carpen...@oracle.com>
Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_core.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_core.c 
b/drivers/crypto/chelsio/chcr_core.c
index b6dd9cb..4f677b3 100644
--- a/drivers/crypto/chelsio/chcr_core.c
+++ b/drivers/crypto/chelsio/chcr_core.c
@@ -154,15 +154,15 @@ static void *chcr_uld_add(const struct cxgb4_lld_info 
*lld)
struct uld_ctx *u_ctx;
 
/* Create the device and add it in the device list */
+   if (!(lld->ulp_crypto & ULP_CRYPTO_LOOKASIDE))
+   return ERR_PTR(-EOPNOTSUPP);
+
+   /* Create the device and add it in the device list */
u_ctx = kzalloc(sizeof(*u_ctx), GFP_KERNEL);
if (!u_ctx) {
u_ctx = ERR_PTR(-ENOMEM);
goto out;
}
-   if (!(lld->ulp_crypto & ULP_CRYPTO_LOOKASIDE)) {
-   u_ctx = ERR_PTR(-ENOMEM);
-   goto out;
-   }
u_ctx->lldi = *lld;
 out:
return u_ctx;
-- 
2.1.4

[PATCH v2 1/7] crypto:chelsio: Remove unused parameter

[Harsh Jain]

From: Yeshaswi M R Gowda <yesha...@chelsio.com>

Remove unused parameter sent to latest fw.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 43 +++---
 drivers/crypto/chelsio/chcr_algo.h | 12 +--
 2 files changed, 23 insertions(+), 32 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 0e81607..bdb1014 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -577,36 +577,27 @@ static int chcr_cipher_fallback(struct crypto_skcipher 
*cipher,
 static inline void create_wreq(struct chcr_context *ctx,
   struct chcr_wr *chcr_req,
   void *req, struct sk_buff *skb,
-  int kctx_len, int hash_sz,
-  int is_iv,
+  int hash_sz,
   unsigned int sc_len,
   unsigned int lcb)
 {
struct uld_ctx *u_ctx = ULD_CTX(ctx);
-   int iv_loc = IV_DSGL;
int qid = u_ctx->lldi.rxq_ids[ctx->rx_qidx];
-   unsigned int immdatalen = 0, nr_frags = 0;
+   unsigned int immdatalen = 0;
 
-   if (is_ofld_imm(skb)) {
+   if (is_ofld_imm(skb))
immdatalen = skb->data_len;
-   iv_loc = IV_IMMEDIATE;
-   } else {
-   nr_frags = skb_shinfo(skb)->nr_frags;
-   }
 
-   chcr_req->wreq.op_to_cctx_size = FILL_WR_OP_CCTX_SIZE(immdatalen,
-   ((sizeof(chcr_req->key_ctx) + kctx_len) >> 4));
+   chcr_req->wreq.op_to_cctx_size = FILL_WR_OP_CCTX_SIZE;
chcr_req->wreq.pld_size_hash_size =
-   htonl(FW_CRYPTO_LOOKASIDE_WR_PLD_SIZE_V(sgl_lengths[nr_frags]) |
- FW_CRYPTO_LOOKASIDE_WR_HASH_SIZE_V(hash_sz));
+   htonl(FW_CRYPTO_LOOKASIDE_WR_HASH_SIZE_V(hash_sz));
chcr_req->wreq.len16_pkd =
htonl(FW_CRYPTO_LOOKASIDE_WR_LEN16_V(DIV_ROUND_UP(
(calc_tx_flits_ofld(skb) * 8), 16)));
chcr_req->wreq.cookie = cpu_to_be64((uintptr_t)req);
chcr_req->wreq.rx_chid_to_rx_q_id =
FILL_WR_RX_Q_ID(ctx->dev->rx_channel_id, qid,
-   is_iv ? iv_loc : IV_NOP, !!lcb,
-   ctx->tx_qidx);
+   !!lcb, ctx->tx_qidx);
 
chcr_req->ulptx.cmd_dest = FILL_ULPTX_CMD_DEST(ctx->dev->tx_channel_id,
   qid);
@@ -616,7 +607,7 @@ static inline void create_wreq(struct chcr_context *ctx,
chcr_req->sc_imm.cmd_more = FILL_CMD_MORE(immdatalen);
chcr_req->sc_imm.len = cpu_to_be32(sizeof(struct cpl_tx_sec_pdu) +
   sizeof(chcr_req->key_ctx) +
-  kctx_len + sc_len + immdatalen);
+  sc_len + immdatalen);
 }
 
 /**
@@ -706,8 +697,8 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
write_buffer_to_skb(skb, , reqctx->iv, ivsize);
write_sg_to_skb(skb, , wrparam->srcsg, wrparam->bytes);
atomic_inc(>chcr_stats.cipher_rqst);
-   create_wreq(ctx, chcr_req, &(wrparam->req->base), skb, kctx_len, 0, 1,
-   sizeof(struct cpl_rx_phys_dsgl) + phys_dsgl,
+   create_wreq(ctx, chcr_req, &(wrparam->req->base), skb, 0,
+   sizeof(struct cpl_rx_phys_dsgl) + phys_dsgl + kctx_len,
ablkctx->ciph_mode == CHCR_SCMD_CIPHER_MODE_AES_CBC);
reqctx->skb = skb;
skb_get(skb);
@@ -1417,8 +1408,8 @@ static struct sk_buff *create_hash_wr(struct 
ahash_request *req,
if (param->sg_len != 0)
write_sg_to_skb(skb, , req->src, param->sg_len);
atomic_inc(>chcr_stats.digest_rqst);
-   create_wreq(ctx, chcr_req, >base, skb, kctx_len,
-   hash_size_in_response, 0, DUMMY_BYTES, 0);
+   create_wreq(ctx, chcr_req, >base, skb, hash_size_in_response,
+   DUMMY_BYTES + kctx_len, 0);
req_ctx->skb = skb;
skb_get(skb);
return skb;
@@ -2080,8 +2071,8 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
write_buffer_to_skb(skb, , req->iv, ivsize);
write_sg_to_skb(skb, , src, req->cryptlen);
atomic_inc(>chcr_stats.cipher_rqst);
-   create_wreq(ctx, chcr_req, >base, skb, kctx_len, size, 1,
-  sizeof(struct cpl_rx_phys_dsgl) + dst_size, 0);
+   create_wreq(ctx, chcr_req, >base, skb, size,
+  sizeof(struct cpl_rx_phys_dsgl) + dst_size + kctx_len, 0);
reqctx->skb = skb;
skb_get(skb);
 
@@ -2396,8 +2387,8 @@ static struct 

[PATCH 1/7] crypto:chelsio: Remove unused parameter

[Harsh Jain]

Remove unused parameter sent to latest fw.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 43 +++---
 drivers/crypto/chelsio/chcr_algo.h | 12 +--
 2 files changed, 23 insertions(+), 32 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 0e81607..bdb1014 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -577,36 +577,27 @@ static int chcr_cipher_fallback(struct crypto_skcipher 
*cipher,
 static inline void create_wreq(struct chcr_context *ctx,
   struct chcr_wr *chcr_req,
   void *req, struct sk_buff *skb,
-  int kctx_len, int hash_sz,
-  int is_iv,
+  int hash_sz,
   unsigned int sc_len,
   unsigned int lcb)
 {
struct uld_ctx *u_ctx = ULD_CTX(ctx);
-   int iv_loc = IV_DSGL;
int qid = u_ctx->lldi.rxq_ids[ctx->rx_qidx];
-   unsigned int immdatalen = 0, nr_frags = 0;
+   unsigned int immdatalen = 0;
 
-   if (is_ofld_imm(skb)) {
+   if (is_ofld_imm(skb))
immdatalen = skb->data_len;
-   iv_loc = IV_IMMEDIATE;
-   } else {
-   nr_frags = skb_shinfo(skb)->nr_frags;
-   }
 
-   chcr_req->wreq.op_to_cctx_size = FILL_WR_OP_CCTX_SIZE(immdatalen,
-   ((sizeof(chcr_req->key_ctx) + kctx_len) >> 4));
+   chcr_req->wreq.op_to_cctx_size = FILL_WR_OP_CCTX_SIZE;
chcr_req->wreq.pld_size_hash_size =
-   htonl(FW_CRYPTO_LOOKASIDE_WR_PLD_SIZE_V(sgl_lengths[nr_frags]) |
- FW_CRYPTO_LOOKASIDE_WR_HASH_SIZE_V(hash_sz));
+   htonl(FW_CRYPTO_LOOKASIDE_WR_HASH_SIZE_V(hash_sz));
chcr_req->wreq.len16_pkd =
htonl(FW_CRYPTO_LOOKASIDE_WR_LEN16_V(DIV_ROUND_UP(
(calc_tx_flits_ofld(skb) * 8), 16)));
chcr_req->wreq.cookie = cpu_to_be64((uintptr_t)req);
chcr_req->wreq.rx_chid_to_rx_q_id =
FILL_WR_RX_Q_ID(ctx->dev->rx_channel_id, qid,
-   is_iv ? iv_loc : IV_NOP, !!lcb,
-   ctx->tx_qidx);
+   !!lcb, ctx->tx_qidx);
 
chcr_req->ulptx.cmd_dest = FILL_ULPTX_CMD_DEST(ctx->dev->tx_channel_id,
   qid);
@@ -616,7 +607,7 @@ static inline void create_wreq(struct chcr_context *ctx,
chcr_req->sc_imm.cmd_more = FILL_CMD_MORE(immdatalen);
chcr_req->sc_imm.len = cpu_to_be32(sizeof(struct cpl_tx_sec_pdu) +
   sizeof(chcr_req->key_ctx) +
-  kctx_len + sc_len + immdatalen);
+  sc_len + immdatalen);
 }
 
 /**
@@ -706,8 +697,8 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
write_buffer_to_skb(skb, , reqctx->iv, ivsize);
write_sg_to_skb(skb, , wrparam->srcsg, wrparam->bytes);
atomic_inc(>chcr_stats.cipher_rqst);
-   create_wreq(ctx, chcr_req, &(wrparam->req->base), skb, kctx_len, 0, 1,
-   sizeof(struct cpl_rx_phys_dsgl) + phys_dsgl,
+   create_wreq(ctx, chcr_req, &(wrparam->req->base), skb, 0,
+   sizeof(struct cpl_rx_phys_dsgl) + phys_dsgl + kctx_len,
ablkctx->ciph_mode == CHCR_SCMD_CIPHER_MODE_AES_CBC);
reqctx->skb = skb;
skb_get(skb);
@@ -1417,8 +1408,8 @@ static struct sk_buff *create_hash_wr(struct 
ahash_request *req,
if (param->sg_len != 0)
write_sg_to_skb(skb, , req->src, param->sg_len);
atomic_inc(>chcr_stats.digest_rqst);
-   create_wreq(ctx, chcr_req, >base, skb, kctx_len,
-   hash_size_in_response, 0, DUMMY_BYTES, 0);
+   create_wreq(ctx, chcr_req, >base, skb, hash_size_in_response,
+   DUMMY_BYTES + kctx_len, 0);
req_ctx->skb = skb;
skb_get(skb);
return skb;
@@ -2080,8 +2071,8 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
write_buffer_to_skb(skb, , req->iv, ivsize);
write_sg_to_skb(skb, , src, req->cryptlen);
atomic_inc(>chcr_stats.cipher_rqst);
-   create_wreq(ctx, chcr_req, >base, skb, kctx_len, size, 1,
-  sizeof(struct cpl_rx_phys_dsgl) + dst_size, 0);
+   create_wreq(ctx, chcr_req, >base, skb, size,
+  sizeof(struct cpl_rx_phys_dsgl) + dst_size + kctx_len, 0);
reqctx->skb = skb;
skb_get(skb);
 
@@ -2396,8 +2387,8 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_req

[PATCH 4/7] crypto:chelsio:Use x8_ble gf multiplication to calculate IV.

[Harsh Jain]

gf128mul_x8_ble() will reduce gf Multiplication iteration by 8.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 11 +--
 drivers/crypto/chelsio/chcr_crypto.h |  1 +
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index e4bf32d..e0ab34a 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -888,9 +888,11 @@ static int chcr_update_tweak(struct ablkcipher_request 
*req, u8 *iv)
int ret, i;
u8 *key;
unsigned int keylen;
+   int round = reqctx->last_req_len / AES_BLOCK_SIZE;
+   int round8 = round / 8;
 
cipher = ablkctx->aes_generic;
-   memcpy(iv, req->info, AES_BLOCK_SIZE);
+   memcpy(iv, reqctx->iv, AES_BLOCK_SIZE);
 
keylen = ablkctx->enckey_len / 2;
key = ablkctx->key + keylen;
@@ -899,7 +901,10 @@ static int chcr_update_tweak(struct ablkcipher_request 
*req, u8 *iv)
goto out;
 
crypto_cipher_encrypt_one(cipher, iv, iv);
-   for (i = 0; i < (reqctx->processed / AES_BLOCK_SIZE); i++)
+   for (i = 0; i < round8; i++)
+   gf128mul_x8_ble((le128 *)iv, (le128 *)iv);
+
+   for (i = 0; i < (round % 8); i++)
gf128mul_x_ble((le128 *)iv, (le128 *)iv);
 
crypto_cipher_decrypt_one(cipher, iv, iv);
@@ -1040,6 +1045,7 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
CRYPTO_ALG_SUB_TYPE_CTR)
bytes = adjust_ctr_overflow(reqctx->iv, bytes);
reqctx->processed += bytes;
+   reqctx->last_req_len = bytes;
wrparam.qid = u_ctx->lldi.rxq_ids[ctx->rx_qidx];
wrparam.req = req;
wrparam.bytes = bytes;
@@ -1132,6 +1138,7 @@ static int process_cipher(struct ablkcipher_request *req,
goto error;
}
reqctx->processed = bytes;
+   reqctx->last_req_len = bytes;
reqctx->dst = reqctx->dstsg;
reqctx->op = op_type;
wrparam.qid = qid;
diff --git a/drivers/crypto/chelsio/chcr_crypto.h 
b/drivers/crypto/chelsio/chcr_crypto.h
index 30af1ee..b3722b3 100644
--- a/drivers/crypto/chelsio/chcr_crypto.h
+++ b/drivers/crypto/chelsio/chcr_crypto.h
@@ -247,6 +247,7 @@ struct chcr_blkcipher_req_ctx {
struct scatterlist *dst;
struct scatterlist *newdstsg;
unsigned int processed;
+   unsigned int last_req_len;
unsigned int op;
short int dst_nents;
u8 iv[CHCR_MAX_CRYPTO_IV_LEN];
-- 
2.1.4

[PATCH 3/7] crypto:gf128mul: The x8_ble multiplication functions

[Harsh Jain]

It multiply GF(2^128) elements in the ble format.
It will be used by chelsio driver to fasten gf multiplication.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 crypto/gf128mul.c | 13 +
 include/crypto/gf128mul.h |  2 +-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/crypto/gf128mul.c b/crypto/gf128mul.c
index dc01212..24e6019 100644
--- a/crypto/gf128mul.c
+++ b/crypto/gf128mul.c
@@ -156,6 +156,19 @@ static void gf128mul_x8_bbe(be128 *x)
x->b = cpu_to_be64((b << 8) ^ _tt);
 }
 
+void gf128mul_x8_ble(le128 *r, const le128 *x)
+{
+   u64 a = le64_to_cpu(x->a);
+   u64 b = le64_to_cpu(x->b);
+
+   /* equivalent to gf128mul_table_be[b >> 63] (see crypto/gf128mul.c): */
+   u64 _tt = gf128mul_table_be[a >> 56];
+
+   r->a = cpu_to_le64((a << 8) | (b >> 56));
+   r->b = cpu_to_le64((b << 8) ^ _tt);
+}
+EXPORT_SYMBOL(gf128mul_x8_ble);
+
 void gf128mul_lle(be128 *r, const be128 *b)
 {
be128 p[8];
diff --git a/include/crypto/gf128mul.h b/include/crypto/gf128mul.h
index 0977fb1..fa0a63d 100644
--- a/include/crypto/gf128mul.h
+++ b/include/crypto/gf128mul.h
@@ -227,7 +227,7 @@ struct gf128mul_4k *gf128mul_init_4k_lle(const be128 *g);
 struct gf128mul_4k *gf128mul_init_4k_bbe(const be128 *g);
 void gf128mul_4k_lle(be128 *a, const struct gf128mul_4k *t);
 void gf128mul_4k_bbe(be128 *a, const struct gf128mul_4k *t);
-
+void gf128mul_x8_ble(le128 *r, const le128 *x);
 static inline void gf128mul_free_4k(struct gf128mul_4k *t)
 {
kzfree(t);
-- 
2.1.4

[PATCH 0/7]crypto:chelsio: Bugs fixes

[Harsh Jain]

It includes bug fix and performance improvement changes.

Harsh Jain (7):
  crypto:gf128mul: The x8_ble multiplication functions
  crypto:chelsio:Use x8_ble gf multiplication to calculate IV.
  crypto:chelsio:Remove allocation of sg list to implement 2K limit of
dsgl header
  crypto:chelsio:Move DMA un/mapping to chcr from lld  cxgb4 driver
  crypto:chelsio: Fix memory leak
  crypto:chelsio: Remove unused parameter
  crypto:chelsio: Check error code with IS_ERR macro

 crypto/gf128mul.c|   13 +
 drivers/crypto/chelsio/chcr_algo.c   | 1784 +-
 drivers/crypto/chelsio/chcr_algo.h   |   57 +-
 drivers/crypto/chelsio/chcr_core.c   |8 +-
 drivers/crypto/chelsio/chcr_core.h   |2 +-
 drivers/crypto/chelsio/chcr_crypto.h |  121 +-
 drivers/net/ethernet/chelsio/cxgb4/sge.c |8 +-
 include/crypto/gf128mul.h|2 +-
 8 files changed, 1166 insertions(+), 829 deletions(-)

-- 
2.1.4

[PATCH 6/7] crypto:chelsio:Move DMA un/mapping to chcr from lld cxgb4 driver

[Harsh Jain]

Allow chcr to do DMA mapping/Unmapping instead of lld cxgb4.
It moves "Copy AAD to dst buffer" requirement from driver to
firmware.

Signed-off-by: Ganesh Goudar <ganes...@chelsio.com>
Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 1645 ++
 drivers/crypto/chelsio/chcr_algo.h   |   44 +-
 drivers/crypto/chelsio/chcr_crypto.h |  114 ++-
 drivers/net/ethernet/chelsio/cxgb4/sge.c |8 +-
 4 files changed, 1116 insertions(+), 695 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index b13991d..646dfff 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -70,6 +70,8 @@
 #include "chcr_algo.h"
 #include "chcr_crypto.h"
 
+#define IV AES_BLOCK_SIZE
+
 static inline  struct chcr_aead_ctx *AEAD_CTX(struct chcr_context *ctx)
 {
return ctx->crypto_ctx->aeadctx;
@@ -102,7 +104,7 @@ static inline struct uld_ctx *ULD_CTX(struct chcr_context 
*ctx)
 
 static inline int is_ofld_imm(const struct sk_buff *skb)
 {
-   return (skb->len <= CRYPTO_MAX_IMM_TX_PKT_LEN);
+   return (skb->len <= SGE_MAX_WR_LEN);
 }
 
 /*
@@ -117,21 +119,92 @@ static inline unsigned int sgl_len(unsigned int n)
return (3 * n) / 2 + (n & 1) + 2;
 }
 
-static int dstsg_2k(struct scatterlist *sgl, unsigned int reqlen)
+static int sg_nents_xlen(struct scatterlist *sg, unsigned int reqlen,
+unsigned int entlen,
+unsigned int skip)
 {
int nents = 0;
unsigned int less;
+   unsigned int skip_len = 0;
 
-   while (sgl && reqlen) {
-   less = min(reqlen, sgl->length);
-   nents += DIV_ROUND_UP(less, CHCR_SG_SIZE);
-   reqlen -= less;
-   sgl = sg_next(sgl);
+   while (sg && skip) {
+   if (sg_dma_len(sg) <= skip) {
+   skip -= sg_dma_len(sg);
+   skip_len = 0;
+   sg = sg_next(sg);
+   } else {
+   skip_len = skip;
+   skip = 0;
+   }
}
 
+   while (sg && reqlen) {
+   less = min(reqlen, sg_dma_len(sg) - skip_len);
+   nents += DIV_ROUND_UP(less, entlen);
+   reqlen -= less;
+   skip_len = 0;
+   sg = sg_next(sg);
+   }
return nents;
 }
 
+static inline void chcr_handle_ahash_resp(struct ahash_request *req,
+ unsigned char *input,
+ int err)
+{
+   struct chcr_ahash_req_ctx *reqctx = ahash_request_ctx(req);
+   int digestsize, updated_digestsize;
+   struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+   struct uld_ctx *u_ctx = ULD_CTX(h_ctx(tfm));
+
+   if (input == NULL)
+   goto out;
+   reqctx = ahash_request_ctx(req);
+   digestsize = crypto_ahash_digestsize(crypto_ahash_reqtfm(req));
+   if (reqctx->is_sg_map)
+   chcr_hash_dma_unmap(_ctx->lldi.pdev->dev, req);
+   if (reqctx->dma_addr)
+   dma_unmap_single(_ctx->lldi.pdev->dev, reqctx->dma_addr,
+reqctx->dma_len, DMA_TO_DEVICE);
+   reqctx->dma_addr = 0;
+   updated_digestsize = digestsize;
+   if (digestsize == SHA224_DIGEST_SIZE)
+   updated_digestsize = SHA256_DIGEST_SIZE;
+   else if (digestsize == SHA384_DIGEST_SIZE)
+   updated_digestsize = SHA512_DIGEST_SIZE;
+   if (reqctx->result == 1) {
+   reqctx->result = 0;
+   memcpy(req->result, input + sizeof(struct cpl_fw6_pld),
+  digestsize);
+   } else {
+   memcpy(reqctx->partial_hash, input + sizeof(struct cpl_fw6_pld),
+  updated_digestsize);
+   }
+out:
+   req->base.complete(>base, err);
+
+   }
+
+static inline void chcr_handle_aead_resp(struct aead_request *req,
+unsigned char *input,
+int err)
+{
+   struct chcr_aead_reqctx *reqctx = aead_request_ctx(req);
+   struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+   struct uld_ctx *u_ctx = ULD_CTX(a_ctx(tfm));
+
+
+   chcr_aead_dma_unmap(_ctx->lldi.pdev->dev, req, reqctx->op);
+   if (reqctx->b0_dma)
+   dma_unmap_single(_ctx->lldi.pdev->dev, reqctx->b0_dma,
+reqctx->b0_len, DMA_BIDIRECTIONAL);
+   if (reqctx->verify == VERIFY_SW) {
+   chcr_verify_tag(req, input, );
+   reqctx->verify = VERIFY_HW;
+}
+   req->base.complete(>base, err);
+
+}
 static void chcr_verify_tag(struct aead_request *req, u8 

[PATCH 7/7] crypto:chelsio: Fix memory leak

[Harsh Jain]

Fix memory leak when device does not support crypto.

Reported-by: Dan Carpenter <dan.carpen...@oracle.com>
Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_core.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_core.c 
b/drivers/crypto/chelsio/chcr_core.c
index b6dd9cb..4f677b3 100644
--- a/drivers/crypto/chelsio/chcr_core.c
+++ b/drivers/crypto/chelsio/chcr_core.c
@@ -154,15 +154,15 @@ static void *chcr_uld_add(const struct cxgb4_lld_info 
*lld)
struct uld_ctx *u_ctx;
 
/* Create the device and add it in the device list */
+   if (!(lld->ulp_crypto & ULP_CRYPTO_LOOKASIDE))
+   return ERR_PTR(-EOPNOTSUPP);
+
+   /* Create the device and add it in the device list */
u_ctx = kzalloc(sizeof(*u_ctx), GFP_KERNEL);
if (!u_ctx) {
u_ctx = ERR_PTR(-ENOMEM);
goto out;
}
-   if (!(lld->ulp_crypto & ULP_CRYPTO_LOOKASIDE)) {
-   u_ctx = ERR_PTR(-ENOMEM);
-   goto out;
-   }
u_ctx->lldi = *lld;
 out:
return u_ctx;
-- 
2.1.4

[PATCH 2/7] crypto:chelsio: Check error code with IS_ERR macro

[Harsh Jain]

Check and return proper error code.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index bdb1014..e4bf32d 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1455,8 +1455,8 @@ static int chcr_ahash_update(struct ahash_request *req)
req_ctx->result = 0;
req_ctx->data_len += params.sg_len + params.bfr_len;
skb = create_hash_wr(req, );
-   if (!skb)
-   return -ENOMEM;
+   if (IS_ERR(skb))
+   return PTR_ERR(skb);
 
if (remainder) {
u8 *temp;
@@ -1519,8 +1519,8 @@ static int chcr_ahash_final(struct ahash_request *req)
params.more = 0;
}
skb = create_hash_wr(req, );
-   if (!skb)
-   return -ENOMEM;
+   if (IS_ERR(skb))
+   return PTR_ERR(skb);
 
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_qidx);
@@ -1570,8 +1570,8 @@ static int chcr_ahash_finup(struct ahash_request *req)
}
 
skb = create_hash_wr(req, );
-   if (!skb)
-   return -ENOMEM;
+   if (IS_ERR(skb))
+   return PTR_ERR(skb);
 
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_qidx);
@@ -1621,8 +1621,8 @@ static int chcr_ahash_digest(struct ahash_request *req)
}
 
skb = create_hash_wr(req, );
-   if (!skb)
-   return -ENOMEM;
+   if (IS_ERR(skb))
+   return PTR_ERR(skb);
 
skb->dev = u_ctx->lldi.ports[0];
set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_qidx);
-- 
2.1.4

Re: [PATCH] iommu/vt-d: Fix scatterlist offset handling

[Harsh Jain]

Robin,

I tried running patch on our test setup.

With "intel_iommu=on" : I can see single occurrence of DMAR Write failure on 
perf traffic with 10 thread.

    [  749.616480] perf: interrupt took too long (3203 > 3202), lowering 
kernel.perf_event_max_sample_rate to 62000
[  852.500671] DMAR: DRHD: handling fault status reg 2
[  852.506039] DMAR: [DMA Write] Request device [02:00.4] fault addr ef919000 
[fault reason 05] PTE Write access is not set
[root@heptagon linux_t4_build]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.9.51+ root=UUID=ccbb7f18-b3f0-43df-89de-07521e9c02fe ro 
intel_iommu=on crashkernel=auto rhgb quiet rhgb quiet console=ttyS0,115200, 
console=tty0 LANG=en_US.UTF-8

With intel_iommu=sp_off : It works fine for more than 30 minutes without any 
issues.


On 28-09-2017 19:44, Robin Murphy wrote:
> The intel-iommu DMA ops fail to correctly handle scatterlists where
> sg->offset is greater than PAGE_SIZE - the IOVA allocation is computed
> appropriately based on the page-aligned portion of the offset, but the
> mapping is set up relative to sg->page, which means it fails to actually
> cover the whole buffer (and in the worst case doesn't cover it at all):
>
> (sg->dma_address + sg->dma_len) +
> sg->dma_address -+  |
> iov_pfn--+   |  |
>  |   |  |
>  v   v  v
> iova:   abcdef
> ||||||
>   <...calculated>
>  [_mapped__]
> pfn:012345
> ||||||
>  ^   ^  ^
>  |   |  |
> sg->page +   |  |
> sg->offset --+  |
> (sg->offset + sg->length) --+
>
> As a result, the caller ends up overrunning the mapping into whatever
> lies beyond, which usually goes badly:
>
> [  429.645492] DMAR: DRHD: handling fault status reg 2
> [  429.650847] DMAR: [DMA Write] Request device [02:00.4] fault addr f2682000 
> ...
>
> Whilst this is a fairly rare occurrence, it can happen from the result
> of intermediate scatterlist processing such as scatterwalk_ffwd() in the
> crypto layer. Whilst that particular site could be fixed up, it still
> seems worthwhile to bring intel-iommu in line with other DMA API
> implementations in handling this robustly.
>
> To that end, fix the intel_map_sg() path to line up the mapping
> correctly (in units of MM pages rather than VT-d pages to match the
> aligned_nrpages() calculation) regardless of the offset, and use
> sg_phys() consistently for clarity.
>
> Reported-by: Harsh Jain <ha...@chelsio.com>
> Signed-off-by: Robin Murphy <robin.mur...@arm.com>
> ---
>  drivers/iommu/intel-iommu.c | 8 +---
>  1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index 6784a05dd6b2..83f3d4831f94 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -2254,10 +2254,12 @@ static int __domain_mapping(struct dmar_domain 
> *domain, unsigned long iov_pfn,
>   uint64_t tmp;
>  
>   if (!sg_res) {
> + unsigned int pgoff = sg->offset & ~PAGE_MASK;
> +
>   sg_res = aligned_nrpages(sg->offset, sg->length);
> - sg->dma_address = ((dma_addr_t)iov_pfn << 
> VTD_PAGE_SHIFT) + sg->offset;
> + sg->dma_address = ((dma_addr_t)iov_pfn << 
> VTD_PAGE_SHIFT) + pgoff;
>   sg->dma_length = sg->length;
> - pteval = page_to_phys(sg_page(sg)) | prot;
> + pteval = (sg_phys(sg) - pgoff) | prot;
>   phys_pfn = pteval >> VTD_PAGE_SHIFT;
>   }
>  
> @@ -3790,7 +3792,7 @@ static int intel_nontranslate_map_sg(struct device 
> *hddev,
>  
>   for_each_sg(sglist, sg, nelems, i) {
>   BUG_ON(!sg_page(sg));
> - sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
> + sg->dma_address = sg_phys(sg);
>   sg->dma_length = sg->length;
>   }
>   return nelems;

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

On 28-09-2017 18:35, Raj, Ashok wrote:
> Thanks for trying that Harsh. 
>
> sp_off turns of super page support. Which this mode, do you still see offsets 
> greater than 4k?
Yes, offset greater than 4k is still there. Refer below.
[56732.774872] offset 4110 len 76 dma addr 3a531200e dma len 76
[56732.804187] offset 4110 len 84 dma addr 3a63b200e dma len 84
[56732.805104] offset 4110 len 68 dma addr 3a531200e dma len 68
[56732.806870] offset 4110 len 56 dma addr 3a531200e dma len 56
[56732.808987] offset 4110 len 56 dma addr 3a531200e dma len 56
[56732.811215] offset 4110 len 56 dma addr 3a531200e dma len 56
[56732.813155] offset 4110 len 56 dma addr 3a531200e dma len 56
[56732.814823] offset 4110 len 56 dma addr 3a531200e dma len 56
[56732.816481] offset 4110 len 56 dma addr 3a531200e dma len 56
[56732.818159] offset 4110 len 56 dma addr 3a531200e dma len 56
[56732.819712] offset 4110 len 56 dma addr 3a531200e dma len 56
[56732.821629] offset 4110 len 56 dma addr 3a531200e dma len 56
[root@heptagon linux_t4_build]#
[root@heptagon linux_t4_build]#
[root@heptagon linux_t4_build]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.9.51 root=UUID=ccbb7f18-b3f0-43df-89de-07521e9c02fe ro 
intel_iommu=sp_off crashkernel=auto rhgb quiet rhgb quiet console=ttyS0,115200, 
console=tty0 LANG=en_US.UTF-8

>
> On Thu, Sep 28, 2017 at 07:08:21PM +0530, Harsh Jain wrote:
>>
>> Today I tried with "Intel_iommu=sp_off" boot option. Traffic runs without 
>> any error for more than 1 hrs. What magic this option did? :)
> Cheers,
> Ashok

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

On 28-09-2017 02:59, Casey Leedom wrote:
> Hey Raj,
>
>   Let us know if you need help in gathering more debugging information.  For
> the time being we've decided to ERRATA the use of the Intel I/O MMU with
> IPsec till we Root Cause the issue.  But this is still at the top of Harsh's
> bug list.
>  
>   With Robin's comments, I'm almost sure that the:
>
> (iov_pfn + sg->offset) << VTD_PAGE_SHIFT)
>
> in your suggested patch is an issue.  iov_pfn is a Page Frame Number and
> sg->offset is a Byte Offset.  It feels like this should be:
>
> size_t page_off = sg->offset & ~VTD_PAGE_MASK;
> unsigned long pfn_off = sg->offset >> VTD_PAGE_MASK;
> ...
> sg->dma_address = ((dma_addr_t)
>(iov_pfn + pfn_off) << VTD_PAGE_SHIFT) + page_off;
>
> When Harsh tried your original patch, Harsh' test system wouldn't even boot.
Today I tried with "Intel_iommu=sp_off" boot option. Traffic runs without any 
error for more than 1 hrs. What magic this option did? :)
>
> Casey

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

On 26-09-2017 01:41, Dan Williams wrote:
> On Mon, Sep 25, 2017 at 1:05 PM, Casey Leedom  wrote:
>> | From: Dan Williams 
>> | Sent: Monday, September 25, 2017 12:31 PM
>> | ...
>> | IIUC it looks like this has been broken ever since commit e1605495c716
>> | "intel-iommu: Introduce domain_sg_mapping() to speed up
>> | intel_map_sg()". I.e. it looks like the calculation for pte_val should
>> | be:
>> |
>> | pteval = (page_to_phys(sg_page(sg)) + sg->offset) | prot;
>>
>> Hhmmm, shouldn't that be:
>>
>> pteval = (page_to_phys(sg_page(sg)) + (sg->offset>>PAGE_SHIFT)) | prot;
> Yes, I think you're right. We do want to mask off the page-unaligned
> portion of sg->offset.
Tried changing above line in "__domain_mapping" but didn't help.

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

Find attached new set of log. After repeated tries it panics.


On 26-09-2017 09:16, Harsh Jain wrote:
> On 26-09-2017 00:16, Casey Leedom wrote:
>> | From: Raj, Ashok <ashok@intel.com>
>> | Sent: Monday, September 25, 2017 8:54 AM
>> |
>> | Not sure how the page->offset would end up being greater than page-size?
> Refer below
>> |
>> | If you have additional traces, please send them by.
>> |
>> | Is this a new driver? wondering how we didn't run into this?
>>
>>   According to Herbert Xu and one of our own engineers, it's actually legal
>> for Scatter/Gather Lists to have this.  This isn't my area of expertise
>> though so I'm just passing that on.
>>
>>   I've asked our team to produce a detailed trace of the exact
>> Scatter/Gather Lists they're seeing and what ends up coming out of the DMA
>> Mappings, etc.  They're in India, so I expect that they'll have this for you
>> by tomorrow morning.
> Below mentioned log was already there in 1st mail. Copied here for easy 
> reference. Let me know if you need
> additional traces.
>
> 1) IN esp_output() "__skb_to_sgvec()" convert skb frags to scatter gather 
> list. 
> At that moment sg->offset was 4094.
> 2) From esp_output control reaches to "crypto_authenc_encrypt()". Here in 
> "scatterwalk_ffwd()" sg->offset become 4110.
> 3) Same sg list received by chelsio crypto driver(chcr). When chcr try to do 
> DMA mapping it starts giving DMA errors.
>
> Following error observed. first two prints are added for debugging in chcr. 
> Kernel version used to reproduce is 4.9.28 on x86_64 with Page size 4K.
>
> Sep 15 12:40:52 heptagon kernel: process_cipher req src 8803cb41f0a8
> Sep 15 12:40:52 heptagon kernel: = issuehit offset:4110 === 
> dma_addr f24b000e ==> DMA mapped address returned by dma_map_sg()
>
> Sep 15 12:40:52 heptagon kernel: DMAR: DRHD: handling fault status reg 2
> Sep 15 12:40:52 heptagon kernel: DMAR: [DMA Write] Request device [02:00.4] 
> fault addr f24b [fault reason 05] PTE Write access is not set
>
>> Casey

[root@heptagon linux_t4_build]# modprobe xts
[root@heptagon linux_t4_build]# modprobe authenc
[root@heptagon linux_t4_build]# insmod chcr/chcr.ko
[root@heptagon linux_t4_build]# ipsec restart
Stopping strongSwan IPsec failed: starter is not running
Starting strongSwan 5.5.1 IPsec [starter]...
[root@heptagon linux_t4_build]# ipsec up hep-chum
initiating IKE_SA hep-chum[1] to 1.0.0.95
generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) 
N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
sending packet: from 1.0.0.50[500] to 1.0.0.95[500] (804 bytes)
received packet: from 1.0.0.95[500] to 1.0.0.50[500] (617 bytes)
parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ 
N(FRAG_SUP) N(HASH_ALG) N(MULT_AUTH) ]
received cert request for "C=CH, O=Strongswan, CN=Strongswan CA"
sending cert request for "C=CH, O=Strongswan, CN=Strongswan CA"
authentication of 'C=CH, O=Strongswan, CN=host1' (myself) with 
RSA_EMSA_PKCS1_SHA2_256 successful
sending end entity cert "C=CH, O=Strongswan, CN=host1"
establishing CHILD_SA hep-chum
generating IKE_AUTH request 1 [ IDi CERT N(INIT_CONTACT) CERTREQ IDr AUTH SA 
TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_4_ADDR) N(ADD_4_ADDR) N(MULT_AUTH) 
N(EAP_ONLY) ]
splitting IKE message with length of 1440 bytes into 2 fragments
generating IKE_AUTH request 1 [ EF(1/2) ]
generating IKE_AUTH request 1 [ EF(2/2) ]
sending packet: from 1.0.0.50[4500] to 1.0.0.95[4500] (1236 bytes)
sending packet: from 1.0.0.50[4500] to 1.0.0.95[4500] (276 bytes)
received packet: from 1.0.0.95[4500] to 1.0.0.50[4500] (1236 bytes)
parsed IKE_AUTH response 1 [ EF(1/2) ]
received fragment #1 of 2, waiting for complete IKE message
received packet: from 1.0.0.95[4500] to 1.0.0.50[4500] (164 bytes)
parsed IKE_AUTH response 1 [ EF(2/2) ]
received fragment #2 of 2, reassembling fragmented IKE message
parsed IKE_AUTH response 1 [ IDr CERT AUTH SA TSi TSr N(AUTH_LFT) N(MOBIKE_SUP) 
N(ADD_4_ADDR) N(ADD_4_ADDR) ]
received end entity cert "C=CH, O=Strongswan, CN=host2"
  using certificate "C=CH, O=Strongswan, CN=host2"
  using trusted ca certificate "C=CH, O=Strongswan, CN=Strongswan CA"
checking certificate status of "C=CH, O=Strongswan, CN=host2"
certificate status is not available
  reached self-signed root ca with a path length of 0
authentication of 'C=CH, O=Strongswan, CN=host2' with RSA_EMSA_PKCS1_SHA2_256 
successful
IKE_SA hep-chum[1] established between 1.0.0.50[C=CH, O=Strongswan, 
CN=host1]...1.0.0.95[C=CH, O=Strongswan, CN=host2]
scheduling reauthentication in 3246s
maximum IKE_SA lifetime 3426s
CHILD_SA hep-chum{1} established with SPIs cfd0aa80_i c7c9816e_o and TS 
2.0.0.0/24 === 1.0.0.0/24

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

On 26-09-2017 00:15, David Woodhouse wrote:
> On Wed, 2017-09-20 at 16:01 +0800, Herbert Xu wrote:
>> Harsh Jain <ha...@chelsio.com> wrote:
>>>  
>>> While debugging DMA mapping error in chelsio crypto driver we
>> observed that when scatter/gather list received by driver has some
>> entry with page->offset > 4096 (PAGE_SIZE). It starts giving DMA
>> error.  Without IOMMU it works fine.
>>
>> This is not a bug.  The network stack can and will feed us such
>> SG lists.
> Hm? Under what circumstances is the offset permitted to be >
> PAGE_SIZE? 
Its random, Kernel API's don't check offset value after arithmetic operations 
like in "__skb_to_sgvec()", "scatterwalk_ffwd()".

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

On 26-09-2017 00:16, Casey Leedom wrote:
> | From: Raj, Ashok 
> | Sent: Monday, September 25, 2017 8:54 AM
> |
> | Not sure how the page->offset would end up being greater than page-size?
Refer below
> |
> | If you have additional traces, please send them by.
> |
> | Is this a new driver? wondering how we didn't run into this?
>
>   According to Herbert Xu and one of our own engineers, it's actually legal
> for Scatter/Gather Lists to have this.  This isn't my area of expertise
> though so I'm just passing that on.
>
>   I've asked our team to produce a detailed trace of the exact
> Scatter/Gather Lists they're seeing and what ends up coming out of the DMA
> Mappings, etc.  They're in India, so I expect that they'll have this for you
> by tomorrow morning.
Below mentioned log was already there in 1st mail. Copied here for easy 
reference. Let me know if you need
additional traces.

1) IN esp_output() "__skb_to_sgvec()" convert skb frags to scatter gather list. 
At that moment sg->offset was 4094.
2) From esp_output control reaches to "crypto_authenc_encrypt()". Here in 
"scatterwalk_ffwd()" sg->offset become 4110.
3) Same sg list received by chelsio crypto driver(chcr). When chcr try to do 
DMA mapping it starts giving DMA errors.

Following error observed. first two prints are added for debugging in chcr. 
Kernel version used to reproduce is 4.9.28 on x86_64 with Page size 4K.

Sep 15 12:40:52 heptagon kernel: process_cipher req src 8803cb41f0a8
Sep 15 12:40:52 heptagon kernel: = issuehit offset:4110 === 
dma_addr f24b000e ==> DMA mapped address returned by dma_map_sg()

Sep 15 12:40:52 heptagon kernel: DMAR: DRHD: handling fault status reg 2
Sep 15 12:40:52 heptagon kernel: DMAR: [DMA Write] Request device [02:00.4] 
fault addr f24b [fault reason 05] PTE Write access is not set

>
> Casey

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

On 20-09-2017 13:31, Herbert Xu wrote:
> Harsh Jain <ha...@chelsio.com> wrote:
>> While debugging DMA mapping error in chelsio crypto driver we observed that 
>> when scatter/gather list received by driver has some entry with page->offset 
>> > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works fine.
> This is not a bug.  The network stack can and will feed us such
> SG lists.
>
>> 2) It cannot be driver's responsibilty to update received sg entries to 
>> adjust offset and page 
>> because we are not the only one who directly uses received sg list.
> No the driver must deal with this.  Having said that, if we can
> improve our driver helper interface to make this easier then we
> should do that too.  What we certainly shouldn't do is to take a
> whack-a-mole approach like this patch does.
Agreed,I added that patch for understanding purpose only. Today I referred 
other crypto driver for DMA related code. Most of them are using dma_map_sg 
except QAT. In QAT, They are first updating the Page address using offset then 
mapping each page in for loop with dma_map_single(0. I will try the same in 
chelsio driver will see the behavior.
>
> Cheers,

Re: DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

On 20-09-2017 15:42, Robin Murphy wrote:
> On 20/09/17 09:01, Herbert Xu wrote:
>> Harsh Jain <ha...@chelsio.com> wrote:
>>> While debugging DMA mapping error in chelsio crypto driver we observed that 
>>> when scatter/gather list received by driver has some entry with 
>>> page->offset > 4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU 
>>> it works fine.
>> This is not a bug.  The network stack can and will feed us such
>> SG lists.
>>
>>> 2) It cannot be driver's responsibilty to update received sg entries to 
>>> adjust offset and page 
>>> because we are not the only one who directly uses received sg list.
>> No the driver must deal with this.  Having said that, if we can
>> improve our driver helper interface to make this easier then we
>> should do that too.  What we certainly shouldn't do is to take a
>> whack-a-mole approach like this patch does.
> AFAICS this is entirely on intel-iommu - from a brief look it appears
> that all the IOVA calculations would handle the offset correctly, but
> then __domain_mapping() blindly uses sg_page() for the physical address,
> so if offset is larger than a page it would end up with the DMA mapping
> covering the wrong part of the buffer.
>
> Does the diff below help?
>
> Robin.
>
> ->8-
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index b3914fce8254..2ed43d928135 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -2253,7 +2253,7 @@ static int __domain_mapping(struct dmar_domain *domain, 
> unsigned long iov_pfn,
>   sg_res = aligned_nrpages(sg->offset, sg->length);
>   sg->dma_address = ((dma_addr_t)iov_pfn << 
> VTD_PAGE_SHIFT) + sg->offset;
>   sg->dma_length = sg->length;
> - pteval = page_to_phys(sg_page(sg)) | prot;
> + pteval = (sg_phys(sg) & PAGE_MASK) | prot;
>   phys_pfn = pteval >> VTD_PAGE_SHIFT;
>   }
Robin,
Still having following error with above change.

[  429.645492] DMAR: DRHD: handling fault status reg 2
[  429.650847] DMAR: [DMA Write] Request device [02:00.4] fault addr f2682000 [t


>  

DMA error when sg->offset value is greater than PAGE_SIZE in Intel IOMMU

[Harsh Jain]

Hi,

While debugging DMA mapping error in chelsio crypto driver we observed that 
when scatter/gather list received by driver has some entry with page->offset > 
4096 (PAGE_SIZE). It starts giving DMA error.  Without IOMMU it works fine.

Before reaching to chelsio crypto driver(driver/crypto/chelsio) following 
entities change the sg'

1) IN esp_output() "__skb_to_sgvec()" convert skb frags to scatter gather list. 
At that moment sg->offset was 4094.
2) From esp_output control reaches to "crypto_authenc_encrypt()". Here in 
"scatterwalk_ffwd()" sg->offset become 4110.
3) Same sg list received by chelsio crypto driver(chcr). When chcr try to do 
DMA mapping it starts giving DMA errors.

Following error observed. first two prints are added for debugging in chcr. 
Kernel version used to reproduce is 4.9.28 on x86_64.

Sep 15 12:40:52 heptagon kernel: process_cipher req src 8803cb41f0a8
Sep 15 12:40:52 heptagon kernel: = issuehit offset:4110 === 
dma_addr f24b000e ==> DMA mapped address returned by dma_map_sg()

Sep 15 12:40:52 heptagon kernel: DMAR: DRHD: handling fault status reg 2
Sep 15 12:40:52 heptagon kernel: DMAR: [DMA Write] Request device [02:00.4] 
fault addr f24b [fault reason 05] PTE Write access is not set

 By applying following hack in kernel. Things start working.

diff --git a/crypto/scatterwalk.c b/crypto/scatterwalk.c
index c16c94f8..1d75a3a 100644
--- a/crypto/scatterwalk.c
+++ b/crypto/scatterwalk.c
@@ -78,6 +78,8 @@ struct scatterlist *scatterwalk_ffwd(struct scatterlist dst[2]
 struct scatterlist *src,
 unsigned int len)
 {
+   unsigned int mod_page_offset;
+
for (;;) {
if (!len)
return src;
@@ -90,7 +92,9 @@ struct scatterlist *scatterwalk_ffwd(struct scatterlist dst[2]
}

sg_init_table(dst, 2);
-   sg_set_page(dst, sg_page(src), src->length - len, src->offset + len);
+mod_page_offset = (src->offset + len) / PAGE_SIZE;
+   sg_set_page(dst, sg_page(src) + mod_page_offset, src->length - len,
+   (src->offset + len) - (mod_page_offset * PAGE_SIZE));
scatterwalk_crypto_chain(dst, sg_next(src), 0, 2);


1) We are not expecting issue in "scatterwalk_ffwd" because it is not the only 
place where kernel 
updates src->offset without checking page boundary. similar logic used in 
"__skb_to_sgvec".
 
2) It cannot be driver's responsibilty to update received sg entries to adjust 
offset and page 
because we are not the only one who directly uses received sg list.

3) Since Without IOMMU every thing works fine. We are expecting IOMMU bugs.

Regards

Harsh Jain

Updated IV for XTS

[Harsh Jain]

Hi,

XTS template does not seems updating the IV after request completion. Same 
output if we run belwo command in loop

[root@heptagon test]# ./kcapi -x 1 -d 4 -s  -e -c "xts(aes)" -i 
7fbc02ebf5b93322329df9bfccb635af -k 
8d7dd9b0170ce0b5f2f8e1aa768e01e91da8bfc67fd486d081b28254c99eb423 -p `perl -e 
'print "e" x  32'`
4ea328cd5b10d0cb3bbc7ab92d54072d
4ea328cd5b10d0cb3bbc7ab92d54072d
4ea328cd5b10d0cb3bbc7ab92d54072d
4ea328cd5b10d0cb3bbc7ab92d54072d

Is IV update not required for XTS as we did for cbc,ctr mode.

Regards
Harsh Jain

Re: [PATCH v7 00/19] simplify crypto wait for async op

[Harsh Jain]

On Sun, Sep 3, 2017 at 11:47 AM, Gilad Ben-Yossef <gi...@benyossef.com> wrote:
> On Thu, Aug 31, 2017 at 3:31 PM, Harsh Jain <harshjain.p...@gmail.com> wrote:
>> HI Gilad,
>>
>> I think we need an update in ESP also. Now EBUSY return means driver
>> has accepted, Packet should not be dropped in
>>
>> esp_output_tail() function.
>
> Good catch. You are right and the same holds true for ah_output() in ah4.c.
>
> But I do wonder, the code there now treats -EBUSY as a special case
> and returns NET_XMIT_DROP
> but if an AEAD or AHASH transformation return some other error, like
> -ENOMEM or -EINVAL shouldn't
> we return NET_XMIT_DROP in that case too?
I think we should not,  XMIT_DROP implies drop current packet only,
later on when device is recovered from busy state, Upper layer
protocol(TCP) will re-transmit the packet. It helps in flow control.
>
> Any ideas?
>
> Gilad
>
>>
>>
>> On Thu, Aug 24, 2017 at 7:48 PM, Gilad Ben-Yossef <gi...@benyossef.com> 
>> wrote:
>>> Many users of kernel async. crypto services have a pattern of
>>> starting an async. crypto op and than using a completion
>>> to wait for it to end.
>>>
>>> This patch set simplifies this common use case in two ways:
>>>
>>> First, by separating the return codes of the case where a
>>> request is queued to a backlog due to the provider being
>>> busy (-EBUSY) from the case the request has failed due
>>> to the provider being busy and backlogging is not enabled
>>> (-EAGAIN).
>>>
>>> Next, this change is than built on to create a generic API
>>> to wait for a async. crypto operation to complete.
>>>
>>> The end result is a smaller code base and an API that is
>>> easier to use and more difficult to get wrong.
>>>
>>> The patch set was boot tested on x86_64 and arm64 which
>>> at the very least tests the crypto users via testmgr and
>>> tcrypt but I do note that I do not have access to some
>>> of the HW whose drivers are modified nor do I claim I was
>>> able to test all of the corner cases.
>>>
>>> The patch set is based upon linux-next release tagged
>>> next-20170824.
>>>
>>> Changes from v6:
>>> - Fix brown paper bag compile error on marvell/cesa
>>>   code.
>>>
>>> Changes from v5:
>>> - Remove redundant new line as spotted by Jonathan
>>>   Cameron.
>>> - Reworded dm-verity change commit message to better
>>>   clarify potential issue averted by change as
>>>   pointed out by Mikulas Patocka.
>>>
>>> Changes from v4:
>>> - Rebase on top of latest algif changes from Stephan
>>>   Mueller.
>>> - Fix typo in ccp patch title.
>>>
>>> Changes from v3:
>>> - Instead of changing the return code to indicate
>>>   backlog queueing, change the return code to indicate
>>>   transient busy state, as suggested by Herbert Xu.
>>>
>>> Changes from v2:
>>> - Patch title changed from "introduce crypto wait for
>>>   async op" to better reflect the current state.
>>> - Rebase on top of latest linux-next.
>>> - Add a new return code of -EIOCBQUEUED for backlog
>>>   queueing, as suggested by Herbert Xu.
>>> - Transform more users to the new API.
>>> - Update the drbg change to account for new init as
>>>   indicated by Stephan Muller.
>>>
>>> Changes from v1:
>>> - Address review comments from Eric Biggers.
>>> - Separated out bug fixes of existing code and rebase
>>>   on top of that patch set.
>>> - Rename 'ecr' to 'wait' in fscrypto code.
>>> - Split patch introducing the new API from the change
>>>   moving over the algif code which it originated from
>>>   to the new API.
>>> - Inline crypto_wait_req().
>>> - Some code indentation fixes.
>>>
>>> Gilad Ben-Yossef (19):
>>>   crypto: change transient busy return code to -EAGAIN
>>>   crypto: ccp: use -EAGAIN for transient busy indication
>>>   crypto: remove redundant backlog checks on EBUSY
>>>   crypto: marvell/cesa: remove redundant backlog checks on EBUSY
>>>   crypto: introduce crypto wait for async op
>>>   crypto: move algif to generic async completion
>>>   crypto: move pub key to generic async completion
>>>   crypto: move drbg to generic async completion
>>>   crypto: move gcm to generic async completion
>>>   crypto: move testmgr to generic async completion
>

Re: [PATCH v7 00/19] simplify crypto wait for async op

[Harsh Jain]

HI Gilad,

I think we need an update in ESP also. Now EBUSY return means driver
has accepted, Packet should not be dropped in

esp_output_tail() function.

.

Regards
Harsh Jain



On Thu, Aug 24, 2017 at 7:48 PM, Gilad Ben-Yossef <gi...@benyossef.com> wrote:
> Many users of kernel async. crypto services have a pattern of
> starting an async. crypto op and than using a completion
> to wait for it to end.
>
> This patch set simplifies this common use case in two ways:
>
> First, by separating the return codes of the case where a
> request is queued to a backlog due to the provider being
> busy (-EBUSY) from the case the request has failed due
> to the provider being busy and backlogging is not enabled
> (-EAGAIN).
>
> Next, this change is than built on to create a generic API
> to wait for a async. crypto operation to complete.
>
> The end result is a smaller code base and an API that is
> easier to use and more difficult to get wrong.
>
> The patch set was boot tested on x86_64 and arm64 which
> at the very least tests the crypto users via testmgr and
> tcrypt but I do note that I do not have access to some
> of the HW whose drivers are modified nor do I claim I was
> able to test all of the corner cases.
>
> The patch set is based upon linux-next release tagged
> next-20170824.
>
> Changes from v6:
> - Fix brown paper bag compile error on marvell/cesa
>   code.
>
> Changes from v5:
> - Remove redundant new line as spotted by Jonathan
>   Cameron.
> - Reworded dm-verity change commit message to better
>   clarify potential issue averted by change as
>   pointed out by Mikulas Patocka.
>
> Changes from v4:
> - Rebase on top of latest algif changes from Stephan
>   Mueller.
> - Fix typo in ccp patch title.
>
> Changes from v3:
> - Instead of changing the return code to indicate
>   backlog queueing, change the return code to indicate
>   transient busy state, as suggested by Herbert Xu.
>
> Changes from v2:
> - Patch title changed from "introduce crypto wait for
>   async op" to better reflect the current state.
> - Rebase on top of latest linux-next.
> - Add a new return code of -EIOCBQUEUED for backlog
>   queueing, as suggested by Herbert Xu.
> - Transform more users to the new API.
> - Update the drbg change to account for new init as
>   indicated by Stephan Muller.
>
> Changes from v1:
> - Address review comments from Eric Biggers.
> - Separated out bug fixes of existing code and rebase
>   on top of that patch set.
> - Rename 'ecr' to 'wait' in fscrypto code.
> - Split patch introducing the new API from the change
>   moving over the algif code which it originated from
>   to the new API.
> - Inline crypto_wait_req().
> - Some code indentation fixes.
>
> Gilad Ben-Yossef (19):
>   crypto: change transient busy return code to -EAGAIN
>   crypto: ccp: use -EAGAIN for transient busy indication
>   crypto: remove redundant backlog checks on EBUSY
>   crypto: marvell/cesa: remove redundant backlog checks on EBUSY
>   crypto: introduce crypto wait for async op
>   crypto: move algif to generic async completion
>   crypto: move pub key to generic async completion
>   crypto: move drbg to generic async completion
>   crypto: move gcm to generic async completion
>   crypto: move testmgr to generic async completion
>   fscrypt: move to generic async completion
>   dm: move dm-verity to generic async completion
>   cifs: move to generic async completion
>   ima: move to generic async completion
>   crypto: tcrypt: move to generic async completion
>   crypto: talitos: move to generic async completion
>   crypto: qce: move to generic async completion
>   crypto: mediatek: move to generic async completion
>   crypto: adapt api sample to use async. op wait
>
>  Documentation/crypto/api-samples.rst |  52 ++---
>  crypto/af_alg.c  |  27 -
>  crypto/ahash.c   |  12 +--
>  crypto/algapi.c  |   6 +-
>  crypto/algif_aead.c  |   8 +-
>  crypto/algif_hash.c  |  50 +
>  crypto/algif_skcipher.c  |   9 +-
>  crypto/api.c |  13 +++
>  crypto/asymmetric_keys/public_key.c  |  28 +
>  crypto/cryptd.c  |   4 +-
>  crypto/cts.c |   6 +-
>  crypto/drbg.c|  36 ++-
>  crypto/gcm.c |  32 ++
>  crypto/lrw.c |   8 +-
>  crypto/rsa-pkcs1pad.c|  16 +--
>  crypto/tcrypt.c  |  84 +--
>  crypto/testmgr.c | 204 
> ---
>  crypto/xts.c   

Re: [bug report] crypto: chcr - Select device in Round Robin fashion

[Harsh Jain]

On 20-07-2017 16:02, Dan Carpenter wrote:
> Hello Harsh Jain,
>
> The patch 14c19b178a01: "crypto: chcr - Select device in Round Robin
> fashion" from Jun 15, 2017, leads to the following static checker
> warning:
>
>   drivers/crypto/chelsio/chcr_core.c:163 chcr_uld_add()
>   warn: overwrite may leak 'u_ctx'
>
> drivers/crypto/chelsio/chcr_core.c
>152  static void *chcr_uld_add(const struct cxgb4_lld_info *lld)
>153  {
>154  struct uld_ctx *u_ctx;
>155  
>156  /* Create the device and add it in the device list */
>157  u_ctx = kzalloc(sizeof(*u_ctx), GFP_KERNEL);
>158  if (!u_ctx) {
>159  u_ctx = ERR_PTR(-ENOMEM);
>160  goto out;
>161  }
>162  if (!(lld->ulp_crypto & ULP_CRYPTO_LOOKASIDE)) {
>
> Sure, we could move this check before the allocation, to prevent the
> leak but is -ENOMEM really the right error code?  It feels like -EINVAL
> with a WARN_ON_ONCE() message would be better but I don't really
> understand this code.
Will fix both issues in next change set. Thanks.
>
>163  u_ctx = ERR_PTR(-ENOMEM);
>164  goto out;
>165  }
>166  u_ctx->lldi = *lld;
>167  out:
>168  return u_ctx;
>169  }
>
> regards,
> dan carpenter

[PATCH] crypto: chcr: Avoid algo allocation in softirq.

[Harsh Jain]

Thsi patch fixes calling "crypto_alloc_cipher" call in bottom halves.
Pre allocate aes cipher required to update Tweak value for XTS.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 23 +++
 drivers/crypto/chelsio/chcr_crypto.h |  1 +
 2 files changed, 16 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index aa4e5b8..508cbc7 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -899,26 +899,20 @@ static int chcr_update_tweak(struct ablkcipher_request 
*req, u8 *iv)
u8 *key;
unsigned int keylen;
 
-   cipher = crypto_alloc_cipher("aes-generic", 0, 0);
+   cipher = ablkctx->aes_generic;
memcpy(iv, req->info, AES_BLOCK_SIZE);
 
-   if (IS_ERR(cipher)) {
-   ret = -ENOMEM;
-   goto out;
-   }
keylen = ablkctx->enckey_len / 2;
key = ablkctx->key + keylen;
ret = crypto_cipher_setkey(cipher, key, keylen);
if (ret)
-   goto out1;
+   goto out;
 
crypto_cipher_encrypt_one(cipher, iv, iv);
for (i = 0; i < (reqctx->processed / AES_BLOCK_SIZE); i++)
gf128mul_x_ble((le128 *)iv, (le128 *)iv);
 
crypto_cipher_decrypt_one(cipher, iv, iv);
-out1:
-   crypto_free_cipher(cipher);
 out:
return ret;
 }
@@ -1262,6 +1256,17 @@ static int chcr_cra_init(struct crypto_tfm *tfm)
pr_err("failed to allocate fallback for %s\n", alg->cra_name);
return PTR_ERR(ablkctx->sw_cipher);
}
+
+   if (get_cryptoalg_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_XTS) {
+   /* To update tweak*/
+   ablkctx->aes_generic = crypto_alloc_cipher("aes-generic", 0, 0);
+   if (IS_ERR(ablkctx->aes_generic)) {
+   pr_err("failed to allocate aes cipher for tweak\n");
+   return PTR_ERR(ablkctx->aes_generic);
+   }
+   } else
+   ablkctx->aes_generic = NULL;
+
tfm->crt_ablkcipher.reqsize =  sizeof(struct chcr_blkcipher_req_ctx);
return chcr_device_init(crypto_tfm_ctx(tfm));
 }
@@ -1292,6 +1297,8 @@ static void chcr_cra_exit(struct crypto_tfm *tfm)
struct ablk_ctx *ablkctx = ABLK_CTX(ctx);
 
crypto_free_skcipher(ablkctx->sw_cipher);
+   if (ablkctx->aes_generic)
+   crypto_free_cipher(ablkctx->aes_generic);
 }
 
 static int get_alg_config(struct algo_param *params,
diff --git a/drivers/crypto/chelsio/chcr_crypto.h 
b/drivers/crypto/chelsio/chcr_crypto.h
index a4f95b0..30af1ee 100644
--- a/drivers/crypto/chelsio/chcr_crypto.h
+++ b/drivers/crypto/chelsio/chcr_crypto.h
@@ -155,6 +155,7 @@
 
 struct ablk_ctx {
struct crypto_skcipher *sw_cipher;
+   struct crypto_cipher *aes_generic;
__be32 key_ctx_hdr;
unsigned int enckey_len;
unsigned char ciph_mode;
-- 
2.1.4

[PATCH 0/9] Bug fixes and ctr mode of operation

[Harsh Jain]

This series is based on cryptodev2.6 tree and includes bug fix ,ctr(aes), 
rfc3686(ctr(aes)) algo.

Harsh Jain (7):
  crypto: chcr - Pass lcb bit setting to firmware
  crypto: chcr - Set fallback key
  crypto: chcr - Return correct error code
  crypto: chcr - Avoid changing request structure
  crypto:chcr - Add ctr mode and process large sg entries for cipher
  MAINTAINERS:Add maintainer for chelsio crypto driver
  crypto: chcr - Ensure Destination sg entry size less than  2k
Atul Gupta (2):
  chcr - Add debug counters
  crypto: chcr - Select device in Round Robin fashion

 MAINTAINERS|7 +
 drivers/crypto/chelsio/chcr_algo.c | 1096 
 drivers/crypto/chelsio/chcr_algo.h |   30 +-
 drivers/crypto/chelsio/chcr_core.c |   56 +-
 drivers/crypto/chelsio/chcr_core.h |5 +-
 drivers/crypto/chelsio/chcr_crypto.h   |   25 +-
 drivers/net/ethernet/chelsio/cxgb4/cxgb4.h |1 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_debugfs.c |   35 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.c |1 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h |   10 +
 10 files changed, 1020 insertions(+), 246 deletions(-)

-- 
1.8.3.1

[PATCH 2/9] crypto: chcr - Fix fallback key setting

[Harsh Jain]

Set key of fallback tfm for rfc4309.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 12 +++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index e8ff505..14641c6 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -2210,7 +2210,8 @@ static int chcr_aead_rfc4309_setkey(struct crypto_aead 
*aead, const u8 *key,
unsigned int keylen)
 {
struct chcr_context *ctx = crypto_aead_ctx(aead);
-struct chcr_aead_ctx *aeadctx = AEAD_CTX(ctx);
+   struct chcr_aead_ctx *aeadctx = AEAD_CTX(ctx);
+   int error;
 
if (keylen < 3) {
crypto_tfm_set_flags((struct crypto_tfm *)aead,
@@ -2218,6 +2219,15 @@ static int chcr_aead_rfc4309_setkey(struct crypto_aead 
*aead, const u8 *key,
aeadctx->enckey_len = 0;
return  -EINVAL;
}
+   crypto_aead_clear_flags(aeadctx->sw_cipher, CRYPTO_TFM_REQ_MASK);
+   crypto_aead_set_flags(aeadctx->sw_cipher, crypto_aead_get_flags(aead) &
+ CRYPTO_TFM_REQ_MASK);
+   error = crypto_aead_setkey(aeadctx->sw_cipher, key, keylen);
+   crypto_aead_clear_flags(aead, CRYPTO_TFM_RES_MASK);
+   crypto_aead_set_flags(aead, crypto_aead_get_flags(aeadctx->sw_cipher) &
+ CRYPTO_TFM_RES_MASK);
+   if (error)
+   return error;
keylen -= 3;
memcpy(aeadctx->salt, key + keylen, 3);
return chcr_ccm_common_setkey(aead, key, keylen);
-- 
1.8.3.1

[PATCH 3/9] crypto: chcr - Return correct error code

[Harsh Jain]

Return correct error instead of EINVAL.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 76 +-
 1 file changed, 42 insertions(+), 34 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 14641c6..156065d 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1399,7 +1399,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
unsigned short stop_offset = 0;
unsigned int  assoclen = req->assoclen;
unsigned int  authsize = crypto_aead_authsize(tfm);
-   int err = -EINVAL, src_nent;
+   int error = -EINVAL, src_nent;
int null = 0;
gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL :
GFP_ATOMIC;
@@ -1416,9 +1416,9 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
reqctx->dst = src;
 
if (req->src != req->dst) {
-   err = chcr_copy_assoc(req, aeadctx);
-   if (err)
-   return ERR_PTR(err);
+   error = chcr_copy_assoc(req, aeadctx);
+   if (error)
+   return ERR_PTR(error);
reqctx->dst = scatterwalk_ffwd(reqctx->dstffwd, req->dst,
   req->assoclen);
}
@@ -1430,6 +1430,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
 (op_type ? -authsize : authsize));
if (reqctx->dst_nents < 0) {
pr_err("AUTHENC:Invalid Destination sg entries\n");
+   error = -EINVAL;
goto err;
}
dst_size = get_space_for_phys_dsgl(reqctx->dst_nents);
@@ -1443,8 +1444,10 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
return ERR_PTR(chcr_aead_fallback(req, op_type));
}
skb = alloc_skb((transhdr_len + sizeof(struct sge_opaque_hdr)), flags);
-   if (!skb)
+   if (!skb) {
+   error = -ENOMEM;
goto err;
+   }
 
/* LLD is going to write the sge hdr. */
skb_reserve(skb, sizeof(struct sge_opaque_hdr));
@@ -1496,9 +1499,9 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
sg_param.nents = reqctx->dst_nents;
sg_param.obsize = req->cryptlen + (op_type ? -authsize : authsize);
sg_param.qid = qid;
-   sg_param.align = 0;
-   if (map_writesg_phys_cpl(_ctx->lldi.pdev->dev, phys_cpl, reqctx->dst,
- _param))
+   error = map_writesg_phys_cpl(_ctx->lldi.pdev->dev, phys_cpl,
+   reqctx->dst, _param);
+   if (error)
goto dstmap_fail;
 
skb_set_transport_header(skb, transhdr_len);
@@ -1520,7 +1523,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
/* ivmap_fail: */
kfree_skb(skb);
 err:
-   return ERR_PTR(-EINVAL);
+   return ERR_PTR(error);
 }
 
 static int set_msg_len(u8 *block, unsigned int msglen, int csize)
@@ -1730,7 +1733,7 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_request *req,
unsigned int dst_size = 0, kctx_len;
unsigned int sub_type;
unsigned int authsize = crypto_aead_authsize(tfm);
-   int err = -EINVAL, src_nent;
+   int error = -EINVAL, src_nent;
gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL :
GFP_ATOMIC;
 
@@ -1746,10 +1749,10 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_request *req,
reqctx->dst = src;
 
if (req->src != req->dst) {
-   err = chcr_copy_assoc(req, aeadctx);
-   if (err) {
+   error = chcr_copy_assoc(req, aeadctx);
+   if (error) {
pr_err("AAD copy to destination buffer fails\n");
-   return ERR_PTR(err);
+   return ERR_PTR(error);
}
reqctx->dst = scatterwalk_ffwd(reqctx->dstffwd, req->dst,
   req->assoclen);
@@ -1758,11 +1761,11 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_request *req,
 (op_type ? -authsize : authsize));
if (reqctx->dst_nents < 0) {
pr_err("CCM:Invalid Destination sg entries\n");
+   error = -EINVAL;
goto err;
}
-
-
-   if (aead_ccm_validate_input(op_type, req, aeadctx, sub_type))
+   error = aead_ccm_validate_input(op_type, req, aeadctx, sub_type);
+   if (error)
goto err;
 
dst_size = get_space_for_phys_dsgl(reqctx->dst_nents);
@@ 

[PATCH 6/9] chcr - Add debug counters

[Harsh Jain]

Count types of operation done by HW.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 16 +-
 drivers/crypto/chelsio/chcr_core.c |  2 ++
 drivers/net/ethernet/chelsio/cxgb4/cxgb4.h |  1 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_debugfs.c | 35 ++
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h |  9 ++
 5 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 03b817f..2f388af 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -154,6 +154,7 @@ int chcr_handle_resp(struct crypto_async_request *req, 
unsigned char *input,
struct uld_ctx *u_ctx = ULD_CTX(ctx);
struct chcr_req_ctx ctx_req;
unsigned int digestsize, updated_digestsize;
+   struct adapter *adap = padap(ctx->dev);
 
switch (tfm->__crt_alg->cra_flags & CRYPTO_ALG_TYPE_MASK) {
case CRYPTO_ALG_TYPE_AEAD:
@@ -207,6 +208,7 @@ int chcr_handle_resp(struct crypto_async_request *req, 
unsigned char *input,
ctx_req.req.ahash_req->base.complete(req, err);
break;
}
+   atomic_inc(>chcr_stats.complete);
return err;
 }
 
@@ -639,6 +641,7 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
unsigned int ivsize = AES_BLOCK_SIZE, kctx_len;
gfp_t flags = wrparam->req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ?
GFP_KERNEL : GFP_ATOMIC;
+   struct adapter *adap = padap(ctx->dev);
 
phys_dsgl = get_space_for_phys_dsgl(reqctx->dst_nents);
 
@@ -701,6 +704,7 @@ static struct sk_buff *create_cipher_wr(struct 
cipher_wr_param *wrparam)
skb_set_transport_header(skb, transhdr_len);
write_buffer_to_skb(skb, , reqctx->iv, ivsize);
write_sg_to_skb(skb, , wrparam->srcsg, wrparam->bytes);
+   atomic_inc(>chcr_stats.cipher_rqst);
create_wreq(ctx, chcr_req, &(wrparam->req->base), skb, kctx_len, 0, 1,
sizeof(struct cpl_rx_phys_dsgl) + phys_dsgl,
ablkctx->ciph_mode == CHCR_SCMD_CIPHER_MODE_AES_CBC);
@@ -1337,6 +1341,7 @@ static struct sk_buff *create_hash_wr(struct 
ahash_request *req,
u8 hash_size_in_response = 0;
gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL :
GFP_ATOMIC;
+   struct adapter *adap = padap(ctx->dev);
 
iopad_alignment = KEYCTX_ALIGN_PAD(digestsize);
kctx_len = param->alg_prm.result_size + iopad_alignment;
@@ -1393,7 +1398,7 @@ static struct sk_buff *create_hash_wr(struct 
ahash_request *req,
param->bfr_len);
if (param->sg_len != 0)
write_sg_to_skb(skb, , req->src, param->sg_len);
-
+   atomic_inc(>chcr_stats.digest_rqst);
create_wreq(ctx, chcr_req, >base, skb, kctx_len,
hash_size_in_response, 0, DUMMY_BYTES, 0);
req_ctx->skb = skb;
@@ -1873,6 +1878,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
int null = 0;
gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL :
GFP_ATOMIC;
+   struct adapter *adap = padap(ctx->dev);
 
if (aeadctx->enckey_len == 0 || (req->cryptlen == 0))
goto err;
@@ -1911,6 +1917,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
T6_MAX_AAD_SIZE,
transhdr_len + (sgl_len(src_nent + MIN_AUTH_SG) * 8),
op_type)) {
+   atomic_inc(>chcr_stats.fallback);
return ERR_PTR(chcr_aead_fallback(req, op_type));
}
skb = alloc_skb((transhdr_len + sizeof(struct sge_opaque_hdr)), flags);
@@ -1983,6 +1990,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
}
write_buffer_to_skb(skb, , req->iv, ivsize);
write_sg_to_skb(skb, , src, req->cryptlen);
+   atomic_inc(>chcr_stats.cipher_rqst);
create_wreq(ctx, chcr_req, >base, skb, kctx_len, size, 1,
   sizeof(struct cpl_rx_phys_dsgl) + dst_size, 0);
reqctx->skb = skb;
@@ -2206,6 +2214,7 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_request *req,
int error = -EINVAL, src_nent;
gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL :
GFP_ATOMIC;
+   struct adapter *adap = padap(ctx->dev);
 
 
if (op_type && req->cryptlen < crypto_aead_authsize(tfm))
@@ -2245,6 +2254,7 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_request *req,
T6_MAX_AAD_SIZE - 18,
  

[PATCH 7/9] MAINTAINERS:Add maintainer for chelsio crypto driver

[Harsh Jain]

Add myself as maintainer for chcr.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 MAINTAINERS | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 1f20176..504dc65 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3706,6 +3706,13 @@ S:   Supported
 F: drivers/infiniband/hw/cxgb4/
 F: include/uapi/rdma/cxgb4-abi.h
 
+CXGB4 CRYPTO DRIVER (chcr)
+M: Harsh Jain <ha...@chelsio.com>
+L: linux-crypto@vger.kernel.org
+W: http://www.chelsio.com
+S: Supported
+F: drivers/crypto/chelsio
+
 CXGB4VF ETHERNET DRIVER (CXGB4VF)
 M: Casey Leedom <lee...@chelsio.com>
 L: net...@vger.kernel.org
-- 
1.8.3.1

[PATCH 5/9] crypto:chcr - Add ctr mode and process large sg entries for cipher

[Harsh Jain]

It send multiple WRs to H/W to handle large sg lists. Adds ctr(aes)
and rfc(ctr(aes)) modes.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 786 ---
 drivers/crypto/chelsio/chcr_algo.h   |  26 +-
 drivers/crypto/chelsio/chcr_core.c   |   1 -
 drivers/crypto/chelsio/chcr_core.h   |   3 +
 drivers/crypto/chelsio/chcr_crypto.h |  19 +-
 5 files changed, 690 insertions(+), 145 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 9c839c6..03b817f 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -55,6 +55,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 #include 
 #include 
 #include 
@@ -151,12 +153,11 @@ int chcr_handle_resp(struct crypto_async_request *req, 
unsigned char *input,
struct chcr_context *ctx = crypto_tfm_ctx(tfm);
struct uld_ctx *u_ctx = ULD_CTX(ctx);
struct chcr_req_ctx ctx_req;
-   struct cpl_fw6_pld *fw6_pld;
unsigned int digestsize, updated_digestsize;
 
switch (tfm->__crt_alg->cra_flags & CRYPTO_ALG_TYPE_MASK) {
case CRYPTO_ALG_TYPE_AEAD:
-   ctx_req.req.aead_req = (struct aead_request *)req;
+   ctx_req.req.aead_req = aead_request_cast(req);
ctx_req.ctx.reqctx = aead_request_ctx(ctx_req.req.aead_req);
dma_unmap_sg(_ctx->lldi.pdev->dev, ctx_req.ctx.reqctx->dst,
 ctx_req.ctx.reqctx->dst_nents, DMA_FROM_DEVICE);
@@ -169,27 +170,16 @@ int chcr_handle_resp(struct crypto_async_request *req, 
unsigned char *input,
);
ctx_req.ctx.reqctx->verify = VERIFY_HW;
}
+   ctx_req.req.aead_req->base.complete(req, err);
break;
 
case CRYPTO_ALG_TYPE_ABLKCIPHER:
-   ctx_req.req.ablk_req = (struct ablkcipher_request *)req;
-   ctx_req.ctx.ablk_ctx =
-   ablkcipher_request_ctx(ctx_req.req.ablk_req);
-   if (!err) {
-   fw6_pld = (struct cpl_fw6_pld *)input;
-   memcpy(ctx_req.req.ablk_req->info, _pld->data[2],
-  AES_BLOCK_SIZE);
-   }
-   dma_unmap_sg(_ctx->lldi.pdev->dev, ctx_req.req.ablk_req->dst,
-ctx_req.ctx.ablk_ctx->dst_nents, DMA_FROM_DEVICE);
-   if (ctx_req.ctx.ablk_ctx->skb) {
-   kfree_skb(ctx_req.ctx.ablk_ctx->skb);
-   ctx_req.ctx.ablk_ctx->skb = NULL;
-   }
+err = chcr_handle_cipher_resp(ablkcipher_request_cast(req),
+  input, err);
break;
 
case CRYPTO_ALG_TYPE_AHASH:
-   ctx_req.req.ahash_req = (struct ahash_request *)req;
+   ctx_req.req.ahash_req = ahash_request_cast(req);
ctx_req.ctx.ahash_ctx =
ahash_request_ctx(ctx_req.req.ahash_req);
digestsize =
@@ -214,6 +204,7 @@ int chcr_handle_resp(struct crypto_async_request *req, 
unsigned char *input,
   sizeof(struct cpl_fw6_pld),
   updated_digestsize);
}
+   ctx_req.req.ahash_req->base.complete(req, err);
break;
}
return err;
@@ -392,7 +383,7 @@ static void write_phys_cpl(struct cpl_rx_phys_dsgl 
*phys_cpl,
   struct phys_sge_parm *sg_param)
 {
struct phys_sge_pairs *to;
-   int out_buf_size = sg_param->obsize;
+   unsigned int len = 0, left_size = sg_param->obsize;
unsigned int nents = sg_param->nents, i, j = 0;
 
phys_cpl->op_to_tid = htonl(CPL_RX_PHYS_DSGL_OPCODE_V(CPL_RX_PHYS_DSGL)
@@ -409,20 +400,15 @@ static void write_phys_cpl(struct cpl_rx_phys_dsgl 
*phys_cpl,
phys_cpl->rss_hdr_int.hash_val = 0;
to = (struct phys_sge_pairs *)((unsigned char *)phys_cpl +
   sizeof(struct cpl_rx_phys_dsgl));
-
-   for (i = 0; nents; to++) {
-   for (j = 0; j < 8 && nents; j++, nents--) {
-   out_buf_size -= sg_dma_len(sg);
-   to->len[j] = htons(sg_dma_len(sg));
+   for (i = 0; nents && left_size; to++) {
+   for (j = 0; j < 8 && nents && left_size; j++, nents--) {
+   len = min(left_size, sg_dma_len(sg));
+   to->len[j] = htons(len);
to->addr[j] = cpu_to_be64(sg_dma_address(sg));
+   left_size -= len;
sg = sg_next(sg);
}
}
-   if (out_buf_size) {
-   j--;
-

[PATCH 9/9] crypto: chcr - Select device in Round Robin fashion

[Harsh Jain]

When multiple devices are present in system select device
in round-robin fashion for crypto operations

Signed-off-by: Atul Gupta 
Reviewed-by: Ganesh Goudar 
---
 drivers/crypto/chelsio/chcr_algo.c |  8 ++--
 drivers/crypto/chelsio/chcr_core.c | 53 ++
 drivers/crypto/chelsio/chcr_core.h |  2 +-
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.c |  1 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h |  1 +
 5 files changed, 44 insertions(+), 21 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 9a84ffa..aa4e5b8 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1216,7 +1216,7 @@ static int chcr_aes_decrypt(struct ablkcipher_request 
*req)
 
 static int chcr_device_init(struct chcr_context *ctx)
 {
-   struct uld_ctx *u_ctx;
+   struct uld_ctx *u_ctx = NULL;
struct adapter *adap;
unsigned int id;
int txq_perchan, txq_idx, ntxq;
@@ -1224,12 +1224,12 @@ static int chcr_device_init(struct chcr_context *ctx)
 
id = smp_processor_id();
if (!ctx->dev) {
-   err = assign_chcr_device(>dev);
-   if (err) {
+   u_ctx = assign_chcr_device();
+   if (!u_ctx) {
pr_err("chcr device assignment fails\n");
goto out;
}
-   u_ctx = ULD_CTX(ctx);
+   ctx->dev = u_ctx->dev;
adap = padap(ctx->dev);
ntxq = min_not_zero((unsigned int)u_ctx->lldi.nrxq,
adap->vres.ncrypto_fc);
diff --git a/drivers/crypto/chelsio/chcr_core.c 
b/drivers/crypto/chelsio/chcr_core.c
index 5ae659a..b6dd9cb 100644
--- a/drivers/crypto/chelsio/chcr_core.c
+++ b/drivers/crypto/chelsio/chcr_core.c
@@ -29,6 +29,7 @@
 static LIST_HEAD(uld_ctx_list);
 static DEFINE_MUTEX(dev_mutex);
 static atomic_t dev_count;
+static struct uld_ctx *ctx_rr;
 
 typedef int (*chcr_handler_func)(struct chcr_dev *dev, unsigned char *input);
 static int cpl_fw6_pld_handler(struct chcr_dev *dev, unsigned char *input);
@@ -49,25 +50,28 @@
.rx_handler = chcr_uld_rx_handler,
 };
 
-int assign_chcr_device(struct chcr_dev **dev)
+struct uld_ctx *assign_chcr_device(void)
 {
-   struct uld_ctx *u_ctx;
-   int ret = -ENXIO;
+   struct uld_ctx *u_ctx = NULL;
 
/*
-* Which device to use if multiple devices are available TODO
-* May be select the device based on round robin. One session
-* must go to the same device to maintain the ordering.
+* When multiple devices are present in system select
+* device in round-robin fashion for crypto operations
+* Although One session must use the same device to
+* maintain request-response ordering.
 */
-   mutex_lock(_mutex); /* TODO ? */
-   list_for_each_entry(u_ctx, _ctx_list, entry)
-   if (u_ctx->dev) {
-   *dev = u_ctx->dev;
-   ret = 0;
-   break;
+   mutex_lock(_mutex);
+   if (!list_empty(_ctx_list)) {
+   u_ctx = ctx_rr;
+   if (list_is_last(_rr->entry, _ctx_list))
+   ctx_rr = list_first_entry(_ctx_list,
+ struct uld_ctx,
+ entry);
+   else
+   ctx_rr = list_next_entry(ctx_rr, entry);
}
mutex_unlock(_mutex);
-   return ret;
+   return u_ctx;
 }
 
 static int chcr_dev_add(struct uld_ctx *u_ctx)
@@ -82,11 +86,27 @@ static int chcr_dev_add(struct uld_ctx *u_ctx)
u_ctx->dev = dev;
dev->u_ctx = u_ctx;
atomic_inc(_count);
+   mutex_lock(_mutex);
+   list_add_tail(_ctx->entry, _ctx_list);
+   if (!ctx_rr)
+   ctx_rr = u_ctx;
+   mutex_unlock(_mutex);
return 0;
 }
 
 static int chcr_dev_remove(struct uld_ctx *u_ctx)
 {
+   if (ctx_rr == u_ctx) {
+   if (list_is_last(_rr->entry, _ctx_list))
+   ctx_rr = list_first_entry(_ctx_list,
+ struct uld_ctx,
+ entry);
+   else
+   ctx_rr = list_next_entry(ctx_rr, entry);
+   }
+   list_del(_ctx->entry);
+   if (list_empty(_ctx_list))
+   ctx_rr = NULL;
kfree(u_ctx->dev);
u_ctx->dev = NULL;
atomic_dec(_count);
@@ -139,10 +159,11 @@ static void *chcr_uld_add(const struct cxgb4_lld_info 
*lld)
u_ctx = ERR_PTR(-ENOMEM);
goto out;
}
+   if (!(lld->ulp_crypto & ULP_CRYPTO_LOOKASIDE)) {
+   u_ctx = ERR_PTR(-ENOMEM);
+   goto out;
+   }
u_ctx->lldi = *lld;
-  

[PATCH 8/9] crypto: chcr - Ensure Destination sg entry size less than 2k

[Harsh Jain]

Allocate new sg list in case received destination sg list has entry
greater that 2k.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   | 153 +++
 drivers/crypto/chelsio/chcr_crypto.h |   6 ++
 2 files changed, 142 insertions(+), 17 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 2f388af..9a84ffa 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -166,6 +166,8 @@ int chcr_handle_resp(struct crypto_async_request *req, 
unsigned char *input,
kfree_skb(ctx_req.ctx.reqctx->skb);
ctx_req.ctx.reqctx->skb = NULL;
}
+   free_new_sg(ctx_req.ctx.reqctx->newdstsg);
+   ctx_req.ctx.reqctx->newdstsg = NULL;
if (ctx_req.ctx.reqctx->verify == VERIFY_SW) {
chcr_verify_tag(ctx_req.req.aead_req, input,
);
@@ -1068,6 +1070,8 @@ static int chcr_handle_cipher_resp(struct 
ablkcipher_request *req,
chcr_send_wr(skb);
return 0;
 complete:
+   free_new_sg(reqctx->newdstsg);
+   reqctx->newdstsg = NULL;
req->base.complete(>base, err);
return err;
 }
@@ -1083,7 +1087,7 @@ static int process_cipher(struct ablkcipher_request *req,
struct chcr_context *ctx = crypto_ablkcipher_ctx(tfm);
struct ablk_ctx *ablkctx = ABLK_CTX(ctx);
struct  cipher_wr_param wrparam;
-   int bytes, err = -EINVAL;
+   int bytes, nents, err = -EINVAL;
 
reqctx->newdstsg = NULL;
reqctx->processed = 0;
@@ -1097,7 +1101,14 @@ static int process_cipher(struct ablkcipher_request *req,
goto error;
}
wrparam.srcsg = req->src;
-   reqctx->dstsg = req->dst;
+   if (is_newsg(req->dst, )) {
+   reqctx->newdstsg = alloc_new_sg(req->dst, nents);
+   if (IS_ERR(reqctx->newdstsg))
+   return PTR_ERR(reqctx->newdstsg);
+   reqctx->dstsg = reqctx->newdstsg;
+   } else {
+   reqctx->dstsg = req->dst;
+   }
bytes = chcr_sg_ent_in_wr(wrparam.srcsg, reqctx->dstsg, MIN_CIPHER_SG,
 SPACE_LEFT(ablkctx->enckey_len),
 ,
@@ -1150,6 +1161,8 @@ static int process_cipher(struct ablkcipher_request *req,
 
return 0;
 error:
+   free_new_sg(reqctx->newdstsg);
+   reqctx->newdstsg = NULL;
return err;
 }
 
@@ -1808,6 +1821,63 @@ static void chcr_hmac_cra_exit(struct crypto_tfm *tfm)
}
 }
 
+static int is_newsg(struct scatterlist *sgl, unsigned int *newents)
+{
+   int nents = 0;
+   int ret = 0;
+
+   while (sgl) {
+   if (sgl->length > CHCR_SG_SIZE)
+   ret = 1;
+   nents += DIV_ROUND_UP(sgl->length, CHCR_SG_SIZE);
+   sgl = sg_next(sgl);
+   }
+   *newents = nents;
+   return ret;
+}
+
+static inline void free_new_sg(struct scatterlist *sgl)
+{
+   kfree(sgl);
+}
+
+static struct scatterlist *alloc_new_sg(struct scatterlist *sgl,
+  unsigned int nents)
+{
+   struct scatterlist *newsg, *sg;
+   int i, len, processed = 0;
+   struct page *spage;
+   int offset;
+
+   newsg = kmalloc_array(nents, sizeof(struct scatterlist), GFP_KERNEL);
+   if (!newsg)
+   return ERR_PTR(-ENOMEM);
+   sg = newsg;
+   sg_init_table(sg, nents);
+   offset = sgl->offset;
+   spage = sg_page(sgl);
+   for (i = 0; i < nents; i++) {
+   len = min_t(u32, sgl->length - processed, CHCR_SG_SIZE);
+   sg_set_page(sg, spage, len, offset);
+   processed += len;
+   offset += len;
+   if (offset >= PAGE_SIZE) {
+   offset = offset % PAGE_SIZE;
+   spage++;
+   }
+   if (processed == sgl->length) {
+   processed = 0;
+   sgl = sg_next(sgl);
+   if (!sgl)
+   break;
+   spage = sg_page(sgl);
+   offset = sgl->offset;
+   }
+   sg = sg_next(sg);
+   }
+   return newsg;
+}
+
 static int chcr_copy_assoc(struct aead_request *req,
struct chcr_aead_ctx *ctx)
 {
@@ -1870,7 +1940,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
struct scatterlist *src;
unsigned int frags = 0, transhdr_len;
unsigned int ivsize = crypto_aead_ivsize(tfm), dst_size = 0;
-   unsigned int   kctx_len = 0;
+   unsigned int   kctx_len = 0, nents;
unsigned short stop_off

[PATCH 4/9] crypto: chcr - Avoid changing request structure

[Harsh Jain]

Do not update assoclen received in aead_request.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 37 ++---
 1 file changed, 14 insertions(+), 23 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 156065d..9c839c6 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -126,13 +126,13 @@ static void chcr_verify_tag(struct aead_request *req, u8 
*input, int *err)
fw6_pld = (struct cpl_fw6_pld *)input;
if ((get_aead_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_AEAD_RFC4106) ||
(get_aead_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_AEAD_GCM)) {
-   cmp = memcmp(_pld->data[2], (fw6_pld + 1), authsize);
+   cmp = crypto_memneq(_pld->data[2], (fw6_pld + 1), authsize);
} else {
 
sg_pcopy_to_buffer(req->src, sg_nents(req->src), temp,
authsize, req->assoclen +
req->cryptlen - authsize);
-   cmp = memcmp(temp, (fw6_pld + 1), authsize);
+   cmp = crypto_memneq(temp, (fw6_pld + 1), authsize);
}
if (cmp)
*err = -EBADMSG;
@@ -1840,9 +1840,8 @@ static struct sk_buff *create_gcm_wr(struct aead_request 
*req,
struct scatterlist *src;
unsigned int frags = 0, transhdr_len;
unsigned int ivsize = AES_BLOCK_SIZE;
-   unsigned int dst_size = 0, kctx_len;
+   unsigned int dst_size = 0, kctx_len, assoclen = req->assoclen;
unsigned char tag_offset = 0;
-   unsigned int crypt_len = 0;
unsigned int authsize = crypto_aead_authsize(tfm);
int error = -EINVAL, src_nent;
gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL :
@@ -1854,27 +1853,21 @@ static struct sk_buff *create_gcm_wr(struct 
aead_request *req,
 
if (op_type && req->cryptlen < crypto_aead_authsize(tfm))
goto err;
-   src_nent = sg_nents_for_len(req->src, req->assoclen + req->cryptlen);
+   src_nent = sg_nents_for_len(req->src, assoclen + req->cryptlen);
if (src_nent < 0)
goto err;
 
-   src = scatterwalk_ffwd(reqctx->srcffwd, req->src, req->assoclen);
+   src = scatterwalk_ffwd(reqctx->srcffwd, req->src, assoclen);
reqctx->dst = src;
if (req->src != req->dst) {
error = chcr_copy_assoc(req, aeadctx);
if (error)
return  ERR_PTR(error);
reqctx->dst = scatterwalk_ffwd(reqctx->dstffwd, req->dst,
-  req->assoclen);
+  assoclen);
}
 
-   if (!req->cryptlen)
-   /* null-payload is not supported in the hardware.
-* software is sending block size
-*/
-   crypt_len = AES_BLOCK_SIZE;
-   else
-   crypt_len = req->cryptlen;
+
reqctx->dst_nents = sg_nents_for_len(reqctx->dst, req->cryptlen +
 (op_type ? -authsize : authsize));
if (reqctx->dst_nents < 0) {
@@ -1907,19 +1900,19 @@ static struct sk_buff *create_gcm_wr(struct 
aead_request *req,
memset(chcr_req, 0, transhdr_len);
 
if (get_aead_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_AEAD_RFC4106)
-   req->assoclen -= 8;
+   assoclen = req->assoclen - 8;
 
tag_offset = (op_type == CHCR_ENCRYPT_OP) ? 0 : authsize;
chcr_req->sec_cpl.op_ivinsrtofst = FILL_SEC_CPL_OP_IVINSR(
ctx->dev->rx_channel_id, 2, (ivsize ?
-   (req->assoclen + 1) : 0));
+   (assoclen + 1) : 0));
chcr_req->sec_cpl.pldlen =
-   htonl(req->assoclen + ivsize + req->cryptlen);
+   htonl(assoclen + ivsize + req->cryptlen);
chcr_req->sec_cpl.aadstart_cipherstop_hi = FILL_SEC_CPL_CIPHERSTOP_HI(
-   req->assoclen ? 1 : 0, req->assoclen,
-   req->assoclen + ivsize + 1, 0);
+   assoclen ? 1 : 0, assoclen,
+   assoclen + ivsize + 1, 0);
chcr_req->sec_cpl.cipherstop_lo_authinsert =
-   FILL_SEC_CPL_AUTHINSERT(0, req->assoclen + ivsize + 1,
+   FILL_SEC_CPL_AUTHINSERT(0, assoclen + ivsize + 1,
tag_offset, tag_offset);
chcr_req->sec_cpl.seqno_numivs =
FILL_SEC_CPL_SCMD0_SEQNO(op_type, (op_type ==
@@ -1955,9 +1948,7 @@ static struct s

[PATCH 1/9] crypto: chcr - Pass lcb bit setting to firmware

[Harsh Jain]

GCM and CBC mode of operation requires Last Cipher Block.
This patch set lcb bit in WR header when required.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c | 18 +++---
 drivers/crypto/chelsio/chcr_algo.h |  4 ++--
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index f00e0d8..e8ff505 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -518,7 +518,8 @@ static inline void create_wreq(struct chcr_context *ctx,
   void *req, struct sk_buff *skb,
   int kctx_len, int hash_sz,
   int is_iv,
-  unsigned int sc_len)
+  unsigned int sc_len,
+  unsigned int lcb)
 {
struct uld_ctx *u_ctx = ULD_CTX(ctx);
int iv_loc = IV_DSGL;
@@ -543,7 +544,8 @@ static inline void create_wreq(struct chcr_context *ctx,
chcr_req->wreq.cookie = cpu_to_be64((uintptr_t)req);
chcr_req->wreq.rx_chid_to_rx_q_id =
FILL_WR_RX_Q_ID(ctx->dev->rx_channel_id, qid,
-   is_iv ? iv_loc : IV_NOP, ctx->tx_qidx);
+   is_iv ? iv_loc : IV_NOP, !!lcb,
+   ctx->tx_qidx);
 
chcr_req->ulptx.cmd_dest = FILL_ULPTX_CMD_DEST(ctx->dev->tx_channel_id,
   qid);
@@ -652,7 +654,8 @@ static inline void create_wreq(struct chcr_context *ctx,
write_buffer_to_skb(skb, , reqctx->iv, ivsize);
write_sg_to_skb(skb, , req->src, req->nbytes);
create_wreq(ctx, chcr_req, req, skb, kctx_len, 0, 1,
-   sizeof(struct cpl_rx_phys_dsgl) + phys_dsgl);
+   sizeof(struct cpl_rx_phys_dsgl) + phys_dsgl,
+   ablkctx->ciph_mode == CHCR_SCMD_CIPHER_MODE_AES_CBC);
reqctx->skb = skb;
skb_get(skb);
return skb;
@@ -923,7 +926,7 @@ static struct sk_buff *create_hash_wr(struct ahash_request 
*req,
write_sg_to_skb(skb, , req->src, param->sg_len);
 
create_wreq(ctx, chcr_req, req, skb, kctx_len, hash_size_in_response, 0,
-   DUMMY_BYTES);
+   DUMMY_BYTES, 0);
req_ctx->skb = skb;
skb_get(skb);
return skb;
@@ -1508,7 +1511,7 @@ static struct sk_buff *create_authenc_wr(struct 
aead_request *req,
write_buffer_to_skb(skb, , req->iv, ivsize);
write_sg_to_skb(skb, , src, req->cryptlen);
create_wreq(ctx, chcr_req, req, skb, kctx_len, size, 1,
-  sizeof(struct cpl_rx_phys_dsgl) + dst_size);
+  sizeof(struct cpl_rx_phys_dsgl) + dst_size, 0);
reqctx->skb = skb;
skb_get(skb);
 
@@ -1804,7 +1807,7 @@ static struct sk_buff *create_aead_ccm_wr(struct 
aead_request *req,
skb_set_transport_header(skb, transhdr_len);
frags = fill_aead_req_fields(skb, req, src, ivsize, aeadctx);
create_wreq(ctx, chcr_req, req, skb, kctx_len, 0, 1,
-   sizeof(struct cpl_rx_phys_dsgl) + dst_size);
+   sizeof(struct cpl_rx_phys_dsgl) + dst_size, 0);
reqctx->skb = skb;
skb_get(skb);
return skb;
@@ -1950,7 +1953,8 @@ static struct sk_buff *create_gcm_wr(struct aead_request 
*req,
write_buffer_to_skb(skb, , reqctx->iv, ivsize);
write_sg_to_skb(skb, , src, req->cryptlen);
create_wreq(ctx, chcr_req, req, skb, kctx_len, size, 1,
-   sizeof(struct cpl_rx_phys_dsgl) + dst_size);
+   sizeof(struct cpl_rx_phys_dsgl) + dst_size,
+   reqctx->verify);
reqctx->skb = skb;
skb_get(skb);
return skb;
diff --git a/drivers/crypto/chelsio/chcr_algo.h 
b/drivers/crypto/chelsio/chcr_algo.h
index 751d06a..9894c7b 100644
--- a/drivers/crypto/chelsio/chcr_algo.h
+++ b/drivers/crypto/chelsio/chcr_algo.h
@@ -185,11 +185,11 @@
FW_CRYPTO_LOOKASIDE_WR_CCTX_LOC_V(1) | \
FW_CRYPTO_LOOKASIDE_WR_CCTX_SIZE_V((ctx_len)))
 
-#define FILL_WR_RX_Q_ID(cid, qid, wr_iv, fid) \
+#define FILL_WR_RX_Q_ID(cid, qid, wr_iv, lcb, fid) \
htonl( \
FW_CRYPTO_LOOKASIDE_WR_RX_CHID_V((cid)) | \
FW_CRYPTO_LOOKASIDE_WR_RX_Q_ID_V((qid)) | \
-   FW_CRYPTO_LOOKASIDE_WR_LCB_V(0) | \
+   FW_CRYPTO_LOOKASIDE_WR_LCB_V((lcb)) | \
FW_CRYPTO_LOOKASIDE_WR_IV_V((wr_iv)) | \
FW_CRYPTO_LOOKASIDE_WR_FQIDX_V(fid))
 
-- 
1.8.3.1

Re: [PATCH 1/1] crypto:drbg- Fixes panic in wait_for_completion call.

[Harsh Jain]

Hi Herbert,

Since It's a panic BUG.  Can we push this fix to stable kernel releases?

Regards
Harsh Jain

On Sat, Jun 10, 2017 at 9:47 AM, Herbert Xu <herb...@gondor.apana.org.au> wrote:
> On Fri, May 26, 2017 at 12:12:51PM +0200, Stephan Müller wrote:
>> Am Donnerstag, 25. Mai 2017, 17:23:47 CEST schrieb Harsh Jain:
>>
>> Hi Harsh,
>>
>> > Initialise ctr_completion variable before use.
>>
>> Thank you very much for catching this.
>>
>> But I think the chosen function is wrong. When we have an HMAC or Hash DRBG, 
>> this completion function does not need to be initialized.
>>
>> May I ask you to check this patch?
>
> Patch applied.  Thanks.
> --
> Email: Herbert Xu <herb...@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Can someone check linux kernel 4.4, and 4.9 rfc4309 test vectors?

[Harsh Jain]

On Wed, Jun 7, 2017 at 7:27 PM, Che-Min Hsieh  wrote:
> Rfc4309 test vectors in testmgr.h have gone through major changes from linux3 
>  to linux4.
> In linux 4.4, linux4.9, there are vectors as such
>
> 23194 static struct aead_testvec aes_ccm_rfc4309_enc_tv_template[] = {
> 23195{ /* Generated using Crypto++ */
> 23196.key   = zeroed_string,
> 23197.klen  = 19,
> 23198.iv   = zeroed_string,
> 23199.input= zeroed_string,
> 23200.ilen   = 16,
> 23201.assoc= zeroed_string,
> 23202.alen  = 16,
> 23203.result   = "\x2E\x9A\xCA\x6B\xDA\x54\xFC\x6F"
> 23204  "\x12\x50\xE8\xDE\x81\x3C\x63\x08"
> 23205  "\x1A\x22\xBA\x75\xEE\xD4\xD5\xB5"
> 23206  "\x27\x50\x01\xAC\x03\x33\x39\xFB",
> 23207.rlen   = 32,
>
>
> I have a test program using open ssl API (-l crypto), and run on Ubuntu Linux 
> PC,   I  get the following  test result:
>
> 2e 9a ca 6b da 54 fc 6f 12 50 e8 de 81 3c 63 08
> fb 64 91 b4 dd dc bf 5d fd 67 e3 a2 f8 7c 0e 6c
>   The first part of encrypted text is correct. But MAC is not the 
> same.
>
> My program is as the following:
>
> void ccmTest()
> {
> /* Initialization */
> EVP_CIPHER_CTX ctx;
> EVP_CIPHER_CTX *cryptCtx = 
> EVP_CIPHER_CTX_init(cryptCtx);
> int i;
>
>unsigned char P[16] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
> int Psize = sizeof(P);
> unsigned char K[16] = {0};
> unsigned char N[11] = {0};
> unsigned char A[16] = {0};
> unsigned char CT[128];
>
> int Nsize = 11;
> int Tsize = 16;
>
> // Initialize the context with the alg only
> EVP_EncryptInit(cryptCtx, EVP_aes_128_ccm(), 0, 0);
>
> // Set nonce and tag sizes
> EVP_CIPHER_CTX_ctrl(cryptCtx, EVP_CTRL_CCM_SET_IVLEN, Nsize, 0);
> EVP_CIPHER_CTX_ctrl(cryptCtx, EVP_CTRL_CCM_SET_TAG, Tsize, 0);
>
> // Finally set the key and the nonce
> EVP_EncryptInit(cryptCtx, 0, K, N);
>
> // Tell the alg we will encrypt Psize bytes
> int outl = 0;
> EVP_EncryptUpdate(cryptCtx, 0, , 0, sizeof(P));
>// Add the AAD
> EVP_EncryptUpdate(cryptCtx, 0, , A, sizeof(A));
>// Now we encrypt the data in P, placing the output in CT
> EVP_EncryptUpdate(cryptCtx, CT, , P, Psize);
> EVP_EncryptFinal(cryptCtx, [outl], );
> // Append the tag to the end of the encrypted output
> EVP_CIPHER_CTX_ctrl(cryptCtx, EVP_CTRL_CCM_GET_TAG, Tsize, [Psize]);
> hexdump(CT, Tsize+Psize);
> }
>
>
> I run "insmod tcrypt.ko mode=45"  rfc4309 test with Qualcomm crypto hardware 
> on Linux4.4. The test fails. The generated output is the same as my openSSL 
> test application in 1.
Older kernel driver also includes IV in Authentication data.  Thats
why MAC is wrong . 4.4 kernel crypto drivers don't need to include IV
in authentication data. Updating driver not to include IV string in
authentication data for latest kernels should work.
>
> My test application runs on Ubuntu with linux 3.10 rfc4309 test vector, and 
> generated MAC as expected from test vectors.  Qualcomm crypto hardware runs 
> "insmod tcrypt.ko mode=45" successfully with linux 3.10.
>
> I am suspicious about the test vectors of 4.4. Can someone verify the Linux 
> 4.4 rfc4309 test vectors with his/her openSSL application on PC?
>
> Chemin

Re: [PATCH 1/1] crypto:drbg- Fixes panic in wait_for_completion call.

[Harsh Jain]

On Fri, May 26, 2017 at 3:42 PM, Stephan Müller <smuel...@chronox.de> wrote:
> Am Donnerstag, 25. Mai 2017, 17:23:47 CEST schrieb Harsh Jain:
>
> Hi Harsh,
>
>> Initialise ctr_completion variable before use.
>
> Thank you very much for catching this.
>
> But I think the chosen function is wrong. When we have an HMAC or Hash DRBG, 
> this completion function does not need to be initialized.
>
> May I ask you to check this patch?
Yup, Its working.
>
> Ciao
> Stephan
>
> ---8<---
>
> From 1ec5f753644eb9005ad758c9bfd75032602c9727 Mon Sep 17 00:00:00 2001
> From: Stephan Mueller <smuel...@chronox.de>
> Date: Fri, 26 May 2017 12:11:31 +0200
> Subject: [PATCH] crypto: drbg- Fixes panic in wait_for_completion call
>
> Initialise ctr_completion variable before use.
>
> Signed-off-by: Harsh Jain <harshjain.p...@gmail.com>
> Signed-off-by: Stephan Mueller <smuel...@chronox.de>
> ---
>  crypto/drbg.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/crypto/drbg.c b/crypto/drbg.c
> index fa749f4..433c425 100644
> --- a/crypto/drbg.c
> +++ b/crypto/drbg.c
> @@ -1691,6 +1691,7 @@ static int drbg_init_sym_kernel(struct drbg_state *drbg)
> return PTR_ERR(sk_tfm);
> }
> drbg->ctr_handle = sk_tfm;
> +   init_completion(>ctr_completion);
>
> req = skcipher_request_alloc(sk_tfm, GFP_KERNEL);
> if (!req) {
> --
> 2.9.3
>
>

[PATCH 1/1] crypto:drbg- Fixes panic in wait_for_completion call.

[Harsh Jain]

Initialise ctr_completion variable before use.

Signed-off-by: Harsh Jain <harshjain.p...@gmail.com>
---
 crypto/drbg.c |1 +
 1 file changed, 1 insertion(+)

diff --git a/crypto/drbg.c b/crypto/drbg.c
index fa749f4..f1db29d 100644
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1840,6 +1840,7 @@ static int drbg_kcapi_init(struct crypto_tfm *tfm)
 struct drbg_state *drbg = crypto_tfm_ctx(tfm);
 
 mutex_init(>drbg_mutex);
+init_completion(>ctr_completion);
 
 return 0;
 }
-- 
1.7.10.1

Re: BUG: drbg: Added nodes from Stack Memory in link list

[Harsh Jain]

On Mon, May 8, 2017 at 2:00 PM, Stephan Müller <smuel...@chronox.de> wrote:
> Am Montag, 8. Mai 2017, 08:30:13 CEST schrieb Harsh Jain:
>
> Hi Harsh,
>>
>> Confusing, I have to dig more for DRBG. Actually we observed following
>> panic in Chcr (Chelsio) when drgb is enabled and Panic trace points
>> some thing wrong
>> with drgb modules. Any idea what are possible reason for this.
>
> Just to confirm: are you using the latest kernel?

No, I tried on 4.9.13. Will let you know the behavior with latest kernel.

The bug you are referring to
> happens in the drbg_kcapi_sym_ctr called by the update operation to process
> seed material. This function had a bug in it where I used stack buffer. This
> is now repaced with heap buffer:
>
> 5102981212454998d549273ff9847f19e97a1794
>
> I am yet wondering why a __list_add is called that causes the bug. In the DRBG
> code path seen below, I am not seeing any list_add calls.
>>
>> alg: No test for authenc(digest_null,rfc3686(ctr(aes)))
>> (authenc(digest_null-generic,rfc3686-ctr-aes-chcr))
>> alg: No test for seqiv(authenc(digest_null,rfc3686(ctr(aes
>> (seqiv(authenc(digest_null-generic,rfc3686-ctr-aes-chcr)))
>> alg: No test for fips(ansi_cprng) (fips_ansi_cprng)
>> BUG: unable to handle kernel NULL pointer dereference at   (null)
>> IP: [] __list_add+0x26/0xd0
>> PGD 0
>> Oops:  [#1] SMP
>> Modules linked in: drbg(+) ansi_cprng seqiv xfrm6_mode_tunnel
>> xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4
>> af_key cbc ccm ctr ghash_generic gf128mul ghash_clmulni_intel cryptd
>> gcm sha512_ssse3 sha512_generic chcr(OE) cxgb4(OE) authenc netconsole
>> configfs xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
>> nf_nat_ipv4 nf_nat nf_conntrack ip_tables nfsd lockd grace nfs_acl
>> auth_rpcgss sunrpc ipv6 crc_ccitt vfat fat joydev iTCO_wdt
>> iTCO_vendor_support mxm_wmi pcspkr sg i2c_i801 i2c_smbus lpc_ich
>> mfd_core shpchp xhci_pci xhci_hcd igb i2c_algo_bit i2c_core ptp
>> pps_core ioatdma dca ipmi_si ipmi_msghandler wmi acpi_cpufreq acpi_pad
>> dm_mod(E) ext4(E) mbcache(E) jbd2(E) sd_mod(E) ahci(E) libahci(E)
>> [last unloaded: scsi_transport_fc]
>> CPU: 9 PID: 3672 Comm: cryptomgr_test Tainted: G   OE   4.9.13 #2
>> Hardware name: Supermicro X10DRi/X10DRi, BIOS 2.0 12/28/2015
>> task: 88103b418a00 task.stack: c90008a7c000
>> RIP: 0010:[]  [] __list_add+0x26/0xd0
>> RSP: 0018:c90008a7f8c8  EFLAGS: 00010046
>> RAX:  RBX: c90008a7f920 RCX: 0001
>> RDX: 88103c8b5ef0 RSI:  RDI: c90008a7f920
>> RBP: c90008a7f8f8 R08:  R09: 8810053200b0
>> R10: 88103caf3100 R11: 0020 R12: 88103c8b5ef0
>> R13:  R14: 88103b418a00 R15: 7fff
>> FS:  () GS:88107f44() knlGS:
>> CS:  0010 DS:  ES:  CR0: 80050033
>> CR2:  CR3: 01c07000 CR4: 001406e0
>> Stack:
>>  c90008a7f8e8 0246 88103caf3040 88103c8b5ee0
>>  88103c8b5ee8 c90008a7f908 c90008a7f968 81654c02
>>  0001 88103b418a00 81097370 
>> Call Trace:
>>  [] wait_for_completion_interruptible+0xc2/0x130
>>  [] ? try_to_wake_up+0x240/0x240
>>  [] drbg_kcapi_sym_ctr+0xeb/0x150 [drbg]
>>  [] drbg_ctr_update+0x1b0/0x2a0 [drbg]
>>  [] drbg_seed+0x1a2/0x2e0 [drbg]
>>  [] ? drbg_init_sym_kernel+0x13f/0x200 [drbg]
>>  [] drbg_instantiate+0x52/0x1e0 [drbg]
>>  [] ? __kmalloc+0xee/0x1d0
>>  [] ? crypto_create_tfm+0x3d/0xd0
>>  [] drbg_kcapi_seed+0xcc/0x118 [drbg]
>>  [] ? crypto_create_tfm+0xa1/0xd0
>>  [] crypto_rng_reset+0x5d/0x80
>>  [] drbg_cavs_test+0xf7/0x370
>>  [] ? dequeue_task_fair+0x68/0x420
>>  [] ? pick_next_task_idle+0x45/0x50
>>  [] alg_test_drbg+0x6b/0xa0
>>  [] alg_test+0x145/0x350
>>  [] ? cryptomgr_probe+0xd0/0xd0
>>  [] ? cryptomgr_probe+0xd0/0xd0
>>  [] cryptomgr_test+0x45/0x50
>>  [] kthread+0xcd/0xf0
>>  [] ? schedule_tail+0x1e/0xc0
>>  [] ? __kthread_init_worker+0x40/0x40
>>  [] ret_from_fork+0x22/0x30
>> Code: 00 00 00 00 00 55 48 89 e5 48 83 ec 30 48 89 5d e8 4c 89 65 f0
>> 48 89 fb 4c 89 6d f8 4c 8b 42 08 49 89 f5 49 89 d4 49 39 f0 75 31 <4d>
>> 8b 45 00 4d 39 c4 75 6f 4c 39 e3 74 45 4c 39 eb 74 40 49 89
>> RIP  [] __list_add+0x26/0xd0
>>  RSP 
>> CR2: 
>> ---[ end trace fbf11c880e8c4c52 ]---
>
> Ciao
> Stephan

Re: BUG: drbg: Added nodes from Stack Memory in link list

[Harsh Jain]

On Wed, May 3, 2017 at 6:41 PM, Stephan Müller <smuel...@chronox.de> wrote:
> Am Mittwoch, 3. Mai 2017, 14:47:24 CEST schrieb Harsh Jain:
>
> Hi Harsh,
>
>> Hi Stephen,
>>
>> In drbg driver local variable of function are added as list nodes. one
>> instance is given below.
>>
>>
>> static inline int __init drbg_healthcheck_sanity(void)
>>
>>
>>
>> ---
>> ---
>>
>> max_addtllen = drbg_max_addtl(drbg);
>> max_request_bytes = drbg_max_request_bytes(drbg);
>> drbg_string_fill(, buf, max_addtllen + 1);
>> /* overflow addtllen with additonal info string */
>> len = drbg_generate(drbg, buf, OUTBUFLEN, );
>>
>> ===> Added "addtllist"  to the tail. which local variable.
>>
>>
>> BUG_ON(0 < len);
>> /* overflow max_bits */
>> len = drbg_generate(drbg, buf, (max_request_bytes + 1), NULL);
>> BUG_ON(0 < len);
>>
>> /* overflow max addtllen with personalization string */
>> ret = drbg_seed(drbg, , false);
>>
>> > Added "seedlist" to the tail of same list. Previous
>> node(addtllist) already invalid(Out of scope when function returns)
>>
>>
>> BUG_ON(0 == ret);
>> /* all tests passed */
>> rc = 0;
>>
>>
>>
>> Is't a BUG? or I missed something.
>
> :-) You are right that in normal processing this is a bug. But this is a
> special function which shall perform a sanity checks for the maximum length
> enforcement. Thus, addtllist or seedlist should never be filled.
>
> To be precise: The called code is expected to return an error before the
> variables are used due to the length checks. If these errors do not appear,
> something is wrong with the DRBG which causes the BUG_ON.

Confusing, I have to dig more for DRBG. Actually we observed following
panic in Chcr (Chelsio) when drgb is enabled and Panic trace points
some thing wrong
with drgb modules. Any idea what are possible reason for this.

alg: No test for authenc(digest_null,rfc3686(ctr(aes)))
(authenc(digest_null-generic,rfc3686-ctr-aes-chcr))
alg: No test for seqiv(authenc(digest_null,rfc3686(ctr(aes
(seqiv(authenc(digest_null-generic,rfc3686-ctr-aes-chcr)))
alg: No test for fips(ansi_cprng) (fips_ansi_cprng)
BUG: unable to handle kernel NULL pointer dereference at   (null)
IP: [] __list_add+0x26/0xd0
PGD 0
Oops:  [#1] SMP
Modules linked in: drbg(+) ansi_cprng seqiv xfrm6_mode_tunnel
xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4
af_key cbc ccm ctr ghash_generic gf128mul ghash_clmulni_intel cryptd
gcm sha512_ssse3 sha512_generic chcr(OE) cxgb4(OE) authenc netconsole
configfs xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4 nf_nat nf_conntrack ip_tables nfsd lockd grace nfs_acl
auth_rpcgss sunrpc ipv6 crc_ccitt vfat fat joydev iTCO_wdt
iTCO_vendor_support mxm_wmi pcspkr sg i2c_i801 i2c_smbus lpc_ich
mfd_core shpchp xhci_pci xhci_hcd igb i2c_algo_bit i2c_core ptp
pps_core ioatdma dca ipmi_si ipmi_msghandler wmi acpi_cpufreq acpi_pad
dm_mod(E) ext4(E) mbcache(E) jbd2(E) sd_mod(E) ahci(E) libahci(E)
[last unloaded: scsi_transport_fc]
CPU: 9 PID: 3672 Comm: cryptomgr_test Tainted: G   OE   4.9.13 #2
Hardware name: Supermicro X10DRi/X10DRi, BIOS 2.0 12/28/2015
task: 88103b418a00 task.stack: c90008a7c000
RIP: 0010:[]  [] __list_add+0x26/0xd0
RSP: 0018:c90008a7f8c8  EFLAGS: 00010046
RAX:  RBX: c90008a7f920 RCX: 0001
RDX: 88103c8b5ef0 RSI:  RDI: c90008a7f920
RBP: c90008a7f8f8 R08:  R09: 8810053200b0
R10: 88103caf3100 R11: 0020 R12: 88103c8b5ef0
R13:  R14: 88103b418a00 R15: 7fff
FS:  () GS:88107f44() knlGS:
CS:  0010 DS:  ES:  CR0: 80050033
CR2:  CR3: 01c07000 CR4: 001406e0
Stack:
 c90008a7f8e8 0246 88103caf3040 88103c8b5ee0
 88103c8b5ee8 c90008a7f908 c90008a7f968 81654c02
 0001 88103b418a00 81097370 
Call Trace:
 [] wait_for_completion_interruptible+0xc2/0x130
 [] ? try_to_wake_up+0x240/0x240
 [] drbg_kcapi_sym_ctr+0xeb/0x150 [drbg]
 [] drbg_ctr_update+0x1b0/0x2a0 [drbg]
 [] drbg_seed+0x1a2/0x2e0 [drbg]
 [] ? drbg_init_sym_kernel+0x13f/0x200 [drbg]
 [] drbg_instantiate+0x52/0x1e0 [drbg]
 [] ? __kmalloc+0xee/0x1d0
 [] ? crypto_create_tfm+0x3d/0xd0
 [] drbg_kcapi_seed+0xcc/0x118 [drbg]
 [] ? crypto_create_tfm+0xa1/0xd0
 [] crypto_rng_reset+0x5d/0x80
 [] drbg_cavs_test+0xf7/0x370
 [] ? dequeue_task_fair+0x68/0x420
 [] ? pick_next_task_idle+0x45/0x50
 [] alg_test_drbg+0x6b/0xa0
 [] alg_test+0x145/0x350
 [] ? cryptomgr_probe+0xd0/0xd0
 [] ? cryptomgr_probe+0xd0/0xd0
 [] cryptomgr_test+0

BUG: drbg: Added nodes from Stack Memory in link list

[Harsh Jain]

Hi Stephen,

In drbg driver local variable of function are added as list nodes. one
instance is given below.


static inline int __init drbg_healthcheck_sanity(void)



---
---

max_addtllen = drbg_max_addtl(drbg);
max_request_bytes = drbg_max_request_bytes(drbg);
drbg_string_fill(, buf, max_addtllen + 1);
/* overflow addtllen with additonal info string */
len = drbg_generate(drbg, buf, OUTBUFLEN, );

===> Added "addtllist"  to the tail. which local variable.


BUG_ON(0 < len);
/* overflow max_bits */
len = drbg_generate(drbg, buf, (max_request_bytes + 1), NULL);
BUG_ON(0 < len);

/* overflow max addtllen with personalization string */
ret = drbg_seed(drbg, , false);

> Added "seedlist" to the tail of same list. Previous
node(addtllist) already invalid(Out of scope when function returns)


BUG_ON(0 == ret);
/* all tests passed */
rc = 0;



Is't a BUG? or I missed something.


Regards
Harsh Jain

Re: [PATCH 08/22] crypto: chcr: Make use of the new sg_map helper function

[Harsh Jain]

On Fri, Apr 14, 2017 at 3:35 AM, Logan Gunthorpe  wrote:
> The get_page in this area looks *highly* suspect due to there being no
> corresponding put_page. However, I've left that as is to avoid breaking
> things.
chcr driver will post the request to LLD driver cxgb4 and put_page is
implemented there. it will no harm. Any how
we have removed the below code from driver.

http://www.mail-archive.com/linux-crypto@vger.kernel.org/msg24561.html

After this merge we can ignore your patch. Thanks

>
> I've also removed the KMAP_ATOMIC_ARGS check as it appears to be dead
> code that dates back to when it was first committed...


>
> Signed-off-by: Logan Gunthorpe 
> ---
>  drivers/crypto/chelsio/chcr_algo.c | 28 +++-
>  1 file changed, 15 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/crypto/chelsio/chcr_algo.c 
> b/drivers/crypto/chelsio/chcr_algo.c
> index 41bc7f4..a993d1d 100644
> --- a/drivers/crypto/chelsio/chcr_algo.c
> +++ b/drivers/crypto/chelsio/chcr_algo.c
> @@ -1489,22 +1489,21 @@ static struct sk_buff *create_authenc_wr(struct 
> aead_request *req,
> return ERR_PTR(-EINVAL);
>  }
>
> -static void aes_gcm_empty_pld_pad(struct scatterlist *sg,
> - unsigned short offset)
> +static int aes_gcm_empty_pld_pad(struct scatterlist *sg,
> +unsigned short offset)
>  {
> -   struct page *spage;
> unsigned char *addr;
>
> -   spage = sg_page(sg);
> -   get_page(spage); /* so that it is not freed by NIC */
> -#ifdef KMAP_ATOMIC_ARGS
> -   addr = kmap_atomic(spage, KM_SOFTIRQ0);
> -#else
> -   addr = kmap_atomic(spage);
> -#endif
> -   memset(addr + sg->offset, 0, offset + 1);
> +   get_page(sg_page(sg)); /* so that it is not freed by NIC */
> +
> +   addr = sg_map(sg, SG_KMAP_ATOMIC);
> +   if (IS_ERR(addr))
> +   return PTR_ERR(addr);
> +
> +   memset(addr, 0, offset + 1);
> +   sg_unmap(sg, addr, SG_KMAP_ATOMIC);
>
> -   kunmap_atomic(addr);
> +   return 0;
>  }
>
>  static int set_msg_len(u8 *block, unsigned int msglen, int csize)
> @@ -1940,7 +1939,10 @@ static struct sk_buff *create_gcm_wr(struct 
> aead_request *req,
> if (req->cryptlen) {
> write_sg_to_skb(skb, , src, req->cryptlen);
> } else {
> -   aes_gcm_empty_pld_pad(req->dst, authsize - 1);
> +   err = aes_gcm_empty_pld_pad(req->dst, authsize - 1);
> +   if (err)
> +   goto dstmap_fail;
> +
> write_sg_to_skb(skb, , reqctx->dst, crypt_len);
>
> }
> --
> 2.1.4
>

Re: [PATCH 2/2] crypto: chcr - Fix error checking

[Harsh Jain]

On Thu, Apr 13, 2017 at 8:20 PM, Christophe JAILLET
 wrote:
> Le 13/04/2017 à 16:04, Dan Carpenter a écrit :
>>
>> On Thu, Apr 13, 2017 at 02:14:30PM +0200, Christophe JAILLET wrote:
>>>
>>> If 'chcr_alloc_shash()' a few lines above fails, 'base_hash' can be an
>>> error pointer when we 'goto out'.
>>> So checking for NULL here is not enough because it is likely that
>>> 'chcr_free_shash' will crash if we pass an error pointer.
>>>
>>> Signed-off-by: Christophe JAILLET 
>>> ---
>>> Another solution, amybe safer, would be to instrument 'chcr_free_shash'
>>> or
>>> 'crypto_free_shash' to accept an error pointer and return immediatelly in
>>> such a case.
>>> ---
>>>   drivers/crypto/chelsio/chcr_algo.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/crypto/chelsio/chcr_algo.c
>>> b/drivers/crypto/chelsio/chcr_algo.c
>>> index f19590ac8775..41750b97f43c 100644
>>> --- a/drivers/crypto/chelsio/chcr_algo.c
>>> +++ b/drivers/crypto/chelsio/chcr_algo.c
>>> @@ -2351,7 +2351,7 @@ static int chcr_authenc_setkey(struct crypto_aead
>>> *authenc, const u8 *key,
>>> }
>>>   out:
>>> aeadctx->enckey_len = 0;
>>> -   if (base_hash)
>>> +   if (!IS_ERR_OR_NULL(base_hash))
>>> chcr_free_shash(base_hash);
>>
>> Ah...  Ok.  Fine, but redo the first patch anyway because it shouldn't
>> ever be NULL.
>>
>> regards,
>> dan carpenter
>
> Hi Dan,
>
> I will update the first patch as you proposed in order to:
>- teach 'chcr_alloc_shash' not to return NULL
>- initialize 'base_hash' with ERR_PTR(-EINVAL)
>- update the above test to !IS_ERR.
> The 2 patches will be merged in only 1.
>
> Thanks for your suggestions.

Thanks for pointing the error. or You can simply return instead of
goto. Just like that.

 1.3 @@ -2455,7 +2455,8 @@ static int chcr_authenc_setkey(struct cr
 1.4   base_hash  = chcr_alloc_shash(max_authsize);
 1.5   if (IS_ERR(base_hash)) {
 1.6   pr_err("chcr : Base driver cannot be loaded\n");
 1.7 - goto out;
 1.8 + aeadctx->enckey_len = 0;
 1.9 + return -EINVAL;
1.10   }
1.11   {
1.12   SHASH_DESC_ON_STACK(shash, base_hash);





>
> Best regards,
> CJ
>

[PATCH 4/4] chcr: Add fallback for AEAD algos

[Harsh Jain]

Fallback to sw when
I AAD length greater than 511
II Zero length payload
II No of sg entries exceeds Request size.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   |  219 ++
 drivers/crypto/chelsio/chcr_algo.h   |4 +
 drivers/crypto/chelsio/chcr_crypto.h |3 +-
 3 files changed, 151 insertions(+), 75 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 5470e4e..53d9ce4 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1343,7 +1343,36 @@ static int chcr_copy_assoc(struct aead_request *req,
 
return crypto_skcipher_encrypt(skreq);
 }
+static int chcr_aead_need_fallback(struct aead_request *req, int src_nent,
+  int aadmax, int wrlen,
+  unsigned short op_type)
+{
+   unsigned int authsize = crypto_aead_authsize(crypto_aead_reqtfm(req));
+
+   if (((req->cryptlen - (op_type ? authsize : 0)) == 0) ||
+   (req->assoclen > aadmax) ||
+   (src_nent > MAX_SKB_FRAGS) ||
+   (wrlen > MAX_WR_SIZE))
+   return 1;
+   return 0;
+}
 
+static int chcr_aead_fallback(struct aead_request *req, unsigned short op_type)
+{
+   struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+   struct chcr_context *ctx = crypto_aead_ctx(tfm);
+   struct chcr_aead_ctx *aeadctx = AEAD_CTX(ctx);
+   struct aead_request *subreq = aead_request_ctx(req);
+
+   aead_request_set_tfm(subreq, aeadctx->sw_cipher);
+   aead_request_set_callback(subreq, req->base.flags,
+ req->base.complete, req->base.data);
+aead_request_set_crypt(subreq, req->src, req->dst, req->cryptlen,
+req->iv);
+aead_request_set_ad(subreq, req->assoclen);
+   return op_type ? crypto_aead_decrypt(subreq) :
+   crypto_aead_encrypt(subreq);
+}
 
 static struct sk_buff *create_authenc_wr(struct aead_request *req,
 unsigned short qid,
@@ -1367,7 +1396,7 @@ static int chcr_copy_assoc(struct aead_request *req,
unsigned short stop_offset = 0;
unsigned int  assoclen = req->assoclen;
unsigned int  authsize = crypto_aead_authsize(tfm);
-   int err = 0;
+   int err = -EINVAL, src_nent;
int null = 0;
gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL :
GFP_ATOMIC;
@@ -1377,8 +1406,8 @@ static int chcr_copy_assoc(struct aead_request *req,
 
if (op_type && req->cryptlen < crypto_aead_authsize(tfm))
goto err;
-
-   if (sg_nents_for_len(req->src, req->assoclen + req->cryptlen) < 0)
+   src_nent = sg_nents_for_len(req->src, req->assoclen + req->cryptlen);
+   if (src_nent < 0)
goto err;
src = scatterwalk_ffwd(reqctx->srcffwd, req->src, req->assoclen);
reqctx->dst = src;
@@ -1396,7 +1425,7 @@ static int chcr_copy_assoc(struct aead_request *req,
}
reqctx->dst_nents = sg_nents_for_len(reqctx->dst, req->cryptlen +
 (op_type ? -authsize : authsize));
-   if (reqctx->dst_nents <= 0) {
+   if (reqctx->dst_nents < 0) {
pr_err("AUTHENC:Invalid Destination sg entries\n");
goto err;
}
@@ -1404,6 +1433,12 @@ static int chcr_copy_assoc(struct aead_request *req,
kctx_len = (ntohl(KEY_CONTEXT_CTX_LEN_V(aeadctx->key_ctx_hdr)) << 4)
- sizeof(chcr_req->key_ctx);
transhdr_len = CIPHER_TRANSHDR_SIZE(kctx_len, dst_size);
+   if (chcr_aead_need_fallback(req, src_nent + MIN_AUTH_SG,
+   T6_MAX_AAD_SIZE,
+   transhdr_len + (sgl_len(src_nent + MIN_AUTH_SG) * 8),
+   op_type)) {
+   return ERR_PTR(chcr_aead_fallback(req, op_type));
+   }
skb = alloc_skb((transhdr_len + sizeof(struct sge_opaque_hdr)), flags);
if (!skb)
goto err;
@@ -1485,24 +1520,6 @@ static int chcr_copy_assoc(struct aead_request *req,
return ERR_PTR(-EINVAL);
 }
 
-static void aes_gcm_empty_pld_pad(struct scatterlist *sg,
- unsigned short offset)
-{
-   struct page *spage;
-   unsigned char *addr;
-
-   spage = sg_page(sg);
-   get_page(spage); /* so that it is not freed by NIC */
-#ifdef KMAP_ATOMIC_ARGS
-   addr = kmap_atomic(spage, KM_SOFTIRQ0);
-#else
-   addr = kmap_atomic(spage);
-#endif
-   memset(addr + sg->offset, 0, offset + 1);
-
-   kunmap_atomic(addr);
-}
-
 static int set_msg_len(u8 *block, unsigned int msglen, int csize)
 {
__be32 data;
@@ -156

[PATCH 3/4] chcr:Fix txq ids.

[Harsh Jain]

The patch fixes a critical issue to map txqid with flows on the hardware 
appropriately,
if tx queues created are more than flows configured then  txqid shall map within
the range of hardware flows configured. This ensure that un-mapped txqid does 
not remain un-handled.
The patch also segregated the rxqid and txqid for clarity.

Signed-off-by: Atul Gupta 
Reviewed-by: Ganesh Goudar 
---
 drivers/crypto/chelsio/chcr_algo.c  |   47 +-
 drivers/crypto/chelsio/chcr_core.h  |2 +
 drivers/crypto/chelsio/chcr_crypto.h|3 +-
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c |9 
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h  |1 +
 drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h   |3 +-
 6 files changed, 44 insertions(+), 21 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 2d61043..5470e4e 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -522,7 +522,7 @@ static inline void create_wreq(struct chcr_context *ctx,
 {
struct uld_ctx *u_ctx = ULD_CTX(ctx);
int iv_loc = IV_DSGL;
-   int qid = u_ctx->lldi.rxq_ids[ctx->tx_channel_id];
+   int qid = u_ctx->lldi.rxq_ids[ctx->rx_qidx];
unsigned int immdatalen = 0, nr_frags = 0;
 
if (is_ofld_imm(skb)) {
@@ -543,7 +543,7 @@ static inline void create_wreq(struct chcr_context *ctx,
chcr_req->wreq.cookie = cpu_to_be64((uintptr_t)req);
chcr_req->wreq.rx_chid_to_rx_q_id =
FILL_WR_RX_Q_ID(ctx->dev->rx_channel_id, qid,
-   is_iv ? iv_loc : IV_NOP, ctx->tx_channel_id);
+   is_iv ? iv_loc : IV_NOP, ctx->tx_qidx);
 
chcr_req->ulptx.cmd_dest = FILL_ULPTX_CMD_DEST(ctx->dev->tx_channel_id,
   qid);
@@ -721,19 +721,19 @@ static int chcr_aes_encrypt(struct ablkcipher_request 
*req)
struct sk_buff *skb;
 
if (unlikely(cxgb4_is_crypto_q_full(u_ctx->lldi.ports[0],
-   ctx->tx_channel_id))) {
+   ctx->tx_qidx))) {
if (!(req->base.flags & CRYPTO_TFM_REQ_MAY_BACKLOG))
return -EBUSY;
}
 
-   skb = create_cipher_wr(req, u_ctx->lldi.rxq_ids[ctx->tx_channel_id],
+   skb = create_cipher_wr(req, u_ctx->lldi.rxq_ids[ctx->rx_qidx],
   CHCR_ENCRYPT_OP);
if (IS_ERR(skb)) {
pr_err("chcr : %s : Failed to form WR. No memory\n", __func__);
return  PTR_ERR(skb);
}
skb->dev = u_ctx->lldi.ports[0];
-   set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_channel_id);
+   set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_qidx);
chcr_send_wr(skb);
return -EINPROGRESS;
 }
@@ -746,19 +746,19 @@ static int chcr_aes_decrypt(struct ablkcipher_request 
*req)
struct sk_buff *skb;
 
if (unlikely(cxgb4_is_crypto_q_full(u_ctx->lldi.ports[0],
-   ctx->tx_channel_id))) {
+   ctx->tx_qidx))) {
if (!(req->base.flags & CRYPTO_TFM_REQ_MAY_BACKLOG))
return -EBUSY;
}
 
-   skb = create_cipher_wr(req, u_ctx->lldi.rxq_ids[0],
+   skb = create_cipher_wr(req, u_ctx->lldi.rxq_ids[ctx->rx_qidx],
   CHCR_DECRYPT_OP);
if (IS_ERR(skb)) {
pr_err("chcr : %s : Failed to form WR. No memory\n", __func__);
return PTR_ERR(skb);
}
skb->dev = u_ctx->lldi.ports[0];
-   set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_channel_id);
+   set_wr_txq(skb, CPL_PRIORITY_DATA, ctx->tx_qidx);
chcr_send_wr(skb);
return -EINPROGRESS;
 }
@@ -766,7 +766,9 @@ static int chcr_aes_decrypt(struct ablkcipher_request *req)
 static int chcr_device_init(struct chcr_context *ctx)
 {
struct uld_ctx *u_ctx;
+   struct adapter *adap;
unsigned int id;
+   int txq_perchan, txq_idx, ntxq;
int err = 0, rxq_perchan, rxq_idx;
 
id = smp_processor_id();
@@ -777,11 +779,18 @@ static int chcr_device_init(struct chcr_context *ctx)
goto out;
}
u_ctx = ULD_CTX(ctx);
+   adap = padap(ctx->dev);
+   ntxq = min_not_zero((unsigned int)u_ctx->lldi.nrxq,
+   adap->vres.ncrypto_fc);
rxq_perchan = u_ctx->lldi.nrxq / u_ctx->lldi.nchan;
+   txq_perchan = ntxq / u_ctx->lldi.nchan;
rxq_idx = ctx->dev->tx_channel_id * rxq_perchan;
rxq_idx += id % rxq_perchan;
+   txq_idx = ctx->dev->tx_channel_id * txq_perchan;
+   txq_idx += id % txq_perchan;

[PATCH 2/4] chcr:Set hmac_ctrl bit to use HW register HMAC_CFG[456]

[Harsh Jain]

Use hmac_ctrl bit value saved in setauthsize callback.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c |   24 +---
 1 files changed, 5 insertions(+), 19 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 7d59591..2d61043 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -1335,19 +1335,6 @@ static int chcr_copy_assoc(struct aead_request *req,
return crypto_skcipher_encrypt(skreq);
 }
 
-static unsigned char get_hmac(unsigned int authsize)
-{
-   switch (authsize) {
-   case ICV_8:
-   return CHCR_SCMD_HMAC_CTRL_PL1;
-   case ICV_10:
-   return CHCR_SCMD_HMAC_CTRL_TRUNC_RFC4366;
-   case ICV_12:
-   return CHCR_SCMD_HMAC_CTRL_IPSEC_96BIT;
-   }
-   return CHCR_SCMD_HMAC_CTRL_NO_TRUNC;
-}
-
 
 static struct sk_buff *create_authenc_wr(struct aead_request *req,
 unsigned short qid,
@@ -1600,13 +1587,13 @@ static void fill_sec_cpl_for_aead(struct cpl_tx_sec_pdu 
*sec_cpl,
  struct chcr_context *chcrctx)
 {
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+   struct chcr_aead_ctx *aeadctx = AEAD_CTX(crypto_aead_ctx(tfm));
unsigned int ivsize = AES_BLOCK_SIZE;
unsigned int cipher_mode = CHCR_SCMD_CIPHER_MODE_AES_CCM;
unsigned int mac_mode = CHCR_SCMD_AUTH_MODE_CBCMAC;
unsigned int c_id = chcrctx->dev->rx_channel_id;
unsigned int ccm_xtra;
unsigned char tag_offset = 0, auth_offset = 0;
-   unsigned char hmac_ctrl = get_hmac(crypto_aead_authsize(tfm));
unsigned int assoclen;
 
if (get_aead_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_AEAD_RFC4309)
@@ -1642,8 +1629,8 @@ static void fill_sec_cpl_for_aead(struct cpl_tx_sec_pdu 
*sec_cpl,
crypto_aead_authsize(tfm));
sec_cpl->seqno_numivs =  FILL_SEC_CPL_SCMD0_SEQNO(op_type,
(op_type == CHCR_ENCRYPT_OP) ? 0 : 1,
-   cipher_mode, mac_mode, hmac_ctrl,
-   ivsize >> 1);
+   cipher_mode, mac_mode,
+   aeadctx->hmac_ctrl, ivsize >> 1);
 
sec_cpl->ivgen_hdrlen = FILL_SEC_CPL_IVGEN_HDRLEN(0, 0, 1, 0,
1, dst_size);
@@ -1820,7 +1807,6 @@ unsigned int fill_aead_req_fields(struct sk_buff *skb,
unsigned char tag_offset = 0;
unsigned int crypt_len = 0;
unsigned int authsize = crypto_aead_authsize(tfm);
-   unsigned char hmac_ctrl = get_hmac(authsize);
int err = 0;
gfp_t flags = req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL :
GFP_ATOMIC;
@@ -1893,8 +1879,8 @@ unsigned int fill_aead_req_fields(struct sk_buff *skb,
FILL_SEC_CPL_SCMD0_SEQNO(op_type, (op_type ==
CHCR_ENCRYPT_OP) ? 1 : 0,
CHCR_SCMD_CIPHER_MODE_AES_GCM,
-   CHCR_SCMD_AUTH_MODE_GHASH, hmac_ctrl,
-   ivsize >> 1);
+   CHCR_SCMD_AUTH_MODE_GHASH,
+   aeadctx->hmac_ctrl, ivsize >> 1);
} else {
chcr_req->sec_cpl.cipherstop_lo_authinsert =
FILL_SEC_CPL_AUTHINSERT(0, 0, 0, 0);
-- 
1.7.1

[PATCH 1/4] chcr: Increase priority of AEAD algos.

[Harsh Jain]

templates(gcm,ccm etc) inherit priority value of driver to
calculate its priority. In some cases template priority becomes
 more than driver priority for same algo.
Without this patch we will not be able to use driver authenc algos. It will
be good if it pushed in stable kernel.

Signed-off-by: Harsh Jain <ha...@chelsio.com>
---
 drivers/crypto/chelsio/chcr_algo.c   |   12 ++--
 drivers/crypto/chelsio/chcr_crypto.h |4 ++--
 2 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/crypto/chelsio/chcr_algo.c 
b/drivers/crypto/chelsio/chcr_algo.c
index 41bc7f4..7d59591 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -2673,6 +2673,7 @@ static int chcr_aead_op(struct aead_request *req,
.cra_name = "gcm(aes)",
.cra_driver_name = "gcm-aes-chcr",
.cra_blocksize  = 1,
+   .cra_priority = CHCR_AEAD_PRIORITY,
.cra_ctxsize =  sizeof(struct chcr_context) +
sizeof(struct chcr_aead_ctx) +
sizeof(struct chcr_gcm_ctx),
@@ -2691,6 +2692,7 @@ static int chcr_aead_op(struct aead_request *req,
.cra_name = "rfc4106(gcm(aes))",
.cra_driver_name = "rfc4106-gcm-aes-chcr",
.cra_blocksize   = 1,
+   .cra_priority = CHCR_AEAD_PRIORITY + 1,
.cra_ctxsize =  sizeof(struct chcr_context) +
sizeof(struct chcr_aead_ctx) +
sizeof(struct chcr_gcm_ctx),
@@ -2710,6 +2712,7 @@ static int chcr_aead_op(struct aead_request *req,
.cra_name = "ccm(aes)",
.cra_driver_name = "ccm-aes-chcr",
.cra_blocksize   = 1,
+   .cra_priority = CHCR_AEAD_PRIORITY,
.cra_ctxsize =  sizeof(struct chcr_context) +
sizeof(struct chcr_aead_ctx),
 
@@ -2728,6 +2731,7 @@ static int chcr_aead_op(struct aead_request *req,
.cra_name = "rfc4309(ccm(aes))",
.cra_driver_name = "rfc4309-ccm-aes-chcr",
.cra_blocksize   = 1,
+   .cra_priority = CHCR_AEAD_PRIORITY + 1,
.cra_ctxsize =  sizeof(struct chcr_context) +
sizeof(struct chcr_aead_ctx),
 
@@ -2747,6 +2751,7 @@ static int chcr_aead_op(struct aead_request *req,
.cra_driver_name =
"authenc-hmac-sha1-cbc-aes-chcr",
.cra_blocksize   = AES_BLOCK_SIZE,
+   .cra_priority = CHCR_AEAD_PRIORITY,
.cra_ctxsize =  sizeof(struct chcr_context) +
sizeof(struct chcr_aead_ctx) +
sizeof(struct chcr_authenc_ctx),
@@ -2768,6 +2773,7 @@ static int chcr_aead_op(struct aead_request *req,
.cra_driver_name =
"authenc-hmac-sha256-cbc-aes-chcr",
.cra_blocksize   = AES_BLOCK_SIZE,
+   .cra_priority = CHCR_AEAD_PRIORITY,
.cra_ctxsize =  sizeof(struct chcr_context) +
sizeof(struct chcr_aead_ctx) +
sizeof(struct chcr_authenc_ctx),
@@ -2788,6 +2794,7 @@ static int chcr_aead_op(struct aead_request *req,
.cra_driver_name =
"authenc-hmac-sha224-cbc-aes-chcr",
.cra_blocksize   = AES_BLOCK_SIZE,
+   .cra_priority = CHCR_AEAD_PRIORITY,
.cra_ctxsize =  sizeof(struct chcr_context) +
sizeof(struct chcr_aead_ctx) +
sizeof(struct chcr_authenc_ctx),
@@ -2807,6 +2814,7 @@ static int chcr_aead_op(struct aead_request *req,
.cra_driver_name =
"authenc-hmac-sha384-cbc-aes-chcr",
.cra_blocksize   = AES_BLOCK_SIZE,
+   .cra_priority = CHCR_AEAD_PRIORITY,
   

[PATCH 0/4] Bug fixes and fallback for AEAD

[Harsh Jain]

This series based on Herbert cryptodev-2.6.
It includes bug fixes and fallback for AEAD algos.

Harsh Jain (3):
  chcr: Increase priority of AEAD algos.
  chcr:Set hmac_ctrl bit to use HW register HMAC_CFG[456].
  chcr: Add fallback for AEAD algos
Atul Gupta (1):
  chcr: Fix txq ids

 drivers/crypto/chelsio/chcr_algo.c  |  298 ++-
 drivers/crypto/chelsio/chcr_algo.h  |4 +
 drivers/crypto/chelsio/chcr_core.h  |2 +
 drivers/crypto/chelsio/chcr_crypto.h|   10 +-
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c |9 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h  |1 +
 drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h   |3 +-
 7 files changed, 210 insertions(+), 117 deletions(-)

Re: What should be the algo priority

[Harsh Jain]

On Tue, Apr 4, 2017 at 6:07 PM, Stephan Müller <smuel...@chronox.de> wrote:
> Am Dienstag, 4. April 2017, 09:53:17 CEST schrieb Harsh Jain:
>
> Hi Harsh,
>
>> Hi,
>>
>> Do we have any guidelines documented to decide what should be the
>> algorithm priority. Specially for authenc implementation.Most of the
>> drivers have fixed priority for all algos. Problem comes in when we
>> have cbc(aes), hmac(sha1) and authenc(cbc(aes),hmac(sha1))
>> implementation in driver. Base authenc driver gets more precedence
>> because of higher priority(enc->base.cra_priority * 10 +
>> auth_base->cra_priority;)
>>
>> What should be the priority of
>> cbc(aes),
>> hmac(sha1)
>> authenc(cbc(aes),hmac(sha1))
>
> There is no general rule about the actual numbers. But commonly, the prios are
> set such that the prios of C implementations < ASM implementations < hardware
> accelerators. The idea is to give users the fastest implementation there is
> for his particular system.

It means cbc, hmac should have smaller(nearly 10 times less) priority
than their authenc implementation otherwise request will not offload
to driver because sw authenc priority is (aes * 10 + hmac).

>
> Ciao
> Stephan

Re: BUG: Seems un-initialed dst pointer received from algif_aead when outlen is zero

[Harsh Jain]

On Tue, Mar 21, 2017 at 5:13 PM, Stephan Müller <smuel...@chronox.de> wrote:
> Am Dienstag, 21. März 2017, 11:59:54 CET schrieb Harsh Jain:
>
> Hi Harsh,
>
>> > Executing this command on a 4.9 kernel, I get:
>> >
>> > bin/kcapi -x 2   -c "gcm(aes)" -i 0d92aa861746b324f20ee6b7 -k
>> > f4a6a5e5f2066f6dd9ec6fc5169c29043560ef595c9e81e76f42d29212cc581c -a "" -t
>> > "5f24c68cbe6f32c29652442bf5d483ad" -q ""
>> > EBADMSG
>>
>> Probably because s/w implementation is not trying to access dst sg
>> pointer because there's nothing to copy in destination buffer.  1
>> question If we don't have data to copy to destination buffer what
>> should dst pointer contains?
>
> The dst SGL should simply be discarded by implementations in the case you
> mention above.
>
> The implementation receives the tag size and the supplied input buffer. If
> that input buffer length is equal to the tag length (i.e. no AAD and no
> ciphertext), why would the dst SGL be ever touched during decrytion?

Yes, Driver can figure out when to discard dst SGL but for that Driver
has to put checks before accessing dst SGL. Isn't better if AF_ALG
sends NULL for dst SGL.

>
> Ciao
> Stephan

Re: BUG: Seems un-initialed dst pointer received from algif_aead when outlen is zero

[Harsh Jain]

On Tue, Mar 21, 2017 at 4:29 PM, Harsh Jain <harshjain.p...@gmail.com> wrote:
> On Tue, Mar 21, 2017 at 3:34 PM, Stephan Müller <smuel...@chronox.de> wrote:
>> Am Dienstag, 21. März 2017, 07:13:53 CET schrieb Harsh Jain:
>>
>> Hi Harsh,
>>
>>> Hi,
>>>
>>> For tag only AEAD decrypt operation(Zero length Payload). The dst sg
>>> list pointer panic with general protection fault. I think it should be
>>> NULL when output buffer is supposed to be empty.
>>>
>>> Kcapi command to re-produce the issue
>>>
>>> ./kcapi -x 2   -c "gcm(aes)" -i 0d92aa861746b324f20ee6b7 -k
>>> f4a6a5e5f2066f6dd9ec6fc5169c29043560ef595c9e81e76f42d29212cc581c -a ""
>>> -t "5f24c68cbe6f32c29652442bf5d483ad" -q ""
>>>
>>> Its decrypt operation. Expected result should be EBADMSG.
>>
>> Executing this command on a 4.9 kernel, I get:
>>
>> bin/kcapi -x 2   -c "gcm(aes)" -i 0d92aa861746b324f20ee6b7 -k
>> f4a6a5e5f2066f6dd9ec6fc5169c29043560ef595c9e81e76f42d29212cc581c -a "" -t
>> "5f24c68cbe6f32c29652442bf5d483ad" -q ""
>> EBADMSG
>
> Probably because s/w implementation is not trying to access dst sg
> pointer because there's nothing to copy in destination buffer.  1
> question If we don't have data to copy to destination buffer what
> should dst pointer contains? I think either NULL or null sg entry.
>
>>
>> There is no GP or other error. Can you please provide some details about your
>> system? I.e. which kernel version and what cipher implementation resolves to
>> gcm(aes)?
>
> I tried with 4.10.13. It's with gcm(aes-chcr). changes which trigger
> issue is not submitted to community yet.
typo Its 4.10.0-rc3+
>
>>
>> Thanks
>>
>> Ciao
>> Stephan

Re: BUG: Seems un-initialed dst pointer received from algif_aead when outlen is zero

[Harsh Jain]

On Tue, Mar 21, 2017 at 3:34 PM, Stephan Müller <smuel...@chronox.de> wrote:
> Am Dienstag, 21. März 2017, 07:13:53 CET schrieb Harsh Jain:
>
> Hi Harsh,
>
>> Hi,
>>
>> For tag only AEAD decrypt operation(Zero length Payload). The dst sg
>> list pointer panic with general protection fault. I think it should be
>> NULL when output buffer is supposed to be empty.
>>
>> Kcapi command to re-produce the issue
>>
>> ./kcapi -x 2   -c "gcm(aes)" -i 0d92aa861746b324f20ee6b7 -k
>> f4a6a5e5f2066f6dd9ec6fc5169c29043560ef595c9e81e76f42d29212cc581c -a ""
>> -t "5f24c68cbe6f32c29652442bf5d483ad" -q ""
>>
>> Its decrypt operation. Expected result should be EBADMSG.
>
> Executing this command on a 4.9 kernel, I get:
>
> bin/kcapi -x 2   -c "gcm(aes)" -i 0d92aa861746b324f20ee6b7 -k
> f4a6a5e5f2066f6dd9ec6fc5169c29043560ef595c9e81e76f42d29212cc581c -a "" -t
> "5f24c68cbe6f32c29652442bf5d483ad" -q ""
> EBADMSG

Probably because s/w implementation is not trying to access dst sg
pointer because there's nothing to copy in destination buffer.  1
question If we don't have data to copy to destination buffer what
should dst pointer contains?

>
> There is no GP or other error. Can you please provide some details about your
> system? I.e. which kernel version and what cipher implementation resolves to
> gcm(aes)?

I tried with 4.10.13. It's with gcm(aes-chcr). changes which trigger
issue is not submitted to community yet.

>
> Thanks
>
> Ciao
> Stephan