The branch main has been updated by markj:
URL:
https://cgit.FreeBSD.org/src/commit/?id=98d788c867b9e1d7a7e290254443b87ea77d8ab1
commit 98d788c867b9e1d7a7e290254443b87ea77d8ab1
Author: Mark Johnston <ma...@freebsd.org>
AuthorDate: 2021-01-20 01:34:35 +0000
Commit: Mark Johnston <ma...@freebsd.org>
CommitDate: 2021-01-20 01:34:35 +0000
opencrypto: Fix assignment of crypto completions to worker threads
Since r336439 we simply take the session pointer value mod the number of
worker threads (ncpu by default). On small systems this ends up
funneling all completion work through a single thread, which becomes a
bottleneck when processing IPSec traffic using hardware crypto drivers.
(Software drivers such as aesni(4) are unaffected since they invoke
completion handlers synchonously.)
Instead, maintain an incrementing counter with a unique value per
session, and use that to distribute work to completion threads.
Reviewed by: cem, jhb
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D28159
---
sys/opencrypto/crypto.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/sys/opencrypto/crypto.c b/sys/opencrypto/crypto.c
index 7bc230140568..fc4169fb2365 100644
--- a/sys/opencrypto/crypto.c
+++ b/sys/opencrypto/crypto.c
@@ -135,6 +135,7 @@ static int crypto_drivers_size = 0;
struct crypto_session {
struct cryptocap *cap;
struct crypto_session_params csp;
+ uint64_t id;
};
/*
@@ -908,6 +909,7 @@ int
crypto_newsession(crypto_session_t *cses,
const struct crypto_session_params *csp, int crid)
{
+ static uint64_t sessid = 0;
crypto_session_t res;
struct cryptocap *cap;
int err;
@@ -944,6 +946,7 @@ crypto_newsession(crypto_session_t *cses,
M_WAITOK | M_ZERO);
res->cap = cap;
res->csp = *csp;
+ res->id = atomic_fetchadd_64(&sessid, 1);
/* Call the driver initialization routine. */
err = CRYPTODEV_NEWSESSION(cap->cc_dev, res, csp);
@@ -1416,7 +1419,7 @@ crypto_dispatch(struct cryptop *crp)
CRYPTOSTAT_INC(cs_ops);
- crp->crp_retw_id = ((uintptr_t)crp->crp_session) % crypto_workers_num;
+ crp->crp_retw_id = crp->crp_session->id % crypto_workers_num;
if (CRYPTOP_ASYNC(crp)) {
if (crp->crp_flags & CRYPTO_F_ASYNC_KEEPORDER) {
_______________________________________________
dev-commits-src-main@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/dev-commits-src-main
To unsubscribe, send any mail to "dev-commits-src-main-unsubscr...@freebsd.org"
