From: Alex Ousherovitch <[email protected]> crypto: cmh - add CRI CryptoManager Hub hardware crypto accelerator
This series adds a driver for the CRI CryptoManager Hub (CMH), a hardware cryptographic accelerator IP from Cryptography Research at Rambus Inc. (https://www.rambus.com/cryptographyresearch/). CMH provides a broad set of symmetric, asymmetric, and post-quantum cryptographic algorithms accelerated in hardware, accessed via a mailbox-based Virtual Command Queue (VCQ) interface. The hardware is a platform device matched via device tree (compatible = "cri,cmh"). It exposes a single MMIO register region (SIC) with per-mailbox doorbell, status, and command registers. Each mailbox has DMA-coherent queue memory for VCQ command submission and completion. Driver architecture: In-kernel users /dev/cmh_mgmt (ioctl) (dm-crypt, IPsec, kTLS, fscrypt) (key management) | | v v +----------------------------------------------------+ | Kernel Crypto API + hwrng (72 total) | | ahash | skcipher | aead | akcipher | sig | kpp | +----------------------------------------------------+ | | v v +------------------+ +------------------------+ | Transaction Mgr |--->| Key / Mgmt subsystem | | (kthread, CMQ) | | (datastore, ioctl ops) | +------------------+ +------------------------+ | v +------------------+ +-------------------+ | MQI (VCQ pack, |---->| Response Handler | | DMA map, submit)| | (threaded IRQ, | +------------------+ | watchdog, unmap) | | +-------------------+ v ^ +-----------+ +-----------+ | Hardware |--- IRQ ----->| Hardware | | (mailbox) | | (mailbox) | +-----------+ +-----------+ The transaction manager runs as a dedicated kthread that pulls requests from a central command queue, packs VCQ entries, maps DMA buffers, and submits to the least-loaded mailbox. Completion is handled by per-mailbox threaded IRQs. The driver returns -EINPROGRESS for async crypto requests and supports the CRYPTO_TFM_REQ_MAY_BACKLOG flag for queue-full backpressure. Registered algorithms (72 total): Type Count Algorithms --------- ----- -------------------------------------------------- ahash 15 SHA-{224,256,384,512}, SHA3-{224,256,384,512}, SHAKE-{128,256}, cSHAKE-{128,256}, KMAC-{128,256}, SM3 ahash(HMAC) 8 HMAC-SHA-{224,256,384,512}, HMAC-SHA3-{224,256,384,512} ahash(MAC) 4 CMAC(AES), CMAC(SM4), XCBC(SM4), Poly1305 skcipher 11 AES-{ECB,CBC,CTR,CFB,XTS}, SM4-{ECB,CBC,CTR,CFB,XTS}, ChaCha20 aead 6 AES-{GCM,CCM}, SM4-{GCM,CCM}, rfc7539(chacha20,poly1305), rfc7539esp(chacha20,poly1305) akcipher 1 RSA (2048--4096 bit; 512/1024 legacy/test) sig 23 ECDSA P-{256,384,521}, SM2 (verify-only), ML-DSA-{44,65,87}, SLH-DSA (12 parameter sets), LMS, LMS-HSS, XMSS, XMSS-MT kpp 3 ECDH P-{256,384}, X25519 hwrng 1 DRBG-backed /dev/hwrng Ioctl-only algorithms (not registered with the crypto API at all): - EdDSA (Ed25519, Ed448): sign and verify - ML-KEM (ML-KEM-512/768/1024): no standard kernel KEM API exists The driver also exposes /dev/cmh_mgmt, a misc device providing 44 ioctl commands. Relative to the in-kernel crypto API these fall into two groups; the distinction matters because some commands name the same primitives the driver also registers, and that overlap is deliberate and bounded: (1) Operations with no crypto API representation - the large majority. The crypto API has no transform type or verb for these, so a character device is the only available UAPI: - hardware key lifecycle: create, import, export, derive, destroy, enumerate (keystore CRUD) - no keystore verb - KIC key derivation (HKDF, AES-CMAC-KDF, DKEK) - asymmetric key generation (RSA, EC, EdDSA, ML-DSA, SLH-DSA) and public-key derivation - the crypto API has no keygen verb - ML-KEM encapsulate/decapsulate - no kernel KEM API exists - SM2 encrypt/decrypt and key exchange (multi-step GM/T 0003) - EdDSA sign/verify - not registered with the crypto API - EAC Chip Authentication and DRBG (re)configuration (2) Hardware-held-key operations on algorithms that ARE also registered (RSA decrypt, ECDSA/ML-DSA/SLH-DSA sign, ECDH). These name the same primitives as the registered akcipher/sig/kpp transforms, but the crypto API's set_priv_key()/set_secret() accept only raw key bytes supplied by the caller; they cannot reference a private key that is generated inside, and never leaves, the hardware datastore - the central security property of this device. The ioctl path keeps the private key hardware-resident, while the registered transforms serve raw-key in-kernel users. The two paths are complementary, not redundant. The device requires CAP_SYS_ADMIN. /dev/cmh_mgmt is built conditionally on CONFIG_CRYPTO_DEV_CMH_MGMT (default n); when disabled the ioctl interface is absent while all kernel crypto API algorithms remain registered. The ML-DSA sig algorithms are registered at priority 5001. The kernel's crypto/mldsa.c registers at priority 5000 with verify-only (sign returns -EOPNOTSUPP). Our driver provides full HW-accelerated sign + verify, so the higher priority ensures the hardware implementation is preferred when the driver is loaded. Power management uses DEFINE_SIMPLE_DEV_PM_OPS. On suspend the transaction manager drains in-flight requests (configurable 10s timeout, returns -ECANCELED on timeout), stops the kthread, and masks IRQs. On resume it re-verifies SIC/boot status and restarts the kthread. Dependencies: - Kernel 7.1+ (based on Herbert Xu's cryptodev-2.6 tree, 7.1.0-rc2) - sig_alg backend (upstream since 6.13) - CRYPTO_AHASH_REQ_VIRT (native support, no fallback needed) - CMH eSW loaded independently by hardware before driver probe The driver registers all algorithms through the standard in-kernel crypto API; in-kernel users (dm-crypt, fscrypt, IPsec, etc.) consume them directly. Key provisioning and hardware-held-key operations are exposed to user space via /dev/cmh_mgmt ioctls. Public hardware documentation: Product brief: https://go.rambus.com/ch-7xx-and-cc-7xx-product-brief No public datasheets are currently available. The driver was developed against the CRI CryptoManager Hub Hardware Reference Manual (Rambus Inc. confidential). Detailed hardware reference is available under NDA from Rambus Inc.; contact the maintainers listed in MAINTAINERS for access during review. Tested on RISC-V and ARM64 QEMU emulation with the CMH hardware model (QEMU TCG, 512 MiB RAM). Also exercised on Xilinx VMK180 FPGA board with real CMH IP. - testmgr: 41 CMH algorithm registrations matched by upstream test vectors, all pass; 30 names report "No test for" (PQC families, KMAC, cSHAKE - no upstream vectors yet). - kselftest tools/testing/selftests/drivers/crypto/cmh: 6 pass, 0 fail. checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks on all files (the only output is the expected per-file "does MAINTAINERS need updating?" reminder, satisfied by the MAINTAINERS patch). sparse (C=2): 0 warnings. W=1 -Werror: clean. make dt_binding_check: clean (dtschema validates the cri,cmh.yaml binding). Tested with the following debug options enabled simultaneously (submit-checklist "Test your code" item 1): CONFIG_PROVE_LOCKING, CONFIG_PROVE_RCU, CONFIG_DEBUG_LOCK_ALLOC, CONFIG_DEBUG_OBJECTS_RCU_HEAD, CONFIG_SLUB_DEBUG, CONFIG_DEBUG_PAGEALLOC, CONFIG_DEBUG_MUTEXES, CONFIG_DEBUG_SPINLOCK, CONFIG_DEBUG_PREEMPT, CONFIG_DEBUG_ATOMIC_SLEEP. Result: no lockdep warnings, no ODEBUG splats, no slab corruption. Additionally tested (separate passes - mutually exclusive configs): - CONFIG_KASAN + CONFIG_UBSAN + CONFIG_DEBUG_KMEMLEAK + CONFIG_KFENCE: no sanitizer findings; KMEMLEAK scan reports 0 unreferenced objects. - CONFIG_KCSAN (arm64; riscv64 lacks HAVE_ARCH_KCSAN): 0 data-race reports attributed to the driver. Stack usage: worst-case under 1 KB on both riscv64 and arm64 (scripts/checkstack.pl). Hardware command buffers live in per-request context (heap-allocated by the crypto framework). Alex Ousherovitch (19): dt-bindings: crypto: add Rambus CryptoManager Hub crypto: cmh - add core platform driver crypto: cmh - add key provisioning and management crypto: cmh - add SHA-2/SHA-3/SHAKE ahash crypto: cmh - add HMAC ahash crypto: cmh - add CSHAKE/KMAC ahash crypto: cmh - add SM3 ahash crypto: cmh - add AES skcipher/aead/cmac crypto: cmh - add SM4 skcipher/aead/cmac/xcbc crypto: cmh - add ChaCha20-Poly1305 crypto: cmh - add DRBG hwrng crypto: cmh - add RSA akcipher crypto: cmh - add ECDSA/SM2 sig crypto: cmh - add ECDH/X25519 kpp crypto: cmh - add ML-KEM/ML-DSA (QSE) crypto: cmh - add SLH-DSA/LMS/XMSS (HCQ) Documentation: ioctl: add CMH ioctl documentation and register 'J' selftests: crypto: cmh - add kselftest for management ioctl MAINTAINERS: add Rambus CryptoManager Hub (CMH) base-commit: 6ea0ce3a19f9c37a014099e2b0a46b27fa164564 -- 2.43.7 ** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying, forwarding or saving it. Please delete the message and attachments and notify the sender immediately. ** Rambus Inc.<http://www.rambus.com>
