Public bug reported: CONFIG_CRYPTO_STATS is enabled in Jammy 5.15.0-xx as well as 6.2.0-xx kernel by default.
# uname -r 5.15.0-69-generic root@r100:~# grep CONFIG_CRYPTO_STATS /boot/config-5.15.0-69-generic CONFIG_CRYPTO_STATS=y However, that CONFIG_CRYPTO_STATS makes big negative performance impacts. Lustre (http://www.lustre.com Open source parallel filesystem) supports client encryption based on fscrypt. I ran some benchmarks of Lustre client encryption on Lustre client with Ubuntu 6.2.0-34-generic kernel below. 1 x client(2 x Gold 6438Y+, 512GB RAM, 2 x HDR200) Ubuntu22.04 (6.2.0-34-generic) # fio -name=iotest-w -ioengine=sync -rw=write -blocksize=16m -iodepth=1 -direct=1 -fallocate=none -size=200g -numjobs=128 -group_reporting -directory=/lustre/encrypted -create_serialize=0 -fiename_format='f.$jobnum.$filenum' -scramble_buffers=0 -runtime=60 6.2.0-34-generic, no encryption 48.5GB/s 6.2.0-34-generic, encryption 21.7GB/s 6.2.0-34-generic-noCryptStats, encryption 48.2GB/s When Lustre client encryption enabled, there were 65% performance degradations compared to non encryption's performance. (21.7GB/s vs 48.5GB/s) I also tested on 6.2.0-34-generic-noCryptStats which I built custom 6.2.0-34-generic kernel with disabling CONFIG_CRYPTO_STATS. The result was pretty good. The performance did boost and it was almost same speed against non encryption case. (48.2GB/s vs 48.5GB/s) I also found same fscrypt performance problem reported in Ceph with RHEL kernel below. https://lists.ceph.io/hyperkitty/list/ceph-us...@ceph.io/thread/44GMO5UGOXDZKFSOQMCPPHYTREUEA3ZI/ Eventually, Redhat disabled CONFIG_CRYPTO_STATS in latest RHEL9 kernel https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/2947 Kernel config option CONFIG_CRYPTO_STATS should be disabled until it is enhanced (BZ#2231850) - https://access.redhat.com/errata/RHSA-2023:5069 I would disable CONFIG_CRYPTO_STATS in all Jammy kernels until proper fixes and new solution is available in upstream kernel. ** Affects: linux (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2039576 Title: Enabling CONFIG_CRYPTO_STATS causes significant fscrypt performance degradation Status in linux package in Ubuntu: New Bug description: CONFIG_CRYPTO_STATS is enabled in Jammy 5.15.0-xx as well as 6.2.0-xx kernel by default. # uname -r 5.15.0-69-generic root@r100:~# grep CONFIG_CRYPTO_STATS /boot/config-5.15.0-69-generic CONFIG_CRYPTO_STATS=y However, that CONFIG_CRYPTO_STATS makes big negative performance impacts. Lustre (http://www.lustre.com Open source parallel filesystem) supports client encryption based on fscrypt. I ran some benchmarks of Lustre client encryption on Lustre client with Ubuntu 6.2.0-34-generic kernel below. 1 x client(2 x Gold 6438Y+, 512GB RAM, 2 x HDR200) Ubuntu22.04 (6.2.0-34-generic) # fio -name=iotest-w -ioengine=sync -rw=write -blocksize=16m -iodepth=1 -direct=1 -fallocate=none -size=200g -numjobs=128 -group_reporting -directory=/lustre/encrypted -create_serialize=0 -fiename_format='f.$jobnum.$filenum' -scramble_buffers=0 -runtime=60 6.2.0-34-generic, no encryption 48.5GB/s 6.2.0-34-generic, encryption 21.7GB/s 6.2.0-34-generic-noCryptStats, encryption 48.2GB/s When Lustre client encryption enabled, there were 65% performance degradations compared to non encryption's performance. (21.7GB/s vs 48.5GB/s) I also tested on 6.2.0-34-generic-noCryptStats which I built custom 6.2.0-34-generic kernel with disabling CONFIG_CRYPTO_STATS. The result was pretty good. The performance did boost and it was almost same speed against non encryption case. (48.2GB/s vs 48.5GB/s) I also found same fscrypt performance problem reported in Ceph with RHEL kernel below. https://lists.ceph.io/hyperkitty/list/ceph-us...@ceph.io/thread/44GMO5UGOXDZKFSOQMCPPHYTREUEA3ZI/ Eventually, Redhat disabled CONFIG_CRYPTO_STATS in latest RHEL9 kernel https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/2947 Kernel config option CONFIG_CRYPTO_STATS should be disabled until it is enhanced (BZ#2231850) - https://access.redhat.com/errata/RHSA-2023:5069 I would disable CONFIG_CRYPTO_STATS in all Jammy kernels until proper fixes and new solution is available in upstream kernel. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2039576/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
