apport information ** Tags added: apport-collected jammy uec-images
** Description changed: CONFIG_CRYPTO_STATS is enabled in Jammy 5.15.0-xx as well as 6.2.0-xx kernel by default. # uname -r 5.15.0-69-generic root@r100:~# grep CONFIG_CRYPTO_STATS /boot/config-5.15.0-69-generic CONFIG_CRYPTO_STATS=y However, that CONFIG_CRYPTO_STATS makes big negative performance impacts. Lustre (http://www.lustre.com Open source parallel filesystem) supports client encryption based on fscrypt. I ran some benchmarks of Lustre client encryption on Lustre client with Ubuntu 6.2.0-34-generic kernel below. 1 x client(2 x Gold 6438Y+, 512GB RAM, 2 x HDR200) Ubuntu22.04 (6.2.0-34-generic) # fio -name=iotest-w -ioengine=sync -rw=write -blocksize=16m -iodepth=1 -direct=1 -fallocate=none -size=200g -numjobs=128 -group_reporting -directory=/lustre/encrypted -create_serialize=0 -fiename_format='f.$jobnum.$filenum' -scramble_buffers=0 -runtime=60 6.2.0-34-generic, no encryption 48.5GB/s 6.2.0-34-generic, encryption 21.7GB/s 6.2.0-34-generic-noCryptStats, encryption 48.2GB/s When Lustre client encryption enabled, there were 65% performance degradations compared to non encryption's performance. (21.7GB/s vs 48.5GB/s) I also tested on 6.2.0-34-generic-noCryptStats which I built custom 6.2.0-34-generic kernel with disabling CONFIG_CRYPTO_STATS. The result was pretty good. The performance did boost and it was almost same speed against non encryption case. (48.2GB/s vs 48.5GB/s) I also found same fscrypt performance problem reported in Ceph with RHEL kernel below. https://lists.ceph.io/hyperkitty/list/ceph-us...@ceph.io/thread/44GMO5UGOXDZKFSOQMCPPHYTREUEA3ZI/ Eventually, Redhat disabled CONFIG_CRYPTO_STATS in latest RHEL9 kernel https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/2947 Kernel config option CONFIG_CRYPTO_STATS should be disabled until it is enhanced (BZ#2231850) - https://access.redhat.com/errata/RHSA-2023:5069 - I would disable CONFIG_CRYPTO_STATS in all Jammy kernels until proper - fixes and new solution is available in upstream kernel. + I would disable CONFIG_CRYPTO_STATS in all Jammy kernels until proper fixes and new solution is available in upstream kernel. + --- + ProblemType: Bug + AlsaDevices: + total 0 + crw-rw---- 1 root audio 116, 1 Oct 17 21:41 seq + crw-rw---- 1 root audio 116, 33 Oct 17 21:41 timer + AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' + ApportVersion: 2.20.11-0ubuntu82.4 + Architecture: amd64 + ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' + AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: + CRDA: N/A + CasperMD5CheckResult: unknown + DistroRelease: Ubuntu 22.04 + IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' + Lsusb: + Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub + Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Lsusb-t: + /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 10000M + /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M + MachineType: Giga Computing H263-S63-AAN1-000 + Package: linux (not installed) + PciMultimedia: + + ProcEnviron: + LC_CTYPE=C.UTF-8 + TERM=vt100 + PATH=(custom, no user) + LANG=C.UTF-8 + SHELL=/bin/bash + ProcFB: 0 astdrmfb + ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-6.2.0-34-generic root=UUID=b48b34d5-f2cf-444d-b376-0ffc987891c1 ro + ProcVersionSignature: Ubuntu 6.2.0-34.34~22.04.1-generic 6.2.16 + RelatedPackageVersions: + linux-restricted-modules-6.2.0-34-generic N/A + linux-backports-modules-6.2.0-34-generic N/A + linux-firmware 20220329.git681281e4-0ubuntu3.19 + RfKill: Error: [Errno 2] No such file or directory: 'rfkill' + Tags: jammy uec-images + Uname: Linux 6.2.0-34-generic x86_64 + UpgradeStatus: No upgrade log present (probably fresh install) + UserGroups: N/A + _MarkForUpload: True + dmi.bios.date: 08/09/2023 + dmi.bios.release: 5.31 + dmi.bios.vendor: GIGABYTE + dmi.bios.version: F20 + dmi.board.asset.tag: 01234567890123456789AB + dmi.board.name: MS63-HD1-000 + dmi.board.vendor: Giga Computing + dmi.board.version: 01000100 + dmi.chassis.asset.tag: 01234567890123456789AB + dmi.chassis.type: 23 + dmi.chassis.vendor: Giga Computing + dmi.chassis.version: 01234567 + dmi.modalias: dmi:bvnGIGABYTE:bvrF20:bd08/09/2023:br5.31:svnGigaComputing:pnH263-S63-AAN1-000:pvr9NH263S63DR000AAN1*:rvnGigaComputing:rnMS63-HD1-000:rvr01000100:cvnGigaComputing:ct23:cvr01234567:sku01234567890123456789AB: + dmi.product.family: Server + dmi.product.name: H263-S63-AAN1-000 + dmi.product.sku: 01234567890123456789AB + dmi.product.version: 9NH263S63DR000AAN1* + dmi.sys.vendor: Giga Computing ** Attachment added: "CurrentDmesg.txt" https://bugs.launchpad.net/bugs/2039576/+attachment/5710490/+files/CurrentDmesg.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2039576 Title: Enabling CONFIG_CRYPTO_STATS causes significant fscrypt performance degradation Status in linux package in Ubuntu: Confirmed Bug description: CONFIG_CRYPTO_STATS is enabled in Jammy 5.15.0-xx as well as 6.2.0-xx kernel by default. # uname -r 5.15.0-69-generic root@r100:~# grep CONFIG_CRYPTO_STATS /boot/config-5.15.0-69-generic CONFIG_CRYPTO_STATS=y However, that CONFIG_CRYPTO_STATS makes big negative performance impacts. Lustre (http://www.lustre.com Open source parallel filesystem) supports client encryption based on fscrypt. I ran some benchmarks of Lustre client encryption on Lustre client with Ubuntu 6.2.0-34-generic kernel below. 1 x client(2 x Gold 6438Y+, 512GB RAM, 2 x HDR200) Ubuntu22.04 (6.2.0-34-generic) # fio -name=iotest-w -ioengine=sync -rw=write -blocksize=16m -iodepth=1 -direct=1 -fallocate=none -size=200g -numjobs=128 -group_reporting -directory=/lustre/encrypted -create_serialize=0 -fiename_format='f.$jobnum.$filenum' -scramble_buffers=0 -runtime=60 6.2.0-34-generic, no encryption 48.5GB/s 6.2.0-34-generic, encryption 21.7GB/s 6.2.0-34-generic-noCryptStats, encryption 48.2GB/s When Lustre client encryption enabled, there were 65% performance degradations compared to non encryption's performance. (21.7GB/s vs 48.5GB/s) I also tested on 6.2.0-34-generic-noCryptStats which I built custom 6.2.0-34-generic kernel with disabling CONFIG_CRYPTO_STATS. The result was pretty good. The performance did boost and it was almost same speed against non encryption case. (48.2GB/s vs 48.5GB/s) I also found same fscrypt performance problem reported in Ceph with RHEL kernel below. https://lists.ceph.io/hyperkitty/list/ceph-us...@ceph.io/thread/44GMO5UGOXDZKFSOQMCPPHYTREUEA3ZI/ Eventually, Redhat disabled CONFIG_CRYPTO_STATS in latest RHEL9 kernel https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/2947 Kernel config option CONFIG_CRYPTO_STATS should be disabled until it is enhanced (BZ#2231850) - https://access.redhat.com/errata/RHSA-2023:5069 I would disable CONFIG_CRYPTO_STATS in all Jammy kernels until proper fixes and new solution is available in upstream kernel. --- ProblemType: Bug AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Oct 17 21:41 seq crw-rw---- 1 root audio 116, 33 Oct 17 21:41 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.11-0ubuntu82.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A CasperMD5CheckResult: unknown DistroRelease: Ubuntu 22.04 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Lsusb-t: /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 10000M /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M MachineType: Giga Computing H263-S63-AAN1-000 Package: linux (not installed) PciMultimedia: ProcEnviron: LC_CTYPE=C.UTF-8 TERM=vt100 PATH=(custom, no user) LANG=C.UTF-8 SHELL=/bin/bash ProcFB: 0 astdrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-6.2.0-34-generic root=UUID=b48b34d5-f2cf-444d-b376-0ffc987891c1 ro ProcVersionSignature: Ubuntu 6.2.0-34.34~22.04.1-generic 6.2.16 RelatedPackageVersions: linux-restricted-modules-6.2.0-34-generic N/A linux-backports-modules-6.2.0-34-generic N/A linux-firmware 20220329.git681281e4-0ubuntu3.19 RfKill: Error: [Errno 2] No such file or directory: 'rfkill' Tags: jammy uec-images Uname: Linux 6.2.0-34-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: N/A _MarkForUpload: True dmi.bios.date: 08/09/2023 dmi.bios.release: 5.31 dmi.bios.vendor: GIGABYTE dmi.bios.version: F20 dmi.board.asset.tag: 01234567890123456789AB dmi.board.name: MS63-HD1-000 dmi.board.vendor: Giga Computing dmi.board.version: 01000100 dmi.chassis.asset.tag: 01234567890123456789AB dmi.chassis.type: 23 dmi.chassis.vendor: Giga Computing dmi.chassis.version: 01234567 dmi.modalias: dmi:bvnGIGABYTE:bvrF20:bd08/09/2023:br5.31:svnGigaComputing:pnH263-S63-AAN1-000:pvr9NH263S63DR000AAN1*:rvnGigaComputing:rnMS63-HD1-000:rvr01000100:cvnGigaComputing:ct23:cvr01234567:sku01234567890123456789AB: dmi.product.family: Server dmi.product.name: H263-S63-AAN1-000 dmi.product.sku: 01234567890123456789AB dmi.product.version: 9NH263S63DR000AAN1* dmi.sys.vendor: Giga Computing To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2039576/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
