** Description changed: I noticed performance issues on the server running some backend applications after updating OS packages with Ubuntu Jammy. After further investigation, the slowness was caused by disk IO. After the update, running on kernel 5.15.0-112, the disk writes take about twice as long. I can reproduce this by changing the GRUB configuration to boot into kernel 5.15.0-107, where the writes are - consistently slower. See below. + consistently faster than on the newer kernel. See below. # uname -a Linux dev03 5.15.0-112-generic #122-Ubuntu SMP Thu May 23 07:48:21 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux # time dd if=/dev/urandom of=deleteme bs=1024 count=1000000 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB, 977 MiB) copied, 6.54711 s, 156 MB/s real 0m6.782s user 0m0.192s sys 0m5.968s # uname -a Linux dev03 5.15.0-107-generic #117-Ubuntu SMP Fri Apr 26 12:26:49 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux # time dd if=/dev/urandom of=deleteme bs=1024 count=1000000 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB, 977 MiB) copied, 3.77936 s, 271 MB/s real 0m3.781s user 0m0.132s sys 0m3.648s - These times are with NVMe drives on ext4 filesystem. I observe the same performance difference for ZFS filesystem, and also with HDDs (spinning drives). The disk performance is also worse for 6.5.0-41 kernel. I also tried 6.5.0-35 and linux-image-generic-hwe-22.04 - same problem - writes are slow. ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: linux-image-5.15.0-112-generic 5.15.0-112.122 ProcVersionSignature: Ubuntu 5.15.0-112.122-generic 5.15.152 Uname: Linux 5.15.0-112-generic x86_64 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair AlsaDevices: - total 0 - drwxr-xr-x 2 root root 60 Jun 24 08:23 by-path - crw-rw---- 1 root audio 116, 2 Jun 24 08:23 controlC0 - crw-rw---- 1 root audio 116, 1 Jun 24 08:23 seq - crw-rw---- 1 root audio 116, 33 Jun 24 08:23 timer + total 0 + drwxr-xr-x 2 root root 60 Jun 24 08:23 by-path + crw-rw---- 1 root audio 116, 2 Jun 24 08:23 controlC0 + crw-rw---- 1 root audio 116, 1 Jun 24 08:23 seq + crw-rw---- 1 root audio 116, 33 Jun 24 08:23 timer AlsaVersion: Advanced Linux Sound Architecture Driver Version k5.15.0-112-generic. AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.11-0ubuntu82.5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer' Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer' CasperMD5CheckResult: unknown Date: Mon Jun 24 10:24:44 2024 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: - Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub - Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub - Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub - Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub + Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub + Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Lsusb-t: - /: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M - /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M - /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M - /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M + /: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M + /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M + /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M + /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M ProcEnviron: - TERM=xterm-256color - PATH=(custom, no user) - XDG_RUNTIME_DIR=<set> - LANG=en_US.UTF-8 - SHELL=/bin/bash + TERM=xterm-256color + PATH=(custom, no user) + XDG_RUNTIME_DIR=<set> + LANG=en_US.UTF-8 + SHELL=/bin/bash ProcFB: - + ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.15.0-112-generic root=UUID=3bff164a-d728-43bd-a926-ad3be6ff4715 ro consoleblank=0 systemd.show_status=true nomodeset consoleblank=0 RelatedPackageVersions: - linux-restricted-modules-5.15.0-112-generic N/A - linux-backports-modules-5.15.0-112-generic N/A - linux-firmware 20220329.git681281e4-0ubuntu3.31 + linux-restricted-modules-5.15.0-112-generic N/A + linux-backports-modules-5.15.0-112-generic N/A + linux-firmware 20220329.git681281e4-0ubuntu3.31 RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 07/09/2020 dmi.bios.release: 5.14 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: L1.02W dmi.board.name: B450D4U-V1L dmi.board.vendor: ASRockRack dmi.chassis.asset.tag: To Be Filled By O.E.M. dmi.chassis.type: 3 dmi.chassis.vendor: To Be Filled By O.E.M. dmi.chassis.version: To Be Filled By O.E.M. dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrL1.02W:bd07/09/2020:br5.14:svnHetzner:pn:pvr1.0:rvnASRockRack:rnB450D4U-V1L:rvr:cvnToBeFilledByO.E.M.:ct3:cvrToBeFilledByO.E.M.:sku: dmi.product.version: 1.0 dmi.sys.vendor: Hetzner
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2070261 Title: Slow disk IO after upgrade to kernel 5.15.0-112-generic To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2070261/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
