[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
** Tags added: cscc -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
This bug was erroneously marked for verification in bionic; verification is not required and verification-needed-bionic is being removed. ** Tags added: verification-done-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
** Tags removed: verification-needed-bionic ** Tags added: kernel-fixup-verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed- bionic'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
This bug was fixed in the package linux - 4.13.0-39.44 --- linux (4.13.0-39.44) artful; urgency=medium * linux: 4.13.0-39.44 -proposed tracker (LP: #1761456) * intel-microcode 3.20180312.0 causes lockup at login screen(w/ linux- image-4.13.0-37-generic) (LP: #1759920) // CVE-2017-5715 (Spectre v2 Intel) // CVE-2017-5754 - x86/mm: Reinitialize TLB state on hotplug and resume * intel-microcode 3.20180312.0 causes lockup at login screen(w/ linux- image-4.13.0-37-generic) (LP: #1759920) // CVE-2017-5715 (Spectre v2 Intel) - Revert "x86/mm: Only set IBPB when the new thread cannot ptrace current thread" - x86/speculation: Use Indirect Branch Prediction Barrier in context switch * DKMS driver builds fail with: Cannot use CONFIG_STACK_VALIDATION=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel (LP: #1760876) - [Packaging] include the retpoline extractor in the headers * retpoline hints: primary infrastructure and initial hints (LP: #1758856) - [Packaging] retpoline-extract: flag *0xNNN(%reg) branches - x86/speculation, objtool: Annotate indirect calls/jumps for objtool - x86/speculation, objtool: Annotate indirect calls/jumps for objtool on 32bit - x86/paravirt, objtool: Annotate indirect calls - [Packaging] retpoline -- add safe usage hint support - [Packaging] retpoline-check -- only report additions - [Packaging] retpoline -- widen indirect call/jmp detection - [Packaging] retpoline -- elide %rip relative indirections - [Packaging] retpoline -- clear hint information from packages - KVM: x86: Make indirect calls in emulator speculation safe - KVM: VMX: Make indirect call speculation safe - x86/boot, objtool: Annotate indirect jump in secondary_startup_64() - SAUCE: early/late -- annotate indirect calls in early/late initialisation code - SAUCE: vga_set_mode -- avoid jump tables - [Config] retpoline -- switch to new format - [Packaging] retpoline hints -- handle missing files when RETPOLINE not enabled - [Packaging] final-checks -- remove check for empty retpoline files * retpoline: ignore %cs:0xNNN constant indirections (LP: #1752655) - [Packaging] retpoline -- elide %cs:0x constants on i386 * zfs system process hung on container stop/delete (LP: #1754584) - SAUCE: Fix non-prefaulted page deadlock (LP: #1754584) * zfs-linux 0.6.5.11-1ubuntu5 ADT test failure with linux 4.15.0-1.2 (LP: #1737761) - SAUCE: (noup) Update zfs to 0.6.5.11-1ubuntu3.2 * AT_BASE_PLATFORM in AUXV is absent on kernels available on Ubuntu 17.10 (LP: #1759312) - powerpc/64s: Fix NULL AT_BASE_PLATFORM when using DT CPU features * btrfs and tar sparse truncate archives (LP: #1757565) - Btrfs: move definition of the function btrfs_find_new_delalloc_bytes - Btrfs: fix reported number of inode blocks after buffered append writes * efifb broken on ThunderX-based Gigabyte nodes (LP: #1758375) - drivers/fbdev/efifb: Allow BAR to be moved instead of claiming it * Intel i40e PF reset due to incorrect MDD detection (continues...) (LP: #1723127) - i40e/i40evf: Account for frags split over multiple descriptors in check linearize * Fix an issue that when system in S3, USB keyboard can't wake up the system. (LP: #1759511) - ACPI / PM: Allow deeper wakeup power states with no _SxD nor _SxW * [8086:3e92] display becomes blank after S3 (LP: #1759188) - drm/i915: Apply Display WA #1183 on skl, kbl, and cfl * add audio kernel patches for Raven (LP: #1758364) - ALSA: hda: Add Raven PCI ID - ALSA: hda/realtek - Fix ALC700 family no sound issue * Cpu utilization showing system time for kvm guests (performance) (sysstat) (LP: #1755979) - KVM: PPC: Book3S HV: Fix guest time accounting with VIRT_CPU_ACCOUNTING_GEN * Kernel panic on a nfsroot system (LP: #1734327) - Revert "UBUNTU: SAUCE: LSM stacking: add stacking support to apparmor network hooks" - Revert "UBUNTU: SAUCE: LSM stacking: LSM: Infrastructure management of the remaining blobs" * can't record sound via front headset port on the Dell Precision 3630 (LP: #1759088) - ALSA: hda/realtek - Fix Dell headset Mic can't record * speaker can't output sound anymore after system resumes from S3 on a lenovo machine with alc257 (LP: #1758829) - ALSA: hda/realtek - Fix speaker no sound after system resume * hda driver initialization takes too much time on the machine with coffeelake audio controller [8086:a348] (LP: #1758800) - ALSA: hda - Force polling mode on CFL for fixing codec communication * Let headset-mode initialization be called on Dell Precision 3930 (LP: #1757584) - ALSA: hda/realtek - Add headset mode support for Dell laptop * ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 (LP: #1755073) - SAUCE: crypto: thunderx_zip: Fix
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
-- before proposed -- Ubuntu 17.10 (GNU/Linux 4.13.0-38-generic aarch64) ubuntu@anuchin:~$ sudo bash root@anuchin:~# modprobe zram root@anuchin:~# echo 1 > /sys/block/zram0/reset root@anuchin:~# echo deflate > /sys/block/zram0/comp_algorithm root@anuchin:~# echo 128M > /sys/block/zram0/disksize root@anuchin:~# mkfs.ext4 -F /dev/zram0 mke2fs 1.43.5 (04-Aug-2017) Discarding device blocks: done Creating filesystem with 32768 4k blocks and 32768 inodes Allocating group tables: done Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: manjo@lazy:~$ ipmitool -H 10.229.48.14 -I lanplus -U admin -P admin sol activate [SOL Session operational. Use ~? for help] [ 1420.012055] watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [mkfs.ext4:3121] [ 1424.023938] watchdog: BUG: soft lockup - CPU#40 stuck for 23s! [kworker/u96:2:406] [ 1454.099071] INFO: rcu_sched self-detected stall on CPU [ 1454.104260] 14-...: (14998 ticks this GP) idle=796/141/0 softirq=4896/4896 fqs=7500 [ 1454.107070] INFO: rcu_sched detected stalls on CPUs/tasks: [ 1454.107076] 14-...: (14998 ticks this GP) idle=796/141/0 softirq=4896/4896 fqs=7500 [ 1454.107077] (detected by 40, t=15002 jiffies, g=3929, c=3928, q=4597) [ 1454.134272] (t=15008 jiffies g=3929 c=3928 q=4597) -- after proposed -- ubuntu@anuchin:~$ uname -a Linux anuchin 4.13.0-39-generic #44-Ubuntu SMP Thu Apr 5 14:20:13 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux ubuntu@anuchin:~$ sudo bash root@anuchin:~# modprobe zram root@anuchin:~# echo 1 > /sys/block/zram0/reset root@anuchin:~# echo deflate > /sys/block/zram0/comp_algorithm root@anuchin:~# echo 128M > /sys/block/zram0/disksize root@anuchin:~# mkfs.ext4 -F /dev/zram0 mke2fs 1.43.5 (04-Aug-2017) Discarding device blocks: done Creating filesystem with 32768 4k blocks and 32768 inodes Allocating group tables: done Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done root@anuchin:~# ** Tags removed: verification-needed-artful ** Tags added: verification-done-artful -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
This bug was fixed in the package linux - 4.15.0-15.16 --- linux (4.15.0-15.16) bionic; urgency=medium * linux: 4.15.0-15.16 -proposed tracker (LP: #1761177) * FFe: Enable configuring resume offset via sysfs (LP: #1760106) - PM / hibernate: Make passing hibernate offsets more friendly * /dev/bcache/by-uuid links not created after reboot (LP: #1729145) - SAUCE: (no-up) bcache: decouple emitting a cached_dev CHANGE uevent * Ubuntu18.04:POWER9:DD2.2 - Unable to start a KVM guest with default machine type(pseries-bionic) complaining "KVM implementation does not support Transactional Memory, try cap-htm=off" (kvm) (LP: #1752026) - powerpc: Use feature bit for RTC presence rather than timebase presence - powerpc: Book E: Remove unused CPU_FTR_L2CSR bit - powerpc: Free up CPU feature bits on 64-bit machines - powerpc: Add CPU feature bits for TM bug workarounds on POWER9 v2.2 - powerpc/powernv: Provide a way to force a core into SMT4 mode - KVM: PPC: Book3S HV: Work around transactional memory bugs in POWER9 - KVM: PPC: Book3S HV: Work around XER[SO] bug in fake suspend mode - KVM: PPC: Book3S HV: Work around TEXASR bug in fake suspend state * Important Kernel fixes to be backported for Power9 (kvm) (LP: #1758910) - powerpc/mm: Fixup tlbie vs store ordering issue on POWER9 * Ubuntu 18.04 - IO Hang on some namespaces when running HTX with 16 namespaces (Bolt / NVMe) (LP: #1757497) - powerpc/64s: Fix lost pending interrupt due to race causing lost update to irq_happened * fwts-efi-runtime-dkms 18.03.00-0ubuntu1: fwts-efi-runtime-dkms kernel module failed to build (LP: #1760876) - [Packaging] include the retpoline extractor in the headers linux (4.15.0-14.15) bionic; urgency=medium * linux: 4.15.0-14.15 -proposed tracker (LP: #1760678) * [Bionic] mlx4 ETH - mlnx_qos failed when set some TC to vendor (LP: #1758662) - net/mlx4_en: Change default QoS settings * AT_BASE_PLATFORM in AUXV is absent on kernels available on Ubuntu 17.10 (LP: #1759312) - powerpc/64s: Fix NULL AT_BASE_PLATFORM when using DT CPU features * Bionic update to 4.15.15 stable release (LP: #1760585) - net: dsa: Fix dsa_is_user_port() test inversion - openvswitch: meter: fix the incorrect calculation of max delta_t - qed: Fix MPA unalign flow in case header is split across two packets. - tcp: purge write queue upon aborting the connection - qed: Fix non TCP packets should be dropped on iWARP ll2 connection - sysfs: symlink: export sysfs_create_link_nowarn() - net: phy: relax error checking when creating sysfs link netdev->phydev - devlink: Remove redundant free on error path - macvlan: filter out unsupported feature flags - net: ipv6: keep sk status consistent after datagram connect failure - ipv6: old_dport should be a __be16 in __ip6_datagram_connect() - ipv6: sr: fix NULL pointer dereference when setting encap source address - ipv6: sr: fix scheduling in RCU when creating seg6 lwtunnel state - mlxsw: spectrum_buffers: Set a minimum quota for CPU port traffic - net: phy: Tell caller result of phy_change() - ipv6: Reflect MTU changes on PMTU of exceptions for MTU-less routes - net sched actions: return explicit error when tunnel_key mode is not specified - ppp: avoid loop in xmit recursion detection code - rhashtable: Fix rhlist duplicates insertion - test_rhashtable: add test case for rhltable with duplicate objects - kcm: lock lower socket in kcm_attach - sch_netem: fix skb leak in netem_enqueue() - ieee802154: 6lowpan: fix possible NULL deref in lowpan_device_event() - net: use skb_to_full_sk() in skb_update_prio() - net: Fix hlist corruptions in inet_evict_bucket() - s390/qeth: free netdevice when removing a card - s390/qeth: when thread completes, wake up all waiters - s390/qeth: lock read device while queueing next buffer - s390/qeth: on channel error, reject further cmd requests - soc/fsl/qbman: fix issue in qman_delete_cgr_safe() - dpaa_eth: fix error in dpaa_remove() - dpaa_eth: remove duplicate initialization - dpaa_eth: increment the RX dropped counter when needed - dpaa_eth: remove duplicate increment of the tx_errors counter - dccp: check sk for closed state in dccp_sendmsg() - ipv6: fix access to non-linear packet in ndisc_fill_redirect_hdr_option() - l2tp: do not accept arbitrary sockets - net: ethernet: arc: Fix a potential memory leak if an optional regulator is deferred - net: ethernet: ti: cpsw: add check for in-band mode setting with RGMII PHY interface - net: fec: Fix unbalanced PM runtime calls - net/iucv: Free memory obtained by kzalloc - netlink: avoid a double skb free in genlmsg_mcast() - net: Only honor ifindex in IP_PKTINFO if non-0 - net: systemport: Rewrite __bcm_sysport_tx_reclaim() -
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- artful' to 'verification-done-artful'. If the problem still exists, change the tag 'verification-needed-artful' to 'verification-failed- artful'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-artful -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
** Changed in: linux (Ubuntu Artful) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
** Changed in: linux (Ubuntu Artful) Status: New => In Progress ** Changed in: linux (Ubuntu Artful) Assignee: (unassigned) => Paolo Pisati (p-pisati) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
Patches posted: https://marc.info/?l=linux-kernel=152224242828223=2 https://marc.info/?l=linux-kernel=152224242228218=2 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
Can be reproduced with X-hwe on ThunderX as well. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
** Also affects: linux (Ubuntu Bionic) Importance: Undecided Assignee: Paolo Pisati (p-pisati) Status: Confirmed ** Also affects: linux (Ubuntu Artful) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Bionic) Status: Confirmed => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
** Description changed: + == SRU Justification == + + Enabling virtual mapped kernel stacks breaks the thunderx_zip driver. On + compression or decompression the executing CPU hangs in an endless loop. + The reason for this is the usage of __pa() by the driver that does not + work for an address that is not part of the 1:1 mapping. + + == Fix == + + The patch in comment #12 fixes it. + To reproduce the bug (and test the new kernel), see comment #9. + + == Regression Potential == + + The patch affects only the thunderx_zip driver (that is already in a + completely broken state), so the regression potential is low. + + --- + This is a POTENTIAL REGRESSION. This test has passed with 4.13.0-36.40, but not 4.13.0-37.42 The test will stuck with error message in dmesg: ubuntu@starmie-kernel:~$ dmesg [ 470.227210] zram: Added device: zram0 [ 472.262544] zram0: detected capacity change from 0 to 134217728 [ 472.396960] EXT4-fs (zram0): mounted filesystem with ordered data mode. Opts: (null) [ 475.761947] zram0: detected capacity change from 134217728 to 0 [ 476.796641] zram0: detected capacity change from 0 to 134217728 [ 476.909118] EXT4-fs (zram0): mounted filesystem with ordered data mode. Opts: (null) [ 480.233817] zram0: detected capacity change from 134217728 to 0 [ 481.239001] zram0: detected capacity change from 0 to 134217728 [ 508.079684] watchdog: BUG: soft lockup - CPU#8 stuck for 23s! [mkfs.ext4:2253] [ 508.086994] Modules linked in: lz4 lz4_compress zram nls_iso8859_1 thunderx_edac i2c_thunderx thunderx_zip i2c_smbus shpchp cavium_rng_vf cavium_rng gpio_keys ipmi_ssif uio_pdrv_genirq uio ipmi_devintf ipmi_msghandler ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear nicvf nicpf ast i2c_algo_bit ttm drm_kms_helper syscopyarea sysfillrect aes_ce_blk sysimgblt aes_ce_cipher fb_sys_fops crc32_ce crct10dif_ce drm ghash_ce sha2_ce sha1_ce ahci libahci thunder_bgx thunder_xcv mdio_thunder thunderx_mmc mdio_cavium aes_neon_bs aes_neon_blk crypto_simd cryptd [ 508.087138] CPU: 8 PID: 2253 Comm: mkfs.ext4 Not tainted 4.13.0-37-generic #42-Ubuntu [ 508.087141] Hardware name: Cavium ThunderX CRB/To be filled by O.E.M., BIOS 5.11 12/12/2012 [ 508.087144] task: 801f57a86900 task.stack: 23ec [ 508.087158] PC is at zip_deflate+0x184/0x2e8 [thunderx_zip] [ 508.087164] LR is at zip_deflate+0x17c/0x2e8 [thunderx_zip] [ 508.087167] pc : [] lr : [] pstate: 20400145 [ 508.087169] sp : 23ec3580 - [ 508.087171] x29: 23ec3580 x28: - [ 508.087177] x27: 801f57a86900 x26: 0001b104 - [ 508.087182] x25: 801f5618c000 x24: 801f6a15bc18 - [ 508.087188] x23: 1000 x22: 23ec3688 - [ 508.087193] x21: 801f5a2d4880 x20: 801f6a15bc18 - [ 508.087199] x19: 23ec3608 x18: - [ 508.087204] x17: 0001 x16: - [ 508.087209] x15: bd4048db x14: fa13 - [ 508.087215] x13: 1000 x12: fa12 - [ 508.087220] x11: x10: - [ 508.087225] x9 : x8 : - [ 508.087230] x7 : x6 : 8000101d6080 - [ 508.087236] x5 : 01a69058 x4 : 0008 - [ 508.087241] x3 : x2 : - [ 508.087246] x1 : 801f6a15be08 x0 : + [ 508.087171] x29: 23ec3580 x28: + [ 508.087177] x27: 801f57a86900 x26: 0001b104 + [ 508.087182] x25: 801f5618c000 x24: 801f6a15bc18 + [ 508.087188] x23: 1000 x22: 23ec3688 + [ 508.087193] x21: 801f5a2d4880 x20: 801f6a15bc18 + [ 508.087199] x19: 23ec3608 x18: + [ 508.087204] x17: 0001 x16: + [ 508.087209] x15: bd4048db x14: fa13 + [ 508.087215] x13: 1000 x12: fa12 + [ 508.087220] x11: x10: + [ 508.087225] x9 : x8 : + [ 508.087230] x7 : x6 : 8000101d6080 + [ 508.087236] x5 : 01a69058 x4 : 0008 + [ 508.087241] x3 : x2 : + [ 508.087246] x1 : 801f6a15be08 x0 : [ 508.087252] Call trace: [ 508.087256] Exception stack(0x23ec3440 to 0x23ec3580) [ 508.087260] 3440: 801f6a15be08 [ 508.087264] 3460: 0008 01a69058 8000101d6080 [ 508.087268] 3480: [ 508.087272] 34a0: fa12
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
This is a regression caused by CONFIG_VMAP_STACK. The driver uses __pa (phys_to_virt) on a stack address which does not work with virtual mapped stacks. To solve this upstream will require a bit more work, I'm attaching a minimal patch to work-around by using kmalloc for the problematic allocation. ** Patch added: "zip.patch" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+attachment/5090802/+files/zip.patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
I can reproduce this issue with 4.13.0-37-generic. The debugfs info for the zip driver shows that 2 requests are pending. Either the requests are not submitted properly (Decomp Req Submitted=0) or not completed (DOORBELL regs = 0). root@crb1s:/sys/kernel/debug/thunderx_zip# cat zip_stats ZIP Device 0 Stats --- Comp Req Submitted: 0 Comp Req Completed: 0 Compress In Bytes : 0 Compressed Out Bytes : 0 Average Chunk size: 0 Average Compression ratio : 0 Decomp Req Submitted : 0 Decomp Req Completed : 0 Decompress In Bytes : 0 Decompressed Out Bytes: 0 Decompress Bad requests : 0 Pending Req : 2 - root@crb1s:/sys/kernel/debug/thunderx_zip# cat zip_regs ZIP Device 0 Registers ZIP_CMD_CTL: 0x0002 ZIP_THROTTLE : 0x0010 ZIP_CONSTANTS : 0x02017c002006 ZIP_QUE0_MAP : 0x0003 ZIP_QUE1_MAP : 0x0003 ZIP_QUE_ENA: 0x0003 ZIP_QUE_PRI: 0x0003 ZIP_QUE0_DONE : 0x ZIP_QUE1_DONE : 0x ZIP_QUE0_DOORBELL : 0x ZIP_QUE1_DOORBELL : 0x ZIP_QUE0_SBUF_ADDR : 0xfa040100 ZIP_QUE1_SBUF_ADDR : 0xfa042000 ZIP_QUE0_SBUF_CTL : 0x03f1 ZIP_QUE1_SBUF_CTL : 0x03f1 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
Ok, i tested it on another board (seuss) with 4.14.27, 4.15.10 and 4.16-rc4: 4.14.0: affected 4.14.27: affected 4.15.10: affected 4.16-rc: doesn't boot at all, so i can't tell so we can rule out the bad hardware variable, and we can say for sure that is not related to kpti at all. Another thing: once building a kernel from upstream, you clearly need to enable ZSMALLOC and ZRAM too. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
Ok, here is what i found so far: the problem lies in the 'thunderx_zip' driver, that is the driver for the hw accelerated zip compressor / decompressor ip block - it kicks in once we select the deflate method for the zram device. How way to reproduce it: # modprobe zram # echo 1 > /sys/block/zram0/reset # echo deflate > /sys/block/zram0/comp_algorithm # echo 128M > /sys/block/zram0/disksize # mkfs.ext4 -F /dev/zram0 [stuck forever here] Two trivial workarounds: -blacklist the thunderx_zip kmod: # rmmod thunderx_zip # echo 'blacklist thunderx_zip' >> /etc/modprobe.d/blacklist.conf or -disable the CRYPTO_DEV_CAVIUM_ZIP kconfig (and void building thunderx_zip kmod) and recompile As to what is causing it, till yesterday, it appeared as the problem was connected to the arm64 kpti but now i'm sure it is not: the problem started to appear in 4.13.0-37-generic #42, while 4.13.0-36-generic #40 was immune and the only difference between those two kernels is the arm64 kpti patchset. You can easily reproduce it in the upstream stable/linux-4.14.y tree too (4.14.26 is affected for example). $ make defconfig $ echo "CONFIG_CRYPTO_DEV_CAVIUM_ZIP=m" >> .config $ make oldconfig build and install as usual, and then try the reproducer above. But what i found this morning, is that even the original 4.14.0 release is affected, but that release clearly doesn't contain the kpti patches. Now what i want to try is: 1) test it on different hardware (one thing that i noticed is that if the thunderx_zip kmod is loaded at boot or later in the board life cycle, that slightly changes the error and that smells a lot like memory corruption) 2) test it with 4.15x and 4.16 I'll write another update when i have more data. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1755073] Re: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64
Tested with a Moonshot system, this ubuntu_zram_smoke test can pass with Artful kernel. So this matches ppisati's finding in comment #5 ** Summary changed: - ubuntu_zram_smoke test will cause soft lockup on Artful ARM64 kernel + ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1755073 Title: ubuntu_zram_smoke test will cause soft lockup on Artful ThunderX ARM64 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1755073/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
