This bug is awaiting verification that the linux-
gcp-5.4/5.4.0-1140.149~18.04.1 kernel in -proposed solves the problem.
Please test the kernel and update this bug with the results. If the
problem is solved, change the tag 'verification-needed-bionic-linux-
gcp-5.4' to 'verification-done-bionic-linux-gcp-5.4'. If the problem
still exists, change the tag 'verification-needed-bionic-linux-gcp-5.4'
to 'verification-failed-bionic-linux-gcp-5.4'.


If verification is not done by 5 working days from today, this fix will
be dropped from the source code, and this bug will be closed.


See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how
to enable and use -proposed. Thank you!


** Tags added: kernel-spammed-bionic-linux-gcp-5.4-v2 
verification-needed-bionic-linux-gcp-5.4

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2081085

Title:
  wbt:wbt_* trace event NULL pointer dereference with GENHD_FL_HIDDEN
  disks

Status in linux package in Ubuntu:
  Invalid
Status in linux source package in Focal:
  Fix Released

Bug description:
  [Impact]
  Systems with storage devices that utilize the GENHD_FL_HIDDEN flag, such as 
NVMe disks declaring support for multiple controllers (aka native 
multipathing), will have a request queue with backing_dev_info->dev set to 
NULL. When tracing is enabled with any of the wbt:wbt_* events enabled, a NULL 
pointer dereference will occur in the corresponding trace function called from 
wb_timer_fn. This occurs when the trace function attempts to access the 
device's name with dev_name.

  On a DGXA100 system, this can be reproduced by running the following, where 
/dev/nvme0n1 is one of the 4 NVMe disks in the system that support native 
multipathing:
  $ echo 1 | sudo tee /sys/kernel/tracing/events/wbt/enable
  $ echo 1 | sudo tee /sys/kernel/tracing/tracing_on
  $ sudo dd if=/dev/zero of=/dev/nvme0n1

  A NULL pointer dereference will occur and the system will become
  unresponsive.

  [Fix]
  The upstream commit d51cfc53ade318 ("bdi: use bdi_dev_name() to get device 
name") resolves this by changing the wbt:wbt_* trace functions to use the 
bdi_dev_name function instead of dev_name. The bdi_dev_name function safely 
handles the case where the supplied device is NULL.

  [Test Case]
  Verified that the commit d51cfc53ade318 ("bdi: use bdi_dev_name() to get 
device name") resolves the issue on DGXA100 when applied to the 
"Ubuntu-5.4.0-196.216" tag. The reproducer no longer causes a NULL pointer 
dereference or otherwise crash the system.

  [Regression Potential]
  There is a low risk of a regression:
  * In the focal K5.4 kernel, the bdi_dev_name function is used in other trace 
event functions for the same purpose of catching the case where bdi->dev is 
NULL.
  * This change is already present in kernel versions 5.7 and newer.

  [Other]
  The patch d51cfc53ade318 ("bdi: use bdi_dev_name() to get device name") is 
already present in Jammy K5.15 and newer.

  -----
  Originally found via kernel regression testing and reported here by 
@cypressyew: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/2072972.

  [ 1073.837085] BUG: kernel NULL pointer dereference, address: 0000000000000050
  [ 1073.844858] #PF: supervisor read access in kernel mode
  [ 1073.850589] #PF: error_code(0x0000) - not-present page
  [ 1073.856318] PGD 0 P4D 0
  [ 1073.859141] Oops: 0000 [#1] SMP NOPTI
  [ 1073.863226] CPU: 9 PID: 0 Comm: swapper/9 Tainted: P           OE     
5.4.0-196-generic #216-Ubuntu
  [ 1073.873319] Hardware name: NVIDIA DGXA100 920-23687-2530-000/DGXA100, BIOS 
1.25 08/31/2023
  [ 1073.882547] RIP: 0010:trace_event_raw_event_wbt_timer+0x6f/0x100
  [ 1073.889248] Code: 59 80 e5 02 0f 85 8f 00 00 00 4c 89 e6 ba 34 00 00 00 48 
8d 7d a0 e8 00 aa c9 ff 49 89 c4 48 85 c0 74 37 49 8b 87 b8 03 00 00 <48> 8b 70 
50 48 85 f6 74 45 49 8d 7c 24 08 ba 20 00 00 00 e8 c9 18
  [ 1073.910200] RSP: 0018:ffffaf08598a8da0 EFLAGS: 00010282
  [ 1073.916029] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 
0000000080000100
  [ 1073.923988] RDX: ffff970f0ba9501c RSI: 0000000000000100 RDI: 
ffff970f0ba95018
  [ 1073.931947] RBP: ffffaf08598a8e08 R08: ffff970f0ba95010 R09: 
0000000000000100
  [ 1073.939906] R10: ffffcf083fdc9a58 R11: 0000000000000386 R12: 
ffff970f0ba9501c
  [ 1073.947864] R13: 0000000000000000 R14: 0000000000000001 R15: 
ffff976efdfba000
  [ 1073.955824] FS:  0000000000000000(0000) GS:ffff970f0f640000(0000) 
knlGS:0000000000000000
  [ 1073.964850] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  [ 1073.971259] CR2: 0000000000000050 CR3: 0000001f7de80000 CR4: 
0000000000340ee0
  [ 1073.979217] Call Trace:
  [ 1073.981942]  <IRQ>
  [ 1073.984188]  ? show_regs.cold+0x1a/0x1f
  [ 1073.988456]  ? __die+0x90/0xd9
  [ 1073.991862]  ? no_context.isra.0+0x12c/0x320
  [ 1073.996626]  ? update_group_capacity+0x2c/0x1d0
  [ 1074.001679]  ? __bad_area_nosemaphore+0x45/0x1a0
  [ 1074.006829]  ? bad_area_nosemaphore+0x16/0x20
  [ 1074.011687]  ? do_user_addr_fault+0x267/0x440
  [ 1074.016547]  ? __enqueue_entity+0x96/0xa0
  [ 1074.021018]  ? enqueue_entity+0x139/0x670
  [ 1074.025490]  ? __do_page_fault+0x58/0x90
  [ 1074.029861]  ? do_page_fault+0x2c/0xe0
  [ 1074.034044]  ? page_fault+0x34/0x40
  [ 1074.037933]  ? trace_event_raw_event_wbt_timer+0x6f/0x100
  [ 1074.043954]  ? enqueue_entity+0x139/0x670
  [ 1074.048426]  wb_timer_fn+0x1d6/0x3c0
  [ 1074.052413]  ? blk_mq_tag_update_depth+0x100/0x100
  [ 1074.057755]  blk_stat_timer_fn+0x13a/0x140
  [ 1074.062326]  call_timer_fn+0x32/0x130
  [ 1074.066409]  __run_timers.part.0+0x180/0x280
  [ 1074.071174]  ? tick_sched_handle+0x33/0x60
  [ 1074.075740]  ? tick_sched_timer+0x3d/0x80
  [ 1074.080211]  ? recalibrate_cpu_khz+0x10/0x10
  [ 1074.084971]  ? ktime_get+0x3e/0xa0
  [ 1074.088765]  ? native_apic_msr_write+0x2b/0x30
  [ 1074.093719]  run_timer_softirq+0x2a/0x50
  [ 1074.098091]  __do_softirq+0xd1/0x2c1
  [ 1074.102078]  irq_exit+0xae/0xb0
  [ 1074.105580]  smp_apic_timer_interrupt+0x7b/0x140
  [ 1074.110728]  apic_timer_interrupt+0xf/0x20
  [ 1074.115295]  </IRQ>
  [ 1074.117634] RIP: 0010:native_safe_halt+0xe/0x10
  [ 1074.122685] Code: 7b ff ff ff eb bd 90 90 90 90 90 90 e9 07 00 00 00 0f 00 
2d e6 0c 50 00 f4 c3 66 90 e9 07 00 00 00 0f 00 2d d6 0c 50 00 fb f4 <c3> 90 0f 
1f 44 00 00 55 48 89 e5 41 55 41 54 53 e8 dd 3e 61 ff 65
  [ 1074.143638] RSP: 0018:ffffaf0840387e70 EFLAGS: 00000246 ORIG_RAX: 
ffffffffffffff13
  [ 1074.152083] RAX: ffffffff90d0ab70 RBX: 0000000000000009 RCX: 
0000000000000001
  [ 1074.160042] RDX: 0000000000065c72 RSI: 0000000000000083 RDI: 
0000000000000009
  [ 1074.168002] RBP: ffffaf0840387e90 R08: 0000000000000001 R09: 
0000000000022f80
  [ 1074.175962] R10: ffff980f3fed5328 R11: 0000000000000000 R12: 
0000000000000009
  [ 1074.183920] R13: ffff970f0bff5e00 R14: 0000000000000000 R15: 
0000000000000000
  [ 1074.191882]  ? __cpuidle_text_start+0x8/0x8
  [ 1074.196546]  ? default_idle+0x20/0x140
  [ 1074.200726]  arch_cpu_idle+0x15/0x20
  [ 1074.204711]  default_idle_call+0x23/0x30
  [ 1074.209085]  do_idle+0x1fb/0x270
  [ 1074.212683]  ? complete+0x49/0x50
  [ 1074.216376]  cpu_startup_entry+0x20/0x30
  [ 1074.220750]  start_secondary+0x178/0x1d0
  [ 1074.225126]  secondary_startup_64+0xa4/0xb0
  [ 1074.229789] Modules linked in: nls_iso8859_1 dm_multipath scsi_dh_rdac 
scsi_dh_emc scsi_dh_alua amd64_edac_mod edac_mce_amd kvm_amd kvm nvidia_uvm(O) 
nvidia_drm(PO) nvidia_modeset(PO) ipmi_ssif nvidia(PO) input_leds mlx5_ib(OE) 
ib_uverbs(OE) binfmt_misc ib_core(OE) ccp k10temp ipmi_si ipmi_devintf 
ipmi_msghandler mac_hid sch_fq_codel msr ramoops reed_solomon efi_pstore 
ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov 
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 
multipath linear ses enclosure crct10dif_pclmul crc32_pclmul 
ghash_clmulni_intel aesni_intel mlx5_core(OE) ast crypto_simd drm_vram_helper 
cryptd glue_helper ttm drm_kms_helper pci_hyperv_intf syscopyarea mlxdevm(OE) 
auxiliary(OE) sysfillrect mpt3sas hid_generic igb uas tls sysimgblt usbhid 
raid_class mlxfw(OE) psample scsi_transport_sas usb_storage dca hid fb_sys_fops 
i2c_algo_bit nvme mlx_compat(OE) drm nvme_core i2c_piix4
  [ 1074.323033] CR2: 0000000000000050
  [ 1074.326858] ---[ end trace f3c3dea0291e7e7a ]---
  [ 1074.472012] RIP: 0010:trace_event_raw_event_wbt_timer+0x6f/0x100
  [ 1074.478714] Code: 59 80 e5 02 0f 85 8f 00 00 00 4c 89 e6 ba 34 00 00 00 48 
8d 7d a0 e8 00 aa c9 ff 49 89 c4 48 85 c0 74 37 49 8b 87 b8 03 00 00 <48> 8b 70 
50 48 85 f6 74 45 49 8d 7c 24 08 ba 20 00 00 00 e8 c9 18
  [ 1074.499668] RSP: 0018:ffffaf08598a8da0 EFLAGS: 00010282
  [ 1074.505497] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 
0000000080000100
  [ 1074.513459] RDX: ffff970f0ba9501c RSI: 0000000000000100 RDI: 
ffff970f0ba95018
  [ 1074.521418] RBP: ffffaf08598a8e08 R08: ffff970f0ba95010 R09: 
0000000000000100
  [ 1074.529378] R10: ffffcf083fdc9a58 R11: 0000000000000386 R12: 
ffff970f0ba9501c
  [ 1074.537336] R13: 0000000000000000 R14: 0000000000000001 R15: 
ffff976efdfba000
  [ 1074.545297] FS:  0000000000000000(0000) GS:ffff970f0f640000(0000) 
knlGS:0000000000000000
  [ 1074.554325] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  [ 1074.560733] CR2: 0000000000000050 CR3: 0000001f7de80000 CR4: 
0000000000340ee0
  [ 1074.568695] Kernel panic - not syncing: Fatal exception in interrupt
  [ 1074.578317] Kernel Offset: 0xf200000 from 0xffffffff81000000 (relocation 
range: 0xffffffff80000000-0xffffffffbfffffff)
  [ 1075.126325] ---[ end Kernel panic - not syncing: Fatal exception in 
interrupt ]---

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2081085/+subscriptions


-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to