[Bug 1854225] Re: Kernel oops and system lock up when invoking wg-quick up
Note that this bug is quite old now, and I upgraded this machine to 20.04 last night. I have no idea if the information collected by apport has any relevance any more. ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1854225 Title: Kernel oops and system lock up when invoking wg-quick up To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1854225/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1854225] Re: Kernel oops and system lock up when invoking wg-quick up
apport information ** Tags added: apport-collected focal ** Description changed: On 2 occasions over the past week I have had full system crashes after running "wg-quick up wg0". On the terminal, the command does not complete (i.e. it does not return to the prompt), the fans on my laptop start whirring and the system gradually becomes unresponsive before my desktop crashes and the system becomes completely unresponsive. On both occasions I opened another window to run "top" to see what process was consuming resources but "top" never actually runs. On the second occasion I managed to run "dmesg" before the system crashed completely and saw multiple lines of text about a kernel oops and red-highlighted text about a null-pointer dereference. I could reboot with "Alt-PrtScr_REISUB". On reboot I was confronted with the "system problem detected" dialog, but selecting the "report" option didn't seem to do anything. I have 2 reports in /var/crash from the last oops which I will attach to this report. I cannot reproduce this on demand. Most of the time, wg-quick performs normally. On both occasions the laptop had recently woken from suspend, but invoking "wg-quick" after waking from suspend doesn't trigger it on demand. On the first occasion I was running with stock boot options. On the second, I was running with "mitigations=off" as an experiment. $ lsb_release -rd Description: Ubuntu 19.10 Release: 19.10 $ apt policy wireguard wireguard: Installed: 0.0.20190913-1ubuntu1 Candidate: 0.0.20190913-1ubuntu1 Version table: *** 0.0.20190913-1ubuntu1 500 500 http://gb.archive.ubuntu.com/ubuntu eoan/universe amd64 Packages 500 http://gb.archive.ubuntu.com/ubuntu eoan/universe i386 Packages 100 /var/lib/dpkg/status $ apt policy wireguard-tools wireguard-tools: Installed: 0.0.20190913-1ubuntu1 Candidate: 0.0.20190913-1ubuntu1 Version table: *** 0.0.20190913-1ubuntu1 500 500 http://gb.archive.ubuntu.com/ubuntu eoan/universe amd64 Packages 100 /var/lib/dpkg/status $ uname -a Linux padbeast 5.3.0-23-generic #25-Ubuntu SMP Tue Nov 12 09:22:33 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux # cat /etc/wireguard/wg0.conf [Interface] PrivateKey = MyPrivateKey= Address = 10.66.66.5/24,fd42:42:42::5/64 DNS = 8.8.8.8,1.1.1.1 [Peer] PublicKey = MyPublicKey= Endpoint = my.domain.com:1195 AllowedIPs = 0.0.0.0/0,::/0 I'm reporting this as a security bug due to the "Null pointer dereference" in the kernel, but don't know if that is relevant. I don't know how to access or send the old dmesg information, so please let me know how to access this or how to collect it if the crash recurs. ProblemType: Bug DistroRelease: Ubuntu 19.10 Package: wireguard 0.0.20190913-1ubuntu1 ProcVersionSignature: Ubuntu 5.3.0-23.25-generic 5.3.7 Uname: Linux 5.3.0-23-generic x86_64 ApportVersion: 2.20.11-0ubuntu8.2 Architecture: amd64 CurrentDesktop: MATE Date: Wed Nov 27 20:44:24 2019 InstallationDate: Installed on 2019-10-11 (47 days ago) InstallationMedia: Ubuntu-MATE 19.10 "Eoan Ermine" - Beta amd64 (20190926.2) PackageArchitecture: all SourcePackage: wireguard UpgradeStatus: No upgrade log present (probably fresh install) + --- + ProblemType: Bug + ApportVersion: 2.20.11-0ubuntu16 + Architecture: amd64 + AudioDevicesInUse: + USERPID ACCESS COMMAND + /dev/snd/controlC0: neil 3007 F pulseaudio + /dev/snd/pcmC0D0p: neil 3007 F...m pulseaudio + CurrentDesktop: MATE + DistroRelease: Ubuntu 20.04 + InstallationDate: Installed on 2019-10-11 (118 days ago) + InstallationMedia: Ubuntu-MATE 19.10 "Eoan Ermine" - Beta amd64 (20190926.2) + MachineType: LENOVO 2325A39 + NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair + Package: linux (not installed) + ProcFB: 0 i915drmfb + ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-12-generic root=/dev/mapper/vgubuntu--mate-root ro quiet splash mitigations=off vt.handoff=7 + ProcVersionSignature: Ubuntu 5.4.0-12.15-generic 5.4.8 + RelatedPackageVersions: + linux-restricted-modules-5.4.0-12-generic N/A + linux-backports-modules-5.4.0-12-generic N/A + linux-firmware1.186 + RfKill: + 0: phy0: Wireless LAN + Soft blocked: no + Hard blocked: no + Tags: focal + Uname: Linux 5.4.0-12-generic x86_64 + UpgradeStatus: Upgraded to focal on 2020-02-07 (0 days ago) + UserGroups: adm audio cdrom dip lpadmin lxd plugdev sambashare sudo + _MarkForUpload: True + dmi.bios.date: 06/19/2018 + dmi.bios.vendor: LENOVO + dmi.bios.version: G2ETB3WW (2.73 ) + dmi.board.asset.tag: Not Available + dmi.board.name: 2325A39 + dmi.board.vendor: LENOVO + dmi.board.version: Not Defined + dmi.chassis.asset.tag: No Asset Information + dmi.chassis.type: 10 + dmi.chassis.vendor: LENOVO + dmi.chassis.version: Not Available + dmi.modalias:
[Bug 1854225] Re: Kernel oops and system lock up when invoking wg-quick up
Doesn't look like a WireGuard bug. ** Package changed: wireguard (Ubuntu) => linux (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1854225 Title: Kernel oops and system lock up when invoking wg-quick up To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1854225/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1854225] Re: Kernel oops and system lock up when invoking wg-quick up
Thanks for the bug report. That kern.log is useful. The relevant part is reproduced below in this comment. Looks like wg-quick(8) invokes sysctl(8), which then uses /proc/sys/, and somehow invokes a null pointer dereference while holding a spinlock, leading to that lock being hit by other cores, eventually locking up your system. Nov 26 23:20:01 padbeast kernel: [16283.030060] BUG: kernel NULL pointer dereference, address: 0011 Nov 26 23:20:01 padbeast kernel: [16283.030064] #PF: supervisor read access in kernel mode Nov 26 23:20:01 padbeast kernel: [16283.030065] #PF: error_code(0x) - not-present page Nov 26 23:20:01 padbeast kernel: [16283.030067] PGD 0 P4D 0 Nov 26 23:20:01 padbeast kernel: [16283.030070] Oops: [#1] SMP NOPTI Nov 26 23:20:01 padbeast kernel: [16283.030073] CPU: 1 PID: 6983 Comm: sysctl Tainted: G OE 5.3.0-23-generic #25-Ubuntu Nov 26 23:20:01 padbeast kernel: [16283.030074] Hardware name: LENOVO 2325A39/2325A39, BIOS G2ETB3WW (2.73 ) 06/19/2018 Nov 26 23:20:01 padbeast kernel: [16283.030080] RIP: 0010:rb_first+0xb/0x20 Nov 26 23:20:01 padbeast kernel: [16283.030082] Code: fe ff ff 4c 89 e9 4c 89 f2 4d 89 ee 49 89 c5 e9 81 fe ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 48 8b 07 48 85 c0 74 10 49 89 c0 <48> 8b 40 10 48 85 c0 75 f4 4c 89 c0 c3 45 31 c0 eb f7 0f 1f 00 48 Nov 26 23:20:01 padbeast kernel: [16283.030083] RSP: 0018:b662c21efe18 EFLAGS: 00010202 Nov 26 23:20:01 padbeast kernel: [16283.030085] RAX: 0001 RBX: b662c21efec0 RCX: Nov 26 23:20:01 padbeast kernel: [16283.030087] RDX: 0001 RSI: b71e1b73 RDI: 9e25445eea50 Nov 26 23:20:01 padbeast kernel: [16283.030088] RBP: b662c21efe70 R08: 0001 R09: 0004 Nov 26 23:20:01 padbeast kernel: [16283.030090] R10: b71e1b71 R11: R12: 9e24f782ead8 Nov 26 23:20:01 padbeast kernel: [16283.030091] R13: 9e24f782ea80 R14: 9e24f75cb400 R15: b60e2ba0 Nov 26 23:20:01 padbeast kernel: [16283.030093] FS: 7f669f9d6580() GS:9e255604() knlGS: Nov 26 23:20:01 padbeast kernel: [16283.030095] CS: 0010 DS: ES: CR0: 80050033 Nov 26 23:20:01 padbeast kernel: [16283.030096] CR2: 0011 CR3: 000147bb8006 CR4: 001606e0 Nov 26 23:20:01 padbeast kernel: [16283.030098] Call Trace: Nov 26 23:20:01 padbeast kernel: [16283.030104] ? proc_sys_readdir+0x11a/0x2c0 Nov 26 23:20:01 padbeast kernel: [16283.030109] iterate_dir+0x9a/0x1b0 Nov 26 23:20:01 padbeast kernel: [16283.030112] ksys_getdents64+0x9c/0x130 Nov 26 23:20:01 padbeast kernel: [16283.030114] ? iterate_dir+0x1b0/0x1b0 Nov 26 23:20:01 padbeast kernel: [16283.030117] __x64_sys_getdents64+0x1a/0x20 Nov 26 23:20:01 padbeast kernel: [16283.030120] do_syscall_64+0x5a/0x130 Nov 26 23:20:01 padbeast kernel: [16283.030124] entry_SYSCALL_64_after_hwframe+0x44/0xa9 Nov 26 23:20:01 padbeast kernel: [16283.030126] RIP: 0033:0x7f669f8c507b Nov 26 23:20:01 padbeast kernel: [16283.030129] Code: 0f 1e fa 48 8b 47 20 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 81 fa ff ff ff 7f b8 ff ff ff 7f 48 0f 47 d0 b8 d9 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 05 c3 0f 1f 40 00 48 8b 15 e1 8d 10 00 f7 d8 Nov 26 23:20:01 padbeast kernel: [16283.030130] RSP: 002b:7ffc1e7e4ef8 EFLAGS: 0293 ORIG_RAX: 00d9 Nov 26 23:20:01 padbeast kernel: [16283.030132] RAX: ffda RBX: 562cc04d9ce0 RCX: 7f669f8c507b Nov 26 23:20:01 padbeast kernel: [16283.030134] RDX: 8000 RSI: 562cc04d9ce0 RDI: 0007 Nov 26 23:20:01 padbeast kernel: [16283.030135] RBP: ff80 R08: 0030 R09: 007c Nov 26 23:20:01 padbeast kernel: [16283.030137] R10: R11: 0293 R12: 562cc04d9cb4 Nov 26 23:20:01 padbeast kernel: [16283.030138] R13: R14: 562cc04d9cb0 R15: 562cc04c1bc0 Nov 26 23:20:01 padbeast kernel: [16283.030140] Modules linked in: binfmt_misc wireguard(OE) ip6_udp_tunnel udp_tunnel acpi_call(OE) msr ccm uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc nls_iso8859_1 mei_hdcp intel_rapl_msr snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec iwldvm snd_hda_core snd_hwdep mac80211 joydev snd_pcm input_leds libarc4 wmi_bmof snd_seq_midi snd_seq_midi_event iwlwifi snd_rawmidi thinkpad_acpi cfg80211 nvram ledtrig_audio snd_seq snd_seq_device snd_timer intel_rapl_common x86_pkg_temp_thermal snd intel_powerclamp coretemp kvm_intel kvm mei_me soundcore mei mac_hid irqbypass intel_cstate serio_raw intel_rapl_perf sch_fq_codel parport_pc ppdev lp parport ip_tables x_tables autofs4 dm_crypt crct10dif_pclmul crc32_pclmul ghash_clmulni_intel i915 aesni_intel i2c_algo_bit aes_x86_64 crypto_simd drm_kms_helper sdhci_pci cryptd glue_helper psmouse cqhci syscopyarea ahci i2c_i801 libahci
[Bug 1854225] Re: Kernel oops and system lock up when invoking wg-quick up
Hi Neil - I think that's a good idea since we haven't seen any progress on this private bug report. I'm not sure of the cause here but I think that we would have received a lot more reports if this was a widespread issue when using wg-quick (as we have in the past). ** Information type changed from Private Security to Public -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1854225 Title: Kernel oops and system lock up when invoking wg-quick up To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/wireguard/+bug/1854225/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
