"K R" can reproduce a hang on a multiple sockets amd64 that has been
first reported in a different thread:
https://marc.info/?l=openbsd-tech&m=176631121132731&w=2
The reports seem to always contain a CPU spinning for `tlb_shoot_wait'
inside pmap_kremove().
In two these reports, including the one below, instead of a hang, a CPU
faulted inside Xipi_invlrange_pcid executing the following instruction:
00000000000006e0 <Xipi_invlrange_pcid>:
[...]
717: 66 0f 38 82 0c 24 invpcid (%rsp),%rcx
Any idea?
Thanks,
Martin
----- Forwarded message from K R <[email protected]> -----
From: K R <[email protected]>
Subject: Re: Debugging the (known) amd64 hang problem under 7.8-current
Date: Mon, 22 Dec 2025 06:54:02 -0300
Another kernel protection fault trap, this time during make build.
Please let me know if you need other commands at the ddb prompt.
Thanks,
--Kor
kernel: protection fault trap, code=0
Stopped at Xipi_invlrange_pcid+0x37:
ddb{0}> show reg
rdi 0
rsi 0xfffffd9c9b2c7880
rbp 0xffff800055df2910
rbx 0x1
rdx 0xffff800055ef6000
rcx 0
rax 0x800000000000
r8 0
r9 0
r10 0
r11 0x8a38de34ad98001f
r12 0
r13 0x3bc6000 __kernel_phys_end+0xfc6000
r14 0
r15 0xfffffd9c9b2c7880
rip 0xffffffff82578717 Xipi_invlrange_pcid+0x37
cs 0x8
rflags 0x10007 __ALIGN_SIZE+0xf007
rsp 0xffff800055df27f0
ss 0
Xipi_invlrange_pcid+0x37:
ddb{0}> tr
Xipi_invlrange_pcid() at Xipi_invlrange_pcid+0x37
uvm_fault_lower_lookup(ffff800055df2b28,ffff800055df2b60,ffff800055df2aa0)
at uvm_fault_lower_lookup+0x126
uvm_fault_lower(ffff800055df2b28,ffff800055df2b60,ffff800055df2aa0) at
uvm_fault_lower+0x5c
uvm_fault(fffffd9ca4fe0d00,3bc5000,0,4) at uvm_fault+0x1c5
upageflttrap(ffff800055df2ca0,3bc5140) at upageflttrap+0x6c
usertrap(ffff800055df2ca0) at usertrap+0x28b
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x787ea5370980, count: -7
ddb{0}> ps /o
TID PID UID PRFLAGS PFLAGS CPU COMMAND
420197 33827 21 0x3 0 11 c++
*496882 38495 21 0x3 0 0 cc
436657 12129 21 0x3 0 4 cc
286050 44426 21 0x3 0 14 cc
304357 94663 21 0x3 0 9 cc
361353 11993 21 0x3 0x4000000 2 ld
207747 11993 21 0x3 0x4000000 6 ld
186926 11993 21 0x3 0x4000000 15 ld
461300 11993 21 0x3 0x4000000 10 ld
85235 10748 21 0x3 0 5 cc
354453 14234 0 0x14000 0x200 3 reaper
ddb{0}> ps
PID TID PPID UID S FLAGS WAIT COMMAND
33827 420197 70854 21 7 0x3 c++
70854 248565 71435 21 3 0x10008b sigsusp sh
*38495 496882 71435 21 7 0x3 cc
12129 436657 71435 21 7 0x3 cc
44426 286050 71435 21 7 0x3 cc
94663 304357 71435 21 7 0x3 cc
27210 465629 71435 21 3 0x3 kmmaplk ld
62460 422721 71435 21 3 0x3 kmmaplk ld
70434 296875 71435 21 3 0x3 kmmaplk ld
61912 252230 71435 21 3 0x3 kmmaplk ld
35064 173786 90751 21 3 0x3 kmmaplk cc
91729 3267 71435 21 3 0x3 kmmaplk cc
90751 317227 71435 21 3 0x10008b sigsusp sh
13844 392836 71435 21 3 0x3 kmmaplk cc
50801 411917 71435 21 3 0x3 uobjlk ld
11993 321094 71435 21 3 0x83 fsleep ld
11993 361353 71435 21 7 0x4000003 ld
11993 207747 71435 21 7 0x4000003 ld
11993 186926 71435 21 7 0x4000003 ld
11993 461300 71435 21 7 0x4000003 ld
10748 85235 52149 21 7 0x3 cc
52149 486885 71435 21 3 0x10008b sigsusp sh
71435 374578 3380 21 3 0x10008b sigsusp make
3380 349362 91865 21 3 0x10008b sigsusp sh
91865 143287 14954 21 3 0x10008b sigsusp make
14954 159995 64832 0 3 0x10008b sigsusp sh
64832 5456 2261 0 3 0x10008b sigsusp make
2261 45292 87953 0 3 0x10008b sigsusp make
87953 463303 84630 0 3 0x100089 sigsusp sh
84630 50290 70674 0 3 0x10008b sigsusp sh
70674 124625 32449 0 3 0x10008b sigsusp make
27203 345957 1 0 3 0x100083 ttyin getty
84008 469274 1 0 3 0x100083 ttyin getty
37669 385804 1 0 3 0x100083 ttyin getty
98787 487823 1 0 3 0x100083 ttyin getty
71013 299934 1 0 3 0x100083 ttyin getty
32449 509352 1 0 3 0x10008b sigsusp ksh
93197 97452 1 0 3 0x100098 kqread cron
69215 74790 1 99 3 0x1100090 kqread sndiod
72396 332320 1 110 3 0x100090 kqread sndiod
63624 233057 38055 95 3 0x1100092 kqread smtpd
68680 52646 38055 103 3 0x1100092 kqread smtpd
20622 95955 38055 95 3 0x1100092 kqread smtpd
55449 111792 38055 95 3 0x100092 kqread smtpd
96276 19463 38055 95 3 0x1100092 kqread smtpd
39340 313589 38055 95 3 0x1100092 kqread smtpd
38055 505655 1 0 3 0x100080 kqread smtpd
64918 202970 1 0 3 0x88 kqread sshd
70976 58238 1 0 3 0x100080 kqread ntpd
859 200146 79811 83 3 0x100092 kqread ntpd
79811 28622 1 83 3 0x1100092 kqread ntpd
42732 195616 20140 74 3 0x1100092 bpf pflogd
20140 184927 1 0 3 0x80 sbwait pflogd
52831 16574 48644 73 3 0x1100090 kqread syslogd
48644 489345 1 0 3 0x100082 sbwait syslogd
17601 3887 1 0 3 0x100080 kqread resolvd
77919 178404 89382 77 3 0x100092 kqread dhcpleased
22973 57680 89382 77 3 0x100092 kqread dhcpleased
89382 494427 1 0 3 0x80 kqread dhcpleased
59447 122354 72236 115 3 0x100092 kqread slaacd
92332 79485 72236 115 3 0x100092 kqread slaacd
72236 479047 1 0 3 0x100080 kqread slaacd
80237 109218 0 0 3 0x14200 bored smr
78638 14804 0 0 3 0x14200 pgzero zerothread
72743 67933 0 0 3 0x14200 aiodoned aiodoned
97676 292567 0 0 3 0x14200 syncer update
65568 467570 0 0 3 0x14200 cleaner cleaner
14234 354453 0 0 7 0x14200 reaper
30359 380080 0 0 3 0x14200 pgdaemon pagedaemon
89012 14387 0 0 3 0x14200 bored wsdisplay0
48338 290288 0 0 3 0x14200 usbtsk usbtask
61041 401336 0 0 3 0x14200 usbatsk usbatsk
89256 247963 0 0 3 0x40014200 acpi0 acpi0
13413 391338 0 0 7 0x40014200 idle31
48495 319953 0 0 7 0x40014200 idle30
65593 264877 0 0 7 0x40014200 idle29
72409 69036 0 0 7 0x40014200 idle28
98496 243188 0 0 7 0x40014200 idle27
55275 521370 0 0 7 0x40014200 idle26
8472 300408 0 0 7 0x40014200 idle25
40905 514270 0 0 7 0x40014200 idle24
91593 338354 0 0 7 0x40014200 idle23
66291 89888 0 0 7 0x40014200 idle22
69132 31945 0 0 7 0x40014200 idle21
64551 197335 0 0 7 0x40014200 idle20
1825 103108 0 0 7 0x40014200 idle19
23981 461922 0 0 7 0x40014200 idle18
46634 138679 0 0 7 0x40014200 idle17
89219 260421 0 0 7 0x40014200 idle16
35440 362506 0 0 3 0x40014200 idle15
98701 220117 0 0 3 0x40014200 idle14
53442 313675 0 0 7 0x40014200 idle13
20028 371900 0 0 7 0x40014200 idle12
96342 458196 0 0 3 0x40014200 idle11
74233 128785 0 0 3 0x40014200 idle10
19430 150209 0 0 3 0x40014200 idle9
8368 274288 0 0 7 0x40014200 idle8
37543 202336 0 0 3 0x40014200 idle7
39201 505457 0 0 3 0x40014200 idle6
77513 14639 0 0 3 0x40014200 idle5
51698 298170 0 0 3 0x40014200 idle4
85808 103679 0 0 3 0x40014200 idle3
31028 37597 0 0 3 0x40014200 idle2
64516 65581 0 0 7 0x40014200 idle1
24199 109001 0 0 2 0x40014200 sensors
87535 511602 0 0 3 0x14200 bored softnet7
21108 487312 0 0 3 0x14200 bored softnet6
36929 125735 0 0 3 0x14200 bored softnet5
87838 128616 0 0 3 0x14200 bored softnet4
54497 421774 0 0 3 0x14200 bored softnet3
33544 387201 0 0 3 0x14200 bored softnet2
34957 240095 0 0 3 0x14200 bored softnet1
59373 260027 0 0 3 0x14200 bored softnet0
81264 198390 0 0 3 0x14200 bored systqmp
85508 88021 0 0 3 0x14200 bored systq
74026 254652 0 0 3 0x14200 tmoslp softclockmp
81844 238982 0 0 3 0x40014200 tmoslp softclock
31225 488152 0 0 3 0x40014200 idle0
1 267925 0 0 3 0x82 wait init
0 0 -1 0 3 0x10200 scheduler swapper
ddb{0}> mach ddb 0t11
Stopped at x86_ipi_db+0x16: leave
ddb{11}> tr
x86_ipi_db(ffff800055314ff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_enter(fffffd9c9b2c7958,2f3b4d000,1cb4c7e000,3,22) at pmap_enter+0x5e2
uvm_fault_upper(ffff800055f9d288,ffff800055f9d2c0,ffff800055f9d180) at
uvm_fault_upper+0x1e0
uvm_fault(fffffd9ca4fe0b90,2f3b4d000,0,2) at uvm_fault+0xce
upageflttrap(ffff800055f9d400,2f3b4d008) at upageflttrap+0x6c
usertrap(ffff800055f9d400) at usertrap+0x28b
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x7f2fcd372a00, count: -9
ddb{11}> mach ddb 0t4
Stopped at x86_ipi_db+0x16: leave
ddb{4}> tr
x86_ipi_db(ffff8000552d5ff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_write_protect(fffffda07ffdfd80,25d5a4000,25d5a5000,1) at
pmap_write_protect+0x272
uvm_map_protect(fffffd9ca4fe0180,25d5a4000,25d5a5000,3,0,0,f2cd35b186f9406d)
at uvm_map_protect+0x481
sys_mprotect(ffff800055d27a28,ffff800055e5de80,ffff800055e5de00) at
sys_mprotect+0x17c
syscall(ffff800055e5de80) at syscall+0x5f9
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x7abd95df9fc0, count: -8
ddb{4}> mach ddb 0t14
Stopped at x86_ipi_db+0x16: leave
ddb{14}> tr
x86_ipi_db(ffff80005532fff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_enter(fffffda07ffdf1b0,26e36c000,1cb3853000,3,22) at pmap_enter+0x5e2
uvm_fault_upper(ffff800055f41c28,ffff800055f41c60,ffff800055f41b20) at
uvm_fault_upper+0x1e0
uvm_fault(fffffd9ca4fe02f0,26e36c000,0,2) at uvm_fault+0xce
upageflttrap(ffff800055f41da0,26e36c008) at upageflttrap+0x6c
usertrap(ffff800055f41da0) at usertrap+0x28b
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x752ba9ba2210, count: -9
ddb{14}> mach ddb 0t9
Stopped at x86_ipi_db+0x16: leave
ddb{9}> tr
x86_ipi_db(ffff800055302ff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_write_protect(fffffd9c9b2c7e68,26d190000,26d191000,1) at
pmap_write_protect+0x272
uvm_map_protect(fffffd9ca4fe05d0,26d190000,26d191000,3,0,0,f2cd35b186f9406d)
at uvm_map_protect+0x481
sys_mprotect(ffff800055f90820,ffff800055f138d0,ffff800055f13850) at
sys_mprotect+0x17c
syscall(ffff800055f138d0) at syscall+0x5f9
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x7f22d44ebd60, count: -8
ddb{9}> mach ddb 0t2
Stopped at x86_ipi_db+0x16: leave
ddb{2}> tr
x86_ipi_db(ffff8000552c3ff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_enter(fffffd9b3c800438,445b000,1c9e579000,4,20) at pmap_enter+0x66a
uvm_fault_lower_lookup(ffff800055f851c8,ffff800055f85200,ffff800055f85140)
at uvm_fault_lower_lookup+0x126
uvm_fault_lower(ffff800055f851c8,ffff800055f85200,ffff800055f85140) at
uvm_fault_lower+0x5c
uvm_fault(fffffd9ccaf295d8,4457000,0,4) at uvm_fault+0x1c5
upageflttrap(ffff800055f85340,4457dc0) at upageflttrap+0x6c
usertrap(ffff800055f85340) at usertrap+0x28b
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x25e4253d0, count: -10
ddb{2}> mach ddb 0t6
Stopped at x86_ipi_db+0x16: leave
ddb{6}> tr
x86_ipi_db(ffff8000552e7ff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_tlb_shootpage(fffffd9b3c800438,445b000,1) at pmap_tlb_shootpage+0x202
pmap_enter(fffffd9b3c800438,445b000,1c9e579000,4,20) at pmap_enter+0x622
uvm_fault_lower_lookup(ffff800055faae28,ffff800055faae60,ffff800055faada0)
at uvm_fault_lower_lookup+0x126
uvm_fault_lower(ffff800055faae28,ffff800055faae60,ffff800055faada0) at
uvm_fault_lower+0x5c
uvm_fault(fffffd9ccaf295d8,4457000,0,4) at uvm_fault+0x1c5
upageflttrap(ffff800055faafa0,4457dc0) at upageflttrap+0x6c
usertrap(ffff800055faafa0) at usertrap+0x28b
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x262cba920, count: -11
ddb{6}> mach ddb 0t15
Stopped at x86_ipi_db+0x16: leave
ddb{15}> tr
x86_ipi_db(ffff800055338ff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_enter(fffffd9b3c800438,4457000,1b4e7f2000,4,24) at pmap_enter+0x662
uvm_fault_lower(ffff800055fb1528,ffff800055fb1560,ffff800055fb14a0) at
uvm_fault_lower+0x255
uvm_fault(fffffd9ccaf295d8,4457000,0,4) at uvm_fault+0x1c5
upageflttrap(ffff800055fb16a0,4457dc0) at upageflttrap+0x6c
usertrap(ffff800055fb16a0) at usertrap+0x28b
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x293ddbb00, count: -9
ddb{15}> mach ddb 0t10
Stopped at x86_ipi_db+0x16: leave
ddb{10}> tr
x86_ipi_db(ffff80005530bff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_enter(fffffd9b3c800438,4458000,1b73bae000,4,20) at pmap_enter+0x662
uvm_fault_lower_lookup(ffff800055fb7788,ffff800055fb77c0,ffff800055fb7700)
at uvm_fault_lower_lookup+0x126
uvm_fault_lower(ffff800055fb7788,ffff800055fb77c0,ffff800055fb7700) at
uvm_fault_lower+0x5c
uvm_fault(fffffd9ccaf295d8,4457000,0,4) at uvm_fault+0x1c5
upageflttrap(ffff800055fb7900,4457dc0) at upageflttrap+0x6c
usertrap(ffff800055fb7900) at usertrap+0x28b
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x24cf701e0, count: -10
ddb{10}> mach ddb 0t5
Stopped at x86_ipi_db+0x16: leave
ddb{5}> tr
x86_ipi_db(ffff8000552deff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_enter(fffffd9b3c800948,6a97000,1c75294000,4,20) at pmap_enter+0x662
uvm_fault_lower_lookup(ffff800055eefc28,ffff800055eefc60,ffff800055eefba0)
at uvm_fault_lower_lookup+0x126
uvm_fault_lower(ffff800055eefc28,ffff800055eefc60,ffff800055eefba0) at
uvm_fault_lower+0x5c
uvm_fault(fffffd9ccaf29748,6a95000,0,4) at uvm_fault+0x1c5
upageflttrap(ffff800055eefda0,6a952e0) at upageflttrap+0x6c
usertrap(ffff800055eefda0) at usertrap+0x28b
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x77fe8e033e10, count: -10
ddb{5}> mach ddb 0t3
Stopped at x86_ipi_db+0x16: leave
ddb{3}> tr
x86_ipi_db(ffff8000552ccff0) at x86_ipi_db+0x16
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
pmap_kremove(ffff800055f7a000,6000) at pmap_kremove+0xa2
km_free(ffff800055f7a000,6000,ffffffff8276bbf0,ffffffff828fd2c0) at
km_free+0x1c6
uvm_uarea_free(ffff8000ffff4d28) at uvm_uarea_free+0x41
reaper(ffff800055ce22b8) at reaper+0x141
end trace frame: 0x0, count: -7
ddb{3}>
[...]
----- End forwarded message -----
