Testing now with my third power-supply and usb-c cable. This time
I caught a new lock order reversal in uvm and a kernel panic
uvm_fault failed.
witness: lock order reversal:
1st 0xffffff8001246998 /sys/dev/rnd.c:321 (/sys/dev/rnd.c:321)
2nd 0xffffff80012081d0 /sys/kern/kern_timeout.c:57
(/sys/kern/kern_timeout.c:57)
lock order data w2 -> w1 missing
lock order "/sys/dev/rnd.c:321"(mutex) -> "/sys/kern/kern_timeout.c:57"(mutex)
first seen at:
#0 mtx_enter+0x48
#1 timeout_del+0x30
#2 dequeue_randomness+0x3c
#3 extract_entropy+0x94
#4 _rs_stir+0x2c
#5 arc4random_buf+0x108
#6 setregs+0x74
#7 sys_execve+0xcbc
#8 svc_handler+0x470
#9 do_el0_sync+0xcc
#10 handle_el0_sync+0x78
witness: lock order reversal:
1st 0xffffff802b928198 uobjlk (&uobj->vmobjlock)
2nd 0xffffff80012edfc0 kmmaplk (&map->lock)
lock order "&map->lock"(rwlock) -> "&uobj->vmobjlock"(rwlock) first seen at:
#0 rw_enter+0x84
#1 uvm_fault_lower_lookup+0x50
#2 uvm_fault_lower+0x58
#3 uvm_fault+0x178
#4 uvm_fault_wire+0x5c
#5 exec_timekeep_map+0xcc
#6 sys_execve+0xc94
#7 start_init+0x22c
#8 proc_trampoline+0x10
lock order "&uobj->vmobjlock"(rwlock) -> "&map->lock"(rwlock) first seen at:
#0 rw_enter_read+0x4c
#1 uvmfault_lookup+0xdc
#2 uvm_fault_check+0x44
#3 uvm_fault+0xd8
#4 kdata_abort+0xd8
#5 handle_el1h_sync+0x70
#6 pmap_copy_page+0xa0
#7 pmap_copy_page+0xa0
#8 uvm_fault_lower+0x180
#9 uvm_fault+0x178
#10 udata_abort+0x13c
#11 do_el0_sync+0x134
#12 handle_el0_sync+0x78
panic: uvm_fault failed: ffffff80009fd020 esr 96000047 far ffffff800264ab13
Stopped at panic+0x14c: cmp w21, #0x0
TID PID UID PRFLAGS PFLAGS CPU COMMAND
*440668 84673 55 0x9000002 0 0 sh
48307 89294 55 0x9100002 0 2 cksum
199941 94945 55 0x9100002 0 3 tar
271254 9549 73 0x19100010 0 1 syslogd
db_enter() at panic+0x148
panic() at kdata_abort+0x170
kdata_abort() at handle_el1h_sync+0x6c
handle_el1h_sync() at pmap_copy_page+0x9c
pmap_copy_page() at pmap_copy_page+0x9c
pmap_copy_page() at uvm_fault_lower+0x17c
uvm_fault_lower() at uvm_fault+0x174
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports. Insufficient info makes it difficult to find and fix bugs.
ddb{0}> show regs
No such command
ddb{0}> show reg
x0 0x4c
x1 0x14
x2 0x14
x3 0xd
x4 0xffffff808d78e8f0
x5 0
x6 0x2
x7 0xffffff808d78ebe0
x8 0x1
x9 0xf1985d06ea64aa62
x10 0xffffff80012e6000 pf_osfp_entry_pl+0x20
x11 0xffffff80012e6000 pf_osfp_entry_pl+0x20
x12 0x300
x13 0x10
x14 0x7c5d0f85
x15 0x7665eac013faa2f2
x16 0xffffff8000259460 agintc_splx
x17 0
x18 0xffffff808d78e920
x19 0xffffff8000e5fbdb $d.3+0xfb
x20 0xffffff8001328c70 cpu_info_primary+0x6b0
x21 0x1
x22 0xffffff808c3d3270
x23 0x2
x24 0xffffff800264a000 _end+0x1320638
x25 0xffffff80012edec8 kernel_map_store
x26 0xffffff808d78a000
x27 0xffffff8001213000 pagetable_end
x28 0
x29 0xffffff808d78e920
x30 0
sp 0xffffff808d78e920
spsr 0x60400305
elr 0xffffff800022bb88 db_enter+0x18
lr 0xffffff80007efa04 panic+0x14c
panic+0x14c: cmp w21, #0x0
ddb{0}> show uvm
Current UVM status:
pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
3956800 VM pages: 13958 active, 118883 inactive, 1 wired, 3374930 free
(421889 zero)
min 10% (25) anon, 10% (25) vnode, 5% (12) vtext
freemin=131893, free-target=175857, inactive-target=175858, wired-max=1318933
faults=1480084, traps=0, intrs=0, ctxswitch=1400722 fpuswitch=0
softint=34105, syscalls=1749851, kmapent=15
fault counts:
noram=0, noanon=0, noamap=0, pgwait=0, pgrele=0
ok relocks(total)=145976(157567), anget(retries)=417537(0), amapcopy=217604
neighbor anon/obj pg=287927/964005, gets(lock/unlock)=460335/157570
cases: anon=349161, anoncow=68376, obj=394019, prcopy=54722, przero=613851
daemon and swap counts:
woke=45, revs=26, scans=0, obscans=0, anscans=0
busy=0, freed=0, reactivate=0, deactivate=112016
pageouts=0, pending=0, nswget=0
nswapdev=1
swpages=4191853, swpginuse=0, swpgonly=0 paging=0
kernel pointers:
objs(kern)=0xffffff8001306e68
ddb{0}> show bcstats
Current Buffer Cache status:
numbufs 71020 busymapped 4, delwri 3548
kvaslots 12159 avail kva slots 12155
bufpages 407054, dmapages 194492, dirtypages 28368
pendingreads 3, pendingwrites 2
highflips 38445, highflops 0, dmaflips 79
ddb{0}> show uvmexp
Current UVM status:
pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
3956800 VM pages: 13958 active, 118883 inactive, 1 wired, 3374930 free
(421889 zero)
min 10% (25) anon, 10% (25) vnode, 5% (12) vtext
freemin=131893, free-target=175857, inactive-target=175858, wired-max=1318933
faults=1480084, traps=0, intrs=0, ctxswitch=1400722 fpuswitch=0
softint=34105, syscalls=1749851, kmapent=15
fault counts:
noram=0, noanon=0, noamap=0, pgwait=0, pgrele=0
ok relocks(total)=145976(157567), anget(retries)=417537(0), amapcopy=217604
neighbor anon/obj pg=287927/964005, gets(lock/unlock)=460335/157570
cases: anon=349161, anoncow=68376, obj=394019, prcopy=54722, przero=613851
daemon and swap counts:
woke=45, revs=26, scans=0, obscans=0, anscans=0
busy=0, freed=0, reactivate=0, deactivate=112016
pageouts=0, pending=0, nswget=0
nswapdev=1
swpages=4191853, swpginuse=0, swpgonly=0 paging=0
kernel pointers:
objs(kern)=0xffffff8001306e68
ddb{0}> show all locks
CPU 3:
exclusive mutex namei r = 0 (0xffffff80012b5de0)
Process 84673 (sh) thread 0xffffff808c3d3270 (440668)
exclusive rwlock uobjlk r = 0 (0xffffff802b928198)
exclusive rwlock amaplk r = 0 (0xffffff808d77f0f8)
shared rwlock vmmaplk r = 0 (0xffffff808c165d68)
Process 89294 (cksum) thread 0xffffff808c3d2550 (48307)
exclusive rrwlock inode r = 0 (0xffffff802ad73f88)
Process 94945 (tar) thread 0xffffff808c3d2a90 (199941)
exclusive kernel_lock &kernel_lock r = 0 (0xffffff80013268b8)
exclusive rrwlock inode r = 0 (0xffffff808d7974e0)
exclusive rrwlock inode r = 0 (0xffffff808d781b40)
ddb{0}> show malloc
Type InUse MemUse HighUse Limit Requests Type Lim
devbuf 3450 3163K 3164K 302191K 4763 0
pcb 15 10K 10K 302191K 15 0
rtable 60 1K 2K 302191K 169 0
pf 15 6K 10K 302191K 26 0
ifaddr 27 33K 33K 302191K 27 0
ifgroup 22 1K 1K 302191K 22 0
counters 182 137K 137K 302191K 182 0
ioctlops 0 0K 4K 302191K 1505 0
mount 11 11K 11K 302191K 11 0
log 0 0K 0K 302191K 5 0
vnodes 1087 68K 68K 302191K 1103 0
UFS quota 1 32K 32K 302191K 1 0
UFS mount 45 94K 94K 302191K 45 0
shm 2 1K 1K 302191K 2 0
VM map 3 1K 1K 302191K 3 0
sem 2 0K 0K 302191K 2 0
dirhash 189 36K 36K 302191K 1137 0
file desc 5 2K 2K 302191K 10 0
proc 114 129K 154K 302191K 527 0
NFS srvsock 1 0K 0K 302191K 1 0
NFS daemon 1 16K 16K 302191K 1 0
in_multi 11 0K 0K 302191K 11 0
ether_multi 1 0K 0K 302191K 1 0
ISOFS mount 1 32K 32K 302191K 1 0
MSDOSFS mount 1 16K 16K 302191K 1 0
ttys 25 122K 122K 302191K 25 0
exec 0 0K 1K 302191K 4964 0
tdb 3 0K 0K 302191K 3 0
VM swap 8 2214K 2214K 302191K 11 0
UVM amap 578 40K 50K 302191K 35033 0
UVM aobj 3 2K 2K 302191K 3 0
pinsyscall 103 206K 219K 302191K 19007 0
USB 100 37K 37K 302191K 116 0
USB device 30 1K 1K 302191K 30 0
USB HC 5 19K 19K 302191K 5 0
crypto data 1 1K 1K 302191K 1 0
NDP 4 0K 0K 302191K 4 0
temp 4 3840K 3968K 302191K 126807 0
kqueue 46 78K 80K 302191K 53 0
SYN cache 2 16K 16K 302191K 2 0
ddb{0}> show all pools
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
plcache 128 104 0 0 4 0 4 4 0 8 0
rtpcb 120 34 0 29 1 0 1 1 0 8 0
rtentry 112 23 0 0 1 0 1 1 0 8 0
unpcb 144 102 0 30 3 0 3 3 0 8 0
syncache 336 3 0 3 2 2 0 1 0 8 0
tcpqe 32 2 0 2 1 1 0 1 0 8 0
tcpcb 808 13 0 6 1 0 1 1 0 8 0
arp 120 3 0 0 1 0 1 1 0 8 0
inpcb 392 111 0 94 2 0 2 2 0 8 0
pfosfp 40 1428 0 1005 5 0 5 5 0 8 0
pfosfpen 112 1428 0 714 21 0 21 21 0 8 0
pfstitem 24 55 0 42 1 0 1 1 0 8 0
pfstkey 128 55 0 42 1 0 1 1 0 8 0
pfstate 376 55 0 42 2 0 2 2 0 8 0
pfrule 1344 21 0 16 2 1 1 2 0 8 0
art_heap8 4096 1 0 0 1 0 1 1 0 8 0
art_heap4 256 98 0 0 7 0 7 7 0 8 0
art_table 32 99 0 0 1 0 1 1 0 8 0
art_node 16 23 0 0 1 0 1 1 0 8 0
dirhash 1024 1164 0 905 41 8 33 33 0 8 0
dino2pl 256 202512 0 184895 1180 78 1102 1102 0 8 0
ffsino 272 202512 0 184895 1259 84 1175 1175 0 8 0
nchpl 144 99366 0 94201 219 27 192 192 0 8 0
uvmvnodes 80 54832 0 0 1120 0 1120 1120 0 8 0
vnodes 216 54832 0 0 3047 0 3047 3047 0 8 0
namei 1024 385598 0 385597 5 4 1 1 0 8 0
percpumem 64 37 0 0 1 0 1 1 0 8 0
scxspl 216 495968 0 495968 33 29 4 4 1 8 4
extentpl 40 21 0 4 1 0 1 1 0 8 0
ohcixfer 256 32 0 30 1 0 1 1 0 8 0
xhcixfer 280 54 0 51 1 0 1 1 0 8 0
ehcixfer 296 58 0 55 1 0 1 1 0 8 0
kstatmem 264 17 0 0 2 0 2 2 0 8 0
plimitpl 152 60 0 34 2 0 2 2 0 8 0
sigapl 424 4920 0 4833 11 1 10 10 0 8 0
futexpl 64 19254 0 19254 1 1 0 1 0 8 0
knotepl 120 132 0 0 4 0 4 4 0 8 0
kqueuepl 216 44 0 7 3 0 3 3 0 8 0
pipepl 320 391 0 388 3 2 1 1 0 8 0
fdescpl 496 4892 0 4833 8 0 8 8 0 8 0
filepl 152 100470 0 100278 8 0 8 8 0 8 0
lockfpl 104 67 0 64 1 0 1 1 0 8 0
lockfspl 48 35 0 32 1 0 1 1 0 8 0
sessionpl 144 40 0 16 1 0 1 1 0 8 0
pgrppl 48 50 0 24 1 0 1 1 0 8 0
ucredpl 104 25491 0 25434 2 0 2 2 0 8 0
zombiepl 144 4833 0 4833 6 5 1 1 0 8 1
processpl 1136 4920 0 4833 7 0 7 7 0 8 0
procpl 672 4969 0 4882 9 1 8 8 0 8 0
sockpl 584 247 0 153 7 0 7 7 0 8 0
mcl9k 9216 174 0 0 10 0 10 10 0 8 0
mcl8k 8192 3 0 0 1 0 1 1 0 8 0
mcl4k 4096 3 0 0 1 0 1 1 0 8 0
mcl2k 2048 31 0 0 4 0 4 4 0 8 0
mtagpl 96 5 0 0 1 0 1 1 0 8 0
mbufpl 256 271 0 0 11 0 11 11 0 8 0
bufpl 280 114100 0 43080 5073 0 5073 5073 0 8 0
anonpl 24 736949 0 720376 322 192 130 226 0 1490 0
amapchunkpl 152 96851 0 95813 151 107 44 101 0 158 0
amappl16 200 1437 0 1418 33 31 2 21 0 8 0
amappl15 192 115 0 115 6 5 1 1 0 8 1
amappl14 184 1179 0 1132 4 1 3 3 0 8 0
amappl13 176 220 0 218 4 3 1 1 0 8 0
amappl12 168 10659 0 10591 7 3 4 5 0 8 0
amappl11 160 374 0 328 5 3 2 3 0 8 0
amappl10 152 180 0 176 3 2 1 1 0 8 0
amappl9 144 122 0 119 2 1 1 1 0 8 0
amappl8 136 2078 0 2033 20 18 2 19 0 8 0
amappl7 128 273 0 261 2 1 1 2 0 8 0
amappl6 120 4768 0 4713 3 0 3 3 0 8 0
amappl5 112 1392 0 1267 7 2 5 7 0 8 0
amappl4 104 8206 0 8037 32 27 5 31 0 8 0
amappl3 96 17356 0 17021 22 12 10 13 0 8 0
amappl2 88 14522 0 13593 112 88 24 72 0 8 0
amappl1 80 88813 0 87108 37 0 37 37 0 8 1
amappl 88 31581 0 31232 23 13 10 20 0 92 0
dma4096 4096 1 0 1 1 1 0 1 0 8 0
dma256 256 12 0 12 1 1 0 1 0 8 0
dma128 128 2 0 2 1 1 0 1 0 8 0
dma64 64 2 0 2 1 1 0 1 0 8 0
dma32 32 4 0 4 1 1 0 1 0 8 0
dma16 16 10 0 10 1 1 0 1 0 8 0
vp 8192 6311 56 0 6250 155 6095 6095 20 8 8
pted 40 140584 0 0 1420 0 1420 1420 1 8 1
pmap 192 4892 0 4833 4 0 4 4 1 8 0
aobjpl 72 2 0 0 1 0 1 1 0 8 0
uaddrrnd 24 4892 0 4833 1 0 1 1 0 8 0
uaddrbest 32 2 0 0 1 0 1 1 0 8 0
uaddr 24 4892 0 4833 1 0 1 1 0 8 0
vmmpekpl 168 81045 0 80935 8 0 8 8 0 8 0
vmmpepl 168 615097 0 609099 597 258 339 485 0 357 74
vmsppl 448 4891 0 4833 7 0 7 7 0 8 0
rwobjpl 56 170483 0 111940 904 79 825 825 0 8 0
phpool 112 8780 0 333 242 0 242 242 0 8 0
ddb{0}> show all procs
PID TID PPID UID S FLAGS WAIT COMMAND
*84673 440668 29431 55 7 0x9000002 sh
29431 125452 38445 55 3 0x1910008a sigsusp make
89294 48307 67430 55 7 0x9100002 cksum
67430 249928 63054 55 3 0x910008a sigsusp sh
63054 401051 90819 55 3 0x1910008a sigsusp make
90819 188166 47272 55 3 0x1910008a sigsusp make
47272 29437 65603 55 3 0x910008a sigsusp sh
65603 511616 61852 55 3 0x1910008a sigsusp make
61852 282969 16503 55 3 0x910008a sigsusp sh
16503 101108 33823 55 3 0x1910008a sigsusp make
38445 446963 75627 55 3 0x910008a sigsusp sh
75627 289784 33823 55 3 0x1910008a sigsusp make
94945 199941 93135 55 7 0x9100002 tar
56903 423197 93135 55 3 0x19100082 pipewr gzip
93135 477110 54673 55 3 0x910008a sigsusp sh
54673 352878 96396 55 3 0x1910008a sigsusp make
96396 487991 93609 55 3 0x1910008a sigsusp make
93609 166528 27673 55 3 0x910008a sigsusp sh
27673 155795 98976 55 3 0x1910008a sigsusp make
98976 509358 32379 55 3 0x910008a sigsusp sh
32379 82471 33823 55 3 0x1910008a sigsusp make
33823 498000 77208 0 3 0x18000093 piperd perl
26572 306428 62705 1000 3 0x18100083 kqread top
20671 235597 1 1000 3 0x181000b0 kqread ssh-agent
62705 231951 40941 1000 3 0x810008b sigsusp ksh
40941 509948 30413 1000 3 0x18000098 kqread sshd
30413 151545 67846 0 3 0x18000082 kqread sshd
25265 140143 1 1000 3 0x181000b0 kqread ssh-agent
77208 217920 37811 1000 3 0x810008b sigsusp ksh
37811 393894 32245 1000 3 0x18000098 kqread sshd
32245 301999 67846 0 3 0x18000082 kqread sshd
75580 241081 1 0 3 0x18100083 ttyin getty
52175 308586 1 0 3 0x18100098 kqread cron
45541 23407 1 99 3 0x19100090 kqread sndiod
3040 6822 1 110 3 0x18100090 kqread sndiod
95659 353512 58929 95 3 0x19100092 kqread smtpd
96056 152383 58929 103 3 0x19100092 kqread smtpd
42609 438184 58929 95 3 0x19100092 kqread smtpd
15604 11990 58929 95 3 0x18100092 kqread smtpd
14074 443775 58929 95 3 0x19100092 kqread smtpd
87043 136782 58929 95 3 0x19100092 kqread smtpd
58929 258992 1 0 3 0x18100080 kqread smtpd
67846 521141 1 0 3 0x18000088 kqread sshd
10244 244213 1 0 3 0x18100080 kqread ntpd
31841 158802 70750 83 3 0x18100092 kqread ntpd
70750 333823 1 83 3 0x19100092 kqread ntpd
6604 107877 92010 74 3 0x19100092 bpf pflogd
92010 112681 1 0 3 0x18000080 netio pflogd
9549 271254 7118 73 7 0x19100010 syslogd
7118 205994 1 0 3 0x18100082 netio syslogd
81746 366695 1 0 3 0x18100080 kqread resolvd
15621 465939 90505 77 3 0x18100092 kqread dhcpleased
54592 79698 90505 77 3 0x18100092 kqread dhcpleased
90505 337821 1 0 3 0x18000080 kqread dhcpleased
76703 193164 83436 115 3 0x18100092 kqread slaacd
92709 405231 83436 115 3 0x18100092 kqread slaacd
83436 483664 1 0 3 0x18100080 kqread slaacd
78591 122808 0 0 3 0x14200 bored smr
57160 325927 0 0 3 0x14200 pgzero zerothread
10636 129688 0 0 3 0x14200 aiodoned aiodoned
2415 299333 0 0 3 0x14200 syncer update
34661 60256 0 0 3 0x14200 cleaner cleaner
40795 127684 0 0 3 0x14200 reaper reaper
86706 285162 0 0 3 0x14200 pgdaemon pagedaemon
24282 249781 0 0 3 0x14200 mmctsk sdmmc1
12104 40349 0 0 3 0x14200 mmctsk sdmmc0
70784 207962 0 0 3 0x14200 usbtsk usbtask
12687 431479 0 0 3 0x14200 usbatsk usbatsk
31299 396990 0 0 3 0x14200 bored sleep
90191 33494 0 0 7 0x40014200 idle7
302 4446 0 0 7 0x40014200 idle6
82162 71818 0 0 7 0x40014200 idle5
89206 295569 0 0 7 0x40014200 idle4
75393 262891 0 0 3 0x40014200 idle3
82242 167699 0 0 3 0x40014200 idle2
42267 222384 0 0 3 0x40014200 idle1
17769 65724 0 0 3 0x14200 bored softnet3
82526 493562 0 0 3 0x14200 bored softnet2
63417 478182 0 0 3 0x14200 bored softnet1
10022 71996 0 0 3 0x14200 bored softnet0
97332 237393 0 0 3 0x14200 bored systqmp
57081 509181 0 0 3 0x14200 bored systq
43768 394388 0 0 3 0x14200 tmoslp softclockmp
55825 476622 0 0 2 0x40014200 softclock
3270 194614 0 0 3 0x40014200 idle0
78280 42567 0 0 3 0x14200 kmalloc kmthread
1 109828 0 0 3 0x8000082 wait init
0 0 -1 0 3 0x10200 scheduler swapper
ddb{0}> bt
db_enter() at panic+0x148
panic() at kdata_abort+0x170
kdata_abort() at handle_el1h_sync+0x6c
handle_el1h_sync() at pmap_copy_page+0x9c
pmap_copy_page() at pmap_copy_page+0x9c
pmap_copy_page() at uvm_fault_lower+0x17c
uvm_fault_lower() at uvm_fault+0x174
uvm_fault() at udata_abort+0x138
udata_abort() at do_el0_sync+0x130
do_el0_sync() at handle_el0_sync+0x74
handle_el0_sync() at 0x13fbaef100
panic: attempt to access user address 0x6cc8b8f7f8 from EL1
Stopped at panic+0x14c: cmp w21, #0x0
db_enter() at panic+0x148
panic() at kdata_abort+0x180
do_el0_sync() at handle_el1h_sync+0x6c
handle_el1h_sync() at db_read_bytes+0x7c
db_read_bytes() at db_stack_trace_print+0x1b8
db_stack_trace_print() at db_command+0x29c
db_command() at db_command_loop+0xf4
ddb{0}> machine ddbcpu 1
Stopped at agintc_ipi_ddb+0x20: ldr x15, [sp,#16]
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at __mp_lock+0x60
__mp_lock() at scsi_iopool_run+0xf0
scsi_iopool_run() at scsi_xsh_runqueue+0x17c
ddb{1}> bt
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at __mp_lock+0x60
__mp_lock() at scsi_iopool_run+0xf0
scsi_iopool_run() at scsi_xsh_runqueue+0x17c
scsi_xsh_runqueue() at scsi_xsh_add+0x84
scsi_xsh_add() at sdstrategy+0x124
sdstrategy() at spec_strategy+0x38
spec_strategy() at VOP_STRATEGY+0x44
VOP_STRATEGY() at ufs_strategy+0xf0
ufs_strategy() at VOP_STRATEGY+0x44
VOP_STRATEGY() at bwrite+0x1f4
bwrite() at VOP_BWRITE+0x34
VOP_BWRITE() at ffs_fsync+0xb0
ffs_fsync() at VOP_FSYNC+0x54
VOP_FSYNC() at sys_fsync+0xf4
sys_fsync() at svc_handler+0x46c
svc_handler() at do_el0_sync+0xc8
do_el0_sync() at handle_el0_sync+0x74
handle_el0_sync() at 0x8d92248d0
panic: attempt to access user address 0x71ffdf00a8 from EL1
Stopped at panic+0x14c: cmp w21, #0x0
db_enter() at panic+0x148
panic() at kdata_abort+0x180
do_el0_sync() at handle_el1h_sync+0x6c
handle_el1h_sync() at db_read_bytes+0x7c
db_read_bytes() at db_stack_trace_print+0x1b8
db_stack_trace_print() at db_command+0x29c
db_command() at db_command_loop+0xf4
ddb{1}> machine ddbcpu 2
Stopped at agintc_ipi_ddb+0x20: ldr x15, [sp,#16]
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at __mp_lock+0x60
__mp_lock() at __mp_acquire_count+0x34
__mp_acquire_count() at tsleep+0x120
ddb{2}> bt
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at __mp_lock+0x60
__mp_lock() at __mp_acquire_count+0x34
__mp_acquire_count() at tsleep+0x120
tsleep() at bread_cluster+0x160
bread_cluster() at ffs_read+0x160
ffs_read() at VOP_READ+0x3c
VOP_READ() at vn_read+0xac
vn_read() at dofilereadv+0x118
dofilereadv() at sys_read+0x50
sys_read() at svc_handler+0x4a0
svc_handler() at do_el0_sync+0xc8
do_el0_sync() at handle_el0_sync+0x74
handle_el0_sync() at 0x1beb0789a4
panic: attempt to access user address 0x60526f0008 from EL1
Stopped at panic+0x14c: cmp w21, #0x0
db_enter() at panic+0x148
panic() at kdata_abort+0x180
do_el0_sync() at handle_el1h_sync+0x6c
handle_el1h_sync() at db_read_bytes+0x7c
db_read_bytes() at db_stack_trace_print+0x1b8
db_stack_trace_print() at db_command+0x29c
db_command() at db_command_loop+0xf4
ddb{2}> machine ddbcpu 3
Stopped at agintc_ipi_ddb+0x20: ldr x15, [sp,#16]
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at __mp_lock+0x60
__mp_lock() at __mp_acquire_count+0x34
__mp_acquire_count() at tsleep+0x120
ddb{3}> bt
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at __mp_lock+0x60
__mp_lock() at __mp_acquire_count+0x34
__mp_acquire_count() at tsleep+0x120
tsleep() at bwrite+0x240
bwrite() at ffs_update+0x19c
ffs_update() at ufs_makeinode+0x23c
ufs_makeinode() at ufs_create+0x48
ufs_create() at VOP_CREATE+0x3c
VOP_CREATE() at vn_open+0x138
vn_open() at doopenat+0x16c
doopenat() at svc_handler+0x46c
svc_handler() at do_el0_sync+0xc8
do_el0_sync() at handle_el0_sync+0x74
handle_el0_sync() at 0xd2972cae0
panic: attempt to access user address 0x772726d3f8 from EL1
Stopped at panic+0x14c: cmp w21, #0x0
db_enter() at panic+0x148
panic() at kdata_abort+0x180
do_el0_sync() at handle_el1h_sync+0x6c
handle_el1h_sync() at db_read_bytes+0x7c
db_read_bytes() at db_stack_trace_print+0x1b8
db_stack_trace_print() at db_command+0x29c
db_command() at db_command_loop+0xf4
ddb{3}> machine ddbcpu 4
Stopped at agintc_ipi_ddb+0x20: ldr x15, [sp,#16]
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at cpu_idle_cycle+0x24
cpu_idle_cycle() at sched_idle+0x294
sched_idle() at proc_trampoline+0xc
ddb{4}> bt
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at cpu_idle_cycle+0x24
cpu_idle_cycle() at sched_idle+0x294
sched_idle() at proc_trampoline+0xc
ddb{4}> machine ddbcpu 5
Stopped at agintc_ipi_ddb+0x20: ldr x15, [sp,#16]
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at cpu_idle_cycle+0x24
cpu_idle_cycle() at sched_idle+0x294
sched_idle() at proc_trampoline+0xc
ddb{5}> bt
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at cpu_idle_cycle+0x24
cpu_idle_cycle() at sched_idle+0x294
sched_idle() at proc_trampoline+0xc
ddb{5}> machine ddbcpu 6
Stopped at agintc_ipi_ddb+0x20: ldr x15, [sp,#16]
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at cpu_idle_cycle+0x24
cpu_idle_cycle() at sched_idle+0x294
sched_idle() at proc_trampoline+0xc
ddb{6}> bt
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at cpu_idle_cycle+0x24
cpu_idle_cycle() at sched_idle+0x294
sched_idle() at proc_trampoline+0xc
ddb{6}> machine ddbcpu 7
Stopped at agintc_ipi_ddb+0x20: ldr x15, [sp,#16]
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at cpu_idle_cycle+0x24
cpu_idle_cycle() at sched_idle+0x294
sched_idle() at proc_trampoline+0xc
ddb{7}> bt
db_enter() at agintc_ipi_ddb+0x1c
agintc_ipi_ddb() at agintc_irq_handler+0x2bc
agintc_irq_handler() at arm_cpu_irq+0x34
arm_cpu_irq() at handle_el1h_irq+0x6c
handle_el1h_irq() at cpu_idle_cycle+0x24
cpu_idle_cycle() at sched_idle+0x294
sched_idle() at proc_trampoline+0xc
ddb{7}>
