https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=290570
Bug ID: 290570
Summary: [zfs] panic: buffer modified while frozen!
Product: Base System
Version: 14.3-STABLE
Hardware: Any
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: [email protected]
Reporter: [email protected]
running stable/14-087fc6ae0a8c, amd64 with debug configuration (std.debug):
got panic on snapshot removal, while resilver was in progress, pool state was
online:
(kgdb) bt
#0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57
#1 doadump (textdump=textdump@entry=1) at
/usr/src/sys/kern/kern_shutdown.c:405
#2 0xffffffff80b0d420 in kern_reboot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:523
#3 0xffffffff80b0d939 in vpanic (fmt=0xffffffff828755e0 "buffer modified while
frozen!", ap=ap@entry=0xfffffe01590f5ce0) at
/usr/src/sys/kern/kern_shutdown.c:967
#4 0xffffffff80b0d6c3 in panic (fmt=<unavailable>) at
/usr/src/sys/kern/kern_shutdown.c:891
#5 0xffffffff825e34a1 in arc_cksum_verify (buf=0xfffff80603cb39a0) at
/usr/src/sys/contrib/openzfs/module/zfs/arc.c:1446
#6 0xffffffff825f14d8 in arc_write_done (zio=0xfffffe0344bd9980,
zio@entry=<error reading variable: value is not available>) at
/usr/src/sys/contrib/openzfs/module/zfs/arc.c:6593
#7 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bd9980) at
/usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988
#8 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at
/usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#9 zio_execute (zio=<optimized out>) at
/usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#10 0xffffffff80b75392 in taskqueue_run_locked
(queue=queue@entry=0xfffff8000e2f2e00) at
/usr/src/sys/kern/subr_taskqueue.c:518
#11 0xffffffff80b76283 in taskqueue_thread_loop
(arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830
#12 0xffffffff80ac0022 in fork_exit (callout=0xffffffff80b761b0
<taskqueue_thread_loop>, arg=0xfffff80004bafef0, frame=0xfffffe01590f5f40) at
/usr/src/sys/kern/kern_fork.c:1153
#13 <signal handler called>
#14 0x6bec65f367ac65f7 in ?? ()
Backtrace stopped: Cannot access memory at address 0xccdbfebfc09bfebb
from core.txt:
__curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57
57 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct
pcpu,
(kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57
td = <optimized out>
#1 doadump (textdump=textdump@entry=1)
at /usr/src/sys/kern/kern_shutdown.c:405
error = 0
coredump = <optimized out>
#2 0xffffffff80b0d420 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:523
once = 0
__pc = 0x0
#3 0xffffffff80b0d939 in vpanic (
fmt=0xffffffff828755e0 "buffer modified while frozen!",
ap=ap@entry=0xfffffe01590f5ce0) at /usr/src/sys/kern/kern_shutdown.c:967
buf = "buffer modified while frozen!", '\000' <repeats 226 times>
__pc = 0x0
__pc = 0x0
__pc = 0x0
other_cpus = {__bits = {4093, 0 <repeats 15 times>}}
td = 0xfffff8000e2e2000
bootopt = <unavailable>
newpanic = <optimized out>
#4 0xffffffff80b0d6c3 in panic (fmt=<unavailable>)
at /usr/src/sys/kern/kern_shutdown.c:891
ap = {{gp_offset = 8, fp_offset = 48,
overflow_arg_area = 0xfffffe01590f5d10,
reg_save_area = 0xfffffe01590f5cb0}}
#5 0xffffffff825e34a1 in arc_cksum_verify (buf=0xfffff80603cb39a0)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:1446
zc = {zc_word = {9183713730946906330, 4141878062941643272,
14528101124437143189, 16728594463488619406}}
hdr = 0xfffff804979abbc0
#6 0xffffffff825f14d8 in arc_write_done (zio=0xfffffe0344bd9980,
zio@entry=<error reading variable: value is not available>)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6593
hash_lock = 0x0
exists = <optimized out>
callback = 0xfffff8026374a140
buf = 0xfffff80603cb39a0
hdr = 0xfffff804979abbc0
#7 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bd9980)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988
psize = 8192
zl = 0x0
pio = <optimized out>
pio_next = <optimized out>
next_to_execute = <optimized out>
#8 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
stage = ZIO_STAGE_DONE
pipeline = <optimized out>
#9 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
cookie = 0
#10 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2e00)
at /usr/src/sys/kern/subr_taskqueue.c:518
et = {et_link = {tqe_next = 0xffffffff812183c3, tqe_prev = 0x0},
et_td = 0x0, et_section = {bucket = 2166642066},
et_old_priority = 255 '\377'}
tb = {tb_running = 0xfffffe038b8c1bb8, tb_seq = 6978482,
tb_canceling = false, tb_link = {le_next = 0xfffffe0159104e78,
le_prev = 0xfffffe01590fae88}}
in_net_epoch = false
pending = 1
task = <optimized out>
#11 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830
tqp = <optimized out>
tq = 0xfffff8000e2f2e00
#12 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafef0, frame=0xfffffe01590f5f40)
at /usr/src/sys/kern/kern_fork.c:1153
__pc = 0x0
__pc = 0x0
td = 0xfffff8000e2e2000
p = 0xffffffff81b80fa0 <proc0>
dtd = <optimized out>
#13 <signal handler called>
No locals.
#14 0x6bec65f367ac65f7 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0xccdbfebfc09bfebb
(kgdb) (kgdb) Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid
100266 (CPU 0)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xffffffff81cef340 <nmi0_stack+3888>)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 _sx_xunlock_hard (sx=sx@entry=0xfffff804976ca298, x=<optimized out>,
x@entry=18446735277854429184, file=<optimized out>, line=<optimized out>)
at /usr/src/sys/kern/kern_sx.c:937
#5 0xffffffff80b197a9 in _sx_xunlock (sx=sx@entry=0xfffff804976ca298,
file=<optimized out>, line=line@entry=1447)
at /usr/src/sys/kern/kern_sx.c:412
#6 0xffffffff825e343d in arc_cksum_verify (buf=buf@entry=0xfffff804343bf620)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:1447
#7 0xffffffff825f14d8 in arc_write_done (zio=0xfffffe0344be0ac0)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6593
#8 0xffffffff827c45cf in zio_done (zio=0xfffffe0344be0ac0)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988
#9 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#10 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#11 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2e00)
at /usr/src/sys/kern/subr_taskqueue.c:518
#12 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830
#13 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafef0, frame=0xfffffe01590ebf40)
at /usr/src/sys/kern/kern_fork.c:1153
#14 <signal handler called>
#15 0xc4719ce7c8319ce3 in ?? ()
Backtrace stopped: Cannot access memory at address 0x6b4607ab6f0607af
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100268 (CPU 1)
#0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57
#1 doadump (textdump=textdump@entry=1)
at /usr/src/sys/kern/kern_shutdown.c:405
#2 0xffffffff80b0d420 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:523
#3 0xffffffff80b0d939 in vpanic (
fmt=0xffffffff828755e0 "buffer modified while frozen!",
ap=ap@entry=0xfffffe01590f5ce0) at /usr/src/sys/kern/kern_shutdown.c:967
#4 0xffffffff80b0d6c3 in panic (fmt=<unavailable>)
at /usr/src/sys/kern/kern_shutdown.c:891
#5 0xffffffff825e34a1 in arc_cksum_verify (buf=0xfffff80603cb39a0)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:1446
#6 0xffffffff825f14d8 in arc_write_done (zio=0xfffffe0344bd9980,
zio@entry=<error reading variable: value is not available>)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6593
#7 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bd9980)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988
#8 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#9 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#10 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2e00)
at /usr/src/sys/kern/subr_taskqueue.c:518
#11 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830
#12 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafef0, frame=0xfffffe01590f5f40)
at /usr/src/sys/kern/kern_fork.c:1153
#13 <signal handler called>
#14 0x6bec65f367ac65f7 in ?? ()
Backtrace stopped: Cannot access memory at address 0xccdbfebfc09bfebb
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100258 (CPU 2)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a4cbf30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 _sx_xunlock_hard (sx=sx@entry=0xfffffe0371bde9b0, x=<optimized out>,
x@entry=18446735277854445568, file=<optimized out>, line=<optimized out>)
at /usr/src/sys/kern/kern_sx.c:937
#5 0xffffffff80b197a9 in _sx_xunlock (sx=0xfffffe0371bde9b0,
file=<optimized out>, line=line@entry=742)
at /usr/src/sys/kern/kern_sx.c:412
#6 0xffffffff827b7f51 in zio_add_child_first (pio=<optimized out>,
cio=cio@entry=0xfffffe0336bac000)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:742
#7 0xffffffff827b8464 in zio_create (pio=pio@entry=0xfffffe0371bde600,
spa=0xfffffe010f784000, txg=51255565, bp=0x0, data=0x0,
lsize=lsize@entry=4096, psize=4096, done=0x0, private=0x0,
type=ZIO_TYPE_WRITE, priority=ZIO_PRIORITY_ASYNC_WRITE, flags=1314944,
vd=0xfffffe01595c7000, offset=1105181822976, zb=0xfffffe0371bde600,
stage=ZIO_STAGE_READY,
pipeline=(ZIO_STAGE_ENCRYPT | ZIO_STAGE_CHECKSUM_GENERATE |
ZIO_STAGE_BRT_FREE | ZIO_STAGE_DDT_READ_START | ZIO_STAGE_DDT_WRITE |
ZIO_STAGE_GANG_ISSUE | ZIO_STAGE_DVA_FREE | ZIO_STAGE_DVA_CLAIM |
ZIO_STAGE_VDEV_IO_START | unknown: 0x28000000)) at
/usr/src/sys/contrib/openzfs/module/zfs/zio.c:973
#8 0xffffffff827b9cfe in zio_vdev_child_io (pio=0xfffffe0371bde600,
bp=<optimized out>, vd=0xfffffe01595c7000, offset=1105181822976,
data=<optimized out>, size=4096, type=2,
priority=ZIO_PRIORITY_ASYNC_WRITE, flags=1314944, done=0x0, private=0x0)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:1589
#9 0xffffffff82704fca in vdev_raidz_io_start_write (zio=0xfffffe0371bde600,
rr=0xfffff80057111000, ashift=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1649
#10 vdev_raidz_io_start (zio=0xfffffe0371bde600)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1748
#11 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe0371bde600)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042
#12 0xffffffff827ba208 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#13 zio_nowait (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437
#14 0xffffffff826fd684 in vdev_mirror_io_start (zio=0xfffffe0344d06000)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_mirror.c:687
#15 0xffffffff827c28d3 in zio_vdev_io_start (zio=0xfffffe0344d06000)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:3917
#16 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#17 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#18 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f1100)
at /usr/src/sys/kern/subr_taskqueue.c:518
#19 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004baff30) at /usr/src/sys/kern/subr_taskqueue.c:830
#20 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004baff30, frame=0xfffffe01590c3f40)
at /usr/src/sys/kern/kern_fork.c:1153
#21 <signal handler called>
#22 0x94c89f3998889f3d in ?? ()
Backtrace stopped: Cannot access memory at address 0x33ff04753fbf0471
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100254 (CPU 3)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a4daf30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 0xffffffff80ebeb13 in trash_ctor (mem=mem@entry=0xfffff80580056900,
size=size@entry=384, arg=arg@entry=0x0, flags=flags@entry=257)
at /usr/src/sys/vm/uma_dbg.c:76
#5 0xffffffff80eb71d6 in item_ctor (zone=0xfffffe00e2400000,
uz_flags=1090584576, size=384, udata=0x0, flags=257,
item=0xfffff80580056900) at /usr/src/sys/vm/uma_core.c:3471
#6 0xffffffff80a402ad in uma_zalloc (zone=0xdeadc0dedeadc0de,
zone@entry=0xfffff8000e2b2300, flags=257) at /usr/src/sys/vm/uma.h:367
#7 g_clone_bio (bp=bp@entry=0xfffff8000e2b2300)
at /usr/src/sys/geom/geom_io.c:197
#8 0xffffffff80a429a9 in g_slice_start (bp=0xfffff8000e2b2300)
at /usr/src/sys/geom/geom_slice.c:276
#9 0xffffffff80a40832 in g_io_request (bp=bp@entry=0xfffff8000e2b2300,
cp=cp@entry=0xfffff80004239e00) at /usr/src/sys/geom/geom_io.c:587
#10 0xffffffff825aa0ef in vdev_geom_io_start (zio=0xfffffe02fd667600)
at /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/vdev_geom.c:1255
#11 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe02fd667600)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042
#12 0xffffffff827ba208 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#13 zio_nowait (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437
#14 0xffffffff82704fd6 in vdev_raidz_io_start_write (zio=0xfffffe02efd66140,
rr=0xfffff807adf52000, ashift=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1649
#15 vdev_raidz_io_start (zio=0xfffffe02efd66140)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1748
#16 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe02efd66140)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042
#17 0xffffffff827ba208 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#18 zio_nowait (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437
#19 0xffffffff826fd684 in vdev_mirror_io_start (zio=0xfffffe0344d05ac0)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_mirror.c:687
#20 0xffffffff827c28d3 in zio_vdev_io_start (zio=0xfffffe0344d05ac0)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:3917
#21 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#22 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#23 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f1100)
at /usr/src/sys/kern/subr_taskqueue.c:518
#24 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004baff30) at /usr/src/sys/kern/subr_taskqueue.c:830
#25 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004baff30, frame=0xfffffe010bdf9f40)
at /usr/src/sys/kern/kern_fork.c:1153
#26 <signal handler called>
#27 0xe699f066ead9f062 in ?? ()
Backtrace stopped: Cannot access memory at address 0x41ae6b2a4dee6b2e
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100267 (CPU 4)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a4e9f30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 trash_dtor (mem=mem@entry=0xfffff8042f2b5000, size=size@entry=4096,
arg=arg@entry=0xfffff80006f65240) at /usr/src/sys/vm/uma_dbg.c:101
#5 0xffffffff80eb65a0 in item_dtor (zone=0xfffffe0109828000,
item=0xfffff8042f2b5000, size=4096, udata=0xfffff80006f65240,
skip=SKIP_NONE) at /usr/src/sys/vm/uma_core.c:3513
#6 uma_zfree_arg (zone=0xfffffe0109828000, item=0xfffff8042f2b5000,
udata=0xfffff80006f65240) at /usr/src/sys/vm/uma_core.c:4517
#7 0xffffffff8259ead1 in kmem_cache_free (cache=0xfffff80006f65240,
buf=0x1000)
at /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_kmem.c:209
#8 0xffffffff827b7982 in zio_data_buf_free (buf=0x1000, size=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:421
#9 0xffffffff825e02c6 in abd_free_linear (abd=0xfffff800755d7400)
at /usr/src/sys/contrib/openzfs/module/zfs/abd.c:244
#10 abd_free (abd=0xfffff800755d7400)
at /usr/src/sys/contrib/openzfs/module/zfs/abd.c:306
#11 0xffffffff8270059f in vdev_raidz_row_free (rr=0xfffff801ca082800)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:145
#12 vdev_raidz_map_free (rm=0xfffff80177b75700)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:160
#13 0xffffffff827c2db6 in zio_vdev_io_assess (zio=0xfffffe0344c33e40)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4158
#14 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#15 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#16 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2e00)
at /usr/src/sys/kern/subr_taskqueue.c:518
#17 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830
#18 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafef0, frame=0xfffffe01590faf40)
at /usr/src/sys/kern/kern_fork.c:1153
#19 <signal handler called>
#20 0xa15df4f6ad1df4f2 in ?? ()
Backtrace stopped: Cannot access memory at address 0x66a6fba0a2a6fbe
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100259 (CPU 5)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a4f8f30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 cache_bucket_pop (cache=0xfffffe00e2400400, bucket=0xfffffe00e2400410)
at /usr/src/sys/vm/uma_core.c:916
#5 cache_alloc_item (zone=0xfffffe00e2400000, cache=0xfffffe00e2400400,
bucket=0xfffffe00e2400410, udata=0x0, flags=258)
at /usr/src/sys/vm/uma_core.c:3642
#6 uma_zalloc_arg (zone=0xfffffe00e2400000, udata=udata@entry=0x0,
flags=flags@entry=258) at /usr/src/sys/vm/uma_core.c:3740
#7 0xffffffff80a40268 in uma_zalloc (zone=0x0, flags=258)
at /usr/src/sys/vm/uma.h:367
#8 0xffffffff825a9f90 in vdev_geom_io_start (zio=0xfffffe035d281600)
at /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/vdev_geom.c:1199
#9 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe035d281600)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042
#10 0xffffffff827ba208 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#11 zio_nowait (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437
#12 0xffffffff82704fd6 in vdev_raidz_io_start_write (zio=0xfffffe0388482e40,
rr=0xfffff8079b57c000, ashift=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1649
#13 vdev_raidz_io_start (zio=0xfffffe0388482e40)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1748
#14 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe0388482e40)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042
#15 0xffffffff827ba208 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#16 zio_nowait (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437
#17 0xffffffff826fd684 in vdev_mirror_io_start (zio=0xfffffe0344d064c0)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_mirror.c:687
#18 0xffffffff827c28d3 in zio_vdev_io_start (zio=0xfffffe0344d064c0)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:3917
#19 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#20 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#21 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f1100)
at /usr/src/sys/kern/subr_taskqueue.c:518
#22 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004baff30) at /usr/src/sys/kern/subr_taskqueue.c:830
#23 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004baff30, frame=0xfffffe01590d2f40)
at /usr/src/sys/kern/kern_fork.c:1153
#24 <signal handler called>
#25 0xf1e4f728fda4f72c in ?? ()
Backtrace stopped: Cannot access memory at address 0x56d36c645a936c60
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100256 (CPU 6)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a507f30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 0xffffffff826d0e63 in spa_config_held (spa=spa@entry=0xfffffe010f784000,
locks=locks@entry=22, rw=rw@entry=RW_READER)
at /usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c:585
#5 0xffffffff827b8170 in zio_create (pio=pio@entry=0xfffffe035d2e0980,
spa=0xfffffe010f784000, txg=51255565, bp=0x0, data=0xfffff801e799d278,
lsize=lsize@entry=4096, psize=4096,
done=0xffffffff82701610 <vdev_raidz_child_done>,
private=0xfffff801e799d260, type=ZIO_TYPE_WRITE,
priority=ZIO_PRIORITY_ASYNC_WRITE, flags=1048704, vd=0xfffffe01595cb000,
offset=1105181818880, zb=0xfffffe035d2e0980, stage=ZIO_STAGE_READY,
pipeline=(ZIO_STAGE_VDEV_IO_START | ZIO_STAGE_VDEV_IO_DONE |
ZIO_STAGE_VDEV_IO_ASSESS | ZIO_STAGE_DONE))
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:903
#6 0xffffffff827b9cfe in zio_vdev_child_io (pio=0xfffffe035d2e0980,
bp=<optimized out>, vd=0xfffffe01595cb000, offset=1105181818880,
data=<optimized out>, size=4096, type=2,
priority=ZIO_PRIORITY_ASYNC_WRITE, flags=1048704,
done=0xffffffff82701610 <vdev_raidz_child_done>,
private=0xfffff801e799d260)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:1589
#7 0xffffffff82704fca in vdev_raidz_io_start_write (zio=0xfffffe035d2e0980,
rr=0xfffff801e799d000, ashift=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1649
#8 vdev_raidz_io_start (zio=0xfffffe035d2e0980)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1748
#9 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe035d2e0980)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042
#10 0xffffffff827ba208 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#11 zio_nowait (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437
#12 0xffffffff826fd684 in vdev_mirror_io_start (zio=0xfffffe0344d06980)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_mirror.c:687
#13 0xffffffff827c28d3 in zio_vdev_io_start (zio=0xfffffe0344d06980)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:3917
#14 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#15 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#16 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f1100)
at /usr/src/sys/kern/subr_taskqueue.c:518
#17 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004baff30) at /usr/src/sys/kern/subr_taskqueue.c:830
#18 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004baff30, frame=0xfffffe01590b9f40)
at /usr/src/sys/kern/kern_fork.c:1153
#19 <signal handler called>
#20 0x1a7a3fee163a3fea in ?? ()
Backtrace stopped: Cannot access memory at address 0xbd4da4a2b10da4a6
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100269 (CPU 7)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a516f30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 trash_dtor (mem=mem@entry=0xfffff8028709ddc8, size=size@entry=392,
arg=arg@entry=0xfffff80006f66200) at /usr/src/sys/vm/uma_dbg.c:101
#5 0xffffffff80eb65a0 in item_dtor (zone=0xfffffe010984b000,
item=0xfffff8028709ddc8, size=392, udata=0xfffff80006f66200,
skip=SKIP_NONE) at /usr/src/sys/vm/uma_core.c:3513
#6 uma_zfree_arg (zone=0xfffffe010984b000,
item=item@entry=0xfffff8028709ddc8, udata=0xfffff80006f66200)
at /usr/src/sys/vm/uma_core.c:4517
#7 0xffffffff8259ead1 in kmem_cache_free (cache=0xfffff80006f66200,
buf=0x188, buf@entry=0xfffff8028709ddc8)
at /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_kmem.c:209
#8 0xffffffff8260f28a in dbuf_destroy (db=0xfffff8028709ddc8)
at /usr/src/sys/contrib/openzfs/module/zfs/dbuf.c:3116
#9 0xffffffff8261374e in dbuf_rele_and_unlock (db=0xfffff8028709df00,
tag=<optimized out>, evicting=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/dbuf.c:3999
#10 0xffffffff8261b196 in dbuf_write_done (zio=0xfffffe0344bcaac0,
buf=<optimized out>, vdb=0xfffff8028709ddc8)
at /usr/src/sys/contrib/openzfs/module/zfs/dbuf.c:4858
#11 0xffffffff825f196f in arc_write_done (zio=0xfffffe0344bcaac0)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6639
#12 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bcaac0)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988
#13 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#14 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#15 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2e00)
at /usr/src/sys/kern/subr_taskqueue.c:518
#16 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830
#17 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafef0, frame=0xfffffe0159104f40)
at /usr/src/sys/kern/kern_fork.c:1153
#18 <signal handler called>
#19 0xa9ff18e6a5bf18e2 in ?? ()
Backtrace stopped: Cannot access memory at address 0xec883aa028883ae
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100271 (CPU 8)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a525f30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 arc_write_done (zio=zio@entry=0xfffffe0344bf2980)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6564
#5 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bf2980)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988
#6 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#7 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#8 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2d00)
at /usr/src/sys/kern/subr_taskqueue.c:518
#9 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafee0) at /usr/src/sys/kern/subr_taskqueue.c:830
#10 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafee0, frame=0xfffffe015910ef40)
at /usr/src/sys/kern/kern_fork.c:1153
#11 <signal handler called>
#12 0xbe5f9160b21f9164 in ?? ()
Backtrace stopped: Cannot access memory at address 0x19684a3c15280a28
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100273 (CPU 9)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a534f30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 slab_item_index (slab=0xfffff80483ec7be8, keg=0xfffff80006f38a80,
item=0xfffff804810a0000) at /usr/src/sys/vm/uma_int.h:435
#5 uma_dbg_free (zone=zone@entry=0xfffffe0109828000, slab=0xfffff80483ec7be8,
slab@entry=0xfffff80006f65240, item=item@entry=0xfffff804810a0000)
at /usr/src/sys/vm/uma_core.c:5816
#6 0xffffffff80eb6566 in item_dtor (zone=0xfffffe0109828000,
item=0xfffff804810a0000, size=4096, udata=0xfffff80006f65240,
skip=SKIP_NONE) at /usr/src/sys/vm/uma_core.c:3501
#7 uma_zfree_arg (zone=0xfffffe0109828000, item=0xfffff804810a0000,
udata=0xfffff80006f65240) at /usr/src/sys/vm/uma_core.c:4517
#8 0xffffffff8259ead1 in kmem_cache_free (cache=0xfffff804810a0000,
buf=0xfffff80006f38a80)
at /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_kmem.c:209
#9 0xffffffff827b7982 in zio_data_buf_free (buf=0xfffff80006f38a80,
size=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:421
#10 0xffffffff825e02c6 in abd_free_linear (abd=0xfffff8061b8a3e00)
at /usr/src/sys/contrib/openzfs/module/zfs/abd.c:244
#11 abd_free (abd=0xfffff8061b8a3e00)
at /usr/src/sys/contrib/openzfs/module/zfs/abd.c:306
#12 0xffffffff8270059f in vdev_raidz_row_free (rr=0xfffff803e72eb000)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:145
#13 vdev_raidz_map_free (rm=0xfffff804f1092200)
at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:160
#14 0xffffffff827c2db6 in zio_vdev_io_assess (zio=0xfffffe0383e9a7c0)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4158
#15 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#16 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#17 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2d00)
at /usr/src/sys/kern/subr_taskqueue.c:518
#18 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafee0) at /usr/src/sys/kern/subr_taskqueue.c:830
#19 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafee0, frame=0xfffffe0159118f40)
at /usr/src/sys/kern/kern_fork.c:1153
#20 <signal handler called>
#21 0xc4527342c8127346 in ?? ()
Backtrace stopped: Cannot access memory at address 0x6365e80e6f25e80a
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100272 (CPU 10)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a543f30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 0xffffffff826d48b4 in bp_get_dsize_sync (spa=0xfffffe010f784000,
bp=bp@entry=0xfffffe0327b7da00)
at /usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c:2140
#5 0xffffffff82652161 in dsl_dataset_block_born (
ds=ds@entry=0xfffff80037ed2000, bp=bp@entry=0xfffffe0327b7da00,
tx=tx@entry=0xfffff8043315e200)
at /usr/src/sys/contrib/openzfs/module/zfs/dsl_dataset.c:140
#6 0xffffffff8261ad22 in dbuf_write_done (zio=0xfffffe0344be5ac0,
buf=<optimized out>, vdb=0xfffff8021e32dc40)
at /usr/src/sys/contrib/openzfs/module/zfs/dbuf.c:4810
#7 0xffffffff825f196f in arc_write_done (zio=0xfffffe0344be5ac0)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6639
#8 0xffffffff827c45cf in zio_done (zio=0xfffffe0344be5ac0)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988
#9 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#10 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#11 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2d00)
at /usr/src/sys/kern/subr_taskqueue.c:518
#12 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafee0) at /usr/src/sys/kern/subr_taskqueue.c:830
#13 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafee0, frame=0xfffffe0159109f40)
at /usr/src/sys/kern/kern_fork.c:1153
#14 <signal handler called>
#15 0xa17e1b53ad3e1b57 in ?? ()
Backtrace stopped: Cannot access memory at address 0x1649801f0a09801b
Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100270 (CPU 11)
#0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530
#1 0xffffffff81009388 in ipi_nmi_handler ()
at /usr/src/sys/x86/x86/mp_x86.c:1487
#2 0xffffffff810406a6 in trap (frame=0xfffffe003a552f30)
at /usr/src/sys/amd64/amd64/trap.c:248
#3 <signal handler called>
#4 zfs_refcount_remove_many (rc=rc@entry=0xfffff8048def2098,
number=number@entry=1, holder=0xfffff8048def2000)
at /usr/src/sys/contrib/openzfs/module/zfs/refcount.c:176
#5 0xffffffff826ab722 in zfs_refcount_remove (rc=rc@entry=0xfffff8048def2098,
holder=0xfffff8048def2000)
at /usr/src/sys/contrib/openzfs/module/zfs/refcount.c:212
#6 0xffffffff825e7200 in remove_reference (hdr=hdr@entry=0xfffff8048def2000,
tag=tag@entry=0xfffff8048def2000)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:2314
#7 0xffffffff825f182c in arc_write_done (zio=0xfffffe0344be4c80)
at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6629
#8 0xffffffff827c45cf in zio_done (zio=0xfffffe0344be4c80)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988
#9 0xffffffff827baea8 in __zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350
#10 zio_execute (zio=<optimized out>)
at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261
#11 0xffffffff80b75392 in taskqueue_run_locked (
queue=queue@entry=0xfffff8000e2f2d00)
at /usr/src/sys/kern/subr_taskqueue.c:518
#12 0xffffffff80b76283 in taskqueue_thread_loop (
arg=arg@entry=0xfffff80004bafee0) at /usr/src/sys/kern/subr_taskqueue.c:830
#13 0xffffffff80ac0022 in fork_exit (
callout=0xffffffff80b761b0 <taskqueue_thread_loop>,
arg=0xfffff80004bafee0, frame=0xfffffe01590fff40)
at /usr/src/sys/kern/kern_fork.c:1153
#14 <signal handler called>
#15 0xfc4cec75700cec71 in ?? ()
Backtrace stopped: Cannot access memory at address 0xdb7b7739d73b773d
(kgdb)
--
You are receiving this mail because:
You are the assignee for the bug.
