panic: uma_zalloc_arg: Returning an empty bucket.
I just got this panic on my 11.0-CURRENT machine while building ports: panic: uma_zalloc_arg: Returning an empty bucket. cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper(c113330c,2,1000,ee3fd850,ee3fd848,...) at db_trace_self_wrapper+0x2d/frame 0xee3fd810 kdb_backtrace(c12f131f,0,c116b406,ee3fd8e4,c116b406,...) at kdb_backtrace+0x30/frame 0xee3fd878 vpanic(c1410ab8,100,c116b406,ee3fd8e4,ee3fd8e4,...) at vpanic+0x11f/frame 0xee3fd8b4 kassert_panic(c116b406,0,c116b3eb,883,1,...) at kassert_panic+0xea/frame 0xee3fd8d8 uma_zalloc_arg(c26d58c0,0,101,0,0,...) at uma_zalloc_arg+0x4bd/frame 0xee3fd918 vm_radix_insert(d25b6780,c403c194,c116d974,3c7,716,...) at vm_radix_insert+0xfa/frame 0xee3fd960 vm_page_insert_after(6348,0,c403c150,637,0,...) at vm_page_insert_after+0xc1/frame 0xee3fd98c vm_page_alloc(d25b6750,6348,0,40,ee3fdacc,...) at vm_page_alloc+0x4fe/frame 0xee3fd9b8 vm_fault_hold(c99b7000,8e9f1000,2,0,0,...) at vm_fault_hold+0x3cb/frame 0xee3fdb18 vm_fault(c99b7000,8e9f1000,2,0,c99b7000,...) at vm_fault+0x82/frame 0xee3fdb40 trap_pfault(8e9f1000,ee3fdd08,0,c9252620,c141e780,...) at trap_pfault+0x198/frame 0xee3fdbb8 trap(ee3fdd08) at trap+0x5e2/frame 0xee3fdcfc calltrap() at calltrap+0x6/frame 0xee3fdcfc --- trap 0xc, eip = 0x88607643, esp = 0xbfbfce98, ebp = 0xbfbfced8 --- KDB: enter: panic FreeBSD scratch.catspoiler.org 11.0-CURRENT FreeBSD 11.0-CURRENT #60 r258423M: Thu Nov 21 02:27:29 PST 2013 d...@scratch.catspoiler.org:/usr/obj/usr/src/sys/GENERICSMB i386 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: uma_zalloc_arg: Returning an empty bucket.
on 10/11/2010 07:58 Andriy Gapon said the following: > > I've been doing some stress-testing for VM+ZFS on a kernel with > WITNESS+INVARIANTS and I have already stumbled into some curious panics. > > Here's another one: > Unread portion of the kernel message buffer: > panic: uma_zalloc_arg: Returning an empty bucket. Looks like this particular panic was caused by my local changes for draining per-CPU caches in very low memory conditions. > cpuid = 0 > KDB: stack backtrace: > db_trace_self_wrapper() at 0x801b8b3a = db_trace_self_wrapper+0x2a > kdb_backtrace() at 0x803a7a8a = kdb_backtrace+0x3a > panic() at 0x80370632 = panic+0x1d2 > uma_zalloc_arg() at 0x80507850 = uma_zalloc_arg+0x300 > kmem_cache_alloc() at 0x80dac385 = kmem_cache_alloc+0x15 > arc_buf_alloc() at 0x80c016e4 = arc_buf_alloc+0x34 > arc_read_nolock() at 0x80c0389d = arc_read_nolock+0x2bd > arc_read() at 0x80c03dff = arc_read+0x7f > dbuf_prefetch() at 0x80c0826a = dbuf_prefetch+0x15a > dmu_zfetch_dofetch() at 0x80c21444 = dmu_zfetch_dofetch+0x104 > dmu_zfetch() at 0x80c21c89 = dmu_zfetch+0x5e9 > dbuf_read() at 0x80c06c04 = dbuf_read+0x454 > dmu_buf_hold_array_by_dnode() at 0x80c0a58d = > dmu_buf_hold_array_by_dnode+0x27d > dmu_read() at 0x80c0a80f = dmu_read+0xbf > zfs_freebsd_getpages() at 0x80c6c9b5 = zfs_freebsd_getpages+0x8c5 > VOP_GETPAGES_APV() at 0x805a6618 = VOP_GETPAGES_APV+0xe8 > vnode_pager_getpages() at 0x80523c09 = vnode_pager_getpages+0xb9 > vm_fault() at 0x8050a67a = vm_fault+0xf5a > trap_pfault() at 0x80545d6f = trap_pfault+0x11f > trap() at 0x80546448 = trap+0x4a8 > calltrap() at 0x80530628 = calltrap+0x8 > > (kgdb) bt > #0 doadump () at pcpu.h:224 > #1 0x8036fc9a in kern_reboot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:306 > #2 0x80370668 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:609 > #3 0x80507850 in uma_zalloc_arg (zone=0xff00071c6e00, > udata=0xff00071c8380, flags=2) at /usr/src/sys/vm/uma_core.c:2132 > #4 0x80dac385 in kmem_cache_alloc (cache=Variable "cache" is not > available. > ) at > /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/opensolaris_kmem.c:196 > #5 0x80c016e4 in arc_buf_alloc (spa=0xff00071e5000, size=131072, > tag=Variable "tag" is not available. > ) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1344 > #6 0x80c0389d in arc_read_nolock (pio=0x0, spa=0xff00071e5000, > bp=0xff8002c71e80, done=0, private=0x0, priority=6, zio_flags=3, > arc_flags=0xff812673b2bc, zb=0xff812673b280) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2931 > #7 0x80c03dff in arc_read (pio=0x0, spa=0xff00071e5000, > bp=0xff8002c71e80, pbuf=0xff00abaa31b0, done=0, private=0x0, > priority=6, > zio_flags=3, arc_flags=0xff812673b2bc, zb=0xff812673b280) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2832 > #8 0x80c0826a in dbuf_prefetch (dn=0xff00b440f000, blkid=221) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1641 > #9 0x80c21444 in dmu_zfetch_dofetch (zf=0xff00b440f2b8, > zs=0xff0126bb6480) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_zfetch.c:308 > #10 0x80c21c89 in dmu_zfetch (zf=0xff00b440f2b8, offset=Variable > "offset" is not available. > ) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_zfetch.c:526 > #11 0x80c06c04 in dbuf_read (db=0xff0031970b60, > zio=0xff0078bb2328, flags=54) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:568 > #12 0x80c0a58d in dmu_buf_hold_array_by_dnode (dn=0xff00b440f000, > offset=Variable "offset" is not available. > ) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:235 > #13 0x80c0a80f in dmu_read (os=Variable "os" is not available. > ) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:572 > #14 0x80c6c9b5 in zfs_freebsd_getpages (ap=Variable "ap" is not > available. > ) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4348 > #15 0x805a6618 in VOP_GETPAGES_APV (vop=0x80cd8680, >
panic: uma_zalloc_arg: Returning an empty bucket.
I've been doing some stress-testing for VM+ZFS on a kernel with WITNESS+INVARIANTS and I have already stumbled into some curious panics. Here's another one: Unread portion of the kernel message buffer: panic: uma_zalloc_arg: Returning an empty bucket. cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at 0x801b8b3a = db_trace_self_wrapper+0x2a kdb_backtrace() at 0x803a7a8a = kdb_backtrace+0x3a panic() at 0x80370632 = panic+0x1d2 uma_zalloc_arg() at 0x80507850 = uma_zalloc_arg+0x300 kmem_cache_alloc() at 0x80dac385 = kmem_cache_alloc+0x15 arc_buf_alloc() at 0x80c016e4 = arc_buf_alloc+0x34 arc_read_nolock() at 0x80c0389d = arc_read_nolock+0x2bd arc_read() at 0x80c03dff = arc_read+0x7f dbuf_prefetch() at 0x80c0826a = dbuf_prefetch+0x15a dmu_zfetch_dofetch() at 0x80c21444 = dmu_zfetch_dofetch+0x104 dmu_zfetch() at 0x80c21c89 = dmu_zfetch+0x5e9 dbuf_read() at 0x80c06c04 = dbuf_read+0x454 dmu_buf_hold_array_by_dnode() at 0x80c0a58d = dmu_buf_hold_array_by_dnode+0x27d dmu_read() at 0x80c0a80f = dmu_read+0xbf zfs_freebsd_getpages() at 0x80c6c9b5 = zfs_freebsd_getpages+0x8c5 VOP_GETPAGES_APV() at 0x805a6618 = VOP_GETPAGES_APV+0xe8 vnode_pager_getpages() at 0x80523c09 = vnode_pager_getpages+0xb9 vm_fault() at 0x8050a67a = vm_fault+0xf5a trap_pfault() at 0x80545d6f = trap_pfault+0x11f trap() at 0x80546448 = trap+0x4a8 calltrap() at 0x80530628 = calltrap+0x8 (kgdb) bt #0 doadump () at pcpu.h:224 #1 0x8036fc9a in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:306 #2 0x80370668 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:609 #3 0x80507850 in uma_zalloc_arg (zone=0xff00071c6e00, udata=0xff00071c8380, flags=2) at /usr/src/sys/vm/uma_core.c:2132 #4 0x80dac385 in kmem_cache_alloc (cache=Variable "cache" is not available. ) at /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/opensolaris_kmem.c:196 #5 0x80c016e4 in arc_buf_alloc (spa=0xff00071e5000, size=131072, tag=Variable "tag" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1344 #6 0x80c0389d in arc_read_nolock (pio=0x0, spa=0xff00071e5000, bp=0xff8002c71e80, done=0, private=0x0, priority=6, zio_flags=3, arc_flags=0xff812673b2bc, zb=0xff812673b280) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2931 #7 0x80c03dff in arc_read (pio=0x0, spa=0xff00071e5000, bp=0xff8002c71e80, pbuf=0xff00abaa31b0, done=0, private=0x0, priority=6, zio_flags=3, arc_flags=0xff812673b2bc, zb=0xff812673b280) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2832 #8 0x80c0826a in dbuf_prefetch (dn=0xff00b440f000, blkid=221) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1641 #9 0x80c21444 in dmu_zfetch_dofetch (zf=0xff00b440f2b8, zs=0xff0126bb6480) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_zfetch.c:308 #10 0x80c21c89 in dmu_zfetch (zf=0xff00b440f2b8, offset=Variable "offset" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_zfetch.c:526 #11 0x80c06c04 in dbuf_read (db=0xff0031970b60, zio=0xff0078bb2328, flags=54) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:568 #12 0x80c0a58d in dmu_buf_hold_array_by_dnode (dn=0xff00b440f000, offset=Variable "offset" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:235 #13 0x80c0a80f in dmu_read (os=Variable "os" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:572 #14 0x80c6c9b5 in zfs_freebsd_getpages (ap=Variable "ap" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4348 #15 0x805a6618 in VOP_GETPAGES_APV (vop=0x80cd8680, a=0xff812673b740) at vnode_if.c:2666 #16 0x80523c09 in vnode_pager_getpages (object=Variable "object" is not available. ) at vnode_if.h:1154 #17 0x8050a67a in vm_fault (map=0xff00ab3d5c40, vaddr=35585523712, fault_type=1 '\001', fault_flags=Variable "fault_flags" is not available. ) at vm_pager.h:130 #18 0x80545d6f in trap_pfault (frame=0xff812673bc40, usermode=1) at /usr/src/sys/amd64/amd64/trap.c:729 #19 0x80546448 in trap (frame=0xff812673bc40) at /usr/src/sys/amd64/amd64/trap.c:403 #20 0x8053062
panic: uma_zalloc_arg: Returning an empty bucket.
I just saw a strange panic on a dual CPU x86:
(kernel from Tuesday)
panic: uma_zalloc_arg: Returning an empty bucket.
cpuid = 0; lapic.id =
Debugger("panic")
Stopped at Debugger+0x55: xchgl %ebx,in_Debugger.0
db> tr
Debugger(c03477a6,0,c035a2ba,d69b6bc0,1) at Debugger+0x55
panic(c035a2ba,0,c035a07d,54c,0) at panic+0x11f
uma_zalloc_arg(c082da00,0,5,4b,c082da00) at uma_zalloc_arg+0x241
malloc(78,c0376da0,5,4,d69b6c48) at malloc+0x75
g_new_bio(4b,d69b6c5c,c01e,c03aae88,4) at g_new_bio+0x37
g_clone_bio(c4559980,c03aae80,d69b6cb0,d69b6c74,c01b794c) at g_clone_bio+0x14
g_slice_start(c4559980,d69b6d0c,c01b8278,c1581c30,d69b6cb0) at g_slice_start+0x6f
g_io_schedule_down(c1581c30,d69b6cb0,4c,c0342e7a,a) at g_io_schedule_down+0x2e
g_down_procbody(0,d69b6d48,c0344f96,34c,90140004) at g_down_procbody+0x88
fork_exit(c01b81f0,0,d69b6d48) at fork_exit+0xa5
fork_trampoline() at fork_trampoline+0x1a
--- trap 0x1, eip = 0, esp = 0xd69b6d7c, ebp = 0 ---
db> show locks
exclusive sleep mutex 128 (UMA zone) r = 0 (0xc082da24) locked @
../../../vm/uma_core.c:1356
exclusive sleep mutex PCPU 128 (UMA cpu) r = 0 (0xc082dae8) locked @
../../../vm/uma_core.c:1312
exclusive sleep mutex g_down r = 0 (0xd69b6cb0) locked @ ../../../geom/geom_kern.c:113
db> sho pcpu
cpuid= 0
curthread= 0xc1581c30: pid 4 "g_down"
curpcb = 0xd69b6da0
fpcurthread = none
idlethread = 0xc1580a90: pid 12 "idle: cpu0"
currentldt = 0x28
spin locks held:
db> c
boot() called on cpu#1
syncing disks... panic: KSE not on run queue
cpuid = 0; lapic.id =
Debugger("panic")
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
