ia64 r255488: panic: uma_zfree: Freeing to non free bucket index. - textdump provided

2013-10-23 Thread Anton Shterenlikht
After updating to r256624, the system is very unresponsive,
and really unusable - any command (ps, df, top, ls, etc.)
might take up to 1 min to return. However, I could not get
it to panic.

So I reverted back to r255488.
This time I managed to obtain a textdump:

http://www.freebsd.org/cgi/query-pr.cgi?pr=183227

Anton
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ia64 r255488: panic: uma_zfree: Freeing to non free bucket index. - textdump provided

2013-10-23 Thread Adrian Chadd
Hi!


On 23 October 2013 02:18, Anton Shterenlikht me...@bris.ac.uk wrote:

 After updating to r256624, the system is very unresponsive,
 and really unusable - any command (ps, df, top, ls, etc.)
 might take up to 1 min to return. However, I could not get
 it to panic.

 So I reverted back to r255488.
 This time I managed to obtain a textdump:

 http://www.freebsd.org/cgi/query-pr.cgi?pr=183227


A bunch has changed between those two revisions; are you able to bisect the
kernel between those two versions to narrow down the scope of the bad
change?

Thanks!


-adrian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ia64 r255488: panic: uma_zfree: Freeing to non free bucket index. - textdump provided

2013-10-23 Thread Anton Shterenlikht
From adrian.ch...@gmail.com Wed Oct 23 14:22:00 2013
On 23 October 2013 02:18, Anton Shterenlikht me...@bris.ac.uk wrote:

 After updating to r256624, the system is very unresponsive,
 and really unusable - any command (ps, df, top, ls, etc.)
 might take up to 1 min to return. However, I could not get
 it to panic.

 So I reverted back to r255488.
 This time I managed to obtain a textdump:

 http://www.freebsd.org/cgi/query-pr.cgi?pr=183227


A bunch has changed between those two revisions; are you able to bisect the
kernel between those two versions to narrow down the scope of the bad
change?

Both revisions are bad.

r255488 gives panics of at least 3 kinds.
r256624 doesn't give panics but is virtually unresponsive -
I have to wait for over minute in some cases for simple
commands to complete (ls, df, ps, top, ssh, etc.)

So since I cannot send back anything certain from r256624,
I decided to revert to r255488,
where at least I can get textdumps.

Anton
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic: uma_zfree: Freeing to non free bucket index.

2013-10-15 Thread John Baldwin
On Monday, October 14, 2013 4:44:28 am Anton Shterenlikht wrote:
 
 BTW, I see in dmesg:
 
 Starting ddb.
 ddb: sysctl: debug.ddb.scripting.scripts: Invalid argument
 /etc/rc: WARNING: failed to start ddb
 
 What is that about?
 
 
 panic: uma_zfree: Freeing to non free bucket index.
 cpuid = 0
 KDB: stack backtrace:
 db_trace_self(0x9ffc00158380) at db_trace_self+0x40
 db_trace_self_wrapper(0x9ffc00607370) at db_trace_self_wrapper+0x70
 kdb_backtrace(0x9ffc00ed0e10, 0x9ffc0058e660, 0x40c, 
0x9ffc010a44a0) at kdb_backtrace+0xc0
 vpanic(0x9ffc00dfc468, 0xa000e26e0fd8, 0x9ffc00ef9670, 
0x9ffc00ed0bc0) at vpanic+0x260
 kassert_panic(0x9ffc00dfc468, 0xe00015e25f90, 0xe00015e243e0, 
0xe0027ffd5200) at kassert_panic+0x120
 uma_zfree_arg(0xe0027ffccfc0, 0xe00015e243e0, 0x0) at 
uma_zfree_arg+0x2d0
 g_destroy_bio(0xe00015e243e0, 0x9ffc004ad4a0, 0x30a, 0x30a) at 
g_destroy_bio+0x30
 g_disk_done(0xe00015e243e0, 0xe00015e15d10, 0xe00012672700, 
0x9ffc006b18c0) at g_disk_done+0x140
 biodone(0xe00015e243e0, 0x9ffc00e0e150, 0xe00010c24030, 0x0, 
0x0, 0x0, 0x9ffc00066890, 0x614) at biodone+0x180
 dadone(0xe00012672600, 0xe00012541000, 0xe00015e243e0, 0x7) at 
dadone+0x620
 camisr_runqueue(0xe00011a2dc00, 0xe00012541054, 0xe00012541000, 
0x135d) at camisr_runqueue+0x6c0
 camisr(0xe00011a2dc20, 0xe00011a2dc00, 0x9ffc00bee9d0, 
0xa000e26e1548) at camisr+0x260
 intr_event_execute_handlers(0xe000111764a0, 0xe0001118d998, 
0xe00011191c00, 0x0) at intr_event_execute_handlers+0x280
 ithread_loop(0xe00011192f00, 0xa000e26e1550, 0xe00011192f14, 
0xe0001118d99c) at ithread_loop+0x1b0
 fork_exit(0x9ffc00e12a90, 0xe00011192f00, 0xa000e26e1550) at 
fork_exit+0x120
 enter_userland() at enter_userland
 KDB: enter: panic
 [ thread pid 12 tid 100015 ]
 Stopped at  kdb_enter+0x92: [I2]addl r14=0xffe2c990,gp ;;
 db 
 
 db scripts
 lockinfo=show locks; show alllocks; show lockedvnods
 db run lockinfo
 db:0:lockinfo show locks
 db:0:locks  show alllocks
 db:0:alllocks  show lockedvnods
 Locked vnodes
 
 0xe0001ab39ba8: tag ufs, type VDIR
 usecount 1, writecount 0, refcount 3 mountedhere 0
 flags (VI_ACTIVE)
 v_object 0xe0001cd30900 ref 0 pages 0
 lock type ufs: EXCL by thread 0xe000183d9680 (pid 41389, cpdup, tid 
100121)
 ino 5467932, on dev da5p1
 
 0xe00015ed3ba8: tag ufs, type VDIR
 usecount 1, writecount 0, refcount 3 mountedhere 0
 flags (VI_ACTIVE)
 v_object 0xe0001cd33e00 ref 0 pages 0
 lock type ufs: EXCL by thread 0xe00012a28900 (pid 41421, cpdup, tid 
100092)
 ino 5467948, on dev da5p1
 
 0xe0001ab16938: tag ufs, type VREG
 usecount 1, writecount 0, refcount 3 mountedhere 0
 flags (VI_ACTIVE)
 v_object 0xe0001cd98a00 ref 0 pages 1
 lock type ufs: EXCL by thread 0xe00018494000 (pid 41337, cpdup, tid 
100137)
 ino 5469420, on dev da5p1
 
 0xe0001b2503b0: tag ufs, type VREG
 usecount 1, writecount 0, refcount 1 mountedhere 0
 flags (VI_ACTIVE)
 lock type ufs: EXCL by thread 0xe00012a28900 (pid 41421, cpdup, tid 
100092)
 ino 5469421, on dev da5p1
 
 0xe0001ab2a760: tag ufs, type VREG
 usecount 1, writecount 0, refcount 1 mountedhere 0
 flags (VI_ACTIVE)
 lock type ufs: EXCL by thread 0xe000183d9680 (pid 41389, cpdup, tid 
100121)
 ino 5469422, on dev da5p1
 db 
 db script zzz=textdump set; capture on; run lockinfo; show pcpu; bt; ps; 
alltrace; capture off; reset
 db run zzz

I think 'reset' is going to reset without doing a dump?

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


panic: uma_zfree: Freeing to non free bucket index.

2013-10-14 Thread Anton Shterenlikht

BTW, I see in dmesg:

Starting ddb.
ddb: sysctl: debug.ddb.scripting.scripts: Invalid argument
/etc/rc: WARNING: failed to start ddb

What is that about?


panic: uma_zfree: Freeing to non free bucket index.
cpuid = 0
KDB: stack backtrace:
db_trace_self(0x9ffc00158380) at db_trace_self+0x40
db_trace_self_wrapper(0x9ffc00607370) at db_trace_self_wrapper+0x70
kdb_backtrace(0x9ffc00ed0e10, 0x9ffc0058e660, 0x40c, 
0x9ffc010a44a0) at kdb_backtrace+0xc0
vpanic(0x9ffc00dfc468, 0xa000e26e0fd8, 0x9ffc00ef9670, 
0x9ffc00ed0bc0) at vpanic+0x260
kassert_panic(0x9ffc00dfc468, 0xe00015e25f90, 0xe00015e243e0, 
0xe0027ffd5200) at kassert_panic+0x120
uma_zfree_arg(0xe0027ffccfc0, 0xe00015e243e0, 0x0) at 
uma_zfree_arg+0x2d0
g_destroy_bio(0xe00015e243e0, 0x9ffc004ad4a0, 0x30a, 0x30a) at 
g_destroy_bio+0x30
g_disk_done(0xe00015e243e0, 0xe00015e15d10, 0xe00012672700, 
0x9ffc006b18c0) at g_disk_done+0x140
biodone(0xe00015e243e0, 0x9ffc00e0e150, 0xe00010c24030, 0x0, 0x0, 
0x0, 0x9ffc00066890, 0x614) at biodone+0x180
dadone(0xe00012672600, 0xe00012541000, 0xe00015e243e0, 0x7) at 
dadone+0x620
camisr_runqueue(0xe00011a2dc00, 0xe00012541054, 0xe00012541000, 
0x135d) at camisr_runqueue+0x6c0
camisr(0xe00011a2dc20, 0xe00011a2dc00, 0x9ffc00bee9d0, 
0xa000e26e1548) at camisr+0x260
intr_event_execute_handlers(0xe000111764a0, 0xe0001118d998, 
0xe00011191c00, 0x0) at intr_event_execute_handlers+0x280
ithread_loop(0xe00011192f00, 0xa000e26e1550, 0xe00011192f14, 
0xe0001118d99c) at ithread_loop+0x1b0
fork_exit(0x9ffc00e12a90, 0xe00011192f00, 0xa000e26e1550) at 
fork_exit+0x120
enter_userland() at enter_userland
KDB: enter: panic
[ thread pid 12 tid 100015 ]
Stopped at  kdb_enter+0x92: [I2]addl r14=0xffe2c990,gp ;;
db 

db scripts
lockinfo=show locks; show alllocks; show lockedvnods
db run lockinfo
db:0:lockinfo show locks
db:0:locks  show alllocks
db:0:alllocks  show lockedvnods
Locked vnodes

0xe0001ab39ba8: tag ufs, type VDIR
usecount 1, writecount 0, refcount 3 mountedhere 0
flags (VI_ACTIVE)
v_object 0xe0001cd30900 ref 0 pages 0
lock type ufs: EXCL by thread 0xe000183d9680 (pid 41389, cpdup, tid 
100121)
ino 5467932, on dev da5p1

0xe00015ed3ba8: tag ufs, type VDIR
usecount 1, writecount 0, refcount 3 mountedhere 0
flags (VI_ACTIVE)
v_object 0xe0001cd33e00 ref 0 pages 0
lock type ufs: EXCL by thread 0xe00012a28900 (pid 41421, cpdup, tid 
100092)
ino 5467948, on dev da5p1

0xe0001ab16938: tag ufs, type VREG
usecount 1, writecount 0, refcount 3 mountedhere 0
flags (VI_ACTIVE)
v_object 0xe0001cd98a00 ref 0 pages 1
lock type ufs: EXCL by thread 0xe00018494000 (pid 41337, cpdup, tid 
100137)
ino 5469420, on dev da5p1

0xe0001b2503b0: tag ufs, type VREG
usecount 1, writecount 0, refcount 1 mountedhere 0
flags (VI_ACTIVE)
lock type ufs: EXCL by thread 0xe00012a28900 (pid 41421, cpdup, tid 
100092)
ino 5469421, on dev da5p1

0xe0001ab2a760: tag ufs, type VREG
usecount 1, writecount 0, refcount 1 mountedhere 0
flags (VI_ACTIVE)
lock type ufs: EXCL by thread 0xe000183d9680 (pid 41389, cpdup, tid 
100121)
ino 5469422, on dev da5p1
db 
db script zzz=textdump set; capture on; run lockinfo; show pcpu; bt; ps; 
alltrace; capture off; reset
db run zzz

Then I see lots of info on the console.

However, on reboot, there's nothing:

from dmesg:
*skip*
Starting syslogd.
No core dumps found.
^^^
Clearing /tmp (X related).
*skip*

# savecore -C -v
unable to open bounds file, using 0
checking for kernel dump on device /dev/da4p1
mediasize = 72834820096
sectorsize = 512
magic mismatch on last dump header on /dev/da4p1
No dump exists
# ls -al /var/crash/
total 16
drwxr-x---   2 root  wheel  512 Oct 13 21:55 .
drwxr-xr-x  28 root  wheel  512 Oct 14 09:37 ..
-rw-r--r--   1 root  wheel2 Oct 13 21:54 bounds
-rw-r--r--   1 root  wheel5 Jun 12  2009 minfree
# 

Where did my textdump go?

Thanks

Anton
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org