Re: [zfs-discuss] zfs mount stuck in zil_replay

[Neil Perrin]

Hi Robert,

Yes, it could be related, or even the bug. Certainly the replay
was (prior to this bug fix) extremely slow.  I don't really have enough
information to determine if it's the exact problem, though after
re-reading your original post I strongly suspect it is.

I also putback a companion fix which should be helpful in determining
if zil_replay is making progress, or hung:

6486496 zil_replay() useful debug

Neil.

Robert Milkowski wrote On 11/09/06 18:10,:


Hello Neil,

I can see http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6478388
integrated. I guess it could be related to problem I described here,
right?

 


___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] zfs mount stuck in zil_replay

[Robert Milkowski]

Hello Robert,

Monday, July 31, 2006, 12:48:30 AM, you wrote:

RM Hello ZFS,

RMSystem was rebooted and after reboot server again

RM System is snv_39, SPARC, T2000

RM bash-3.00# ptree
RM 7 /lib/svc/bin/svc.startd -s
RM   163   /sbin/sh /lib/svc/method/fs-local
RM 254   /usr/sbin/zfs mount -a
RM [...]

RM bash-3.00# zfs list|wc -l
RM   46

RM Using df I can see most file systems are already mounted.

RM   
 ::ps!grep zfs
RM R254163  7  7  0 0x4a004000 0600219e1800 zfs
 0600219e1800::walk thread|::findstack -v
RM stack pointer for thread 300013026a0: 2a10069ebb1
RM [ 02a10069ebb1 cv_wait+0x40() ]
RM   02a10069ec61 txg_wait_synced+0x54(352f0d0, 2f2216d,
RM 3000107fb90, 352f110, 352f112, 352f0c8)
RM   02a10069ed11 zil_replay+0xbc(60022609c08, 600226a1840,
RM 600226a1870, 700d13d8, 7ba2d000, 60006821200)
RM   02a10069ee01 zfs_domount+0x1f0(7ba35000, 700d1000, 60021a8bae0, 1, 0, 
0)
RM   02a10069eed1 zfs_mount+0x160(600225f6080, 1868800, 2a10069f9d8, 
60021a8bae0, 2f, 0)
RM   02a10069efb1 domount+0x9b0(2a10069f8b0, 1, 600225f6080, 0, 0, 0)
RM   02a10069f121 mount+0x110(600013fbcf8, 2a10069fad8, 0, 0, ff38e40c, 
100)
RM   02a10069f221 syscall_ap+0x44(2a0, ffbfeca0, 1118bf4, 600013fbc40, 15, 
0)
RM   02a10069f2e1 syscall_trap32+0xcc(52b3c8, ffbfeca0, 100, ff38e40c, 0, 
0)




RM # iostat -xnz 1
RM extended device statistics
RM r/sw/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
RM 3.0  513.0  192.0 2822.1  0.0  2.10.04.0   0  95
RM c4t600C0FF009258F3E4C4C5601d0
RM extended device statistics
RM r/sw/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
RM 7.0  598.1  388.6 1832.9  0.0  2.00.03.4   0  93
RM c4t600C0FF009258F3E4C4C5601d0
RM ^C
RM bash-3.00#

RM However for many seconds no IOs at all are issued.

RM # mpstat 1
RM [...]

RM CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt idl
RM   00   0   74   209  10900000 00  1   0  99
RM   10   00 1000000 00  0   0 100
RM   20   00 1000000 00  0   0 100
RM   30   00 1000000 00 100   0   0
RM   40   00 1000000 00  0   0 100
RM   50   00 6180000 00  0   0 100
RM   60   00 1000000 00  0   0 100
RM   70   00 1000000 00  0   0 100
RM   83   01 70   120020   2451  1   0  98
RM   90   00 1000000 00  0   0 100
RM  100   00 1000000 00  0   0 100
RM  110   00 1000000 00  0   0 100
RM  120   00 1000000 00  0   0 100
RM  130   00 60   100000 00  0   0 100
RM  140   00 3021000   1880  1   0  99
RM  150   00 1000000 00  0   0 100
RM  160   00 81   120000 00  0   0 100
RM  170   00 70   120000 00  0   0 100
RM  180   00230   430010 00  0   0 100
RM  190   00 90   160000 00  0   0 100
RM  200   00 80   140000 00  0   0 100
RM  210   00 7280100 10  1   0  99
RM  220   0   1738   3470110 50  0   0 100
RM  230   00 70   120000 00  0   0 100
RM  240   00 4060100 00  0   0 100
RM  250   01 5080100 00  0   0 100
RM  260   00 2020000 00  0   0 100
RM  270   00 2020000 00  0   0 100
RM  280   00 2020010 00  0   0 100
RM  290   00 1000000 00  0   0 100
RM  300   00 1000000 00  0   0 100
RM  310   00 6080000 00  0   0 100
RM ^C
RM bash-3.00#

RM # dtrace -n fbt:::entry'{self-t=timestamp;}' -n
RM fbt:::return'/self-t/[EMAIL PROTECTED](timestamp-self-t);self-t=0;}' -n 
tick-5s'{printa(@);exit(0);}'
RM [...]
RM   syscall_mstate 12636328
RM   callout_schedule_1 14428108
RM   hwblkclr   16656308
RM   avl_rotation