Re: panic, seems related to r234386

2012-05-16 Thread Pedro Giffuni

Hello;

On 05/12/12 17:49, Mateusz Guzik wrote:


Gave this a spin and found what looks like a deadlock:

http://people.freebsd.org/~pho/stress/log/ext2fs.txt

Not a new problem, it would seem. Same issue with 8.3-PRERELEASE r232656M.


pid 2680 (fts) holds lock for vnode cb4be414 and tries to lock cc0ac15c
pid 2581 (openat) holds lock for vnode cc0ac15c and tries to lock cb4be414

openat calls rmdir foo/bar and ext2_rmdir unlocks and tries to lock
again foo's vnode.

This is fairly easly reproducible with concurrently running mkdir and fts
testcase programs that are provided by stress2.

I'll try to come up with a patch by the end of the week.


Easier way to reproduce: mkdir from stress2 and while true; do find /mnt
/dev/null; done on another terminal.

Assuming foo/bar directory tree, deadlock happens during removal of bar
with simultaneous lookup of .. in bar.

Proposed trivial patch:
http://student.agh.edu.pl/~mjguzik/patches/ext2fs_rmdir-deadlock.patch

If the lock cannot be acquired immediately unlocks 'bar' vnode and then
locks both vnodes in order.

After patching this I ran into another issue - wrong vnode type panics
from cache_enter_time after calls by ext2_lookup. (It takes some time to
reproduce this, testcase as before.)

It looks like ext2_lookup is actually adapted version of ufs_lookup and
lacks some bugfixes present in current ufs_lookup. I believe those
bugfixes address this bug.

Here is my attempt to fix the problem (based on ufs_lookup changes):
http://student.agh.edu.pl/~mjguzik/patches/ext2fs_lookup-relookup.patch



It is indeed extremely useful that UFS and ext2fs are so similar.

The two bugfixes were committed as revision 235508, thanks!

Pedro.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-13 Thread Peter Holm
On Sun, May 13, 2012 at 12:49:38AM +0200, Mateusz Guzik wrote:
 On Thu, May 10, 2012 at 12:39:00PM +0200, Peter Holm wrote:
  On Thu, May 10, 2012 at 12:21:18PM +0200, Mateusz Guzik wrote:
   On Tue, May 08, 2012 at 09:45:14PM +0200, Peter Holm wrote:
On Mon, May 07, 2012 at 10:11:53PM +0200, Mateusz Guzik wrote:
 On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
  On 05/06/2012 15:19, Sergey Kandaurov wrote:
   On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
   I got this with today's current, previous (working) kernel is 
   r232719.
  
   panic: _mtx_lock_sleep: recursed on non-recursive mutex struct 
   mount mtx
   @ /frontier/svn/head/sys/kern/vfs_subr.c:4595
  
  ...
  
   Please try this patch.
   
   Index: fs/ext2fs/ext2_vfsops.c
   ===
   --- fs/ext2fs/ext2_vfsops.c (revision 235108)
   +++ fs/ext2fs/ext2_vfsops.c (working copy)
   @@ -830,7 +830,6 @@
   /*
* Write back each (modified) inode.
*/
   -   MNT_ILOCK(mp);
loop:
   MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
   if (vp-v_type == VNON) {
   
  
  Didn't help, sorry. I put 234385 through some pretty heavy load
  yesterday, and everything was fine. As soon as I move up to 234386, 
  the
  panic triggered again. So I cleaned everything up, applied your 
  patch,
  built a kernel from scratch, and rebooted. It was Ok for a few 
  seconds
  after boot, then panic'ed again, I think in a different place, but 
  I'm
  not sure because subsequent attempts to fsck the file systems 
  caused new
  panics which overwrote the old ones before they could be saved.
  
 
 Another MNT_ILOCK was hiding few lines below, try this patch:
 
 http://student.agh.edu.pl/~mjguzik/patches/ext2fs-ilock.patch
 
 I've tested this a bit and I believe this fixes your problem.
 

Gave this a spin and found what looks like a deadlock:

http://people.freebsd.org/~pho/stress/log/ext2fs.txt

Not a new problem, it would seem. Same issue with 8.3-PRERELEASE 
r232656M.

   
   pid 2680 (fts) holds lock for vnode cb4be414 and tries to lock cc0ac15c
   pid 2581 (openat) holds lock for vnode cc0ac15c and tries to lock cb4be414
   
   openat calls rmdir foo/bar and ext2_rmdir unlocks and tries to lock
   again foo's vnode.
   
   This is fairly easly reproducible with concurrently running mkdir and fts
   testcase programs that are provided by stress2.
   
   I'll try to come up with a patch by the end of the week.
   
  
 
 Easier way to reproduce: mkdir from stress2 and while true; do find /mnt 
 /dev/null; done on another terminal.
 
 Assuming foo/bar directory tree, deadlock happens during removal of bar
 with simultaneous lookup of .. in bar.
 
 Proposed trivial patch:
 http://student.agh.edu.pl/~mjguzik/patches/ext2fs_rmdir-deadlock.patch
 
 If the lock cannot be acquired immediately unlocks 'bar' vnode and then
 locks both vnodes in order.
 
 After patching this I ran into another issue - wrong vnode type panics
 from cache_enter_time after calls by ext2_lookup. (It takes some time to
 reproduce this, testcase as before.)
 
 It looks like ext2_lookup is actually adapted version of ufs_lookup and
 lacks some bugfixes present in current ufs_lookup. I believe those
 bugfixes address this bug.
 
 Here is my attempt to fix the problem (based on ufs_lookup changes):
 http://student.agh.edu.pl/~mjguzik/patches/ext2fs_lookup-relookup.patch
 

I have tested these two patches for a few hours and they do indeed
seem to fix the problem I had seen before.

Regards,

- Peter
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-12 Thread Mateusz Guzik
On Thu, May 10, 2012 at 12:39:00PM +0200, Peter Holm wrote:
 On Thu, May 10, 2012 at 12:21:18PM +0200, Mateusz Guzik wrote:
  On Tue, May 08, 2012 at 09:45:14PM +0200, Peter Holm wrote:
   On Mon, May 07, 2012 at 10:11:53PM +0200, Mateusz Guzik wrote:
On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
 On 05/06/2012 15:19, Sergey Kandaurov wrote:
  On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
  I got this with today's current, previous (working) kernel is 
  r232719.
 
  panic: _mtx_lock_sleep: recursed on non-recursive mutex struct 
  mount mtx
  @ /frontier/svn/head/sys/kern/vfs_subr.c:4595
 
 ...
 
  Please try this patch.
  
  Index: fs/ext2fs/ext2_vfsops.c
  ===
  --- fs/ext2fs/ext2_vfsops.c (revision 235108)
  +++ fs/ext2fs/ext2_vfsops.c (working copy)
  @@ -830,7 +830,6 @@
  /*
   * Write back each (modified) inode.
   */
  -   MNT_ILOCK(mp);
   loop:
  MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
  if (vp-v_type == VNON) {
  
 
 Didn't help, sorry. I put 234385 through some pretty heavy load
 yesterday, and everything was fine. As soon as I move up to 234386, 
 the
 panic triggered again. So I cleaned everything up, applied your patch,
 built a kernel from scratch, and rebooted. It was Ok for a few seconds
 after boot, then panic'ed again, I think in a different place, but I'm
 not sure because subsequent attempts to fsck the file systems caused 
 new
 panics which overwrote the old ones before they could be saved.
 

Another MNT_ILOCK was hiding few lines below, try this patch:

http://student.agh.edu.pl/~mjguzik/patches/ext2fs-ilock.patch

I've tested this a bit and I believe this fixes your problem.

   
   Gave this a spin and found what looks like a deadlock:
   
   http://people.freebsd.org/~pho/stress/log/ext2fs.txt
   
   Not a new problem, it would seem. Same issue with 8.3-PRERELEASE r232656M.
   
  
  pid 2680 (fts) holds lock for vnode cb4be414 and tries to lock cc0ac15c
  pid 2581 (openat) holds lock for vnode cc0ac15c and tries to lock cb4be414
  
  openat calls rmdir foo/bar and ext2_rmdir unlocks and tries to lock
  again foo's vnode.
  
  This is fairly easly reproducible with concurrently running mkdir and fts
  testcase programs that are provided by stress2.
  
  I'll try to come up with a patch by the end of the week.
  
 

Easier way to reproduce: mkdir from stress2 and while true; do find /mnt 
/dev/null; done on another terminal.

Assuming foo/bar directory tree, deadlock happens during removal of bar
with simultaneous lookup of .. in bar.

Proposed trivial patch:
http://student.agh.edu.pl/~mjguzik/patches/ext2fs_rmdir-deadlock.patch

If the lock cannot be acquired immediately unlocks 'bar' vnode and then
locks both vnodes in order.

After patching this I ran into another issue - wrong vnode type panics
from cache_enter_time after calls by ext2_lookup. (It takes some time to
reproduce this, testcase as before.)

It looks like ext2_lookup is actually adapted version of ufs_lookup and
lacks some bugfixes present in current ufs_lookup. I believe those
bugfixes address this bug.

Here is my attempt to fix the problem (based on ufs_lookup changes):
http://student.agh.edu.pl/~mjguzik/patches/ext2fs_lookup-relookup.patch

-- 
Mateusz Guzik mjguzik gmail.com
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-10 Thread Mateusz Guzik
On Tue, May 08, 2012 at 09:45:14PM +0200, Peter Holm wrote:
 On Mon, May 07, 2012 at 10:11:53PM +0200, Mateusz Guzik wrote:
  On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
   On 05/06/2012 15:19, Sergey Kandaurov wrote:
On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
I got this with today's current, previous (working) kernel is r232719.
   
panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount 
mtx
@ /frontier/svn/head/sys/kern/vfs_subr.c:4595
   
   ...
   
Please try this patch.

Index: fs/ext2fs/ext2_vfsops.c
===
--- fs/ext2fs/ext2_vfsops.c (revision 235108)
+++ fs/ext2fs/ext2_vfsops.c (working copy)
@@ -830,7 +830,6 @@
/*
 * Write back each (modified) inode.
 */
-   MNT_ILOCK(mp);
 loop:
MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
if (vp-v_type == VNON) {

   
   Didn't help, sorry. I put 234385 through some pretty heavy load
   yesterday, and everything was fine. As soon as I move up to 234386, the
   panic triggered again. So I cleaned everything up, applied your patch,
   built a kernel from scratch, and rebooted. It was Ok for a few seconds
   after boot, then panic'ed again, I think in a different place, but I'm
   not sure because subsequent attempts to fsck the file systems caused new
   panics which overwrote the old ones before they could be saved.
   
  
  Another MNT_ILOCK was hiding few lines below, try this patch:
  
  http://student.agh.edu.pl/~mjguzik/patches/ext2fs-ilock.patch
  
  I've tested this a bit and I believe this fixes your problem.
  
 
 Gave this a spin and found what looks like a deadlock:
 
 http://people.freebsd.org/~pho/stress/log/ext2fs.txt
 
 Not a new problem, it would seem. Same issue with 8.3-PRERELEASE r232656M.
 

pid 2680 (fts) holds lock for vnode cb4be414 and tries to lock cc0ac15c
pid 2581 (openat) holds lock for vnode cc0ac15c and tries to lock cb4be414

openat calls rmdir foo/bar and ext2_rmdir unlocks and tries to lock
again foo's vnode.

This is fairly easly reproducible with concurrently running mkdir and fts
testcase programs that are provided by stress2.

I'll try to come up with a patch by the end of the week.

-- 
Mateusz Guzik mjguzik gmail.com
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-10 Thread Peter Holm
On Thu, May 10, 2012 at 12:21:18PM +0200, Mateusz Guzik wrote:
 On Tue, May 08, 2012 at 09:45:14PM +0200, Peter Holm wrote:
  On Mon, May 07, 2012 at 10:11:53PM +0200, Mateusz Guzik wrote:
   On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
On 05/06/2012 15:19, Sergey Kandaurov wrote:
 On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
 I got this with today's current, previous (working) kernel is 
 r232719.

 panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount 
 mtx
 @ /frontier/svn/head/sys/kern/vfs_subr.c:4595

...

 Please try this patch.
 
 Index: fs/ext2fs/ext2_vfsops.c
 ===
 --- fs/ext2fs/ext2_vfsops.c (revision 235108)
 +++ fs/ext2fs/ext2_vfsops.c (working copy)
 @@ -830,7 +830,6 @@
 /*
  * Write back each (modified) inode.
  */
 -   MNT_ILOCK(mp);
  loop:
 MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
 if (vp-v_type == VNON) {
 

Didn't help, sorry. I put 234385 through some pretty heavy load
yesterday, and everything was fine. As soon as I move up to 234386, the
panic triggered again. So I cleaned everything up, applied your patch,
built a kernel from scratch, and rebooted. It was Ok for a few seconds
after boot, then panic'ed again, I think in a different place, but I'm
not sure because subsequent attempts to fsck the file systems caused new
panics which overwrote the old ones before they could be saved.

   
   Another MNT_ILOCK was hiding few lines below, try this patch:
   
   http://student.agh.edu.pl/~mjguzik/patches/ext2fs-ilock.patch
   
   I've tested this a bit and I believe this fixes your problem.
   
  
  Gave this a spin and found what looks like a deadlock:
  
  http://people.freebsd.org/~pho/stress/log/ext2fs.txt
  
  Not a new problem, it would seem. Same issue with 8.3-PRERELEASE r232656M.
  
 
 pid 2680 (fts) holds lock for vnode cb4be414 and tries to lock cc0ac15c
 pid 2581 (openat) holds lock for vnode cc0ac15c and tries to lock cb4be414
 
 openat calls rmdir foo/bar and ext2_rmdir unlocks and tries to lock
 again foo's vnode.
 
 This is fairly easly reproducible with concurrently running mkdir and fts
 testcase programs that are provided by stress2.
 
 I'll try to come up with a patch by the end of the week.
 

Great. Thank you for looking at this.

- Peter
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-10 Thread Sergey Kandaurov
On 8 May 2012 00:11, Mateusz Guzik mjgu...@gmail.com wrote:
 On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
 On 05/06/2012 15:19, Sergey Kandaurov wrote:
  On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
  I got this with today's current, previous (working) kernel is r232719.
 
  panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
  @ /frontier/svn/head/sys/kern/vfs_subr.c:4595

 ...

  Please try this patch.
 
  Index: fs/ext2fs/ext2_vfsops.c
  ===
  --- fs/ext2fs/ext2_vfsops.c     (revision 235108)
  +++ fs/ext2fs/ext2_vfsops.c     (working copy)
  @@ -830,7 +830,6 @@
          /*
           * Write back each (modified) inode.
           */
  -       MNT_ILOCK(mp);
   loop:
          MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
                  if (vp-v_type == VNON) {
 

 Didn't help, sorry. I put 234385 through some pretty heavy load
 yesterday, and everything was fine. As soon as I move up to 234386, the
 panic triggered again. So I cleaned everything up, applied your patch,
 built a kernel from scratch, and rebooted. It was Ok for a few seconds
 after boot, then panic'ed again, I think in a different place, but I'm
 not sure because subsequent attempts to fsck the file systems caused new
 panics which overwrote the old ones before they could be saved.


 Another MNT_ILOCK was hiding few lines below, try this patch:

 http://student.agh.edu.pl/~mjguzik/patches/ext2fs-ilock.patch

 I've tested this a bit and I believe this fixes your problem.

I just committed your fix in r235241 since it seems to have no
relation to the problem reported by Peter Holm.

-- 
wbr,
pluknet
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-10 Thread Doug Barton
On 05/07/2012 23:14, Sergey Kandaurov wrote:
  Finally, should my next step be to advance to the latest current + your
  patch and see how I go from there?
 Yep, so that patches will be tested before they go to head.

For the record, I upgraded to r235151 + the removal of those 2 locks and
haven't had any problems yet. I was going to do one more update today
and report back, but you beat me to it. :)

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-08 Thread Sergey Kandaurov
On 8 May 2012 05:14, Doug Barton do...@freebsd.org wrote:
 On 05/07/2012 13:11, Mateusz Guzik wrote:
 On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
 On 05/06/2012 15:19, Sergey Kandaurov wrote:
 On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
 I got this with today's current, previous (working) kernel is r232719.

 panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
 @ /frontier/svn/head/sys/kern/vfs_subr.c:4595

 ...

 Please try this patch.

 Ok, so far so good. Again, thanks for the quick response. I'm
 stress-testing my ext2fs partitions a bit atm, and everything seems Ok.
 I'll let you know if I run into any problems.

 So my next question is, does removing those locks present any risks?
 Should they be replaced by different locks, or were they just safety
 belts to start with?

Unlike in the previously used macro MNT_VNODE_FOREACH_ABORT_ILOCKED(),
the currently used macro MNT_VNODE_FOREACH_ALL_ABORT() manages
mount mutexes itself so you don't need to mess with them. The locking is there,
it's just hidden behind macros.


 Finally, should my next step be to advance to the latest current + your
 patch and see how I go from there?

Yep, so that patches will be tested before they go to head.

-- 
wbr,
pluknet
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-08 Thread Peter Holm
On Mon, May 07, 2012 at 10:11:53PM +0200, Mateusz Guzik wrote:
 On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
  On 05/06/2012 15:19, Sergey Kandaurov wrote:
   On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
   I got this with today's current, previous (working) kernel is r232719.
  
   panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
   @ /frontier/svn/head/sys/kern/vfs_subr.c:4595
  
  ...
  
   Please try this patch.
   
   Index: fs/ext2fs/ext2_vfsops.c
   ===
   --- fs/ext2fs/ext2_vfsops.c (revision 235108)
   +++ fs/ext2fs/ext2_vfsops.c (working copy)
   @@ -830,7 +830,6 @@
   /*
* Write back each (modified) inode.
*/
   -   MNT_ILOCK(mp);
loop:
   MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
   if (vp-v_type == VNON) {
   
  
  Didn't help, sorry. I put 234385 through some pretty heavy load
  yesterday, and everything was fine. As soon as I move up to 234386, the
  panic triggered again. So I cleaned everything up, applied your patch,
  built a kernel from scratch, and rebooted. It was Ok for a few seconds
  after boot, then panic'ed again, I think in a different place, but I'm
  not sure because subsequent attempts to fsck the file systems caused new
  panics which overwrote the old ones before they could be saved.
  
 
 Another MNT_ILOCK was hiding few lines below, try this patch:
 
 http://student.agh.edu.pl/~mjguzik/patches/ext2fs-ilock.patch
 
 I've tested this a bit and I believe this fixes your problem.
 

Gave this a spin and found what looks like a deadlock:

http://people.freebsd.org/~pho/stress/log/ext2fs.txt

Not a new problem, it would seem. Same issue with 8.3-PRERELEASE r232656M.

- Peter
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-07 Thread Doug Barton
On 05/06/2012 15:19, Sergey Kandaurov wrote:
 On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
 I got this with today's current, previous (working) kernel is r232719.

 panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
 @ /frontier/svn/head/sys/kern/vfs_subr.c:4595

...

 Please try this patch.
 
 Index: fs/ext2fs/ext2_vfsops.c
 ===
 --- fs/ext2fs/ext2_vfsops.c (revision 235108)
 +++ fs/ext2fs/ext2_vfsops.c (working copy)
 @@ -830,7 +830,6 @@
 /*
  * Write back each (modified) inode.
  */
 -   MNT_ILOCK(mp);
  loop:
 MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
 if (vp-v_type == VNON) {
 

Didn't help, sorry. I put 234385 through some pretty heavy load
yesterday, and everything was fine. As soon as I move up to 234386, the
panic triggered again. So I cleaned everything up, applied your patch,
built a kernel from scratch, and rebooted. It was Ok for a few seconds
after boot, then panic'ed again, I think in a different place, but I'm
not sure because subsequent attempts to fsck the file systems caused new
panics which overwrote the old ones before they could be saved.

I'd like to see this changed backed out until the proponents can
thoroughly regression test all of the file systems that it affects.

FWIW, the thing that seems to be triggering the panic is that I have the
following setup:

/dev/ad0s2a on / (ufs, local)
/dev/ad3s2 on /home (ext2fs, local)

/etc/namedb is a symlink to /home/named/etc/namedb. When I booted 234386
named failed to start because the symlink was corrupted. When I
recreated the symlink and then started named, it panic'ed.

hth,

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-07 Thread Mateusz Guzik
On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
 On 05/06/2012 15:19, Sergey Kandaurov wrote:
  On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
  I got this with today's current, previous (working) kernel is r232719.
 
  panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
  @ /frontier/svn/head/sys/kern/vfs_subr.c:4595
 
 ...
 
  Please try this patch.
  
  Index: fs/ext2fs/ext2_vfsops.c
  ===
  --- fs/ext2fs/ext2_vfsops.c (revision 235108)
  +++ fs/ext2fs/ext2_vfsops.c (working copy)
  @@ -830,7 +830,6 @@
  /*
   * Write back each (modified) inode.
   */
  -   MNT_ILOCK(mp);
   loop:
  MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
  if (vp-v_type == VNON) {
  
 
 Didn't help, sorry. I put 234385 through some pretty heavy load
 yesterday, and everything was fine. As soon as I move up to 234386, the
 panic triggered again. So I cleaned everything up, applied your patch,
 built a kernel from scratch, and rebooted. It was Ok for a few seconds
 after boot, then panic'ed again, I think in a different place, but I'm
 not sure because subsequent attempts to fsck the file systems caused new
 panics which overwrote the old ones before they could be saved.
 

Another MNT_ILOCK was hiding few lines below, try this patch:

http://student.agh.edu.pl/~mjguzik/patches/ext2fs-ilock.patch

I've tested this a bit and I believe this fixes your problem.

-- 
Mateusz Guzik mjguzik gmail.com
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-07 Thread Doug Barton
On 05/07/2012 13:11, Mateusz Guzik wrote:
 On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
 On 05/06/2012 15:19, Sergey Kandaurov wrote:
 On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
 I got this with today's current, previous (working) kernel is r232719.

 panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
 @ /frontier/svn/head/sys/kern/vfs_subr.c:4595

 ...

 Please try this patch.

Ok, so far so good. Again, thanks for the quick response. I'm
stress-testing my ext2fs partitions a bit atm, and everything seems Ok.
I'll let you know if I run into any problems.

So my next question is, does removing those locks present any risks?
Should they be replaced by different locks, or were they just safety
belts to start with?

Finally, should my next step be to advance to the latest current + your
patch and see how I go from there?


Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


panic, seems related to r234386

2012-05-06 Thread Doug Barton
I got this with today's current, previous (working) kernel is r232719.

panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
@ /frontier/svn/head/sys/kern/vfs_subr.c:4595

FWIW I'm using ufs2, ext2fs, and msdosfs on this system. The panic
occurred right after loading the linux kernel module, but I'm not 100%
sure that's related. Full core.txt.0 is in my home directory on freefall.

#0  doadump (textdump=1) at /frontier/svn/head/sys/kern/kern_shutdown.c:268
268 if (textdump  textdump_pending) {
(kgdb) #0  doadump (textdump=1) at
/frontier/svn/head/sys/kern/kern_shutdown.c:268
#1  0x80482999 in kern_reboot (howto=260)
at /frontier/svn/head/sys/kern/kern_shutdown.c:454
#2  0x804823ca in panic (fmt=0x0)
at /frontier/svn/head/sys/kern/kern_shutdown.c:642
#3  0x80471c25 in _mtx_lock_sleep (m=Variable m is not available.
)
at /frontier/svn/head/sys/kern/kern_mutex.c:363
#4  0x80471cb0 in _mtx_lock_flags (m=Variable m is not available.
)
at /frontier/svn/head/sys/kern/kern_mutex.c:212
#5  0x80517353 in __mnt_vnode_first_all (mvp=0xff819e9f0a58,
mp=0xfe00051dc310) at /frontier/svn/head/sys/kern/vfs_subr.c:4595
#6  0x8042bdd5 in ext2_sync (mp=0xfe00051dc310, waitfor=2)
at /frontier/svn/head/sys/fs/ext2fs/ext2_vfsops.c:835
#7  0x80521446 in sys_sync (td=Variable td is not available.
)
at /frontier/svn/head/sys/kern/vfs_syscalls.c:150
#8  0x806b1d52 in amd64_syscall (td=0xfe0005149480, traced=0)
at subr_syscall.c:135
#9  0x8069d697 in Xfast_syscall ()
at /frontier/svn/head/sys/amd64/amd64/exception.S:387
#10 0x0008008b061c in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)


-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-06 Thread Sergey Kandaurov
On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
 I got this with today's current, previous (working) kernel is r232719.

 panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
 @ /frontier/svn/head/sys/kern/vfs_subr.c:4595

 FWIW I'm using ufs2, ext2fs, and msdosfs on this system. The panic
 occurred right after loading the linux kernel module, but I'm not 100%
 sure that's related. Full core.txt.0 is in my home directory on freefall.

 #0  doadump (textdump=1) at /frontier/svn/head/sys/kern/kern_shutdown.c:268
 268             if (textdump  textdump_pending) {
 (kgdb) #0  doadump (textdump=1) at
 /frontier/svn/head/sys/kern/kern_shutdown.c:268
 #1  0x80482999 in kern_reboot (howto=260)
    at /frontier/svn/head/sys/kern/kern_shutdown.c:454
 #2  0x804823ca in panic (fmt=0x0)
    at /frontier/svn/head/sys/kern/kern_shutdown.c:642
 #3  0x80471c25 in _mtx_lock_sleep (m=Variable m is not available.
 )
    at /frontier/svn/head/sys/kern/kern_mutex.c:363
 #4  0x80471cb0 in _mtx_lock_flags (m=Variable m is not available.
 )
    at /frontier/svn/head/sys/kern/kern_mutex.c:212
 #5  0x80517353 in __mnt_vnode_first_all (mvp=0xff819e9f0a58,
    mp=0xfe00051dc310) at /frontier/svn/head/sys/kern/vfs_subr.c:4595
 #6  0x8042bdd5 in ext2_sync (mp=0xfe00051dc310, waitfor=2)
    at /frontier/svn/head/sys/fs/ext2fs/ext2_vfsops.c:835
 #7  0x80521446 in sys_sync (td=Variable td is not available.
 )
    at /frontier/svn/head/sys/kern/vfs_syscalls.c:150
 #8  0x806b1d52 in amd64_syscall (td=0xfe0005149480, traced=0)
    at subr_syscall.c:135
 #9  0x8069d697 in Xfast_syscall ()
    at /frontier/svn/head/sys/amd64/amd64/exception.S:387
 #10 0x0008008b061c in ?? ()
 Previous frame inner to this frame (corrupt stack?)
 (kgdb)


Please try this patch.

Index: fs/ext2fs/ext2_vfsops.c
===
--- fs/ext2fs/ext2_vfsops.c (revision 235108)
+++ fs/ext2fs/ext2_vfsops.c (working copy)
@@ -830,7 +830,6 @@
/*
 * Write back each (modified) inode.
 */
-   MNT_ILOCK(mp);
 loop:
MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
if (vp-v_type == VNON) {

-- 
wbr,
pluknet
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-06 Thread Doug Barton
On 05/06/2012 15:19, Sergey Kandaurov wrote:

 Please try this patch.
 
 Index: fs/ext2fs/ext2_vfsops.c
 ===
 --- fs/ext2fs/ext2_vfsops.c (revision 235108)
 +++ fs/ext2fs/ext2_vfsops.c (working copy)
 @@ -830,7 +830,6 @@
 /*
  * Write back each (modified) inode.
  */
 -   MNT_ILOCK(mp);
  loop:
 MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
 if (vp-v_type == VNON) {
 

I'm currently in the process of building an r234385 world to verify that
things worked properly before r234386. Once that's done, I'll try
r234386 without the patch to verify the panic, then try with the patch
and report back. Probably not till much later today though.

Thanks for the quick response,

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org