Outstanding NBD requests bug? (Was: Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues)

2007-06-26 Thread Mike Snitzer
Peter, Paul, The original message (below) didn't get any response. Could you elaborate on the impact of this NBD issue? Based on your description, having requests just sit on the queue would _seem_ to be quite serious. Is this purely a bookkeeping issue or is there a more subtle side-effect of

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-29 Thread Peter Zijlstra
On Sun, 2007-04-29 at 21:41 +0200, Rogier Wolff wrote: > On Tue, Apr 17, 2007 at 10:37:38PM -0700, Andrew Morton wrote: > > Florin, can we please see /proc/meminfo as well? > > > > Also the result of `echo m > /proc/sysrq-trigger' > > Hi, > > It's been a while since this thread died out, but may

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-29 Thread Rogier Wolff
On Tue, Apr 17, 2007 at 10:37:38PM -0700, Andrew Morton wrote: > Florin, can we please see /proc/meminfo as well? > > Also the result of `echo m > /proc/sysrq-trigger' Hi, It's been a while since this thread died out, but maybe I'm having the same problem. Networking, large part of memory is b

Re: Success! Was: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-20 Thread Florin Iucha
On Fri, Apr 20, 2007 at 09:37:30AM -0400, Trond Myklebust wrote: > Thanks! Did you ever find out what had happened to the test that hung > last night? Nope. I could not ssh into it and the machine was needed for some windows duty before I got home ;) I'll try again this coming week-end and let y

Re: Success! Was: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-20 Thread Trond Myklebust
On Fri, 2007-04-20 at 08:30 -0500, Florin Iucha wrote: > On Thu, Apr 19, 2007 at 04:49:31PM -0500, Florin Iucha wrote: > > On Thu, Apr 19, 2007 at 05:30:42PM -0400, Trond Myklebust wrote: > > > > I'm far from the machine right now, so I will do some more tests > > > > tonight, but right now, the ne

Success! Was: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-20 Thread Florin Iucha
On Thu, Apr 19, 2007 at 04:49:31PM -0500, Florin Iucha wrote: > On Thu, Apr 19, 2007 at 05:30:42PM -0400, Trond Myklebust wrote: > > > I'm far from the machine right now, so I will do some more tests > > > tonight, but right now, the new patchset is not good. What is the > > > difference between r

Re: Failure! Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-19 Thread Florin Iucha
On Thu, Apr 19, 2007 at 05:30:42PM -0400, Trond Myklebust wrote: > > I'm far from the machine right now, so I will do some more tests > > tonight, but right now, the new patchset is not good. What is the > > difference between reverting the patch you sent yesterday and your > > current fifth patch

Re: Failure! Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-19 Thread Trond Myklebust
On Thu, 2007-04-19 at 14:58 -0500, Florin Iucha wrote: > On Thu, Apr 19, 2007 at 12:09:42PM -0400, Trond Myklebust wrote: > > See > >http://client.linux-nfs.org/Linux-2.6.x/2.6.21-rc7/ > > > > I'm giving the first 5 patches of that series (i.e. > > linux-2.6.21-001-cleanup_unstable_write.dif t

Failure! Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-19 Thread Florin Iucha
On Thu, Apr 19, 2007 at 12:09:42PM -0400, Trond Myklebust wrote: > See >http://client.linux-nfs.org/Linux-2.6.x/2.6.21-rc7/ > > I'm giving the first 5 patches of that series (i.e. > linux-2.6.21-001-cleanup_unstable_write.dif to > linux-2.6.21-005-fix_nfsv4_resend.dif) an extra beating since t

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-19 Thread Trond Myklebust
On Thu, 2007-04-19 at 10:50 -0500, Florin Iucha wrote: > On Thu, Apr 19, 2007 at 11:17:28AM -0400, Trond Myklebust wrote: > > On Thu, 2007-04-19 at 11:12 -0400, Chuck Lever wrote: > > > Perhaps instead of looking at the number of bytes sent, the logic in the > > > last hunk of this patch should ch

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-19 Thread Florin Iucha
On Thu, Apr 19, 2007 at 11:17:28AM -0400, Trond Myklebust wrote: > On Thu, 2007-04-19 at 11:12 -0400, Chuck Lever wrote: > > Perhaps instead of looking at the number of bytes sent, the logic in the > > last hunk of this patch should check which queue the request is sitting on. > > ??? It would be

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-19 Thread Chuck Lever
Trond Myklebust wrote: On Wed, 2007-04-18 at 20:52 -0500, Florin Iucha wrote: On Wed, Apr 18, 2007 at 10:11:46AM -0400, Trond Myklebust wrote: Do you have a copy of wireshark or ethereal on hand? If so, could you take a look at whether or not any NFS traffic is going between the client and serv

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-19 Thread Trond Myklebust
On Thu, 2007-04-19 at 11:12 -0400, Chuck Lever wrote: > Perhaps instead of looking at the number of bytes sent, the logic in the > last hunk of this patch should check which queue the request is sitting on. ??? It would be a bug for the request to be sitting on _any_ queue when it enters xprt_tra

Success! Was: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Florin Iucha
On Wed, Apr 18, 2007 at 10:45:13PM -0400, Trond Myklebust wrote: > On Wed, 2007-04-18 at 20:52 -0500, Florin Iucha wrote: > > It seems that my original problem report had a big mistake! There is > > no hang, but at some point the write slows down to a trickle (from > > 40,000 blocks/s to 22 blocks

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Trond Myklebust
On Wed, 2007-04-18 at 20:52 -0500, Florin Iucha wrote: > On Wed, Apr 18, 2007 at 10:11:46AM -0400, Trond Myklebust wrote: > > Do you have a copy of wireshark or ethereal on hand? If so, could you > > take a look at whether or not any NFS traffic is going between the > > client and server once the h

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Florin Iucha
On Wed, Apr 18, 2007 at 10:11:46AM -0400, Trond Myklebust wrote: > Do you have a copy of wireshark or ethereal on hand? If so, could you > take a look at whether or not any NFS traffic is going between the > client and server once the hang happens? I used the following command tcpdump -w nfs-

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Peter Zijlstra
On Wed, 2007-04-18 at 10:19 +0200, Peter Zijlstra wrote: > On Tue, 2007-04-17 at 21:19 -0400, Trond Myklebust wrote: > > I've split the issues introduced by the 2.6.21-rcX write code up into 4 > > subproblems. > > > > The first patch is just a cleanup in order to ease review. > > > > Patch number

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Trond Myklebust
On Wed, 2007-04-18 at 09:17 -0500, Florin Iucha wrote: > On Wed, Apr 18, 2007 at 10:11:46AM -0400, Trond Myklebust wrote: > > On Wed, 2007-04-18 at 08:42 -0500, Florin Iucha wrote: > > > Could the port in CLOSE_WAIT state be the culprit? (FWIW > > > the server has been up for 38 days and subjected

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Florin Iucha
On Wed, Apr 18, 2007 at 10:11:46AM -0400, Trond Myklebust wrote: > On Wed, 2007-04-18 at 08:42 -0500, Florin Iucha wrote: > > Could the port in CLOSE_WAIT state be the culprit? (FWIW > > the server has been up for 38 days and subjected to > > this nfs test quite a bit without showing any stress).

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Florin Iucha
On Wed, Apr 18, 2007 at 08:42:25AM -0500, Florin Iucha wrote: > On Wed, Apr 18, 2007 at 09:15:31AM -0400, Trond Myklebust wrote: > The netstat outputs are stable (not changed in 5 minutes): > >http://iucha.net/nfs/21-rc7-nfs3/netstat-server : > > tcp1 0 hermes.iucha.org:nfsze

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Trond Myklebust
On Wed, 2007-04-18 at 08:42 -0500, Florin Iucha wrote: > On Wed, Apr 18, 2007 at 09:15:31AM -0400, Trond Myklebust wrote: > > There is only one request on the 'pending' queue. That would usually > > indicate that the connection to the server is down. Can you check using > > "netstat -t" whether or

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Florin Iucha
On Wed, Apr 18, 2007 at 09:15:31AM -0400, Trond Myklebust wrote: > There is only one request on the 'pending' queue. That would usually > indicate that the connection to the server is down. Can you check using > "netstat -t" whether or not there is a connection in the 'ESTABLISHED' > state to the s

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Trond Myklebust
On Wed, 2007-04-18 at 07:38 -0500, Florin Iucha wrote: > On Tue, Apr 17, 2007 at 10:37:38PM -0700, Andrew Morton wrote: > > Florin, can we please see /proc/meminfo as well? > >http://iucha.net/nfs/21-rc7-nfs2/meminfo > > > Also the result of `echo m > /proc/sysrq-trigger' > >http://iucha

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Florin Iucha
On Tue, Apr 17, 2007 at 10:37:38PM -0700, Andrew Morton wrote: > Florin, can we please see /proc/meminfo as well? http://iucha.net/nfs/21-rc7-nfs2/meminfo > Also the result of `echo m > /proc/sysrq-trigger' http://iucha.net/nfs/21-rc7-nfs2/big-copy This has 'echo m > /proc/sysrq-trigger',

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Trond Myklebust
On Tue, 2007-04-17 at 23:07 -0500, Florin Iucha wrote: > When 'big-copy' hangs, if I switch to a different console and run > 'lsof', '[u]mount', or use shell completion on a network mount then that > process goes into D state. I cannot umount the network shares nor > stop autofs. I cannot do a cl

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread OGAWA Hirofumi
Trond Myklebust <[EMAIL PROTECTED]> writes: > I was mainly interested in feedback from Peter, Florin and Ogawa-san to > find out if this series fixes their problems. You were unfortunate > enough to have been on earlier Ccs, so I didn't dare trim you off. :-) Sorry. I'm trying to reproduce that,

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-18 Thread Peter Zijlstra
On Tue, 2007-04-17 at 21:19 -0400, Trond Myklebust wrote: > I've split the issues introduced by the 2.6.21-rcX write code up into 4 > subproblems. > > The first patch is just a cleanup in order to ease review. > > Patch number 2 ensures that we never release the PG_writeback flag until > _after_

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Andrew Morton
On Tue, 17 Apr 2007 22:14:02 -0700 (PDT) Linus Torvalds <[EMAIL PROTECTED]> wrote: > > > On Tue, 17 Apr 2007, Florin Iucha wrote: > > > > Already did. Traces from vanilla kernel at > >http://iucha.net/nfs/21-rc7/big-copy > > Well, there's a pdflush in io_schedule_timeout/congestion_wait,

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Florin Iucha
On Tue, Apr 17, 2007 at 10:14:02PM -0700, Linus Torvalds wrote: > On Tue, 17 Apr 2007, Florin Iucha wrote: > > > > Already did. Traces from vanilla kernel at > >http://iucha.net/nfs/21-rc7/big-copy > > Well, there's a pdflush in io_schedule_timeout/congestion_wait, and > there's a nfsv4-scv

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Linus Torvalds
On Tue, 17 Apr 2007, Florin Iucha wrote: > > Already did. Traces from vanilla kernel at >http://iucha.net/nfs/21-rc7/big-copy Well, there's a pdflush in io_schedule_timeout/congestion_wait, and there's a nfsv4-scv in svc_recv/nfs_callback_sv, and a lot of processes either just in schedul

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Florin Iucha
On Tue, Apr 17, 2007 at 09:13:50PM -0700, Andrew Morton wrote: > On Tue, 17 Apr 2007 23:07:30 -0500 [EMAIL PROTECTED] (Florin Iucha) wrote: > > > > The process traces are at: > > > > > > > >http://iucha.net/nfs/21-rc7-nfs1/gnome-session > > > >http://iucha.net/nfs/21-rc7-nfs1/big-copy > >

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Andrew Morton
On Tue, 17 Apr 2007 23:07:30 -0500 [EMAIL PROTECTED] (Florin Iucha) wrote: > On Tue, Apr 17, 2007 at 11:54:45PM -0400, Trond Myklebust wrote: > > > The good news is that the Gnome session log-in progresses to the point > > > where both top and bottom bars are painted (gray) and the bottom bar > >

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Florin Iucha
On Tue, Apr 17, 2007 at 11:54:45PM -0400, Trond Myklebust wrote: > > The good news is that the Gnome session log-in progresses to the point > > where both top and bottom bars are painted (gray) and the bottom bar > > is populated with icons (2.6.21-rc7 vanilla stops after displaying the > > splash)

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Trond Myklebust
On Tue, 2007-04-17 at 22:30 -0500, Florin Iucha wrote: > On Tue, Apr 17, 2007 at 11:06:05PM -0400, Trond Myklebust wrote: > > > > I've split the issues introduced by the 2.6.21-rcX write code up into 4 > > > > subproblems. > > > > > > > > The first patch is just a cleanup in order to ease review.

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Florin Iucha
On Tue, Apr 17, 2007 at 11:06:05PM -0400, Trond Myklebust wrote: > > > I've split the issues introduced by the 2.6.21-rcX write code up into 4 > > > subproblems. > > > > > > The first patch is just a cleanup in order to ease review. > > > > > > Patch number 2 ensures that we never release the PG_

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Trond Myklebust
On Tue, 2007-04-17 at 19:58 -0700, Andrew Morton wrote: > On Tue, 17 Apr 2007 21:19:46 -0400 Trond Myklebust <[EMAIL PROTECTED]> wrote: > > > > > I've split the issues introduced by the 2.6.21-rcX write code up into 4 > > subproblems. > > > > The first patch is just a cleanup in order to ease re

Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Andrew Morton
On Tue, 17 Apr 2007 21:19:46 -0400 Trond Myklebust <[EMAIL PROTECTED]> wrote: > > I've split the issues introduced by the 2.6.21-rcX write code up into 4 > subproblems. > > The first patch is just a cleanup in order to ease review. > > Patch number 2 ensures that we never release the PG_writeba

[PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

2007-04-17 Thread Trond Myklebust
I've split the issues introduced by the 2.6.21-rcX write code up into 4 subproblems. The first patch is just a cleanup in order to ease review. Patch number 2 ensures that we never release the PG_writeback flag until _after_ we've either discarded the unstable request altogether, or put it on the