First off, I know that cross posting is evil, but I wanted to try
and make sure developers saw it.
On Tue, 27 Oct 2009, Olaf Seibert wrote:
I see an annoying behaviour with NFS over TCP. It happens both with nfs
and newnfs. This is with FreeBSD/amd64 8.0-RC1 as client. The server is
some Linux or perhaps Solaris, I'm not entirely sure.
After trying to find something in packet traces, I think I have found
something.
The scenario seems to be as follows. Sorry for the width of the lines.
No. Time Source Destination Protocol Info
2296 2992.216855 xxx.xxx.31.43 xxx.xxx.16.142 NFS V3
LOOKUP Call (Reply In 2297), DH:0x3819da36/w
2297 2992.217107 xxx.xxx.16.142 xxx.xxx.31.43 NFS V3
LOOKUP Reply (Call In 2296) Error:NFS3ERR_NOENT
2298 2992.217141 xxx.xxx.31.43 xxx.xxx.16.142 NFS V3
LOOKUP Call (Reply In 2299), DH:0x170cb16a/bin
2299 2992.217334 xxx.xxx.16.142 xxx.xxx.31.43 NFS V3
LOOKUP Reply (Call In 2298), FH:0x61b8eb12
2300 2992.217361 xxx.xxx.31.43 xxx.xxx.16.142 NFS V3
ACCESS Call (Reply In 2301), FH:0x61b8eb12
2301 2992.217582 xxx.xxx.16.142 xxx.xxx.31.43 NFS V3
ACCESS Reply (Call In 2300)
2302 2992.217605 xxx.xxx.31.43 xxx.xxx.16.142 NFS V3
LOOKUP Call (Reply In 2303), DH:0x61b8eb12/w
2303 2992.217860 xxx.xxx.16.142 xxx.xxx.31.43 NFS V3
LOOKUP Reply (Call In 2302) Error:NFS3ERR_NOENT
2304 2992.318770 xxx.xxx.31.43 xxx.xxx.16.142 TCP 934 >
nfs [ACK] Seq=238293 Ack=230289 Win=8192 Len=0 TSV=86492342 TSER=12393434
2306 3011.537520 xxx.xxx.16.142 xxx.xxx.31.43 NFS V3
GETATTR Reply (Call In 2305) Directory mode:2755 uid:4100 gid:4100
2307 3011.637744 xxx.xxx.31.43 xxx.xxx.16.142 TCP 934 >
nfs [ACK] Seq=238429 Ack=230405 Win=8192 Len=0 TSV=86511662 TSER=12395366
2308 3371.534980 xxx.xxx.16.142 xxx.xxx.31.43 TCP nfs >
934 [FIN, ACK] Seq=230405 Ack=238429 Win=49232 Len=0 TSV=12431366 TSER=86511662
The server decides, for whatever reason, to terminate the
connection and sends a FIN.
2309 3371.535018 xxx.xxx.31.43 xxx.xxx.16.142 TCP 934 >
nfs [ACK] Seq=238429 Ack=230406 Win=8192 Len=0 TSV=86871578 TSER=12431366
Client acknowledges this,
2310 3375.379693 xxx.xxx.31.43 xxx.xxx.16.142 NFS V3
ACCESS Call, FH:0x008002a2
but tries to sneak in another call anyway. [A]
Probably not the best behaviour, but I think it is technically allowed by
TCP. (My TCP is very rusty, but I think the socket should be in
TCPS_CLOSE_WAIT at this point and the BSD code will have called
socantrcvmore(), but not socantsndmore().)
2311 3375.474788 xxx.xxx.16.142 xxx.xxx.31.43 TCP nfs >
934 [ACK] Seq=230406 Ack=238569 Win=49232 Len=0 TSV=12431760 TSER=86875423
Server ACKs but doesn't send anything else... [B]
Time passes...
This is where it seems interesting. It looks to me like the socket upcall
for receiving the FIN would have happened before this point, setting the
ct_error.re_status to RPC_CANTRECV, but the code in clnt_vc_call() doesn't
check for this. (It does check for it happening during and after the
sosend(), but not before it, from what I can see.)
[B] would be a bug of the server in my opinion. If it ACKs a call, it
should send a reply. And if it can't, it shouldn't.
I'll leave this one for the TCP wizzards. I'm not sure what the
correct behaviour is when data is received on a connection. (I think
it is waiting for a FIN from the client side at this point.)
If you could try the following patch and see if it helps, that would be
appreciated, rick
ps: I'll try to reproduce the situation here, but I'm not sure if I can.
--- rpc/clnt_vc.c.sav 2009-10-28 15:44:20.000000000 -0400
+++ rpc/clnt_vc.c 2009-10-28 15:49:57.000000000 -0400
@@ -413,6 +413,19 @@
cr->cr_xid = xid;
mtx_lock(&ct->ct_lock);
+ /*
+ * Check to see if the other end has already started to close down
+ * the connection. If it happens after this point, it will be
+ * detected below, when cr->cr_error is checked.
+ */
+ if (ct->ct_error.re_status == RPC_CANTRECV) {
+ if (errp != &ct->ct_error) {
+ errp->re_errno = ct->ct_error.re_errno;
+ errp->re_status = RPC_CANTRECV;
+ }
+ stat = RPC_CANTRECV;
+ goto out;
+ }
TAILQ_INSERT_TAIL(&ct->ct_pending, cr, cr_link);
mtx_unlock(&ct->ct_lock);
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"
